Hi,
I followed the guide here:
http://blogs.splunk.com/2015/10/01/use-custom-polygons-in-your-choropleth-maps/ to create my own KML files.
After hours ... well **days** of diving into openstreetmap data to fetch the content I need and to convert them actually to KML (KMZ) I'm really happy that everything is - at least - working in Splunk for me.
Unfortunately, for bigger files like about 25 MB in KMZ format, I encounter performance issues. 25 MB as KMZ results in about over 2000 MB in "splunk binary" format (grid, ray, seg) which gets automatically created when using the `geom` command with this featurecollection the first time.
Besides the fact that Splunk blows up your RAM (I tested a 60 MB KMZ file and 8 GB of server RAM are **NOT** able to handle the initial read-in) the second `geom` command is way faster, but still slow.
In my case, I have the first time about 99 seconds delay until the geom command completes.
--> That is because of the initial setup in Splunk of that KMZ file. So far so good. Not a problem as it will occurs only once.
The second time it will be still about 10-11 seconds. Ok ways better! Speed improved by factor 9, of course but it is still unusable for dashboards etc.
When I look at the search.log I can see about 7 seconds delay because of:
11-13-2015 20:09:57.687 WARN spatial:PointInPolygonIndex - requested tile was at y=506 but returned tile as at 505
11-13-2015 20:10:04.461 INFO spatial:PointInPolygonIndex - forcing remapping (probably due to transposition)
Afterwards there are hundreds of :
................ spatial:DPRGeneralizer - generalizer removed XXXX points from polygon, and kept X
But those are all done in a second so that doesn't matter much (in comparison to the other above).
I know I could do several things now. E.g. using a report saving all this in a CSV or kvstore what I need but well that is not the idea behind, right? But wait: I could also use smaller KMZ files right? Well... the same as before I don't think that this is really a solution..
Besides that I was already forced to split my 1 KMZ file because of eating-as-much-RAM-as-possible issue while reading the KMZ data in but I'm not able to split them even further.
**So finally my major question is: Why splunk believes a remapping is needed?**
If this wouldn't happen we would have a result in about 3-4 seconds (... or even better). Well not the best but factor 3 better than now so I could live with that if nothing else could be done to speed this up.
I created my KML files with QGIS as described in the blog post by @mporath
Any ideas?
Thanks
sedi
Of course I also read this one here (which is VERY basic..):
http://docs.splunk.com/Documentation/Splunk/6.3.0/Viz/Choroplethmaps#Components_for_building_geographic_visualizations
↧