Hi,
We have reciently updated our customer's enviroment from 6.4.1 to 6.6.2. Ever since, we have a dashboard that is making our search-head to crash due to bad allocation issues. (By the way the server is 24GB)
I'll say that our dashboard has a base search that loads the events and a couple of post-processing searches that represents those events (| chart or | timechart , nothing else).
I've noticed that if I inspect a job straight from one of the panels I got this amount of events streamed from the indexers:
1.91 dispatch.stream.remote 40 - **84,756,499**
1.20 dispatch.stream.remote.lpspk505 23 - 51,314,390
0.71 dispatch.stream.remote.lpspk506 16 - 33,429,913
0.00 dispatch.stream.remote.ldspk601 1 - 12,196
But if a launch the query in a separate search it works like a charm and I got this little amount:
3.64 dispatch.stream.remote 41 - **2,082,890**
0.18 dispatch.process_remote_timeline 34 1,491,360 93,076
0.01 dispatch.remote_timeline_fullevents 2 197,385 200
0.00 dispatch.stream.local 1 - -
0.00 dispatch.stream.remote.ldspk601 1 - 12,030
2.49 dispatch.stream.remote.lpspk505 25 - 1,316,495
1.15 dispatch.stream.remote.lpspk506 15 - 754,365
Why this huge difference?
I suspect that this huge amount of events streamed is the root cause of the issues, and it goes worst the wider the search windows is.
Thanks, and regards.
↧