Background: I created a dashboard (actually a few dashboards) that used many heavy hitting searches. Well, the Splunk servers couldn't handle the load so I redesigned to using scheduled reports. The users lost flexibility and I had to do a lot more work (instead of four dashboards with user flexibility for report time ranges, I'll have 200+ reports with fixed time ranges) to create all these reports and dashboards, but it seemed to be working.
Problem: Two days later I go to redesign one of the dashboards and the server is failing with "In handler 'savedsearch': Search not executed: Your maximum number of concurrent searches has been reached. usage=10 quota=10 user=..."
It seems like the server isn't caching and remembering that it has already run the report. How can I check on this? How can I get it to reliably work and not pound the server over and over again running these reports over and over again?
↧