Working on development of an app with a customer. App is similar in design to ITSI. As such the data collection is the main issue and complexity. System data via PUSH with SNMP / Syslog provides basic event status and situational data but not any data points about the system health that are within threashold.
Example: POWER watts used, thermals by different system zone, data about memory or drive or other system component.
This data is critical to generate historical predictive data points necessary to realize the full value ITSI. But to get this data you have to use a pull technology: IPMITOOLs, SNMPWALK etc. But this runs into scale issues. Time slices with delays between queries, spawning more and more children threads to meet more systems. Forwarders created just for ability to query more systems and this without any tools to help "on-demand" provisioning.
**Question:**
1) It seems ITSI has the same issue. What methodology in a good design can you build so as to meet this issue?
2) Does splunk have any code and or threading recommendations to handle calls for poll based issues and the scale issue it balloons into?
3) Besides home grown CHEF and PUPPET scripts to spawn more "forwarders" with shell scripts to batch out more "pull agents" are their any architectural plans to help with "on demand" forwarder provisioning and scaling?
Thanks,
↧