Some additional information about the environment:
All indexers are running Splunk 6.3.5. The indexers are all cluster peers receiving slave-apps from the cluster master.
I set up some global index retention policies via a distributed configuration bundle. The bundle is being pushed out to the cluster peer nodes and includes an indexes.conf file that should globally set the following:
/opt/splunk/etc/master-apps/defaultindexretentionbundle/default/indexes.conf
indexes.conf:
[default]
maxHotSpanSecs = 7776000
frozenTimePeriodInSecs = 31536000
maxTotalDataSizeMB = 100000
This bundle works for ALL of the indexes on my peer nodes EXCEPT for the internal indexes.
Does not work on:
_internaldb
_introspection
I'm overriding this for the fishbucket and historydb index as I don't want them to follow this global setting.
Strangely enough, running btool --debug on the indexers shows that they're still using the system/default/server.conf for frozenTimePeriodInSecs and maxHotSpanSecs, but are using the custom config bundle for maxTotalDataSizeMB.
/opt/splunk/etc/slave-apps/custominternalindexesbundle/default/indexes.conf [_internal]
/opt/splunk/etc/system/default/indexes.conf frozenTimePeriodInSecs = 2592000
/opt/splunk/etc/system/default/indexes.conf maxHotSpanSecs = 432000
/opt/splunk/etc/slave-apps/defaultindexretentionbundle/default/indexes.conf maxTotalDataSizeMB = 100000
According to the config file order of precedence for cluster peer nodes the apps should take effect in this order:
1. Slave-app local directories (cluster peers only) -- highest priority
2. System local directory
3. App local directories
4. Slave-app default directories (cluster peers only)
5. App default directories
6. System default directory -- lowest priority
With cluster peers, custom settings common to all the peers (those in the slave-app local directories) have the highest precedence.
Can we not override the default settings for these internal indexes? Anyone see something I'm missing? Has anyone else tried doing this?
My next step is to put the config in /defaultindexretentionbundle/default/indexes.conf to see if that makes a difference, but again, according to the order of precedence that shouldn't matter.
↧