What are the conditions that cause splunk to create a new hot bucket? I have maxDataSize=auto (750MB) set on my index. I was expecting that data would get written to the same bucket until the maxDataSize was reached (and it is rolled to warm). But that is not what's happening. It looks like splunk is creating a new bucket 1) when it gets "old" data and 2) it looks randomly. This is my current indexes.conf entry:
[foo]
homePath = $SPLUNK_DB/foo/db
coldPath = $SPLUNK_DB/foo/colddb
thawedPath = $SPLUNK_DB/foo/thaweddb
maxDataSize = auto
maxHotBuckets = 10
This is the results from Fire Brigade Log Activity-->Bucket Lifecycle. (I had to retype it because I am on a closed network):
![alt text][1]
I also have the output from this search: |dbinspect index=foo | convert ctime(startEpoch) | convert ctime(endEpoch) | fields id sizeOnDiskMB startEpoch endEpoch
![alt text][2]
I really need to understand why the new buckets are getting created. I am working to update my indexes.conf entry to ensure that no data > 45 days is stored in an index. I understand that bucket 22 was created because old data came in. But I don't understand why it created #24 instead of just putting those events into bucket #23 and same with #25.
Unfortunately, this data source routinely has "older" data coming in. Typically it's minutes/hours, but sometimes if a system feeding us is down, we can get data that is days old. My end goal is to configure my index so that I can enforce a 45 day retention period. I was toying with adding a maxHotSpanSecs to limit the span of the data in any given bucket to 1 day. But I'm not sure what that will do to the number of buckets since I'm not understanding why multiple buckets are being created.
[1]: /storage/temp/150220-bucketlifecycleoutput.png
[2]: /storage/temp/150221-dbinspectoutput.png
↧