I am working for a client and last year we created some report for the purpose of audit and scheduled them to send data into default summary index.
since last month we are observing that all the data is gone and summary is only retaining data for approx 3 month and few days.
1. I checked the indexes.conf there is nothing changed w.r.t to default summary index.
2. index size will allow it to grow 500 GB in each indexer.
3. Data age in MC is showing on some indexers its 163 days and while on some its 79 days.
4. on one of the SH event count is very high compared to other SH's.
5. Raw to Index Size Ratio* is 4.39:1
6. current index size is around 130 GB & raw data size is around 600 GB.
The team maintaining platform have already raised the vendor case.
##################################
[summary]
homePath = volume:hotwarm/summarydb/db
coldPath = volume:cold/summarydb/colddb
thawedPath = /splunk_data/cold/summarydb/thaweddb
tstatsHomePath = volume:hotwarm/summarydb/datamodel_summary
##################################
volume:hotwarm]
path = /splunk_data/hotwarm
# 4.xTB volume allocated to each index.
# 4.xTB Splunk volume leaving some headroom
maxVolumeDataSizeMB = 4000000
###################################
[volume:cold]
path = /splunk_data/cold
# 4.xTB volume allocated to each index.
# 4.xTB Splunk volume leaving some headroom
maxVolumeDataSizeMB = 4000000
##################################
However i am trying to understand all possible scenarios because of which these might have happened.
Suggestions and investigation tips are welcome.
i think these might come down to storage of the disk and high number of event. Any suggestion on further investigation?
↧