Hi,
I have setup a file/dir import input to look at a folder and injest the contents of the log files into splunk, there are a huge number of existing files (5000+) I'd like to import to analyse for history going back 10 years.
What I have noticed is that there appear to be large gaps in the data over periods of time over the last 10 years. When I query for the source as the files in the missing time period there is no data for that file which shows up, but it's marked in the system logs as being imported. The data in the file looks ok, so not sure why it wasn't imported. The only thing that I could thing was that because I copied a large number of files into the folder at once it may have gotten something confused in the indexing process, but I'm surprised if that was the case.
I have setup another index and I'm now drip feeding the log files into a folder to see if it still has issues with the same time periods as before.
Is there any other info on a best practice to import a large number of existing log files?
Thanks.
↧