In my use-case my source log (tailed by a monitor input stanza) is being archived once a day at midnight and the resulting archive file is tailed by the same input stanza and the the original source log is being deleted. What I noticed is, if the splunk instance monitoring that source goes down while new events are still being written to the source log and if the splunk instance comes back up again only after the original file has been archived and the source log deleted, then the Archiving processor doesn’t verify if any new unread events can be found within the archive which the Tailreader couldn’t read (as during that time the splunk instance was down), please check following example:
02-05-2020 12:53:00.442 +0000 INFO ArchiveProcessor - Handling file=/etc/ArchiveFolder/sourcelog5.log.gz
02-05-2020 12:53:00.443 +0000 INFO ArchiveProcessor - reading path=/etc/ArchiveFolder/sourcelog5.log.gz (seek=0 len=784)
02-05-2020 12:53:00.499 +0000 INFO ArchiveProcessor - Archive with path="/etc/ArchiveFolder/sourcelog5.log.gz" was already indexed as a non-archive, skipping.
02-05-2020 12:53:00.499 +0000 INFO ArchiveProcessor - Finished processing file '/etc/ArchiveFolder/sourcelog5.log.gz', removing from stats
02-05-2020 13:01:31.503 +0000 INFO WatchedFile - Will begin reading at offset=12392 for file='/etc/ArchiveFolder/sourcelog5.log.gz'.
Based on the documentation:
https://docs.splunk.com/Documentation/Splunk/8.0.2/Data/Howlogfilerotationishandled
I would understand that both the Tailing and the Archiving processor should behave the same, but apparently that is not the case here. I also did the complementary test and extracted again the source log within the archive and at that point the Tailing processor realises that there are effectively still some new unread events and it it will start ingesting those at that stage. Why is the Archiving processor missing those new unread events?
↧