When our license_usage.log is rotated to .1 .2 and so it splunk is not correctly handling the rotation as it should.
This log is rotating every 4 minutes and is around 22MB when it rotates.
Looking at a specific clients events (~ one every 10 mins or so) shows that sometimes the events are obtained from license_usage.log AND license_usage.log.1
My best guess is that the license_usage.log is rotated to license_usage.log.1 BEFORE the monitor has read in the new event. The monitor is a default wildcard for all logs in splunk/var/log/splunk so the rolled version is being monitored also. This means that there will always be events that can potentially show up as coming from either source. This actually breaks quite a few of the stock DMC license savedsearches that we have (they look for license_usage.log and not license_usage.*).
I could reduce this effect but increasing the size of the file but i'd rather know why this is occuring so I can diagose this for other customers also.
↧