Hi
I'm attempting to configure my universal forwarder to read log files from a single directory with multiple subdirectories. We use log rotate so the files will be renamed with (1) up to (4) before starting again. I'm also trying to push those into the right index based on the file name. For example the top level directory is /srv/logs which has multiple subdirectories i.e
application
fileservice
proxyserver
each of these subdirectories contains multiple files from each environment (dev, int, prod etc) Here is an example file name.
application-prod.prod.log, i'm using the following inputs.conf which seems to work(ish). I've changed the monitor names to ensure they are treated as separate and i'm trying to blacklist anything I don't want to appear in each index.
[monitor:///srv/./logs]
blacklist = ppd.*\.log$|prod.*\.log$
sourcetype = service_log
index = nonprod
crcSalt =
[monitor:///srv/logs]
blacklist = devint.*\.log$|int.*\.log$|ft.*\.log$|infradev.*\.log$|nonprod.*\.log$
sourcetype = service_log
index = prod
crcSalt =
So in prod, I only want files that contain .prod and ppd, in nonprod I want devint, int, ft, infradev and nonprod.
So i'm wondering
- Are there better or more performant ways to configure these inputs
- Is there anyway I can check the data is correct in my indexes is correct (no prod data in non prod etc)
- If there are subdirectories should I be using recursive = true?
- The documentation says don't use crcSalt = with log rotate - however I see a number of initcrc errors - should I be setting a initcrclen = 2000 etc?
Sorry this is a long one, thanks for any help.
Thanks
↧