We have logs in the following format:
E Thu Jul 28 01:05:30 2016 progname1 cid1
msg1
E Thu Jul 28 01:05:30 2016 progname2 cid1
msg2
E Thu Jul 28 01:05:30 2016 progname3 cid2
Some other msg
E Thu Jul 28 01:05:30 2016 progname1 cid1
msg4
The notable fields here are:
- Correlation ID, for example "cid1"
- Error message, for example "msg1"
- Program name, for example "progname1"
Trying to figure out how to do the following with Splunk:
Correlate / aggregate the error stacks for a particular correlation ID and have ability to show when similar error stacks happened in the past, and alert whenever a new, not seen before, stack happens. As unique identifier for a transaction/stack we use a combination of program name and message, like this:
progname1:msg1
progname2:msg2
progname1:msg4
Any ideas / suggestions are appreciated.
↧