I have a file that I am monitoring on a Heavy Forwarder(HF). The file is JSON logs. On the HF I have the following props.conf:
[EC-json]
KV_MODE=JSON
TIME_PREFIX="timestamp":"
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
SHOULD_LINEMERGE=false
TRUNCATE=0
After the file gets to the indexers, from the SH, I am trying to create several search time extractions.
I first tested from the search bar using this search:
sourcetype=EC-json | rex field =_raw "userid\":.+?,ou=(?\w+)," | rex field = _raw "SourceName.+?:.+?\/\/.+\/(?.+?\.pdf)"
This was successful, I was able to create two new fields user_org and PDF.
Then I tried using props.conf in /etc/apps/search/local/ on the SH:
[EC-json]
EXTRACT-user_org = userid\".+?,ou=(?\w+),
EXTRACT-PDF = SourceName\".+?:.+?\/\/.+\/(?.+?\.pdf)
Here is a sample of my data:
{"timestamp":"02/16/2018 08:02:23","Accountid":"userj", ,"SourceName":"https://share.org.com/sites/reports/ORGReports/report1.pdf","userid":"cn= joe user,ou=SOC,ou=org,ou=company,ou=us"}
I tried using the suggestions here: https://www.splunk.com/blog/2016/06/28/eureka-extracting-key-value-pairs-from-json-fields.html
and added the following to the props.conf on my SH to pull all the information from the userid:
EXTRACT-KVPS = (?:\\[rnt]|:")(?<_KEY_1>[^="\\]+)=(?:\\")?(?<_VAL_1>[^="\\]+)
But that doesn't seem to pull the info into the right fields. And all I care about is the first OU anyway.
Can someone help with my props.conf syntax?
Do I need to escape the quote after userid and SourceName or not?
Thanks.
↧