Hi Team
I have an XML dataset that looks like the following
2015-08-08T00:00:00 23:58:00 MCP 6th district, Gaithersburg / Montgomery Village GOSHEN/CENTERWAY 39.1631533333333 -77.1921333333333 No No |
The entries have been broken up correctly in props.conf with line breaking & timestamps applied. I have also used the transforms.conf to remove log header before the events. However the fields automatically extracted by Splunk are of an ugly format such as row{@_id} & row.agency.
I would like to change to the fields to be more descriptive by leveraging the transforms.conf but have not had enough luck in doing this with my method so far.
My props.conf entry referring to transforms are:
KV_MODE=xml
TRANSFORMS-filter=null_queue_filter
REPORT-TrafficXML=TrafficXMLheader,TrafficXMLbody
The filter stanza is working perfectly but the field & value translations are not. My transforms to conf are:
[null_queue_filter]
# To remove non valid entries
REGEX=
DEST_KEY=queue
FORMAT=nullQueue
[TrafficXMLheader]
# To extract multiple field & values from initial line & rename the fields
REGEX = \]*)”\s_uuid="([^\s\>]*)”\s_position="([^\s\>]*)”\s_address="([^\s\>]*)”\>
FORMAT = row_id::$1 row_uuid::$2 row_position::$3 row_address::$4
[TrafficXMLbody]
# To extract field & value pairs in each line & rename the fields
REGEX = \<([^\s\>]*)\>([^\s\>]*)\<\w+\>
FORMAT = $1::$2
As stated, the null_que_filter stanza is working fine but the other stanzas have had no impact.
FYI, If I remove the KV_MODE=xml from props.conf, I lose the existing automated field extractions. Perhaps this would be fine if the transforms.conf worked to create the field::value pairs.
I would also like to exclude the geolocation .... line item also which similar to the header had multiple values but not useful.
An example of what s begin seen from splunk is:
![alt text][1]
So in summary looking to address :
- Multiple defined fields & values within a single line which matches the TrafficXMLheader stanza
- Single field & values per line which matches the TrafficXMLbody stanza
- Remove reference to the geolocation tag & associated content
Thanks in advance team!
[1]: /storage/temp/160225-screen-shot-2016-09-17-at-31101-pm.png
|
↧