We are developing a generating custom command using the Splunk Python SDK. The issue we are having is that only those fields exported from the first 'yield' are extracted in future events (so only those fields appear as Extracted/Interesting Fields). Given these field names change continually, we don't want to provide a static list of potential field names.
To provide some background, the command runs on the search-head and obtains a list of JSON objects from a third-party. The JSON events returned will have differing fields in them. Once the JSON object has been returned, we append a field 'sourcetype' to the record in the hope it picks up our configuration from PROPS.CONF. The data is not indexed and only acquired at search-time.
We are using PROPS.CONF to define our 'sourcetype' with the following configuration:
[mySourceType]
DATETIME_CONFIG = CURRENT
KV_MODE = json
AUTO_KV_JSON = true
category = Custom
pulldown_type = 1
The custom command builds a record with the JSON fields as distinct attributes and we also append a JSON.DUMP of the record into the *_raw* attribute (this seemed to be the only way it would appear as 'Syntax Highlighted').
By appending SPATH to the end of the custom command correctly extracts all the fields from each event. It seems like the PROPS.CONF is not picked up correctly as we expected the KV_MODE = json to do the same thing as SPATH. We would like this extraction to occur at search-time without the need to append SPATH.
Thanks for your help.
↧