Hello!
I have a distributed deployment of Splunk Enterprise. All my UFs send raw events to two HFs, these send cooked data to three-node IDX cluster. My search interface is three-node SH cluster. I plan to use a few ingest-time eval fields, first of all I tested how it works, placing:
props.conf to HFs:
props.conf
------------------------------------------------------------
[airwatch_iis_flogs]
TRANSFORMS = ingest-eval-rule-size_bytes, ingest-eval-rule-orig_host, ingest-eval-rule-orig_time
transforms.conf to HFs:
transforms.conf
------------------------------------------------------------
[ingest-eval-rule-size_bytes]
INGEST_EVAL = size_bytes=len(_raw)
[ingest-eval-rule-orig_host]
INGEST_EVAL = orig_host=upper(host)
[ingest-eval-rule-orig_time]
INGEST_EVAL = orig_time=_time
fields.conf to SHs:
fields.conf
------------------------------------------------------------
[size_bytes]
INDEXED = True
[orig_host]
INDEXED = True
[orig_time]
INDEXED = True
I put props.conf/transforms.conf to HFs (not to IDXs) as these servers process all the raw events and cook the data for indexers. This configuration works like a charm: querying
index=* sourcetype=airwatch_iis_flogs
I get the events having expected indexed fields size_bytes (calculated), orig_host (uppercase) and orig_host (match _time).
Now, field of interest to me is _indextime. I want to index the latency:
transforms.conf
------------------------------------------------------------
[ingest-eval-rule-latency_sec]
INGEST_EVAL = latency_sec=_indextime-_time
I also put the relevant changes to props.conf and fields.conf, but unfortunately this configuration doesn't work. Is it maybe because _indextime is empty while cooking events on HFs, and actually filled up while writing events to disk on indexers (not sure, where IndexQueue lives - on IDXs or HFs)?
What should I do to use this _indextime field in ingest-time eval - maybe put:
outputs.conf
------------------------------------------------------------
sendCookedData = false
to my HFs, move all props.conf/transforms.conf to IDXs? I feel myself, there are more drawbacks then benefts from this decision.
Are there any more limitations using _indextime ingest-time eval?
↧