I have two indexes that I want to create a summary from every hour.
Index1
request_type, request_guid, request_timestamp, meta_field1, meta_field2, ...
Index1 contains log entries from each processing steps in each service request. Each service request is assigned a unique request_guid and all ~10 logs for the processing of a request have that id. The time the request was made is stored in request_timestamp and also remains the same through all logs for a request..
index2
request_guid, meta_fieldA, meta_fieldB, ...
index2 contains more data for the logs, but is in a separate index so that it can be secured differently from index1. The request_guid is the same value as in index1
I want to summarize by collecting stats for each request type by hour.
The approach I have taken is to select all the logs from Index1 where the request_timestamp is in the hour. I cannot use the log time directly as a request logs might span into the next hour ( as in started at 9:59:59 and ended at 10:00:01)
index=index1 earliest=0
| addinfo | eval timemillis=strftime(strptime(request_timestamp,"%Y-%m-%dT%H:%M:%S.%3N%z"),"%s")
| where timemillis>=info_min_time AND timemillis
↧