I have created a new data model specifically for filtered proxy events. The root object of the data model is an eventtype which filters a list of common/undesired URL hosts (*symantec.com, *.mcafee.com, etc.).
I have created a saved search which looks for "unique" URL hosts, and then saves the results to a lookup table:
***|tstats summariesonly=t values(Filtered_Proxy.src) AS dst values(Filtered_Proxy.http_user_agent) AS http_user_agent count FROM datamodel=Web_Traffic_Filtered by Filtered_Proxy.dhost | where count<=5| outputlookup raredhost_dm.csv***
In an attempt to compile as "unique" a list as possible, I would like to run the search for 7 or more days (Start: -7d@d Finish: -1h@h). Each time that the search runs, it never completes. The most recent time that I ran the search, the following appeared when I looked at the job status: application=search; size=586.73MB; events=172,454,312; run time = 00:01:30; status = Running(92%). Four hours after the search started, it still has not completed.
I had attempted to perform a similar search with tscollect and tstats - not with a data model. This worked without issue. It was an issue, however, because I do not want the tsidx files collecting on a search head.
Any input/suggestions/solution would be greatly appreciated.
Thank you.
↧