is there a easy way to create a alert that uses standard deviation to alert us when we see sourcetypes and/or indexes grow more than a certain percentage within the license data in the _internal index.
i am thinking about the following search:
index=_internal source=*license_usage.log type="Usage"
| stats sum(b) as b by _time, pool, st
| eval "b"=round (b/1024/1024/1024, 2)
| timechart span=7d sum(b) by st useother=f
*then add the last weeks data and a " | where % >20*"
Goals for this search:
1. Quickly identify and alert when a data source is blowing up our licenses.
2. Quickly identify and alert when a data source is experiencing logging issues such as a whole environment/sourcetype/index is longer sending logs by reversing the logic.
↧