I want to maintain a lot of data in my KV Store, but in order to do so I have to keep it clean; but aging out old data.
The problem with | inputlookup mylookup | where _time>relative_time(now(),"-7d@h") | outputlookup append=false mylookup is that it would cause the full database to be replicated again to other search heads and indexers.
Thus I created a script that will issue delete commands when it runs for any records that are too old. However it can only interact with the API and delete one entry at a time.
If I were able to connect directly to the MongoDB, I could possible issue a "delete from mytable where _time>value" and it would be 1000% more efficient than deleting one record at a time.
Further, I don't think I can delete records fast enough using Python and the API to keep up with what is being added.
Can anyone shed some light on how I can go about connecting directly to the MongoDB?
↧