This is my first time trying out the kvstore, so learning by fire. I set up a collection in myapp/default/collections.conf, and set up a number of lookups within that single collection using stanzas in myapp/local/transforms.conf:
[lookup1]
external_type=kvstore
collection=mycoll
fields_list = _key, value, range
[lookup2]
external_type=kvstore
collection=mycoll
fields_list = _key, value, range
[lookup3]
external_type=kvstore
collection=mycoll
fields_list = _key, maxStat, range
I guess my first question is: can you have multiple lookups within a single kvstore collection, where the field names are repeated among lookups?
Here's the issue i'm having: If I try outputting to a single lookup file, and then see if I can read it back in with another search, it works fine. E.g.:
search=... | outputlookup lookup1
I can then see it with:
| inputlookup lookup1
However, if i try writing out to all of the lookups, they will all write fine, but only the last one has data, and the rest are blank. It's as if an outputlookup command will erase all other lookups in the collection except the one it's writing out to.
Maybe i'm not using these as intended - i'm actually doing it as a way to store data from costly searches on data sources that change once daily, where I have a saved search that I run on a schedule and writes to the kvstore lookup. Alternatively I guess I could use a summary index but not sure which is better.
Thanks
↧