I want to leverage a huge lookup which will likely have a size in the range of 50GB. The size of target data (to which this lookup will be applied) is over a few tera bytes, and the number of nodes in the indexer cluster will be 150+.
The end-up objective is to have de-normalized events which will be the same number of original TBs' events.
For maximizing the performance, which method of lookup would be recommended between kvstore and custom scripted lookup? For kvstore lookup, I'm concerned about the of the size bundle that will be replicated to indexers.
↧