I understand from Splunk documentation that because the master keeps track of which bucket copies are on each node and if they are primary or searchable copies, even if a peer goes down, the master can coordinate remedial activities (compensating for any primary copies, compensating for any searchable copies and compensating bucket copies).
However, it seems that the master does not perform remedial activities unless it detects that a peer is offline.
What could happen if a primary bucket (the copy participating in searches) got corrupted or possibly deleted? If a search which uses the copy is executed, search head doesn't report data contained in the bucket? What if non searchable copies got deleted by mistake? Unless we take a peer down (triggering the master to perform remedial activities), the master does not know that the cluster is incomplete and the master dashboard doesn't report about the incomplete status.
Thanks
↧