Has anyone seen what happens to a Universal Forwarder when the filesystem it is running from goes away?
I just found out about some weekend maintenance to our network storage that will cause connectivity issues with the SAN mount points we have our Splunk UFs installed on. I'm not sure what Splunk will do when the mount disappears, and may not have a lot of time to test this scenario.
A few basic thoughts I have on what would occur:
- Splunk can’t log its own internal log files
- Splunk can’t update its fishbucket data
- Splunk can't read/run scripted inputs (not too worried about this, though - it is ok if we are missing that data since it is mostly *nix)
- Will Splunk continue forwarding data during this scenario?
How I could approach the handling of this:
- Manually Splunk Forwarders down before hand, and manually start up after filesystem comes back (most work for *jhupka*, but safest scenario)
- Let things be, then trick Splunk Forwarders into restarting via Deployment Server after filesystem comes back to start fresh (minimal work/coordination)
- Do absolutely nothing (least work, *jhupka* gets to sleep in on Saturday morning)
↧