So here’s my situation:
Multiple CentOS boxes running Suricata IDS.
Suricata logs events to both:
`/opt/log/suricata/eve.json` (basically raw JSON objects)
And
`/opt/log/suricata/fast.log` (a syslog style summary of events)
The Suricata boxes have a UF on them to forward over the contents of those files to my indexers. That works fine - everything’s indexed and searchable and is great.
However, I also need to send these same logs off via syslog to a third destination. For reasons, the easiest way to do this is to setup syslog forwarding *on the indexers* (I know I can’t do it on the UFs, and indexers are basically HFs with extra stuff), which is something I’ve done before for other things and has been fine.
Here’s the relevant snippets of config on the indexers:
> outputs.conf:
#[syslog]
#defaultGroup = syslogtest
[syslog:syslogtest]
type = tcp
server = 10.x.x.x:9997
priority = <182>
maxEventSize = 8192
timestampformat = %b %e %H:%M:%S> props.conf:
#[suricata]
#[host::*SM7-Z*]
[source::/opt/log/suricata/*]
TRANSFORMS-routing=dc_suricata
> transforms.conf:
[dc_suricata]
REGEX=SM7
DEST_KEY=_SYSLOG_ROUTING
FORMAT=syslogtest
Couple of quick notes:
- Yes, I know I’m using port 9997 as the output port. That’s just me being creative with the firewall rules in place. There’s an rsyslog listening on that server on port 9997, not Splunk.
- The `REGEX` is there in transforms to match specific hostnames. It behaves the same with `REGEX=.`
With this, the events from `fast.log` arrive on the destination syslog server just fine. However, the events from `eve.json` do not - they’re just nowhere to be found.
If I use the `[syslog]` stanza in outputs.conf with `defaultGroup` (the piece that’s commented out) the events *do* come over, but as part of a giant flood of everything. Not great. But with the regular `syslog:foo` stanzas, it just won’t work.
Similarly, I have another application which writes its output to ‘raw’ files (but not JSON objects, just raw log data in KV pairs). Those files show up in Splunk too, but the events don’t get forwarded over when I try to send them by syslog.
Any ideas? It smells like a bug to me but I don’t know if I’m missing anything.
↧