Quantcast
Channel: Questions in topic: "splunk-enterprise"
Viewing all articles
Browse latest Browse all 47296

How to eliminate and prevent duplicate logs in my environment?

$
0
0
I have an application that has 2 environments. Specifically, there are 2 databases that replicate to each other so that each environment has the same data. However, I only replicate one time per day. In the application, I have it set to dump to log files, which I then have the Splunk forwarder send to my Splunk server. For the one server, this is working fine and all the data in the SEPM is real time for the most part. The issue is this second server. When replication occurs, the server with the Splunk forwarder dumps the data to file, but depending on the daily changes, my dump files may overwrite themselves before the event are forwarded. As a result, I could miss data from my second server. My thought was to set up the forwarder on the second server as well, to have it forward data real-time which I can do without much issue. However, my concern is the nightly replication cycle. When the replication occurs, it means that data that was forwarded earlier will be dumped and forwarded again from each server with the forwarder a second time. Is there any way to ignore this second set of data as it is simply a duplication of what will have been forwarded real-time during the day, prior to the nightly database replication cycle?

Viewing all articles
Browse latest Browse all 47296

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>