Currently I am facing a file system /opt issue in the splunk heavyforwarder server, this server is used to monitor and forward the syslogs information to the indexer clusters.
Files size is keep on increasing in this folder /opt/syslogs/generic/... under generic folder there are many subdirectories and each subdirectoris contains some .logs. When validated the splunkd.log I could see the below Info
04-22-2016 12:18:07.201 -0400 INFO AggregatorMiningProcessor - Got done message for: source::/opt/syslogs/generic/xxxx/2838552.log|host::xxxx|syslog|9947901
04-22-2016 12:18:12.904 -0400 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/opt/syslogs/generic/xxxopt/sport.log|host::xxxopt|syslog|9946037
I have tried toexecute the log rotate but when execute the log rotate its consuming the Swap memory has many log rotate process are running and if I kill the process then space increases, as temprorary solution i am trying to add space to this opt file system. Since all the data are critical (network data) deleting the files will be create a probelm while auditing.
My question what will be the permanent solution to fix this issue.
1) Do I need to change any configuration inside the /opt/splunk/etc/apps/local/input.conf
2) Can I move the files to some other location in the same server?
↧