Dear splunk community,
We get the following error message trying to get SPLUNK_TA_jmx 3.1.0 working on a 64Bit SLES 11 SP3 system with splunk 6.2.1.2 build 259063:
Unable to initialize modular input "jmx" defined inside the app "Splunk_TA_jmx": Introspecting scheme=jmx: script running failed (exited with code 1).
Some background:
We set up SPLUNK_TA_jmx for testing on a Windows server 2008 R2 running Splunk 6.2.4 build 271043 and it works without any problems.
Now we want to install it on a heavy forwarder running on 64Bit SLES 11 SP3 in our production Splunk infrastructure with search head and indexer clustering.
We set up Splunk 6.2.1 as a heavy forwarder with
[indexAndForward]
index = false
[tcpout:default-autolb-group]
server =
-
We created an index for the events on the indexer cluster servers and the heavy forwarder.
From the splunkd.log:
12-29-2015 14:52:50.917 +0000 ERROR ModularInputs - Introspecting scheme=jmx: script running failed (exited with code 1).
12-29-2015 14:52:50.918 +0000 ERROR ModularInputs - Unable to initialize modular input "jmx" defined inside the app "Splunk_TA_jmx": Introspecting scheme=jmx: script running failed (exited with code 1).
We found the following information on answers.splunk.com with similar problems, but they don't solve our issue:
https://answers.splunk.com/answers/236224/how-do-i-troubleshoot-a-problem-with-jmx-add-on.html
https://answers.splunk.com/answers/152040/unable-to-get-jmx-data-with-app-monitoring-of-java-virtual-machines-with-jmx.html
https://answers.splunk.com/answers/217742/monitoring-of-java-virtual-machines-with-jmx-how-t-2.html?utm_source=typeahead&utm_medium=newquestion&utm_campaign=no_votes_sort_relev
https://answers.splunk.com/answers/118243/unable-to-initialize-modular-input-jmx-defined-inside-the-app-jmx-ta.html?utm_source=typeahead&utm_medium=newquestion&utm_campaign=no_votes_sort_relev
See for example the following output for reference:
splunk@xxxxxx:~/etc/apps/Splunk_TA_jmx> $SPLUNK_HOME/bin/splunk cmd /opt/splunk/bin/python $SPLUNK_HOME/etc/apps/Splunk_TA_jmx/bin/jmx.py --scheme