We're using the [Splunk App for Jenkins](https://splunkbase.splunk.com/app/3332/) and [Splunk Plugin for Jenkins](https://wiki.jenkins.io/display/JENKINS/Splunk+Plugin+for+Jenkins) in order to ingest Jenkins logs into Splunk. By default, this app / plugin combo provides console logs to `index=jenkins_console sourcetype=text:jenkins`. By and large this is working as expected. However, we are seeing some Jenkins builds that are simply forwarding blank lines for console lines that are *not* empty when you review them in Jenkins.
I've looked at other Splunk Answers, but wasn't able to find another question that had our same setup. I know that Jenkins Pipeline code needs to explicitly call a `sendSplunkConsoleLog` function (e.g., https://answers.splunk.com/answers/691730/how-to-get-jenkins-console-into-splunk.html). However, we are *not* using Jenkins Pipeline. We are using Jenkins Job Builder.
Here is an example of what we are seeing when running a search like `index=jenkins_console sourcetype=text:jenkins`:
**Console output in Splunk**
...
2019-10-08T17:45:11.517-0400 line:6 [WS-CLEANUP] Deleting project workspace...
2019-10-08T17:45:11.526-0400 line:7 Run condition [Boolean condition] enabling prebuild for step [Trigger/call builds on other projects]
2019-10-08T17:45:11.527-0400 line:8 Run condition [Not] preventing prebuild for step [Windows PowerShell]
2019-10-08T17:45:11.560-0400 line:9
2019-10-08T17:45:12.286-0400 line:10
2019-10-08T17:45:12.737-0400 line:11
2019-10-08T17:45:13.623-0400 line:12
2019-10-08T17:45:13.695-0400 line:13
2019-10-08T17:45:13.696-0400 line:14 items.find({
2019-10-08T17:45:13.696-0400 line:15 "@build.name": "F5-master",
2019-10-08T17:45:13.696-0400 line:16 "repo": "F5-master",
2019-10-08T17:45:13.696-0400 line:17 "@gitBranch": "master",
2019-10-08T17:45:13.696-0400 line:18 "@project.name": "F5ManagementClient-master-Tool"
...
**Actual console output in Jenkins**
...
17:45:11 [WS-CLEANUP] Deleting project workspace...
17:45:11 Run condition [Boolean condition] enabling prebuild for step [Trigger/call builds on other projects]
17:45:11 Run condition [Not] preventing prebuild for step [Windows PowerShell]
17:45:11 [CompareGroupsAppFamily-staging-idleDeploy-qwest] $ powershell.exe -NonInteractive -ExecutionPolicy ByPass "& 'C:\Users\Hudson\AppData\Local\Temp\jenkins8757820116422196227.ps1'"
17:45:12 ###### start macro - shared-splunk-createCustomMetadataLog
17:45:12 [CompareGroupsAppFamily-staging-idleDeploy-qwest] $ powershell.exe -NonInteractive -ExecutionPolicy ByPass "& 'C:\Users\Hudson\AppData\Local\Temp\jenkins5520032045158571584.ps1'"
17:45:13 ###### start macro - shared-artifactory-downloadLatestBuild
17:45:13 #################### Query To Be Invoked ####################
17:45:13 items.find({
17:45:13 "@build.name": "F5-master",
17:45:13 "repo": "F5-master",
17:45:13 "@gitBranch": "master",
17:45:13 "@project.name": "F5ManagementClient-master-Tool"
...
As you can see from above, Splunk recognizes that lines 9 through 13 *exist*, but Splunk only receives them as empty lines. When looking at the output in Jenkins, you can see that the lines are not empty and in fact contain crucial information for us.
Does anyone know why this might be happening? Please let me know if you need any more information.
↧