Is it possible to use regex in the file_path setting for the File/Directory Information Input app.
Here is what I am trying to get to
- E:\Folder\Folder2\20160808\InvalidFile\\*.cdi_Error1
- E:\Folder\Folder2\20160809\InvalidFile\\*.cdi_Error1
- E:\Folder\Folder2\20160810\InvalidFile\\*.cdi_Error1
etc.
I have tried
- file_path = E:\Folder\Folder2\\*\InvalidFiles\\*.cdi_Error1
- file_path = E:\Folder\Folder2\\...\InvalidFiles\\*.cdi_Error1
I have also tried several different regex options for *.cdi_Error1. To many to list.
When I try the above options I am receiving this message in the file_meta_data_modular_input.log
- 2016-08-26 10:34:45,864 WARNING Unable to access path="E:\Folder\Folder2\\*\InvalidFiles\\*.cdi_Error1", reason="[Error 123] The filename, directory name, or volume label syntax is incorrect: 'E:\\Folder\\Folder2\\\*\\InvalidFiles\\\*.cdi_Error1'"
- 2016-08-26 10:34:45,864 INFO Completed retrieval of file data, count=0, path=E:\Folder\Folder2\\*\InvalidFiles\\*.cdi_Error1
Not sure why the 2nd message shows it was complete but it definitely did not pull in the information.
I also tried using whitelist
- file_path = E:\Folder\Folder2
- recurse = 1
- whitelist = *.cdi_Error1
But then I get this message
- 2016-08-26 12:54:28,592 ERROR The input stanza 'file_meta_data://APPNAME' is invalid: The parameter 'whitelist' is not a valid argument
I know that I can set the file_path setting to E:\Folder\Folder2 and set recurse = 1 but this then pulls in some 50000 files and I only need the .cdi_Error1 files.
I also know that if I pull in the 50000 files I can just use logic in the search parameters to filter out only the .cdi_Error1 files but this server is already heavily used and I do not want to put more stress on it by grabbing metadata for 50000 files. Plus its just a lot of data that I do not need to index.
I did try restarting splunk on both the indexer, search head and forwarder many times but it did not help.
Any help is appreciated. Thank you
↧