Despite having recently finished the Splunk Admin course, I'm still fuzzy on the terms "index-time" and "search-time" especially when it comes to actually configuring the indexer and search head in a distributed search environment. When determining where to put certain modifications to the props.conf and transforms.conf, should "index-time" options only reside on the indexer and "search-time" options only reside on the search head? Or am I just conflating different ideas with unfortunately similar names?
I currently have an issue on a production Splunk deployment (which due to politics cannot simply be rebuilt from scratch). We are indexing some IIS logs, and following a 5.0.4 to 6.2.2 upgrade, there is a problem of log entries being truncated where seemingly random charactars are lost resulting in fields containing the wrong data, or fragments of other fields. There is also an issue of missing events and duplicated events. When I compare the actual IIS log files to a generic sourcetype=iis search, the log file is fine, but the search results do not match.
I have tried multiple times to re-work the props.conf and transforms.conf in etc/system/local on the indexer and search head, but my new configurations don't seem to take effect (or at least only partially). Based on the way .conf precedence works, nothing should override my new conf files, but my changes just don't seem to be taking effect. I feel like I am missing something very fundamental, but I can't understand what it is. Help?
↧