I'm having trouble pulling in my CloudTrail log files in an S3 bucket that's being populated by multiple accounts. I've got the AWS account and IAM user set up and working which I know because I can at least pull in some logs. The problem is that I only see logs coming in from the numerically first account in the list of accounts in the S3 bucket and no new logs are coming in.
I found a few errors and stack traces in the logs, but nothing that really helps me figure out where the issue is:
2016-05-27 18:01:05,404 INFO pid=26154 tid=Thread-7 file=aws_s3_data_loader.py:_do_index_data:98 | Start processing datainput=CloudTrail Logging_20160526134056, bucket_name=cloudtrail-bucket-name, job_uid=c504a3bc-5b4d-42ec-beba-493e769e89c4
2016-05-27 18:01:07,914 INFO pid=26154 tid=Thread-7 file=aws_s3_data_loader.py:collect_data:180 | Start from last_modified=1971-01-01T00:00:00.000Z for datainput=CloudTrail Logging_20160526134056, bucket_name=cloudtrail-bucket-name, job_uid=c504a3bc-5b4d-42ec-beba-493e769e89c4
2016-05-27 18:01:13,797 ERROR pid=26154 tid=Thread-7 file=aws_s3_data_loader.py:index_data:84 | Failed to collect S3 data from datainput=CloudTrail Logging_20160526134056, bucket_name=cloudtrail-bucket-name, error=Traceback (most recent call last):
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3_mod/aws_s3_data_loader.py", line 79, in index_data
self._do_index_data()
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3_mod/aws_s3_data_loader.py", line 101, in _do_index_data
self.collect_data()
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3_mod/aws_s3_data_loader.py", line 186, in collect_data
for key in keys:
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3_mod/aws_s3_data_loader.py", line 164, in _get_keys
for key in keys:
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3_mod/aws_s3_common.py", line 211, in get_keys
white_matcher = _build_regex(whitelist)
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3_mod/aws_s3_common.py", line 169, in _build_regex
return re.compile(exact_str)
File "/opt/splunk/lib/python2.7/re.py", line 194, in compile
return _compile(pattern, flags)
File "/opt/splunk/lib/python2.7/re.py", line 251, in _compile
raise error, v # invalid expression
error: nothing to repeat
After turning on debug logs for the S3 script, I can see that it's churning through log files and indexing them, but the overall number of events for the index and sourcetype isn't increasing if I search over 'all time' for:
index=aws sourcetype="aws:cloudtrail"
Anyone else had this issue or know where I could look for better logging on what's going wrong?
↧