**The background:**
I have multiple types of logs from multiple groups being piped into Splunk into 1 index
The index=index1
The sourcetype=syslog
**The scenario:**
One of my groups wants to access their VPN data to generate reports (active/closed, duration, user_auth stuff, etc)
In the future, each group will want the same access (about 25-30 groups)
**The Goal:**
Here's my approach
1) isolate logs to just show VPN traffic
2) within isolated VPN_LOGS I need to be able to isolate the group_traffic
3) and Lastly, provide group access to their data while keeping each other groups data separate and private
4) and then create fields to visualize (session_ID, username, creation time, bytes in/out, duration, OS, Browser, Client IP, assigned IP)
I'm having a lot of fun trying to figure this out, but I'm hoping for some assistance, direction, advice on a solution and/or if my approach can work, but is not optimized or maybe it's just wrong...
First thought is to optimize/change the way I am receiving logs to help make things easier for me to figure out, but i'm not sure how to tackle that or suggest what needs to be done, so I am working with what I was given..
-------------------------
**Current work:**
The below search string removes all unnecessary logs using the process field. I then use the transaction command to assemble all the events by session_id field (which is unique only to the VPN logs) and I then pipe into a search to grab the transactions for a specific group. (Not sure if this is the best way)
sourcetype=syslog process=* NOT (logger OR cron OR crond OR syslog-ng OR tmsh OR snmpd OR sshd) | transaction session_id | search group=GROUP1
I was hoping to put this search string into the "restrict_search_terms" field within user permissions/roles and that would provide the solution to data privacy and ability to create reports/dashboard for each group... problem solved... but it a search like this isn't allowed in the "restrict_search_terms"
I've provided some modified logs:
[process=] built in sourcetype field. This field I found target events for each log type
[session_id] I created
<\d+>\w{3}\s\d+\s\d{2}:\d{2}:\d{2}\sf5-sslvpn-\d\s\S+\s\d+:\d:\s(?\w+)
[group=] i created to tell me which group the session_id transaction belongs too
(?=[^R]*(?:Resource: /|R.*Resource: /))^[^/\n]*/(?P\w+)
[username=]
(?=[^U]*(?:Username '|U.*Username '))^[^'\n]*'(?P[^']+)
[user_auth=] I will grab the "Received client info - Type: to make this expression
=========================================== sample logs=================================
_raw Data
<182>Jan 12 16:51:44 f5_vpn-1 logger: [ssl_acc] 127.0.0.1 - - [12/Jan/1980:16:51:43 -0700] "/igeen/igeenPortal.cgi" 200 8787<182>Jan 12 16:51:44 f5_vpn-1 logger: [ssl_req][12/Jan/1980:16:51:43 -0700] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "/igeen/igeenPortal.cgi" 8787<182>Jan 12 16:51:38 f5_vpn-1 logger: [ssl_req][12/Jan/1980:16:51:38 -0700] 999.999.999.991 TLSv1.2 DHE-RSA-AES256-SHA "/xui/update/configuration/alert/statusmenu/coloradvisory" 3762<182>Jan 12 16:51:38 f5_vpn-1 logger: [ssl_acc] 198.238.211.12 - user1 [12/Jan/1980:16:51:38 -0700] "/xui/update/configuration/alert/statusmenu/coloradvisory" 200 3762<182>Jan 12 16:51:38 f5_vpn-1 logger: [ssl_acc] 198.238.121.17 - user2 [12/Jan/1980:16:51:38 -0700] "/xui/update/configuration/alert/statusmenu/coloradvisory" 200 3758<182>Jan 12 16:51:38 f5_vpn-1 logger: [ssl_req][12/Jan/1980:16:51:38 -0700] 999.999.999.100 TLSv1.2 AES256-SHA "/xui/update/configuration/alert/statusmenu/coloradvisory" 3758<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_req][12/Jan/1980:16:51:36 -0700] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "/iControl/iControlPortal.cgi" 8341<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_acc] 127.0.0.1 - - [12/Jan/1980:16:51:36 -0700] "/igeen/igeenPortal.cgi" 200 8341<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_req][12/Jan/1980:16:51:36 -0700] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "/iControl/iControlPortal.cgi" 2818<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_acc] 127.0.0.1 - - [12/Jan/1980:16:51:36 -0700] "/igeen/igeenPortal.cgi" 200 2818<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_req][12/Jan/1980:16:51:36 -0700] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "/iControl/iControlPortal.cgi" 8787<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_acc] 127.0.0.1 - - [12/Jan/1980:16:51:36 -0700] "/igeen/igeenPortal.cgi" 200 8787<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_req][12/Jan/1980:16:51:36 -0700] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "/iControl/iControlPortal.cgi" 8787<182>Jan 12 16:51:36 f5_vpn-2 logger: [ssl_acc] 127.0.0.1 - - [12/Jan/1980:16:51:36 -0700] "/igeen/igeenPortal.cgi" 200 8787<141>Jan 12 16:51:35 f5_vpn-1 sbb3[3645]: 01490502:5: 696fdb86: Session deleted due to user inactivity or errors.<141>Jan 12 16:51:35 f5_vpn-1 sbb[3645]: 01490521:5: 648bd53e: Session statistics - bytes in: 471704, bytes out: 777008
SessionID2 = 648bd53e<182>Jan 12 16:51:32 f5_vpn-1 logger: [ssl_req][12/Jan/1980:16:51:32 -0700] 999.999.999.991 TLSv1.2 DHE-RSA-AES256-SHA "/xui/update/configuration/alert/statusmenu/coloradvisory" 3762<182>Jan 12 16:51:32 f5_vpn-1 logger: [ssl_acc] 999.999.999.991 - user21 [12/Jan/1980:16:51:32 -0700] "/xui/update/configuration/alert/statusmenu/coloradvisory" 200 3762<141>Jan 12 16:51:31 f5_vpn-1 sbb2[3645]: 01490502:5: 4ebfa5a3: Session deleted due to user inactivity or errors.
1 user sessionID -
<141>Jan 12 13:52:28 f5-vpn-1 tmm[3645]: 01490521:5: a620f620: Session statistics - bytes in: 17611113, bytes out: 58663535 process = tmm session_id = a620f620<141>Jan 12 13:51:47 f5-vpn-1 tmm[3645]: 01490502:5: a620f620: Session deleted due to user inactivity or errors. process = tmm session_id = a620f620<141>Jan 12 13:36:49 f5-vpn-1 tmm1[3645]: 01490505:5: a620f620: PPP tunnel 0x57008bd86200 closed. process = tmm1 session_id = a620f620<141>Jan 12 08:01:40 f5-vpn-1 tmm1[3645]: 01490505:5: a620f620: PPP tunnel 0x57008bd86200 started. process = tmm1 session_id = a620f620<141>Jan 12 08:01:40 f5-vpn-1 tmm1[3645]: 01490549:5: a620f620: Assigned PPP Dynamic IPv4: 181.231.31.123 Tunnel Type: VPN_TUNNELTYPE_TLS NA Resource: /GROUP1/NA_GROUP1_CMDCENTER GROUP1 = GROUP1 process = tmm1 session_id = a620f620<141>Jan 12 08:01:23 f5-vpn-2 apd[3594]: 01490102:5: a620f620: Access policy result: Full process = apd session_id = a620f620<141>Jan 12 08:01:23 f5-vpn-2 apd[3594]: 01490005:5: a620f620: Following rule 'Out' from item 'resources_pc' to ending 'Allow'process = apd session_id = a620f620<141>Jan 12 08:01:23 f5-vpn-2 apd[3594]: 01490115:5: a620f620: Following rule 'fallback' from item 'Route Domain and SNAT Selection' to terminalout 'Out' process = apd session_id = a620f620<141>Jan 12 08:01:23 f5-vpn-2 apd[3594]: 01490128:5: a620f620: Webtop '/GROUP1/WT_GROUP1_SSLVPN' assigned process = apd session_id = a620f620 <141>Jan 12 08:01:23 f5-vpn-1 apd[3594]: 01490008:5: a620f620: Connectivity resource '/GROUP1/NA_GROUP1_CMDCENTER' assigned
process = apd session_id = a620f620<141>Jan 12 08:01:23 f5-vpn-2 apd[3594]: 01490115:5: a620f620: Following rule 'fallback' from item 'GROUP1 role group mapping' to terminalout 'Out'process = apd session_id = a620f620<141>Jan 12 08:01:21 f5-vpn-2 apd[3594]: 01490010:5: a620f620: Username 'user1'process = apd session_id = a620f620 username = user1<141>Jan 12 08:01:04 f5-vpn-1 tmm[3645]: 01490500:5: a620f620: New session from client IP 999.999.100.200 (ST=STATE/CC=US/C=NA) at VIP 999.999.166.200 Listener /GROUP1/VS_GROUP1_WC_SSLVPN (Reputation=Unknown)process = tmm session_id = a620f620<141>Jan 12 08:01:04 f5-vpn-1 tmm[3645]: 01490544:5: a620f620: Received client info - Type: IE Version: 8 Platform: Win7 CPU: WOW64 UI Mode: Full Javascript Support: 1 ActiveX Support: 1 Plugin Support: 0 process = tmm session_id = a620f620<141>Jan 12 08:01:04 f5-vpn-1 tmm[3645]: 01490506:5: a620f620: Received User-Agent header: Mozilla%2f4.0%20(compatible%3b%20MSIE%208.0%3b%20Windows%20NT%206.1%3b%20WOW64%3b%20Trident%2f4.0%3b%20SLCC2%3b%20.NET%20CLR%202.0.50727%3b%20.NET%20CLR%203.5.30729%3b%20.NET%20CLR%203.0.30729%3b%20Media%20Center%20PC%206.0%3b%20.NET4.0C%3b%20.NET4.0E%3b%20InfoPath.3).
↧