Hello All,
I am working the below query - When i am running these two main which joined using join command are giving me intended results. But, when the queries are joined, the results from second query doesn’t come up and yields incorrect results. This "join Hour_Of_Day" join i have highlighted in the query Could any one help or let me know my mistake.
Little description of query,
This query captures user login information from VPC oss_logging of the all the users who have successfully authenticated via OKTA.
index=auto_vsc host=dfw* sourcetype=oss_logging userid | rex "userId\=(?P\w+)" | rex "userId\=\s+(?P\w+)"| rex "cuid\=(?P\w+)" | transaction userId source maxspan=1s startswith="SAMLSessionInterceptor(getLoginUser:247)" |search userId!=null| eval time1=_time | eval time=round(time1)| convert ctime(time)| eval fullname =time.".".cuid| table fullname
Description: Once user is authenticated via OKTA and gathers information successfully from VPC. There are chances that iWarranty response fails(e.g., iWarranty DB is down) and this query gives failures results from iWarranty managed logs.
index=auto_iwarranty sourcetype="*managed_log*" "Content-Type: application/json" | rex "ID\:\s(?P[^\s]+)\s" | rex "dealerId\"\:\"(?P\w*)" | rex "dealerBAC\"\:\"(?P\w*)" | rex "Address\:\s(?P.*)" | rex "cuid\"\:\"(?P\w*)" | rex "resultMessage\"\:\"(?P(.*?)(?=\"))" | transaction ID host requestURL maxspan=10m "Content-Type: application/json" keepevicted=true | where requestURL="https://prism-ins-osb-prod.int.ally.com:443/cps-iw-services/services/cps-businessPartner/businesspartner/dealer/" | search resultMessage="E9010: Error – unable to establish a connection to the service. If this continues, please call the NAMSC or your account manager." | eval customer=if(match(cuid, "^(?!G65UFBC9$)(?!BLINKER1$)(?!UserId00$)(?!BLINKER1$)(?!ROUTE001$)(?!Test1234$)(?!CARVANA1$)(?!CREDITIQ$)(?!SHIFT123$)([A-Z]+\d+.*)$"), "UI", "ThirdParty") | search customer="UI" | eval time=_time| convert ctime(time)| eval fullname =time.".".cuid| dedup fullname | table fullname | union [ search index=auto_iwarranty sourcetype="*managed_log*" "Content-Type: application/json" | rex "ID\:\s(?P[^\s]+)\s" | rex "dealerId\"\:\"(?P\w*)" | rex "dealerBAC\"\:\"(?P\w*)" | rex "Address\:\s(?P.*)" | rex "cuid\"\:\"(?P\w*)" | rex "resultMessage\"\:\"(?P(.*?)(?=\"))" | transaction ID host requestURL maxspan=10m "Content-Type: application/json" keepevicted=true | where requestURL="https://prism-ins-osb-prod.int.ally.com:443/cps-iw-services/services/cps-rating/dealer/programs" | search resultMessage="null: java.sql.SQLException: weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CLMDataSource is Suspended, cannot allocate resources to applications.." OR resultMessage="null: java.sql.SQLException: weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CLMDataSourceWithTimeout is Suspended, cannot allocate resources to applications.." | eval customer=if(match(cuid, "^(?!G65UFBC9$)(?!BLINKER1$)(?!UserId00$)(?!BLINKER1$)(?!ROUTE001$)(?!Test1234$)(?!CARVANA1$)(?!CREDITIQ$)(?!SHIFT123$)(?!ABCDEFGH$)([A-Z]+\d+.*)$"), "UI", "ThirdParty")|search customer="UI" | eval time1=_time | eval time=round(time1)|convert ctime(time)| eval fullname =time.".".cuid| eval fullname1 =time.".".dealer.".".cuid] | dedup fullname1 |table fullname
Description: Once user is authenticated via OKTA, there are chances that VPC request fails (e.g., session issue) . This query gives failure results from VPC managed logs.
index=auto_vsc sourcetype="*managed_log*" IllegalStateException earliest=-4w latest=@d | rex "session\sid\s\"(?P\w+)" | rex "invalid\:\s(?P\w+)" | join mysession [search index=auto_vsc host=dfw* sourcetype=oss_logging earliest=-4w latest=@d | rex "sessionId\s\=\s(?P.*?)\!" | rex "userId\s=\s(?P\w+)"| rex "cuid\=(?P\w+)" | rex "sessionId\s\=\s(?P\w+)"| transaction userId source maxspan=1s startswith="SAMLSessionInterceptor(getLoginUser:247)" ]| eval time=_time| convert ctime(time)|eval fullname =time.".".cuid | table fullname
**Below is the full query**
index=auto_vsc host=dfw* sourcetype=oss_logging userid earliest=-4w latest=@d | rex "userId\=(?P\w+)" | rex "userId\=\s+(?P\w+)"| rex "cuid\=(?P\w+)" | transaction userId source maxspan=1s startswith="SAMLSessionInterceptor(getLoginUser:247)" |search userId!=null| eval time1=_time | eval time=round(time1)| convert ctime(time)| eval fullname =time.".".cuid| table fullname | append [search index=auto_iwarranty sourcetype="*managed_log*" "Content-Type: application/json" earliest=-4w latest=@d | rex "ID\:\s(?P[^\s]+)\s" | rex "dealerId\"\:\"(?P\w*)" | rex "dealerBAC\"\:\"(?P\w*)" | rex "Address\:\s(?P.*)" | rex "cuid\"\:\"(?P\w*)" | rex "resultMessage\"\:\"(?P(.*?)(?=\"))" | transaction ID host requestURL maxspan=10m "Content-Type: application/json" keepevicted=true | where requestURL="https://prism-ins-osb-prod.int.ally.com:443/cps-iw-services/services/cps-businessPartner/businesspartner/dealer/" | search resultMessage="E9010: Error – unable to establish a connection to the service. If this continues, please call the NAMSC or your account manager." | eval customer=if(match(cuid, "^(?!G65UFBC9$)(?!BLINKER1$)(?!UserId00$)(?!BLINKER1$)(?!ROUTE001$)(?!Test1234$)(?!CARVANA1$)(?!CREDITIQ$)(?!SHIFT123$)([A-Z]+\d+.*)$"), "UI", "ThirdParty") | search customer="UI" | eval time=_time| convert ctime(time)| eval fullname =time.".".cuid| dedup fullname | table fullname | union [ search index=auto_iwarranty sourcetype="*managed_log*" "Content-Type: application/json" earliest=-4w latest=@d | rex "ID\:\s(?P[^\s]+)\s" | rex "dealerId\"\:\"(?P\w*)" | rex "dealerBAC\"\:\"(?P\w*)" | rex "Address\:\s(?P.*)" | rex "cuid\"\:\"(?P\w*)" | rex "resultMessage\"\:\"(?P(.*?)(?=\"))" | transaction ID host requestURL maxspan=10m "Content-Type: application/json" keepevicted=true | where requestURL="https://prism-ins-osb-prod.int.ally.com:443/cps-iw-services/services/cps-rating/dealer/programs" | search resultMessage="null: java.sql.SQLException: weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CLMDataSource is Suspended, cannot allocate resources to applications.." OR resultMessage="null: java.sql.SQLException: weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CLMDataSourceWithTimeout is Suspended, cannot allocate resources to applications.." | eval customer=if(match(cuid, "^(?!G65UFBC9$)(?!BLINKER1$)(?!UserId00$)(?!BLINKER1$)(?!ROUTE001$)(?!Test1234$)(?!CARVANA1$)(?!CREDITIQ$)(?!SHIFT123$)(?!ABCDEFGH$)([A-Z]+\d+.*)$"), "UI", "ThirdParty")|search customer="UI" | eval time1=_time | eval time=round(time1)|convert ctime(time)| eval fullname =time.".".cuid| eval fullname1 =time.".".dealer.".".cuid] | dedup fullname1 |table fullname] | append [search index=auto_vsc sourcetype="*managed_log*" IllegalStateException earliest=-4w latest=@d | rex "session\sid\s\"(?P\w+)" | rex "invalid\:\s(?P\w+)" | join mysession [search index=auto_vsc host=dfw* sourcetype=oss_logging earliest=-4w latest=@d | rex "sessionId\s\=\s(?P.*?)\!" | rex "userId\s=\s(?P\w+)"| rex "cuid\=(?P\w+)" | rex "sessionId\s\=\s(?P\w+)"| transaction userId source maxspan=1s startswith="SAMLSessionInterceptor(getLoginUser:247)" ]| eval time=_time| convert ctime(time)|eval fullname =time.".".cuid | table fullname]| eventstats count as result by fullname | where result=1 | eval name=split(fullname, ".")| eval FieldB=mvindex(name,0)| eval FieldC=mvindex(name,1)|eval Hour_Of_Day = strftime(strptime(FieldB, "%m/%d/%Y %H:%M:%S"), "%H") |eval Week_Day = strftime(strptime(FieldB, "%m/%d/%Y %H:%M:%S"),"%a") |eval Today=strftime(now(),"%a") |eval Current_Day=if(Week_Day=Today,Today,null()) |stats count AS "Total_Login" by Hour_Of_Day,Current_Day |eval DailyLogins=(Total_Login/4) | stats values(DailyLogins) AS "Current_Day_Average" by Hour_Of_Day,Current_Day
| eval Current_Day_Average=ceil(Current_Day_Average)| ***join Hour_Of_Day*** [search index=auto_vsc host=dfw* sourcetype=oss_logging userid earliest=-d@d latest=@d | rex "userId\=(?P\w+)" | rex "userId\=\s+(?P\w+)"| rex "cuid\=(?P\w+)" | transaction userId source maxspan=1s startswith="SAMLSessionInterceptor(getLoginUser:247)" |search userId!=null| eval time1=_time | eval time=round(time1)| convert ctime(time)| eval completename =time.".".cuid| table completename | append [search index=auto_iwarranty sourcetype="*managed_log*" "Content-Type: application/json" earliest=-d@d latest=@d | rex "ID\:\s(?P[^\s]+)\s" | rex "dealerId\"\:\"(?P\w*)" | rex "dealerBAC\"\:\"(?P\w*)" | rex "Address\:\s(?P.*)" | rex "cuid\"\:\"(?P\w*)" | rex "resultMessage\"\:\"(?P(.*?)(?=\"))" | transaction ID host requestURL maxspan=10m "Content-Type: application/json" keepevicted=true | where requestURL="https://prism-ins-osb-prod.int.ally.com:443/cps-iw-services/services/cps-businessPartner/businesspartner/dealer/" | search resultMessage="E9010: Error – unable to establish a connection to the service. If this continues, please call the NAMSC or your account manager." | eval customer=if(match(cuid, "^(?!G65UFBC9$)(?!BLINKER1$)(?!UserId00$)(?!BLINKER1$)(?!ROUTE001$)(?!Test1234$)(?!CARVANA1$)(?!CREDITIQ$)(?!SHIFT123$)([A-Z]+\d+.*)$"), "UI", "ThirdParty") | search customer="UI" | eval time=_time| convert ctime(time)| eval completename =time.".".cuid| dedup completename | table completename | union [ search index=auto_iwarranty sourcetype="*managed_log*" "Content-Type: application/json" earliest=-d@d latest=@d | rex "ID\:\s(?P[^\s]+)\s" | rex "dealerId\"\:\"(?P\w*)" | rex "dealerBAC\"\:\"(?P\w*)" | rex "Address\:\s(?P.*)" | rex "cuid\"\:\"(?P\w*)" | rex "resultMessage\"\:\"(?P(.*?)(?=\"))" | transaction ID host requestURL maxspan=10m "Content-Type: application/json" keepevicted=true | where requestURL="https://prism-ins-osb-prod.int.ally.com:443/cps-iw-services/services/cps-rating/dealer/programs" | search resultMessage="null: java.sql.SQLException: weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CLMDataSource is Suspended, cannot allocate resources to applications.." OR resultMessage="null: java.sql.SQLException: weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CLMDataSourceWithTimeout is Suspended, cannot allocate resources to applications.." | eval customer=if(match(cuid, "^(?!G65UFBC9$)(?!BLINKER1$)(?!UserId00$)(?!BLINKER1$)(?!ROUTE001$)(?!Test1234$)(?!CARVANA1$)(?!CREDITIQ$)(?!SHIFT123$)(?!ABCDEFGH$)([A-Z]+\d+.*)$"), "UI", "ThirdParty")|search customer="UI" | eval time1=_time | eval time=round(time1)|convert ctime(time)| eval completename =time.".".cuid| eval fullname1 =time.".".dealer.".".cuid] | dedup fullname1 |table completename] | append [search index=auto_vsc sourcetype="*managed_log*" IllegalStateException earliest=-d@d latest=@d | rex "session\sid\s\"(?P\w+)"| rex "invalid\:\s(?P\w+)"| join mysession [search index=auto_vsc host=dfw* sourcetype=oss_logging earliest=-d@d latest=@d | rex field=_raw "sessionId\s\=\s(?P.*?)\!" | rex "userId\s=\s(?P\w+)"| rex "cuid\=(?P\w+)" | rex "sessionId\s\=\s(?P\w+)"| transaction userId source maxspan=1s startswith="SAMLSessionInterceptor(getLoginUser:247)"]| eval time=_time| convert ctime(time)|eval completename =time.".".cuid | table completename]| eventstats count as result by completename | where result=1 | eval name1=split(completename, ".")| eval FieldD=mvindex(name1,0)| eval FieldE=mvindex(name1,1)| eval Hour_Of_Day = strftime(strptime(FieldD, "%m/%d/%Y %H:%M:%S"), "%H") |eval time_date = strftime(strptime(FieldD, "%m/%d/%Y %H:%M:%S"),"%w") | stats count AS "Current_Day_Actual" by Hour_Of_Day, time_date |chart values(Current_Day_Actual) AS Current_Day_Actual by Hour_Of_Day] | eval DifferenceFromAverage=(Current_Day_Actual-Current_Day_Average) | table Hour_Of_Day, Current_Day_Actual,Current_Day_Average,DifferenceFromAverage
↧