So I've found many questions that are similar to what I'm trying to do here but not quite the same and I've not been able to get any of them to work right for me. Apologies if the answer is out there and I just can't put it together. Hope someone can enlighten me on the best way to accomplish this:
I have two inputlookups that potentially have matching data (hostname, ip, mac). input1 is the 'master' I want to check against and I'm trying to get a count on how many records have hostname OR ip OR mac and then give me basic statistics about it. Basically answer the question 'are any of these in the master?'.
Field names are different in each input and hostname formats vary somewhat.
If input1.field1 = inputA.fieldA then A=Match
If input1.field2 = inputA.fieldB then B=Match
If input1.field3 = inputA.fieldC then C=Match
-Count all events from input1 = Total
(input1 is the 'master')
-Count where A OR B OR C=Match = ItsThere
(1 Match count per record regardless of how many matches it had)
-Percent of how much of input1 have matches from inputA = percentage
Also note that input1 has ~225,000 records and the other has 95,000+ records. When I got this as close as I think I could to 'working' it was capping my output to 50k. I increased various limits and added maxout to append but its still an issue. I'm hoping I can do this matching logic more efficiently to avoid this altogether.
I feel like there is a much better way to do this instead of using null checking and I'm not really trusting my results anyhow, seems like the logic is checking against the values in the fields of its own record, not all the other records - which is what it needs to do.
| inputlookup MASTER_LOOKUP
| append [|inputlookup append=t COMPARE_LOOKUP ]
| eval MASTER_hostname=rtrim(MASTER_hostname,".domain.com") | eval MASTER_hostname=rtrim(MASTER_hostname,".DOMAIN.COM")
| eval COMPARE_name=rtrim(COMPARE_name,".domain.com") | eval COMPARE_name=rtrim(COMPARE_name,".DOMAIN.COM")
| eval MASTER_name=rtrim(MASTER_name,".domain.com") | eval MASTER_name=rtrim(MASTER_name,".DOMAIN.COM")
| eval ip_result=nullif('COMPARE_ip','MASTER_ip') | eval ip_result=if(isnull(ip_result), "MASTER", "Match")
| eval name_result=nullif('COMPARE_name','MASTER_name') | eval name_result=if(isnull(name_result), "MASTER", "Match")
| eval mac_result=nullif('COMPARE_mac_address','MASTER_mac_address') | eval mac_result=if(isnull(mac_result), "MASTER", "Match")
| stats count AS Total, count(eval(ip_result="Match" OR mac_result="Match" OR name_result="Match")) AS Matches
| eval percent=round(Matches*100/Total,4) | table percent Total Matches
↧