Quantcast
Channel: Questions in topic: "splunk-enterprise"
Viewing all articles
Browse latest Browse all 47296

Precision of arithmetic operations

$
0
0
When I execute the following search index="does not matter" | stats count AS value | eval value=123456.0 | eval x=value/(1024*1024*1024) | eval y=(value*1.0)/(1024*1024*1024) I get results which do not quite meet my expectations: value: 123456.0 x: 0.0001149774 y: 0.00011 Seems like splunk truncates the value. But why? Multiplying by 1.0 should (could, might, ...) transform the value into a floating point number but strangely this results in a less precise result. Is there any documentation on which precision splunk uses for arithmetic operations?

Viewing all articles
Browse latest Browse all 47296

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>