Sunday 10 February 2019

floating point - Dealing with float precision in Javascript




I have a large amount of numeric values y in javascript. I want to group them by rounding them down to the nearest multiple of x and convert the result to a string.



How do I get around the annoying floating point precision?



For example:



0.2 + 0.4 = 0.6000000000000001



Two things I have tried:



>>> y = 1.23456789 
>>> x = 0.2
>>> parseInt(Math.round(Math.floor(y/x))) * x;
1.2000000000000002


and:




>>> y = 1.23456789 
>>> x = 0.2
>>> y - (y % x)
1.2000000000000002

Answer



From this post: How to deal with floating point number precision in JavaScript?



You have a few options:





  • Use a special datatype for decimals, like decimal.js

  • Format your result to some fixed number of significant digits, like this:
    (Math.floor(y/x) * x).toFixed(2)

  • Convert all your numbers to integers


No comments:

Post a Comment

php - file_get_contents shows unexpected output while reading a file

I want to output an inline jpg image as a base64 encoded string, however when I do this : $contents = file_get_contents($filename); print &q...