151
pretty small subtotal. (thelemmy.club)
submitted 1 year ago by [M] to c/software_gore@programming.dev
26 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[+] -13 points 1 year ago (5 children)

A f64 is a double, floating point precision type, aka not precise

f64 is perfectly precise.

The correct format to store money in? A string

Lol ok.

$3.99999999898998888181

If only we had this thing called rounding...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 12 points 1 year ago (3 children)

    Double is not perfectly precise. It is quite literally a function that calculates what the value should be. There are converters to show the drift: https://www.h-schmidt.net/FloatConverter/IEEE754.html

    $40.01 is literally stored as 01000010001000000000101000111101 in a float type, which is literally stored as 40.009998321533203125. The margin of error goes up the larger the number.

    Either you're just trolling, or incredibly arrogant. I have been coding in the FinTech space for over a decade and can tell you firsthand that these are real issues, and have real consequences.

    But hey, don't take my word for it:

    There have been some famous disasters thanks to floating point math

    As for rounding, I can't even think of a way to describe what a terrible idea that is. I'd suggest reading those articles and asking the stock exchange why they don't "just round it" when dealing with millions of transactions that deal with fractions of pennies. "Just round it"

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 1 year ago*

    Double is not perfectly precise. It is quite literally a function that calculates what the value should be. There are converters to show the drift: https://www.h-schmidt.net/FloatConverter/IEEE754.html

    $40.01 is literally stored as 01000010001000000000101000111101 in a float type, which is literally stored as 40.009998321533203125. The margin of error goes up the larger the number.

    No you're fundamentally misunderstanding what's going on. Float isn't "storing 40.01 but imprecisely". That's like saying an integer "stores 1.1 but imprecisely". Floating point can't represent the number 40.01. But that doesn't matter because 4.0009999999999998 is easily accurate enough for any real world use.

    The story of Ariane 5

    This was caused by converting an f64 to i16 which caused an overflow and hardware exception. Irrelevant.

    How a Minor Calculation Error Cost Intel Half a Billion Dollars

    Hardware bug. Irrelevant.

    Wikipedia

    Literally says "The failure to intercept arose not from using floating point specifically"

    and asking the stock exchange

    I'm not talking about stock exchanges. Obviously they have extreme requirements. For most normal businesses and people f64 is fine.

    You're repeating dogma without thinking about it.

  • permalink
  • fedilink
  • source
  • parent