464
IEEE 754 (cdn.fosstodon.org)
submitted 2 years ago by to c/science_memes@mander.xyz
37 comments fedilink hide all child comments
 

cross-posted from: https://lemmy.ml/post/24332731

Stolen Cross-posted from here: https://fosstodon.org/@foo/113731569632505985

Omni-Man looking at the IEEE 754 Single Precision Floating-Point Standard and saying "look what they need to mimic a fraction" with the rest of the meme whited out
you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 2 years ago*

Eh, if you use doubles and you add 0.000314 (just over 0.03 cents) to ten billion dollars you have an error of 1/10000 of a cent, and thats a deliberately perverse transaction. Its not ideal but its not the waiting disaster that using single precision is.

  • permalink
  • fedilink
  • source
  • parent