464
IEEE 754 (cdn.fosstodon.org)
submitted 2 years ago by to c/science_memes@mander.xyz
37 comments fedilink hide all child comments
 

cross-posted from: https://lemmy.ml/post/24332731

Stolen Cross-posted from here: https://fosstodon.org/@foo/113731569632505985

Omni-Man looking at the IEEE 754 Single Precision Floating-Point Standard and saying "look what they need to mimic a fraction" with the rest of the meme whited out
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 years ago (3 children)

So if you handle different precisions you also need to store the precision/exponent explicitly for every value. Or would you sanitise this at input and throw an exception if someone wants more precision than the program is made for?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 6 points 2 years ago (1 child)

    It depends on the requirements of your app and what programming language you use. Sometimes you can get away with using a fixed precision that you can assume everywhere, but most common programming languages will have some implementation of a decimal type with variable precision if needed, so you won't need to implement it on your own outside of university exercises.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 years ago

    Okay thank you. I was wondering because for stuff like buying electricity, gas or certain resources, parts etc. there is prices with higher precision in cents, but the precision would not be identical over all use cases in a large company.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 5 points 2 years ago

    No exponent, or at least a common fixed exponent. The technique is called "fixed point" as opposed to "floating point". The rationale is always to have a known level of precision.

  • permalink
  • fedilink
  • source
  • parent