• nogrub@lemmy.world
    1·
    1 year ago

    it’s almost like computers are not that accurate when calculating floating point numbers

    • afraid_of_zombies@lemmy.world
      1·
      1 year ago

      About a year ago I ended up with a floating point value that was something like 1.0000000000078 when it should have been 1. Tore my hair out for hours trying to get the piece of crap embedded vendor locked device to just make it 1.

  • halvar@lemm.ee
    1·
    1 year ago

    well 0.9999… is actually 1 because

    x = 0.9999...

10x = 9.9999...

10x (9.9999...) - x (0.9999...) = 9

9x = 9

x = 1

so 0.9999... is 1
    • RedditWanderer@lemmy.world
      1·
      1 year ago

      This is muuuch better demonstrated by

      1/3 = .33… 2/3 = .66… 3/3 = 0.99…

      “Repeating” matters in approximations