GamerBoy705@lemmy.worldM to Software Gore@lemmy.worldEnglish · 1 year agoNot good enough for 100%pxscdn.comimagemessage-square4fedilinkarrow-up14arrow-down10
arrow-up14arrow-down1imageNot good enough for 100%pxscdn.comGamerBoy705@lemmy.worldM to Software Gore@lemmy.worldEnglish · 1 year agomessage-square4fedilink
minus-squarenogrub@lemmy.worldlinkfedilinkarrow-up1·1 year agoit’s almost like computers are not that accurate when calculating floating point numbers
minus-squareafraid_of_zombies@lemmy.worldlinkfedilinkarrow-up1·1 year agoAbout a year ago I ended up with a floating point value that was something like 1.0000000000078 when it should have been 1. Tore my hair out for hours trying to get the piece of crap embedded vendor locked device to just make it 1.
it’s almost like computers are not that accurate when calculating floating point numbers
About a year ago I ended up with a floating point value that was something like 1.0000000000078 when it should have been 1. Tore my hair out for hours trying to get the piece of crap embedded vendor locked device to just make it 1.