View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.programming
Keith Willshaw Keith Willshaw is offline
external usenet poster
 
Posts: 170
Default Can excel calculate?


"Jerry W. Lewis" wrote in message
...
64.1 (and most non-integer numbers) cannot be represented exactly in
(IEEE standard) binary instead you get
64.099999999999994315658113919198513031005859375
or roughly
64.1000000000000???
where the uncertain portion (???) can be either positive or negative,
but rounds to the desired number.
64.1000000000000???
-64.0000000000000
--------------------
0.1000000000000???
you got
0.0999999999999943
which, when multiplied by 10 gives ...

The math is correct, but the input numbers are only approximate, so it
should be no surprise that the answer is also only approximate.

C++ appears to give the right answer for one of 4 possible reasons.

Either

- you are not displaying full precision in C


This is actually the case, he was using cout which defaults to 6 decimal
places.

Keith