View Single Post
  #6   Report Post  
Posted to microsoft.public.excel.misc
Mike H Mike H is offline
external usenet poster
 
Posts: 11,501
Default Excel 2003 Calculation Problems

Luke,

For that matter, since when does division of a positve number by a postive
number yield a negative number?!?


It's all about precision and there being no precise binary representation of
the numbers your using. In the case of your numbers you need ~12 decimal
places before we get .0000000000001705.

Mike

"Luke M" wrote:

Strange. This isn't a solution, but I did find that
=MOD(314.56*6-314.56,314.56)
also returns 1.7053E-13 (answer to one of your equations)

So, afraid I don't have any clue as to why XL is messing up, but wanted to
get an idea as to why the difference in 1.7053E-13 vs -5.68434E-14

For that matter, since when does division of a positve number by a postive
number yield a negative number?!?



--
Best Regards,

Luke M
*Remember to click "yes" if this post helped you!*


"ak_edm" wrote:

I have Excel 2003 (11.8237.8221) SP3 according to the ABOUT screen in the
HELP menu. The three numbers and their cell locations a

F17 = 1887.36
F18 = 314.56
F21 = 314.56

Note that 314.56 x 6 = 1887.36, so why is this happening?

The formula =(1887.36-314.56)/314.56 equals 5
but the formula =MOD(1887.36-314.56,314.56) equals -5.68434E-14

(for this next one I formatted the digits in the Excel cell as far as I
thought good)
The formula =(F17-F18)/F21 equals 5.000000000000000
but the formula =MOD(F17-F18,F21) equals 0.000000000000171

Why?
How do I fix it?

- Eric