View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.misc
Jim Thomlinson Jim Thomlinson is offline
external usenet poster
 
Posts: 5,939
Default Unacceptable floating point errors

You will have to take it up with IEEE (Institute of Electrical and
Electronics Engineers). It is the international standard for floating point
arithmetic. XL complies with this standard. Once you have convinced them that
they are wrong I am sure the XL will fix your issue.
--
HTH...

Jim Thomlinson


"Jeff in GA" wrote:

This is NOT some calculation to the nth place or an atomic weight.

This is: .29 - .28

Instead of returning the correct answer of 0.01, Microsoft Excel returns
0.00999999999999995

This floating point excuse for can not be blamed for poor product quality.

Does Microsoft ever plan to repair its product, or should it only be used
for calculating federal budget numbers?