View Single Post
  #1   Report Post  
Posted to microsoft.public.excel.worksheet.functions
Bill Owens
 
Posts: n/a
Default What is the roundoff error in the subtraction function?

Why does 1.00000000000000000000-0.99950000000000000000=0.000499999999999945
instead of 0.0005? I have 50,000 records that are evenly spaced by 0.0005 and
they appear that way in the column even when viewed with 20 places, however,
after subtracting one number from the other the difference is not 0.0005. I
tried using round off to 4 places and that doesnt resolve the problem. Any
help is appreciated.