View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.worksheet.functions
JE McGimpsey JE McGimpsey is offline
external usenet poster
 
Posts: 4,624
Default EXCEL IF FUNCTION

You can't make that analogy, since that's not how double precision
floating point values are stored. Your comparison is in decimal, not
binary. Take a look at

http://cpearson.com/excel/rounding.htm

for a good basic explanation of the IEEE standard.


In article ,
"MH" wrote:
If that were the case then the same error would occur for the values of 1
and 0.7

MH

"Sandy Mann" wrote in message
...
"MH" wrote in message
...
It looks like this is a bug


It is not a bug, it is the result of the approximation that all computers
using binary representation of fractions makes.

=10209-10208.7
results in 0.299999999999272 although it displays as 0.3