View Single Post
  #3   Report Post  
Posted to microsoft.public.excel.worksheet.functions
Mike H Mike H is offline
external usenet poster
 
Posts: 11,501
Default Problem with = in function

I should have added

=ROUND(E11,1)=0.7

rounding E11 to 1 decimal place makes it evaluate as TRUE

"Mike H" wrote:

Hi,

If you format E10, E11 & E12 to a number with lots of decimal places you'll
finf that E10 is 0.7 but E11 & E12 are actually 0.6999999999999999 so hence
the evaluation as false when tested for =0.7.

The reason for this is in the way Excel does arithmetic and there being no
precise binary equivalent of 0.7. For a full explanation have a look here

http://support.microsoft.com/default.aspx/kb/78113

Mike

"Raj" wrote:

Hi,

Please help in understanding this:

Cells d10,d11,d12 have the value 10.7 in them
Cell e10 has the value .7 directly input
Cell e11 has the value .7 derived by the formula =d11-int(d11)
Cell e12 has the value .7 dervied by the formula = e11-trunc(e11)
When values in cells e10,e11,e12 are tested for =.7, cell f10 shows
true, f11 and f12 show false. Why?

Thanks in Advance for the Help.

Raj