View Single Post
  #5   Report Post  
Posted to microsoft.public.excel.newusers
joeu2004 joeu2004 is offline
external usenet poster
 
Posts: 2,059
Default ABS <= and logic functions

"DrPhil" wrote previously:
I'm having trouble with this function
=IF(ABS(G30-F30)<=0.2,"PASS","FAIL")


You should use:

=if(round(abs(G30-F30),1)<=0.2,"PASS","FAIL")


"DrPhil" wrote in a follow-up:
G30 F30
1.2 1.0 works
30 30.2 works
32.2 32 does not work


Because most decimal fractions cannot be represented exactly in the binary
format that Excel uses internally. This sometimes causes small numerical
abberations in arithmetic results. In this case, you can see the problem if
you enter =ABS(G30-F30) in a cell formatted as Scientific with 14 decimal
places. In your last example, you will see that ABS() returns
2.00000000000003E-01, not 2.00000000000000E-01.