View Single Post
  #4   Report Post  
Posted to microsoft.public.excel.programming
Bernard Liengme Bernard Liengme is offline
external usenet poster
 
Posts: 4,393
Default Floating point number comparison

Not sure what you mean.
Case 1, I want to know if x=y
=IF(ABS(x-y)<=1E-6, "equal", "unequal")
Of course, by "equal" I mean that x is within y +- epsilon, and "unequal"
means x is at least epsilon larger/smaller than y

Case 2 I want to know if x is 10 units larger than y
Without worrying about IEEE precision =IF(x-y=10, "true", "false")
Would I have need to worry about IEEE here?
Suppose x=20 and y = 10.000000000009
Would I want the test to pas or fail?
If I think this should pass: =IF(ROUND(x-y,5)=10, "true", "false")
I have decided that any difference that round to 10 at 5 place precision is
OK.

Any help?
--
Bernard V Liengme
www.stfx.ca/people/bliengme
remove caps from email

"Edward Ulle" wrote in message
...
Yes, but what about other operations <, <, , <= and =? Has anyone
developed efficient routines to do this? Any floating point number can
be x +/- EPSILON.

*** Sent via Developersdex http://www.developersdex.com ***