Home |
Search |
Today's Posts |
#1
Posted to microsoft.public.excel.programming
|
|||
|
|||
Floating point number comparison
I know this is the subject of many prior threads, but, I'm having
trouble comparing floatinp point numbers. Values in registers and values in memory are not exactly equal, even though they were calculated identically. I was wondering if there is a library of functions to compare floating point numbers, equal, less than, greater than, positive numbers and negative numbers. So far I only have one that I have confidence in. x=y is Abs(x-y)<EPSILON. Any leads would be appreciated. *** Sent via Developersdex http://www.developersdex.com *** |
#2
Posted to microsoft.public.excel.programming
|
|||
|
|||
Floating point number comparison
One should never compare two real numbers (aka floating point) for EXACT
equality. Your approach =ABS(x-y)<=EPSILON is the way to go. best wishes -- Bernard V Liengme www.stfx.ca/people/bliengme remove caps from email "Edward Ulle" wrote in message ... I know this is the subject of many prior threads, but, I'm having trouble comparing floatinp point numbers. Values in registers and values in memory are not exactly equal, even though they were calculated identically. I was wondering if there is a library of functions to compare floating point numbers, equal, less than, greater than, positive numbers and negative numbers. So far I only have one that I have confidence in. x=y is Abs(x-y)<EPSILON. Any leads would be appreciated. *** Sent via Developersdex http://www.developersdex.com *** |
#3
Posted to microsoft.public.excel.programming
|
|||
|
|||
Floating point number comparison
Yes, but what about other operations <, <, , <= and =? Has anyone
developed efficient routines to do this? Any floating point number can be x +/- EPSILON. *** Sent via Developersdex http://www.developersdex.com *** |
#4
Posted to microsoft.public.excel.programming
|
|||
|
|||
Floating point number comparison
Not sure what you mean.
Case 1, I want to know if x=y =IF(ABS(x-y)<=1E-6, "equal", "unequal") Of course, by "equal" I mean that x is within y +- epsilon, and "unequal" means x is at least epsilon larger/smaller than y Case 2 I want to know if x is 10 units larger than y Without worrying about IEEE precision =IF(x-y=10, "true", "false") Would I have need to worry about IEEE here? Suppose x=20 and y = 10.000000000009 Would I want the test to pas or fail? If I think this should pass: =IF(ROUND(x-y,5)=10, "true", "false") I have decided that any difference that round to 10 at 5 place precision is OK. Any help? -- Bernard V Liengme www.stfx.ca/people/bliengme remove caps from email "Edward Ulle" wrote in message ... Yes, but what about other operations <, <, , <= and =? Has anyone developed efficient routines to do this? Any floating point number can be x +/- EPSILON. *** Sent via Developersdex http://www.developersdex.com *** |
#5
Posted to microsoft.public.excel.programming
|
|||
|
|||
Floating point number comparison
Basically what I mean is are there any efficient replacements for =, <,
<, , <= and = for floating point number comparison that take into account EPSILON. I have developed 6 functions FPEqual, FPNotEqual, etc. that do this but they may not be the most efficient. FYI, the macro I've developed is used for an engineering application in which floating point numbers are the norm and there are thousands of floating point number comparison performed. *** Sent via Developersdex http://www.developersdex.com *** |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Unacceptable floating point errors | Excel Discussion (Misc queries) | |||
Converting 2-place decimal value to floating point decimal number with leading zero | Excel Discussion (Misc queries) | |||
Floating point problem?? | Excel Programming | |||
setting a floating decimel point | New Users to Excel | |||
Floating Point Functions | Excel Programming |