Home |
Search |
Today's Posts |
#6
![]()
Posted to microsoft.public.excel.programming
|
|||
|
|||
![]()
Hi Dana,
I like using Currency since most comparisons like this are "usually" not over 2 places to the right of the decimal point. Indeed, in one routine I am only checking for <= 0 or =1 Just done a similar speed check as I mentioned in my reply to JE. Currency vs Single vs Constant all about the same speed, the differences in time are trivial. Thanks for this and for your other comments. Regards, Peter "Dana DeLouis" wrote in message ... I like using Currency since most comparisons like this are "usually" not over 2 places to the right of the decimal point. Sub Demo() Dim n As Currency [A1] = 0 [A2].Formula = "=A1+0.05" [A2].AutoFill [A2:A21] n = [A21] Debug.Print "[A21]=1", n = 1 'Now True Debug.Print 1 - [A21] '-2.22044604925031E-16 Debug.Print End Sub [a21] '-2.22044604925031E-16 For the Op, although cell A21 shows 1.000..., Excel works with only 15 digits. Your math coprocessor is using 17. Excel can not display your coprocessor's internal number of 1.0000000000000002. However, when you do the math subtraction, Excel is padding the hidden ending 2 with zero's in base two. When converted to decimal, the displayed numbers is not very meaningful. Therefore, looking at you hidden '2' ? 2*2^-53 2.22044604925031E-16 -- Dana DeLouis Win XP & Office 2003 "JE McGimpsey" wrote in message ... Don't know about the fastest, most efficient, but Const dEPSILON As Double = 1E-10 'set appropriately If Abs(Cells(21, 1).Value - 1) < dEPSILON Then ... will work. Using Range references (Cells(21,1)) instead of the evaluate method ([A21]) will likely speed up your code more than which method of testing you use. In article , "Peter T" <peter_t@discussions wrote: The various methods I've tried to eliminate "false" results slow the process at least to some extent, inevitably I suppose. My purpose and the point of this post is to do exactly what you suggest: "design your programs to be tolerant of the approximations inherent in floating point numbers" But in the fastest most efficient way possible. |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Unacceptable floating point errors | Excel Discussion (Misc queries) | |||
setting a floating decimel point | New Users to Excel | |||
Handle Compile Errors | Excel Programming | |||
REPOST: How to handle errors | Excel Programming | |||
REPOST: How to handle errors | Excel Programming |