LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #6   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 5,600
Default Handle Floating point rounding errors, efficiently

Hi Dana,

I like using Currency since most comparisons like this are "usually" not
over 2 places to the right of the decimal point.


Indeed, in one routine I am only checking for <= 0 or =1

Just done a similar speed check as I mentioned in my reply to JE.
Currency vs Single vs Constant all about the same speed, the differences in
time are trivial.

Thanks for this and for your other comments.

Regards,
Peter

"Dana DeLouis" wrote in message
...
I like using Currency since most comparisons like this are "usually" not
over 2 places to the right of the decimal point.

Sub Demo()
Dim n As Currency

[A1] = 0
[A2].Formula = "=A1+0.05"
[A2].AutoFill [A2:A21]

n = [A21]
Debug.Print "[A21]=1", n = 1 'Now True
Debug.Print 1 - [A21] '-2.22044604925031E-16
Debug.Print
End Sub

[a21] '-2.22044604925031E-16


For the Op, although cell A21 shows 1.000..., Excel works with only 15
digits. Your math coprocessor is using 17. Excel can not display your
coprocessor's internal number of 1.0000000000000002.
However, when you do the math subtraction, Excel is padding the hidden
ending 2 with zero's in base two. When converted to decimal, the

displayed
numbers is not very meaningful.
Therefore, looking at you hidden '2'

? 2*2^-53
2.22044604925031E-16

--
Dana DeLouis
Win XP & Office 2003


"JE McGimpsey" wrote in message
...
Don't know about the fastest, most efficient, but

Const dEPSILON As Double = 1E-10 'set appropriately
If Abs(Cells(21, 1).Value - 1) < dEPSILON Then ...


will work. Using Range references (Cells(21,1)) instead of the evaluate
method ([A21]) will likely speed up your code more than which method of
testing you use.



In article ,
"Peter T" <peter_t@discussions wrote:

The various methods I've tried to eliminate "false" results slow the
process
at least to some extent, inevitably I suppose. My purpose and the

point
of
this post is to do exactly what you suggest:

"design your programs to be tolerant of the approximations inherent in
floating point numbers"

But in the fastest most efficient way possible.





 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unacceptable floating point errors Jeff in GA Excel Discussion (Misc queries) 32 September 25th 09 11:26 AM
setting a floating decimel point Rose New Users to Excel 2 April 29th 05 06:10 PM
Handle Compile Errors JeffT Excel Programming 1 October 14th 04 01:38 PM
REPOST: How to handle errors Bruccce Excel Programming 1 August 8th 03 03:22 PM
REPOST: How to handle errors Bob Kilmer Excel Programming 0 August 5th 03 11:24 PM


All times are GMT +1. The time now is 09:11 AM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"