LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #4   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 27,285
Default Comparing double values

for doubles, VBA/Excel uses the IEEE standard which is approximately 15
digits of precision, so the error is usually around the last couple of
digits. How that translates into your "epsilon" would depend on your
numbers.

Here is an illustrative example of accumulating the results of imprecision:

Sub abcd()
Dim v As Double
v = 0#
For i = 1 To 300
v = v + CDbl(1 / 3)
Next
Debug.Print Format(v, "0.0000000000000000000")
Debug.Print Format(CDbl(1 / 3) * 300, "0.0000000000000000000")

End Sub


--
Regards,
Tom Ogilvy

"Edward Ulle" wrote in message
...
Tom,

I suspected as much but in your opinion, if I am dealing with small
numbers what is a minimum tolerance, say 10 to the minus 10 or is that
too small?


*** Sent via Developersdex http://www.developersdex.com ***



 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Macro to double values in a cell MS Excel Worksheet Functions 9 May 13th 07 03:20 AM
Help with Comparing values and retrieving values in Excel!!!!!! [email protected] Excel Worksheet Functions 1 November 17th 06 12:21 AM
Comparing values between columns only when there are values in bot Mark K Excel Worksheet Functions 1 February 19th 06 06:47 PM
Comparing values in two columns and displaying missing values in n cpetta Excel Programming 1 April 2nd 05 06:18 AM
Comparing Values Todd Huttenstine[_2_] Excel Programming 7 December 1st 03 06:37 AM


All times are GMT +1. The time now is 02:51 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"