View Single Post
  #8   Report Post  
Posted to microsoft.public.excel.programming
Jerry W. Lewis Jerry W. Lewis is offline
external usenet poster
 
Posts: 837
Default Increment for or do while loop by 0.001 (not 1)

Excel displays a maximum of 15 digits, but it takes 17 digits to uniquely
specify a binary floating point value. The discrepancys begin at 1.2, but
don't accumulate sufficiently to be visible in the 15th digit until after 6.

You can see this by changing your output statement to
Debug.Print i - Round(10 * i, 0) / 10
or by using the D2D function from
http://groups.google.com/group/micro...06871cf92f8465

Jerry

"NickHK" wrote:

I fully expected such approximation, but seemed strange why it did not
become apparent until 6.2
But also the loop cannot really be 1+0.1+0.1........+0.1 because then "i"
would fixed at maximum 1 decimal place

NickHK

"Jerry W. Lewis" ...
That is because 0.1, 0.001, and the vast majority of other terminating
decimal fractions are non-terminating binary fractions that can only be
approximated.

Using the Currency type does not avoid the issue altogether, it just
restricts it to where you would expect it. For intance, presumably you
would
understand why

Dim i As Currency
i=1/3
i=1-3*i

does not have i as zero at the end.

Jerry

"NickHK" wrote:

Tom,
I have noticed that using something like :
Dim i As Double
For i = 1 To 10 Step 0.1
Debug.Print i
Next

you cannot use :
If i=6.2 Then
because of accuracy issues. e.g.
............
5.8
5.9
6
6.1
6.19999999999999
6.29999999999999
6.39999999999999
..........
Using a Currency type avoids this .

NickHK
P.S. Seems strange that if you add a line "If i 6 Then Stop" in the
loop,
then in the immediate window "?i+0.1" the result is 6.2 exactly. But
"?i+0.1=6.2" shows False. Confused ?
So a For/Next loop like this is not really adding 0.1 each time, but
using
integers really then scaling the counter ?