View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.programming
K Dales K Dales is offline
external usenet poster
 
Posts: 131
Default Is this rounding error?

It is a little more complex than just rounding error. Try
this:

Public Sub foo()
Dim r
Dim i

For r = 2 To 16

i = Fix(Log(r) / Log(r))
Debug.Print "r:" & r & " i:" & i _
& " Log(r):" & Log(r) & " Log(r)/Log(r):" _
& Log(r) / Log(r) & " Fix:" & Fix(Log(r) / Log(r))
Next r


End Sub

You will see that when the results of the formula Fix(Log
(r) / Log(r)) are printed directly it gives the result 1
but when it is assigned to the variable i it gives the
result 0! This seems to occur even if I explicitly define
r and i to be either long integers or a double precision
floating point variables!


-----Original Message-----
In the below routine, are the instances where i=0 due to

not being able to
express certain values exactly in binary?

And can it be reliably solved for any 'r' by ROUNDing to

15 places?

Or is there something else I'm missing?

Thanks.

===================
Sub foo()
Dim r
Dim i

For r = 2 To 16

i = Fix(Log(r) / Log(r))
Debug.Print ("For r = " & r & ", i = " & i)
Next r

End Sub
================


--ron
.