View Single Post
  #6   Report Post  
Posted to microsoft.public.excel.programming
Ron Rosenfeld Ron Rosenfeld is offline
external usenet poster
 
Posts: 5,651
Default Strange Log base 10 results

On 4 Feb 2005 13:51:18 -0800, "Chip Hankley" wrote:

I've written a function in VBA that requires the Base-10 Log of a
number to be rounded DOWN to the closest integer. E.G. 10.1 would be 1.
10.9 would be 1, etc.

What I'm finding in VBA (whether I run this from Excel, Access, etc) is
that when x (below) is a power of 10, the result is wrong. In this
snippet, z SHOULD evaluate to 1 (i.e., log-base10(10)=1, int(1) = 1).
But it's not working that way. The last line shows why. The variable y
is evaluating to a number *just below* - 1-y is 1.11022E-16.

Anyone ever seen this, or know a good workaround. The function really
depends on this working correctly.

TIA!

Sub test()
x = 10
'Return Log base 10 of x,
' 1 in this case
y = Log(x) / Log(10)
'Round y DOWN to the nearest
' integer. Should be ONE
' in this case
z = Int(y)
Debug.Print z
Debug.Print Format(1 - y, "#.#####E-###")
End Sub


I'm not sure if this will work in all instances, or if your range of values is
compatible, but using the CDec type conversion function seems to result in
acceptable results:

y = CDec(Log(x) / Log(10))


--ron