Single vs Double
Jeff wrote...
I have the code
Sub Test()
Dim X as Single
Dim Y as DOuble
Dim Z as Double
x=.54
Y=300
z=X*Y
End Sub()
Z should equal 162, but it is 162.000006437302
Why is that? . . .
Classic floating point rounding error. 0.54 can't be represented
exactly as a binary fraction.
. . . Does that mean that all the variables have to be double to
get the exact answer?
Not necessarily. If you had declared z as a single, z would have wound
up with the exact value 162 fortuitously.
When using floating point values, you need to check for APPROXIMATE
equality rather than exact equality, e.g.,
Abs(z - 162) < 0.5
rather than
z = 162
This has always been so, and it'll continue to be so for finite
precision floating point.
|