incorrect results when working with doubles or singles
 

Hi everybody,

I am puzzled by the following strange things:

Public Sub test1()
Dim a As Double
a = 16.4
a = a - 0.1

MsgBox a
If (a = 16.3) Then
MsgBox "1"
Else
MsgBox "2"
End If
End Sub

Public Sub test2()
Dim a As Single
a = 16.4
a = a - 0.1

MsgBox a
If (a = 16.3) Then
MsgBox "1"
Else
MsgBox "2"
End If
End Sub

When I run test1 the first MsgBox shows 16.3 and the second shows 2.
This means dat 16.3 doesn't equal to 16.3. Now when i change the
datatype to single (test2) everything works fine. Can anyone explain
me the reason of this?

Public Sub test3()
Dim a As Single
Dim b As Single
a = 16.4

Dim i As Single
For i = 0.1 To 20 Step 0.1
b = a + i
Next i

MsgBox b
End Sub

The result of test3 is: 36.30004. Why isn't it exactly 36.3, and what
to do?

Public Sub test4()
Dim a As Single
Dim b As Single
a = 27.1

Dim i As Single
For i = 0.1 To 5.9 Step 0.1
b = a + i
Next i

MsgBox b
If b = 33 Then
MsgBox "1"
Else
MsgBox "2"
End If
End Sub

When I run test4 the first MsgBox shows 33 and the second one shows 2.
This means that 33 isn't equal to 33. Why not?

Any information about this would be greatly appreciated.

Patrice

incorrect results when working with doubles or singles
 

Check out

http://cpearson.com/excel/rounding.htm

If you're making exact comparisons with floating point numbers you're
best to either round them or to compare them within a small tolerance,
e.g., instead of

If a = 16.3 Then

use

Const epsilon As Double = 1e-10
Dim a As Double
a = 16.4 - 0.1
MsgBox a = 16.3
If Abs(a - 16.3) < epsilon Then MsgBox "Same"

where epsilon is a sufficiently small number that rounding errors won't
cause a problem

In article ,
(Patrice) wrote:

Hi everybody,

I am puzzled by the following strange things: