View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.programming
JE McGimpsey JE McGimpsey is offline
external usenet poster
 
Posts: 4,624
Default incorrect results when working with doubles or singles

Check out

http://cpearson.com/excel/rounding.htm

If you're making exact comparisons with floating point numbers you're
best to either round them or to compare them within a small tolerance,
e.g., instead of

If a = 16.3 Then

use

Const epsilon As Double = 1e-10
Dim a As Double
a = 16.4 - 0.1
MsgBox a = 16.3
If Abs(a - 16.3) < epsilon Then MsgBox "Same"

where epsilon is a sufficiently small number that rounding errors won't
cause a problem

In article ,
(Patrice) wrote:

Hi everybody,

I am puzzled by the following strange things: