Home |
Search |
Today's Posts |
#1
![]()
Posted to microsoft.public.excel.programming
|
|||
|
|||
![]()
In Excel 2003, I compute the following, with the
indicated results. A1: =(1.4434 + 1.4434 + 1.4434) / 3 B1: =3*(1.4434 - A1)^2 / 2 C1: =(1.4434 - A1)^2 A1: 1.44340000000000E+00 B1: 7.39557098644699E-32 C1: 4.93038065763132E-32 I can understand why B1 and C1 are not zero. I am very familiar with the vagaries of IEEE 754 representation. But I do not understand why the results are different when I make the same computations in VBA. Can anyone offer some insight? A2: =myavg() B2: =myvar() C2: =mydiff() D2: =(1.4434 - A2)^2 A2: 1.44340000000000E+00 B2: 0.00000000000000E+00 C2: 0.00000000000000E+00 D2: 0.00000000000000E+00 The VBA functions a Function myavg() As Double myavg = (1.4434 + 1.4434 + 1.4434) / 3 End Function Function myvar() As Double Dim avg As Double avg = (1.4434 + 1.4434 + 1.4434) / 3 myvar = 3 * (1.4434 - avg) ^ 2 / 2 End Function Function mydiff() As Double Dim avg As Double avg = (1.4434 + 1.4434 + 1.4434) / 3 mydiff = (1.4434 - avg) ^ 2 End Function |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Unacceptable floating point errors | Excel Discussion (Misc queries) | |||
Convert floating point to Hours and Minutes ?? | Excel Discussion (Misc queries) | |||
Floating point problem?? | Excel Programming | |||
setting a floating decimel point | New Users to Excel | |||
Floating Point Functions | Excel Programming |