LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 20
Default Why do VBA and Excel floating point results differ?

In Excel 2003, I compute the following, with the
indicated results.

A1: =(1.4434 + 1.4434 + 1.4434) / 3
B1: =3*(1.4434 - A1)^2 / 2
C1: =(1.4434 - A1)^2

A1: 1.44340000000000E+00
B1: 7.39557098644699E-32
C1: 4.93038065763132E-32

I can understand why B1 and C1 are not zero.
I am very familiar with the vagaries of IEEE 754
representation.

But I do not understand why the results are
different when I make the same computations in
VBA. Can anyone offer some insight?

A2: =myavg()
B2: =myvar()
C2: =mydiff()
D2: =(1.4434 - A2)^2

A2: 1.44340000000000E+00
B2: 0.00000000000000E+00
C2: 0.00000000000000E+00
D2: 0.00000000000000E+00

The VBA functions a

Function myavg() As Double
myavg = (1.4434 + 1.4434 + 1.4434) / 3
End Function

Function myvar() As Double
Dim avg As Double
avg = (1.4434 + 1.4434 + 1.4434) / 3
myvar = 3 * (1.4434 - avg) ^ 2 / 2
End Function

Function mydiff() As Double
Dim avg As Double
avg = (1.4434 + 1.4434 + 1.4434) / 3
mydiff = (1.4434 - avg) ^ 2
End Function



 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unacceptable floating point errors Jeff in GA Excel Discussion (Misc queries) 32 September 25th 09 11:26 AM
Convert floating point to Hours and Minutes ?? [email protected] Excel Discussion (Misc queries) 1 August 19th 06 02:26 AM
Floating point problem?? Frederick Chow Excel Programming 6 January 3rd 06 03:39 AM
setting a floating decimel point Rose New Users to Excel 2 April 29th 05 06:10 PM
Floating Point Functions Joel Excel Programming 6 March 30th 05 10:29 PM


All times are GMT +1. The time now is 01:54 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"