XL2003 also returns 2.71947991102104E-16, but per
http://support.microsoft.com/kb/826349/
XL2003 was the first version that changed the STDEV calculation to
=SQRT(SUMSQ(1.4434-AVERAGE(1.4434,1.4434,1.4434),1.4434-AVERAGE(1.4434,1.4434,1.4434),1.4434-AVERAGE(1.4434,1.4434,1.4434))/2)
Prior to XL2003, STDEV was calculated as
=SQRT((SUMSQ(1.4434,1.4434,1.4434)-SUM(1.4434,1.4434,1.4434)^2/3)/2)
which returns the value that the OP reported.
These two formulas are mathematically but not numerically equivalent.
A third approach would use a one-pass updating algorithm
http://groups.google.com/group/micro...6ee0c636ad016a
which would correctly return zero for this standard deviation, even though
1.4434 cannot be exactly represented in binary.
Jerry
"JE McGimpsey" wrote:
It is pretty large, but it's not simply due to rounding...
XL04:
=STDEV(1.4434,1.4434,1.4434) === 2.71947991102104E-16
(I don't have XL03 available right now, but I suspect that the result
would be the same).
The improvement was undoubtedly part of the overhaul of stats functions
for XL03/04:
http://support.microsoft.com/kb/828888/en-us
In article ,
"David K" <David wrote:
the function
=STDEV(1.4434,1.4434,1.4434)
gives
2.98023E-08
(at least on my computer using Excel 2002, sp3)
Is this just rounding error due to IEEE double precision. It seems pretty
large.