Excel Calculation is Incorrect
 

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly

I've seen this b4. It's not an actual bug in XL, from memory, it's really
got to do with the the way the CPU stores the numbers.

Check these:

http://www.cpearson.com/excel/rounding.htm
http://www.bugnet.com/analysis/excel_calc.html
http://support.microsoft.com/default...13&Product=xlw
http://support.microsoft.com/default...18&Product=xlw


--
|
+-- Julian
|

"Overlanda" wrote in message
...
Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly

See

http://www.mcgimpsey.com/excel/pennyoff.html

In article ,
Overlanda wrote:

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly

On Thu, 14 Apr 2005 19:48:02 -0700, Overlanda
wrote:

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly


I'm not a math wiz, but I think it has something to do with binary
(computer) calculation of decimal (base 10) numbers. Use
=round(toberounded) & you'll have fewer "errors".

Don S