Home |
Search |
Today's Posts |
#1
|
|||
|
|||
Excel Calculation is Incorrect
Put this into a cell and change the format of the cell to number with 16
decimal places... =1131.97-1123 The answer is 8.9700000000000300 Notice the 3 at the 14th place after the decimal. Haha what a joke. I have tried on two Excel 2003 boxes - one with SP1 and one without. Both calculated it incorrectly |
#2
|
|||
|
|||
I've seen this b4. It's not an actual bug in XL, from memory, it's really
got to do with the the way the CPU stores the numbers. Check these: http://www.cpearson.com/excel/rounding.htm http://www.bugnet.com/analysis/excel_calc.html http://support.microsoft.com/default...13&Product=xlw http://support.microsoft.com/default...18&Product=xlw -- | +-- Julian | "Overlanda" wrote in message ... Put this into a cell and change the format of the cell to number with 16 decimal places... =1131.97-1123 The answer is 8.9700000000000300 Notice the 3 at the 14th place after the decimal. Haha what a joke. I have tried on two Excel 2003 boxes - one with SP1 and one without. Both calculated it incorrectly |
#3
|
|||
|
|||
See
http://www.mcgimpsey.com/excel/pennyoff.html In article , Overlanda wrote: Put this into a cell and change the format of the cell to number with 16 decimal places... =1131.97-1123 The answer is 8.9700000000000300 Notice the 3 at the 14th place after the decimal. Haha what a joke. I have tried on two Excel 2003 boxes - one with SP1 and one without. Both calculated it incorrectly |
#4
|
|||
|
|||
On Thu, 14 Apr 2005 19:48:02 -0700, Overlanda
wrote: Put this into a cell and change the format of the cell to number with 16 decimal places... =1131.97-1123 The answer is 8.9700000000000300 Notice the 3 at the 14th place after the decimal. Haha what a joke. I have tried on two Excel 2003 boxes - one with SP1 and one without. Both calculated it incorrectly I'm not a math wiz, but I think it has something to do with binary (computer) calculation of decimal (base 10) numbers. Use =round(toberounded) & you'll have fewer "errors". Don S |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Excel 2003 FAILS, but Excel 2000 SUCCEEDS ??? | Excel Discussion (Misc queries) | |||
I get a program error when I download an excel template | Excel Discussion (Misc queries) | |||
Opening an Excel file | Excel Discussion (Misc queries) | |||
Difference in number of Excel NewsGroups | Excel Discussion (Misc queries) | |||
Incorrect SUM Values in Excel Spreadsheets | Excel Worksheet Functions |