Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Overlanda
 
Posts: n/a
Default Excel Calculation is Incorrect

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly
  #2   Report Post  
Thief_
 
Posts: n/a
Default

I've seen this b4. It's not an actual bug in XL, from memory, it's really
got to do with the the way the CPU stores the numbers.

Check these:

http://www.cpearson.com/excel/rounding.htm
http://www.bugnet.com/analysis/excel_calc.html
http://support.microsoft.com/default...13&Product=xlw
http://support.microsoft.com/default...18&Product=xlw


--
|
+-- Julian
|

"Overlanda" wrote in message
...
Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly



  #3   Report Post  
JE McGimpsey
 
Posts: n/a
Default

See

http://www.mcgimpsey.com/excel/pennyoff.html

In article ,
Overlanda wrote:

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly

  #4   Report Post  
Don S
 
Posts: n/a
Default

On Thu, 14 Apr 2005 19:48:02 -0700, Overlanda
wrote:

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly



I'm not a math wiz, but I think it has something to do with binary
(computer) calculation of decimal (base 10) numbers. Use
=round(toberounded) & you'll have fewer "errors".

Don S
Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Excel 2003 FAILS, but Excel 2000 SUCCEEDS ??? Richard Excel Discussion (Misc queries) 2 May 13th 23 11:46 AM
I get a program error when I download an excel template Ladybug Excel Discussion (Misc queries) 3 March 4th 05 12:02 AM
Opening an Excel file Sheldon Zaklow Excel Discussion (Misc queries) 0 March 3rd 05 10:29 PM
Difference in number of Excel NewsGroups Hari Prasadh Excel Discussion (Misc queries) 1 January 25th 05 11:32 AM
Incorrect SUM Values in Excel Spreadsheets Julie Watson Excel Worksheet Functions 2 January 4th 05 01:08 PM


All times are GMT +1. The time now is 01:51 AM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"