Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.misc
JEV JEV is offline
external usenet poster
 
Posts: 9
Default SUM OF NUMBERS APPEAR INCORRECT?

EXCEL 2003

30,625.010000000000000000000000000000
(30,570.020000000000000000000000000000)

54.989999999998

Why does the sum of these 2 numbers yield the answer shown? The answer is
being compared to the following separate sum below and is yeilding a
different answer as well. The 2 numbers being added or subtracted are not
calculated- they're entered manually.

=15340-15285.01
54.990000000000

If the answer to the second equation is formatted out to more than 12 places
you get something less than the 54.99 I would have expected in both instances.

Appreciate your insight.
  #2   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 11,058
Default SUM OF NUMBERS APPEAR INCORRECT?

This is simple rounding error. If you require a precise calculate to two
decimal digits, then use the =ROUND() function.
--
Gary''s Student - gsnu200832


"JEV" wrote:

EXCEL 2003

30,625.010000000000000000000000000000
(30,570.020000000000000000000000000000)

54.989999999998

Why does the sum of these 2 numbers yield the answer shown? The answer is
being compared to the following separate sum below and is yeilding a
different answer as well. The 2 numbers being added or subtracted are not
calculated- they're entered manually.

=15340-15285.01
54.990000000000

If the answer to the second equation is formatted out to more than 12 places
you get something less than the 54.99 I would have expected in both instances.

Appreciate your insight.

  #3   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 4,393
Default SUM OF NUMBERS APPEAR INCORRECT?

Excel, and most other computer apps, follow the IEEE convention to store
digital numbers (number to base 10 that we use because we have 10 fingers)
as binary numbers (numbers to base 2, because a compute memory state is
either ON or OFF). This results is round-off errors like the one you found.

Work around: =ROUND(A1-B1,12) to round to 12 places

Note that there is seldom any good reason to work with more than 12 decimal
places; there are few things even a scientist can measure with that
precision.


More details on IEEE 754:
Chip's clear explanation
http://www.cpearson.com/excel/rounding.htm

Floating-point arithmetic may give inaccurate results in Excel
http://support.microsoft.com/kb/78113/en-us

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/kb/42980

What Every Computer Scientist Should Know About Floating Point
http://docs.sun.com/source/806-3568/ncg_goldberg.html
http://www.cpearson.com/excel/rounding.htm

Visual Basic and Arithmetic Precision
http://support.microsoft.com/default...NoWebContent=1

Others:
http://support.microsoft.com/kb/214118

http://docs.sun.com/source/806-3568/ncg_goldberg.html

--
Bernard V Liengme
Microsoft Excel MVP
http://people.stfx.ca/bliengme
remove caps from email

"JEV" wrote in message
...
EXCEL 2003

30,625.010000000000000000000000000000
(30,570.020000000000000000000000000000)

54.989999999998

Why does the sum of these 2 numbers yield the answer shown? The answer is
being compared to the following separate sum below and is yeilding a
different answer as well. The 2 numbers being added or subtracted are not
calculated- they're entered manually.

=15340-15285.01
54.990000000000

If the answer to the second equation is formatted out to more than 12
places
you get something less than the 54.99 I would have expected in both
instances.

Appreciate your insight.



Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
incorrect calculation Markies Excel Worksheet Functions 2 May 28th 08 08:27 PM
Incorrect Am and Pm MrBlackForest Excel Discussion (Misc queries) 2 January 26th 06 03:11 AM
Incorrect Sum ashlandpmac Excel Discussion (Misc queries) 1 April 15th 05 01:25 AM
Incorrect Sum AshlandPmac Excel Discussion (Misc queries) 1 April 14th 05 10:41 PM
incorrect totals using rounded numbers Alexis Excel Discussion (Misc queries) 1 March 23rd 05 11:19 PM


All times are GMT +1. The time now is 02:48 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"