#1   Report Post  
Jerry W. Lewis
 
Posts: n/a
Default

The math is correct (you can trust it), but the orginal numbers
themselves had to be approximated, with the result that the final result
is only approximate.

Excel and almost all other general purpose computer software do math in
binary, but the only 2-decimal numbers that can be exactly represented
in binary are .00, .25, .50, and .75. All others are non-terminating
binary fractions (just as 1/3 is a non-terminating decimal fraction).
As a result, all other decimal fractions must be approximated. The
degree of approximation is governed by the IEEE standard that is
discussed in Chip Pearson's article that Harald suggested to you.

To see what happened subsquently, consider a hypothetical decimal
computer that carries only 4 decimal figures
=((123+1/3)-123)-1/3
would be calculated as
=(123.3-123)-0.3333
=0.3-0.3333
=-0.0333
instead of zero. Once you introduce finite precision approximations,
many calculations will reveal that the numbers were not exactly represented.

Given that the issue is approximation to inputs, not the subsequent
math, Rounding the final result is not obscuring a serious flaw, as you
seem to fear. Another approach would be to work in pennies instead of
dollars (i.e. multiply everything by 100), since integers are exactly
represented.

Jerry

MattShoreson wrote:

cheers guys for the response.

Have gone with rounding as values are all currencies.

Little bit worrying though as it makes you unsure of your faith in
comps in general to perform accurate calculations!!!!!!!


Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
"Group" function very slow with Excel 2003 :( ... While very quick with Excel2000 :O) Alain79 Excel Discussion (Misc queries) 4 June 14th 05 07:34 AM
Stop Excel Rounding Dates leinad512 Excel Discussion (Misc queries) 1 April 20th 05 04:19 PM
Hints And Tips For New Posters In The Excel Newsgroups Gary Brown Excel Worksheet Functions 0 April 15th 05 05:47 PM
Excel error - Startup (and Acrobat PDFMaker) gxdata Setting up and Configuration of Excel 0 February 4th 05 03:44 AM
Excel 2002 and 2000 co-install. Control Which Starts ? cnuk Excel Discussion (Misc queries) 2 January 17th 05 08:07 PM


All times are GMT +1. The time now is 01:06 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"