Home |
Search |
Today's Posts |
|
#1
![]() |
|||
|
|||
![]()
The math is correct (you can trust it), but the orginal numbers
themselves had to be approximated, with the result that the final result is only approximate. Excel and almost all other general purpose computer software do math in binary, but the only 2-decimal numbers that can be exactly represented in binary are .00, .25, .50, and .75. All others are non-terminating binary fractions (just as 1/3 is a non-terminating decimal fraction). As a result, all other decimal fractions must be approximated. The degree of approximation is governed by the IEEE standard that is discussed in Chip Pearson's article that Harald suggested to you. To see what happened subsquently, consider a hypothetical decimal computer that carries only 4 decimal figures =((123+1/3)-123)-1/3 would be calculated as =(123.3-123)-0.3333 =0.3-0.3333 =-0.0333 instead of zero. Once you introduce finite precision approximations, many calculations will reveal that the numbers were not exactly represented. Given that the issue is approximation to inputs, not the subsequent math, Rounding the final result is not obscuring a serious flaw, as you seem to fear. Another approach would be to work in pennies instead of dollars (i.e. multiply everything by 100), since integers are exactly represented. Jerry MattShoreson wrote: cheers guys for the response. Have gone with rounding as values are all currencies. Little bit worrying though as it makes you unsure of your faith in comps in general to perform accurate calculations!!!!!!! |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
"Group" function very slow with Excel 2003 :( ... While very quick with Excel2000 :O) | Excel Discussion (Misc queries) | |||
Stop Excel Rounding Dates | Excel Discussion (Misc queries) | |||
Hints And Tips For New Posters In The Excel Newsgroups | Excel Worksheet Functions | |||
Excel error - Startup (and Acrobat PDFMaker) | Setting up and Configuration of Excel | |||
Excel 2002 and 2000 co-install. Control Which Starts ? | Excel Discussion (Misc queries) |