Home |
Search |
Today's Posts |
#1
|
|||
|
|||
mathmatical error
Enter the formula =52.8-8.8*6. The answer is, of course, zero. Now enter it
as =(52.8-8.8*6). Excel will return a non zero number. Also try =70.4-8.8+61.6 with and without parentheses. If you happen to divide by any of these you will get a very large number. |
#2
|
|||
|
|||
http://www.cpearson.com/excel/rounding.htm
-- Regards, Peo Sjoblom "CRatt" wrote in message ... Enter the formula =52.8-8.8*6. The answer is, of course, zero. Now enter it as =(52.8-8.8*6). Excel will return a non zero number. Also try =70.4-8.8+61.6 with and without parentheses. If you happen to divide by any of these you will get a very large number. |
#3
|
|||
|
|||
Most decimal fractions, including .8 and .6 have no exact binary
representation (much as 1/3 has no exact decimal representation). Excel and almost all other software follows the IEEE standard for double precision storage to approximate your input numbers. The nonzero values are the result of exact calculation from the approximate inputs. The exact results for =(52.8-8.8*6) and =(70.4-8.8-61.6) are -/+ 1/140737488355328 which in decimal is exactly -/+ 7.10542735760100185871124267578125E-15 Excel correctly reports this to its documented limit of 15 decimal digits as +/- 7.105427357601E-15 Without the parentheses, you get zero because the final operation calculates the difference between two numbers that agree to 15 decimal digits. Under that circumstance, Excel arbitrarily zeroes the result on the assumption that any possible difference from zero is the result of binary approximations. Jerry CRatt wrote: Enter the formula =52.8-8.8*6. The answer is, of course, zero. Now enter it as =(52.8-8.8*6). Excel will return a non zero number. Also try =70.4-8.8+61.6 with and without parentheses. If you happen to divide by any of these you will get a very large number. |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
ERROR | Excel Discussion (Misc queries) | |||
Excel error - Startup (and Acrobat PDFMaker) | Setting up and Configuration of Excel | |||
Findlink Error | Links and Linking in Excel | |||
Error when entering and exiting excel | Excel Discussion (Misc queries) | |||
#REF error | Excel Worksheet Functions |