Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
CRatt
 
Posts: n/a
Default mathmatical error

Enter the formula =52.8-8.8*6. The answer is, of course, zero. Now enter it
as =(52.8-8.8*6). Excel will return a non zero number. Also try
=70.4-8.8+61.6 with and without parentheses. If you happen to divide by any
of these you will get a very large number.
  #2   Report Post  
Peo Sjoblom
 
Posts: n/a
Default

http://www.cpearson.com/excel/rounding.htm

--


Regards,

Peo Sjoblom


"CRatt" wrote in message
...
Enter the formula =52.8-8.8*6. The answer is, of course, zero. Now enter
it
as =(52.8-8.8*6). Excel will return a non zero number. Also try
=70.4-8.8+61.6 with and without parentheses. If you happen to divide by
any
of these you will get a very large number.



  #3   Report Post  
Jerry W. Lewis
 
Posts: n/a
Default

Most decimal fractions, including .8 and .6 have no exact binary
representation (much as 1/3 has no exact decimal representation). Excel
and almost all other software follows the IEEE standard for double
precision storage to approximate your input numbers. The nonzero values
are the result of exact calculation from the approximate inputs. The
exact results for =(52.8-8.8*6) and =(70.4-8.8-61.6) are
-/+ 1/140737488355328
which in decimal is exactly
-/+ 7.10542735760100185871124267578125E-15
Excel correctly reports this to its documented limit of 15 decimal digits as
+/- 7.105427357601E-15

Without the parentheses, you get zero because the final operation
calculates the difference between two numbers that agree to 15 decimal
digits. Under that circumstance, Excel arbitrarily zeroes the result on
the assumption that any possible difference from zero is the result of
binary approximations.

Jerry

CRatt wrote:

Enter the formula =52.8-8.8*6. The answer is, of course, zero. Now enter it
as =(52.8-8.8*6). Excel will return a non zero number. Also try
=70.4-8.8+61.6 with and without parentheses. If you happen to divide by any
of these you will get a very large number.


Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
ERROR Pinto1uk Excel Discussion (Misc queries) 1 February 8th 05 03:15 AM
Excel error - Startup (and Acrobat PDFMaker) gxdata Setting up and Configuration of Excel 0 February 4th 05 03:44 AM
Findlink Error D Moniz via OfficeKB.com Links and Linking in Excel 0 January 20th 05 04:53 PM
Error when entering and exiting excel Randy Excel Discussion (Misc queries) 1 January 11th 05 03:17 PM
#REF error Christen Excel Worksheet Functions 5 November 3rd 04 07:29 PM


All times are GMT +1. The time now is 08:43 AM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"