ExcelBanter

ExcelBanter (https://www.excelbanter.com/)
-   Excel Discussion (Misc queries) (https://www.excelbanter.com/excel-discussion-misc-queries/)
-   -   Calculation error? (https://www.excelbanter.com/excel-discussion-misc-queries/163649-calculation-error.html)

Silver

Calculation error?
 
I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or
lookup table):

=.69-(ROUND(.03*22,8,2))
or
=.69-ROUND(.684,2)
or
=.69-.68
or
=.01

However, I get 0.0099999999999999
Other calculations yield similar results. Sometimes ending with a 95 or 98
instead of 99.

Now, I know I can work around this by inserting additional ROUND statments
in my formula, but I'd like to understand how this error is getting
introduced.

If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer.
Even =.69-.68 doesn't come out to .01.

I don't recall having this kind of error in previous versions of Excel.
What's up here?

Stephen[_2_]

Calculation error?
 
"silver" wrote in message
...
I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or
lookup table):

=.69-(ROUND(.03*22,8,2))
or
=.69-ROUND(.684,2)
or
=.69-.68
or
=.01

However, I get 0.0099999999999999
Other calculations yield similar results. Sometimes ending with a 95 or
98
instead of 99.

Now, I know I can work around this by inserting additional ROUND statments
in my formula, but I'd like to understand how this error is getting
introduced.

If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer.
Even =.69-.68 doesn't come out to .01.

I don't recall having this kind of error in previous versions of Excel.
What's up here?


Nothing is wrong! Excel, like all computer programs, can only calculate to
finite precision. (Your memory about previous versions of Excel being
different is mistaken.) Rounding errors are inherent - you just have to
learn to live with them and deal with your results accordingly, using ROUND
as you suggest or whatever is appropriate. Just don't expect exact answers!




All times are GMT +1. The time now is 07:26 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com