#1   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 11
Default Calculation error?

I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or
lookup table):

=.69-(ROUND(.03*22,8,2))
or
=.69-ROUND(.684,2)
or
=.69-.68
or
=.01

However, I get 0.0099999999999999
Other calculations yield similar results. Sometimes ending with a 95 or 98
instead of 99.

Now, I know I can work around this by inserting additional ROUND statments
in my formula, but I'd like to understand how this error is getting
introduced.

If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer.
Even =.69-.68 doesn't come out to .01.

I don't recall having this kind of error in previous versions of Excel.
What's up here?
  #2   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 364
Default Calculation error?

"silver" wrote in message
...
I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or
lookup table):

=.69-(ROUND(.03*22,8,2))
or
=.69-ROUND(.684,2)
or
=.69-.68
or
=.01

However, I get 0.0099999999999999
Other calculations yield similar results. Sometimes ending with a 95 or
98
instead of 99.

Now, I know I can work around this by inserting additional ROUND statments
in my formula, but I'd like to understand how this error is getting
introduced.

If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer.
Even =.69-.68 doesn't come out to .01.

I don't recall having this kind of error in previous versions of Excel.
What's up here?


Nothing is wrong! Excel, like all computer programs, can only calculate to
finite precision. (Your memory about previous versions of Excel being
different is mistaken.) Rounding errors are inherent - you just have to
learn to live with them and deal with your results accordingly, using ROUND
as you suggest or whatever is appropriate. Just don't expect exact answers!


Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Day Calculation Error Steve[_5_] Excel Discussion (Misc queries) 11 June 18th 07 11:41 PM
Calculation Error? Wei Ming from singapore Excel Discussion (Misc queries) 3 January 3rd 07 12:39 PM
Calculation Error? Randy Sp Excel Discussion (Misc queries) 1 March 17th 06 08:27 PM
Re-Calculation error 5 Star Excel Discussion (Misc queries) 2 June 16th 05 05:05 PM
calculation error happyearth Excel Worksheet Functions 2 June 16th 05 05:59 AM


All times are GMT +1. The time now is 03:01 AM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"