Home |
Search |
Today's Posts |
|
#1
![]()
Posted to microsoft.public.excel.misc
|
|||
|
|||
![]()
I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or lookup table): =.69-(ROUND(.03*22,8,2)) or =.69-ROUND(.684,2) or =.69-.68 or =.01 However, I get 0.0099999999999999 Other calculations yield similar results. Sometimes ending with a 95 or 98 instead of 99. Now, I know I can work around this by inserting additional ROUND statments in my formula, but I'd like to understand how this error is getting introduced. If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer. Even =.69-.68 doesn't come out to .01. I don't recall having this kind of error in previous versions of Excel. What's up here? |
#2
![]()
Posted to microsoft.public.excel.misc
|
|||
|
|||
![]()
"silver" wrote in message
... I have a number of formulas that are showing incorrect results. One of these works out to be (with each value coming from another cell or lookup table): =.69-(ROUND(.03*22,8,2)) or =.69-ROUND(.684,2) or =.69-.68 or =.01 However, I get 0.0099999999999999 Other calculations yield similar results. Sometimes ending with a 95 or 98 instead of 99. Now, I know I can work around this by inserting additional ROUND statments in my formula, but I'd like to understand how this error is getting introduced. If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer. Even =.69-.68 doesn't come out to .01. I don't recall having this kind of error in previous versions of Excel. What's up here? Nothing is wrong! Excel, like all computer programs, can only calculate to finite precision. (Your memory about previous versions of Excel being different is mistaken.) Rounding errors are inherent - you just have to learn to live with them and deal with your results accordingly, using ROUND as you suggest or whatever is appropriate. Just don't expect exact answers! |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Day Calculation Error | Excel Discussion (Misc queries) | |||
Calculation Error? | Excel Discussion (Misc queries) | |||
Calculation Error? | Excel Discussion (Misc queries) | |||
Re-Calculation error | Excel Discussion (Misc queries) | |||
calculation error | Excel Worksheet Functions |