ExcelBanter

ExcelBanter (https://www.excelbanter.com/)
-   Excel Discussion (Misc queries) (https://www.excelbanter.com/excel-discussion-misc-queries/)
-   -   Calculation Error? (https://www.excelbanter.com/excel-discussion-misc-queries/78096-calculation-error.html)

Randy Sp

Calculation Error?
 
I have a spreadsheet that has 29 numbers with up to 2 decimal place precision
(44.29, 36.89, etc). I am taking the 1st number and subtracting the other 28
numbers from it
(=A1-B1-B2-B3-B4-B5-B6-B7-B8-B9-B10-B11-B12-B13-B14-C1-C2-C3-C4-C5-C6-C7-C8-C9-C10-C11-C12-C13-C14).
When I do this I get a result of 775,817.3700000010. My question is: where is
that 1 coming from in the 9th decimal place? I am displaying all of my 29
input numbers to 10 decimal places, and they are all 0's past the 2nd decimal
place. They are all entered as raw values (none are the result of a formula).

One other point: if I take out ANY of the 28 cells that are being
subtracted, the problem goes away - the 1 in the 9th decimal place becomes a
0.

I can send you the sample spreadsheet if that would help.

Bernard Liengme

Calculation Error?
 
Not a bug but a feature of almost all computer math.
See Excel IEEE error http://support.microsoft.com/kb/78113
best wishes
--
Bernard V Liengme
www.stfx.ca/people/bliengme
remove caps from email


"Randy Sp" <Randy wrote in message
...
I have a spreadsheet that has 29 numbers with up to 2 decimal place
precision
(44.29, 36.89, etc). I am taking the 1st number and subtracting the other
28
numbers from it
(=A1-B1-B2-B3-B4-B5-B6-B7-B8-B9-B10-B11-B12-B13-B14-C1-C2-C3-C4-C5-C6-C7-C8-C9-C10-C11-C12-C13-C14).
When I do this I get a result of 775,817.3700000010. My question is: where
is
that 1 coming from in the 9th decimal place? I am displaying all of my 29
input numbers to 10 decimal places, and they are all 0's past the 2nd
decimal
place. They are all entered as raw values (none are the result of a
formula).

One other point: if I take out ANY of the 28 cells that are being
subtracted, the problem goes away - the 1 in the 9th decimal place becomes
a
0.

I can send you the sample spreadsheet if that would help.





All times are GMT +1. The time now is 03:10 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com