decimal values create inequality
1100 binary is equivalent to 12 in decimal
1100.01 is 12.25
1100.011 is 12.375
1100.011001 is 12.390625
1100.0110011001 is 12.3994140625
Keep trying to add extra digits and see whether it ever becomes exactly
12.40.
The answer is that it doesn't.
It's rather like the fact that you can't represent 1/3 or 1/7 as a finite
decimal number.
If you want things to work out correctly, you'll either need to use the
ROUND function, or work in integers by calculating in cents instead of in
dollars.
--
David Biddulph
Sam wrote:
Thanks for your response David. I'm not sure what you mean by
"calculating the exact binary representation of .40". This formula
has produced a "balanced" flag every day for over 7 months and now
found the inequality. Actually, the numbers that are being summed
a -8.55, .16, 161.00, -150.00. The actual sum of these numbers is
2.61. When the user entered 2.61 in the check total field the formula
matching the two numbers showed an inequality because the sum
function produced a value of 2.61000000001 which was not apparent
because it was formatted as 2 decimal places.
I have seen this happen on rare occasions before. There seems to be no
apparent cause. It doesn't seem reasonable that I should have to use
the Round function every time I want to compare the sum of components
to their total.
"David Biddulph" wrote:
If your number of 25.00 is formed by adding 12.40 to 12.60, you can
find the source of the problem by calculating the *exact* binary
representation of
0.40, for example.
Come back to us when you've done it. :-)
--
David Biddulph
"Sam" wrote in message
...
I have an error-checking formula in a spreadsheet:
"If(Sum(A1:A30)<D1,"ERROR","Balanced"), where A1:A30 are values
that are entered from a source document, while D1 is the Total
listed on the same source document. The purpose is to ensure that
all component parts are correctly entered. All values are formatted
as #,##0.00.
There are random times whereby the sum of A1:A30 results in a
number such as
25.0000000001 which does not equal the Total of 25 that had been
entered into D1. Since the above formatting displays 25 for both
values, the only way
of combatting the issue is to use a Round function for the Sum.
How can this happen when there is no division involved and all
components are entered as dollars and cents?
Excel version 2003.
Your help is greatly appreciated,
Sam
|