Intuitively you can see why 5.3213 and 9.3213 would have differeent
binary approximations by thinking about 4-digit decimal approximations
to 1/3.
0+1/3 ~ 0.3333
1+1/3 ~ 1.333
9+1/3 ~ 9.333
10+1/3 ~ 10.33
You lose more precision at each power of 10. Similarly the binary
approximation to a fraction will lose more precision at each power of 2
(1,2,4,8,...).

All nonterminating decimal fractions are nonterminating binary
fractions. Most terminating decimal fractions are still nonterminating
binary fractions, and hence can only be approximated. The only
terminating decimal fractions that are terminating binary fractions are
0.5, 0.25, 0.75, 0.125, 0.375, 0.625, 0.875, 0.0625, ...

Jerry

nsc wrote:

Thanks for the replies concerning the IEE 754 standard. A quick look and
it sort of makes sense to me. Where it doesn't make sense is with
another set of numbers. 5.3213 minus 5.1173 (note, the decimal numbers
are the same as my initial post) works fine in the same circumstances.
I would think that if 9.3213 and 9.1173 have binary approximations as
Mr. Lewis posted, wouldn't the other pair of numbers have the same
approximation? (this is also true if you replace the 5 with a 2, but if
you replace it with a 13 it again has the unexpected outcome)

I actually have a total of 40 pairs of numbers in my spreadsheet. They
all should subtract to .204, and they do. 26 of these pairs end up with
this binary approximation error.

Thanks again.