Sean wrote:
Actually I found this problem recently and don't know why this could happen.
I did simple minus calculation of these two values: (-0.365)-(-0.3575)
and excel shows -0.0075000000000000100000000.
It is not exactly -0.0075.
Do I need to change some settings or ....?

Thank you

----------------------------

No -- what you're seeing is how computers work. Basically Excel, and
most spreadsheets store their numbers and do math in Binary while you
punch in numbers and think in Decimal. There is not an exact conversion
between the two (for fractions) so errors occur down in the least
significant digit.

The number 0.1 in decimal for example is an irrational number infinitely
long in binary. A computer however only carries and calculates with a
finite number of bits so some rounding occurs.

Three ways around this exist in theory:

1) Use more digits of precision in your application. You'll still have
an error but it will be a lot more digits further down in the decimal.
I don't know that you can tell Excel to use double precision -- I think
you're stuck with 16 digits or so. With direct programming languages
you generally can select different numbers of bytes of precision to work in.

2) Use integer arithmetic only. Covert all numbers to integers and work
with those. If you do things like dividing though that's very hard to
manage and you end up with rounding errors anyhow.

3) Have the computer programmed to operate in decimal rather than
binary. This uses storage rather inefficiently and is very slow
compared to binary so it generally isn't used except in cases where it's
really important for some reason.

Bill