Fill/Extend Series adding decimal points to some numbers
 

Hi,
I set up a worksheet for a Lookup Function that contains numbers increasing
by 0.1 from 100.0 to 50.0. I used the drag and fill to set up the numbers.
I noticed some incorrect entries in the results from the look up and found
that on certain numbers it added many decimal spots to the number for example
66.5 look like 66.5000000000001 when I click into the cell. I can not locate
a pattern for why this is occurring (i.e. does not occur in the 4-8 or 10-15
but does in the 9 and 16 series). Formatting is set to numerical with 1
decimal point.

Any help?

Fill/Extend Series adding decimal points to some numbers
 

This is a limitation of any computer that uses binary representation
of decimal numbers - some fractions cannot be represented with
complete accuracy. A binary fraction is made up of digits to represent
1/2, 1/4, 1/8, 1/16 etc, but some simple decimal fractions (like 1/10
or 0.1) cannot be represented completely in binary as they are
recurring numbers (rather like 1/3 or 2/3 in decimal).

Another thing to consider is that Excel can only represent up to 15
digits of precision, so a recurring fraction will be truncated to this
precision and when it is added repeatedly then errors will accumulate.

I'm sure Joe will come along soon to give a more in-depth explanation.

Hope this helps.

Pete

On Apr 26, 4:44*pm, awalker wrote:
Hi,
I set up a worksheet for a Lookup Function that contains numbers increasing
by 0.1 from 100.0 to 50.0. I used the drag and fill to set up the numbers.. * *
I noticed some incorrect entries in the results from the look up and found
that on certain numbers it added many decimal spots to the number for example
66.5 look like 66.5000000000001 when I click into the cell. *I can not locate
a pattern for why this is occurring (i.e. does not occur in the 4-8 or 10-15
but does in the 9 and 16 series). *Formatting is set to numerical with 1
decimal point. *

Any help?