I am currious to find an explanation for this. In excel I created a
simple formula to calculate 12 to the 19th power written in full
numerical form. At 12 to the 17th excel makes an error (note the
zero's). Try it yourself. Anyone know why this happens?

12
144.00
1728.00
20736.00
248832.00
2985984.00
35831808.00
429981696.00
5159780352.00
61917364224.00
743008370688.00
8916100448256.00
106993205379072.00
1283918464548860.00 <-- here is where the error starts, the number
should end with 864.00
15407021574586400.00
184884258895036000.00
2218611106740440000.00
26623333280885200000.00
319479999370623000000.00

Try this with any really large number. The calculations drop off at the
17th digit. I've tried this on different computers all with the same
result.

XL can only return 15 digits of accuracy.
--
HTH,

RD

---------------------------------------------------------------------------
Please keep all correspondence within the NewsGroup, so all may benefit !
---------------------------------------------------------------------------
wrote in message
ups.com...
I am currious to find an explanation for this. In excel I created a
simple formula to calculate 12 to the 19th power written in full
numerical form. At 12 to the 17th excel makes an error (note the
zero's). Try it yourself. Anyone know why this happens?

12
144.00
1728.00
20736.00
248832.00
2985984.00
35831808.00
429981696.00
5159780352.00
61917364224.00
743008370688.00
8916100448256.00
106993205379072.00
1283918464548860.00 <-- here is where the error starts, the number
should end with 864.00
15407021574586400.00
184884258895036000.00
2218611106740440000.00
26623333280885200000.00
319479999370623000000.00

Try this with any really large number. The calculations drop off at the
17th digit. I've tried this on different computers all with the same
result.

On 22 Sep 2006 16:15:31 -0700, wrote:

I am currious to find an explanation for this. In excel I created a
simple formula to calculate 12 to the 19th power written in full
numerical form. At 12 to the 17th excel makes an error (note the
zero's). Try it yourself. Anyone know why this happens?

If you check HELP for Excel Specifications and Limits, you would discover:

Number precision 15 digits


--ron

Excel functions here exactly as it is supposed to.

Almost all computer software (including Excel) does binary math, and in
particular follows the IEEE standard for double precision representation of
floating point numbers. All 15 digit whole numbers can be exactly
represented in double precision. Only some 16 digit whole numbers can be
exactly represented in double precision; hence the MS decision to display
only 15 digits, where many IEEE DP packages will display up to 17 digits.

You can use the D2D function at
http://groups.google.com/group/micro...06871cf92f8465
to display more digits. For your particular calculation,
12^i = 2^(2*i) * 3^i
which means that Excel's calculation should be exact for all i where 3^i can
be exactly represented. Specifically, that would be for i<=33.
D2D(12^33) = 4.10186270246002225336426103593500672E35
which is exactly correct, whereas
D2D(12^34) = 4.922235242952026408889208063769182208E36
is necessarily only an approximation of
4.922235242952026408889208063769182208064E36

Jerry

" wrote:

I am currious to find an explanation for this. In excel I created a
simple formula to calculate 12 to the 19th power written in full
numerical form. At 12 to the 17th excel makes an error (note the
zero's). Try it yourself. Anyone know why this happens?

12
144.00
1728.00
20736.00
248832.00
2985984.00
35831808.00
429981696.00
5159780352.00
61917364224.00
743008370688.00
8916100448256.00
106993205379072.00
1283918464548860.00 <-- here is where the error starts, the number
should end with 864.00
15407021574586400.00
184884258895036000.00
2218611106740440000.00
26623333280885200000.00
319479999370623000000.00

Try this with any really large number. The calculations drop off at the
17th digit. I've tried this on different computers all with the same
result.