Thread: Power and Sum
View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.worksheet.functions
Niek Otten Niek Otten is offline
external usenet poster
 
Posts: 3,440
Default Power and Sum

Excel's precision in decimals is 15 digits

See

http://support.microsoft.com/kb/78113

--
Kind regards,

Niek Otten
Microsoft MVP - Excel

"John Watt" wrote in message ...
|I have recently been doing some work on a particular type of prime in Excel
| when I ran into a problem with the worksheet POWER function. It became
| obvious that at least on my machine there appears to be a rounding problem
| with some of the calculations. I have a couple of machines and repeated it on
| a second machine with Office 2003, with the same result. (The first machine
| had an earlier version of Excel.) The simplest form of the calculation is
| demonstrated by performing the following in a cell '=Power(100001,3)' the
| result seems to suggest that 1 x 1 = 0, The last digit being zero and not
| one. I repeated the calculation with the more straightforward
| '=sum(100001*100001*100001)' with the same disconcerting result. (You can
| check these calculations with the desktop calculator.) Do I need a fix or is
| there some explanation for these results. I have found similar errors with
| more demanding calculations and the work I am doing is grinding to a halt. I
| am in the curious situation that most of my work I am now doing in Excel is
| done with me doing the calculations manually to ensure the result is correct!
| Yet in the documentation it is suggested that Excel can perform calculations
| of the order of 1 * 10 to the power 307. (Note that all the calculations
| are performed with and displayed as numbers, I do not represent anything as
| floating point because it is the numbers and their properties that I am
| interested in.) Can somebody help!