![]() |
Precision problem with automated formula fill-in
I start with a value of -1 in cell A1 of a worksheet. In
cell A2 I enter the formula =A1+0.1 I repeat the formula downwards. At cell A11 (where I should read 0), I find an approximation -1.4E-16 Changing the formula to =(A1*10+1)/10 (which is basically the same) eliminates the problem. Can this type of problem be avoided through a setup option? |
You could use Tools/Options/Calculation Precision as displayed... Note
that that is a global setting. Or you could change your formula to =ROUND(A1+0.1,1) The problem is that 0.1 cannot be represented exactly in finite binary digits, much as 1/3 can't be represented exactly in finite decimal digits (e.g., 0.3333333...) See http://cpearson.com/excel/rounding.html In article , "Erny Meyer" wrote: I start with a value of -1 in cell A1 of a worksheet. In cell A2 I enter the formula =A1+0.1 I repeat the formula downwards. At cell A11 (where I should read 0), I find an approximation -1.4E-16 Changing the formula to =(A1*10+1)/10 (which is basically the same) eliminates the problem. Can this type of problem be avoided through a setup option? |
All times are GMT +1. The time now is 03:56 AM. |
Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com