| Home |
| Search |
| Today's Posts |
|
#1
Posted to microsoft.public.excel.worksheet.functions
|
|||
|
|||
Excel adds significant digits, resulting in errors in calculations
I have data from an 8th grade science lab - before and after measurements of
mass. We are plotting a histogram of the differences in mass for a set of data. However, when I plot the histogram, the results do not match the data, which is significant to 2 decimal places. Excel says that the result when you subtract 21.45 from 21.43 is NOT -0.02, but is -0.199999999999996. This is CRAZY. I know that Excel carries 15 significant digits. However, is there anything I can do to have Excel perform a simple subtraction of two numbers?! When did -0.02 become -0.01999999999996?!! Any help is much appreciated. Using Excel 2003. |
| Thread Tools | Search this Thread |
| Display Modes | |
|
|
Similar Threads
|
||||
| Thread | Forum | |||
| Excel 2003 FAILS, but Excel 2000 SUCCEEDS ??? | Excel Discussion (Misc queries) | |||
| order of calculations in Excel | Excel Worksheet Functions | |||
| Calculations run slow in Excel 2002 SP-1 | Excel Discussion (Misc queries) | |||
| Are there any significant differences in Excel 2003 from Excel 20. | Excel Discussion (Misc queries) | |||
| Excel 2000 file when opened in Excel 2003 generates errors? | Excel Discussion (Misc queries) | |||
Threaded Mode
