| Home |
| Search |
| Today's Posts |
|
#1
Posted to microsoft.public.excel.worksheet.functions
|
|||
|
|||
What is the roundoff error in the subtraction function?
Why does 1.00000000000000000000-0.99950000000000000000=0.000499999999999945
instead of 0.0005? I have 50,000 records that are evenly spaced by 0.0005 and they appear that way in the column even when viewed with 20 places, however, after subtracting one number from the other the difference is not 0.0005. I tried using round off to 4 places and that doesnt resolve the problem. Any help is appreciated. |
| Thread Tools | Search this Thread |
| Display Modes | |
|
|
Similar Threads
|
||||
| Thread | Forum | |||
| Date & Time | New Users to Excel | |||
| Conversion | Excel Worksheet Functions | |||
| I cant use englisch function names in a swedich version of excel | Excel Discussion (Misc queries) | |||
| HOW CAN I GET OFFICE 2003 EXCEL BASIC TO NEST FUNCTIONS LIKE EXCE. | Excel Worksheet Functions | |||
| Find a Function to use accross different worksheets | Excel Worksheet Functions | |||
Threaded Mode
