Home |
Search |
Today's Posts |
#24
![]()
Posted to microsoft.public.excel.misc
|
|||
|
|||
![]()
Jeff
You can't have a greater degree of accuracy in a result than was present in the arguments ..30 - .29 does = .01 to 2 significant digits, beyond that the result is undefined. .01000000000 is just one of many valid ones ..01 is the 2sd representation of any number from .005 to less than .015. If you have a need for such high precision then you probably need to use specialist tools rather than a general purpose product like Excel. cheers Simon Jeff in GA wrote: David, I appreciate your commitment to the notion that there could not possibly be a solution to the problem, and that we should all accept incorrect mathematical results. Meanwhile, even my fourth grade child knows that .30 - .29 = .0100000000000 But yes, David, let's adapt to the machines, and not make the machines adapt to us. Also: the customer is always wrong. Happy now? ...Jeff "David Biddulph" wrote: Perhaps you could tell us what the exact binary representation of 0.29 would be? -- David Biddulph "Jeff in GA" wrote in message ... This is NOT some calculation to the nth place or an atomic weight. This is: .29 - .28 Instead of returning the correct answer of 0.01, Microsoft Excel returns 0.00999999999999995 This floating point excuse for can not be blamed for poor product quality. Does Microsoft ever plan to repair its product, or should it only be used for calculating federal budget numbers? |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Convert floating point to Hours and Minutes ?? | Excel Discussion (Misc queries) | |||
Excel Throwing Circular Errors When No Errors Exist | Excel Worksheet Functions | |||
Converting 2-place decimal value to floating point decimal number with leading zero | Excel Discussion (Misc queries) | |||
setting a floating decimel point | New Users to Excel | |||
Unresolved Errors in IF Statements - Errors do not show in results | Excel Worksheet Functions |