Home |
Search |
Today's Posts |
#6
![]()
Posted to microsoft.public.excel.worksheet.functions
|
|||
|
|||
![]()
You can't make that analogy, since that's not how double precision
floating point values are stored. Your comparison is in decimal, not binary. Take a look at http://cpearson.com/excel/rounding.htm for a good basic explanation of the IEEE standard. In article , "MH" wrote: If that were the case then the same error would occur for the values of 1 and 0.7 MH "Sandy Mann" wrote in message ... "MH" wrote in message ... It looks like this is a bug It is not a bug, it is the result of the approximation that all computers using binary representation of fractions makes. =10209-10208.7 results in 0.299999999999272 although it displays as 0.3 |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
challenge! javascript function into excel function | Excel Worksheet Functions | |||
FUNCTION GETPIVOTDATA MICROSOFT EXCEL 2003 VS EXCEL 2004 FOR MAC | Excel Worksheet Functions | |||
FUNCTION GETPIVOTDATA EXCEL 2003 v EXCEL 2004 FOR MAC | Excel Worksheet Functions | |||
Excel Workday Function with another function | Excel Discussion (Misc queries) | |||
Can you nest a MID function within a IF function in Excel | Excel Worksheet Functions |