LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #6   Report Post  
Posted to microsoft.public.excel.worksheet.functions
external usenet poster
 
Posts: 4,624
Default EXCEL IF FUNCTION

You can't make that analogy, since that's not how double precision
floating point values are stored. Your comparison is in decimal, not
binary. Take a look at

http://cpearson.com/excel/rounding.htm

for a good basic explanation of the IEEE standard.


In article ,
"MH" wrote:
If that were the case then the same error would occur for the values of 1
and 0.7

MH

"Sandy Mann" wrote in message
...
"MH" wrote in message
...
It looks like this is a bug


It is not a bug, it is the result of the approximation that all computers
using binary representation of fractions makes.

=10209-10208.7
results in 0.299999999999272 although it displays as 0.3

 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
challenge! javascript function into excel function Kamila Excel Worksheet Functions 2 February 19th 07 06:35 AM
FUNCTION GETPIVOTDATA MICROSOFT EXCEL 2003 VS EXCEL 2004 FOR MAC FRANCISCO PEREZ-LANDAETA Excel Worksheet Functions 0 July 6th 06 01:25 PM
FUNCTION GETPIVOTDATA EXCEL 2003 v EXCEL 2004 FOR MAC FRANCISCO PEREZ-LANDAETA Excel Worksheet Functions 0 July 6th 06 12:56 PM
Excel Workday Function with another function Monique Excel Discussion (Misc queries) 2 April 27th 06 01:11 PM
Can you nest a MID function within a IF function in Excel Dawn-Anne Excel Worksheet Functions 2 March 4th 05 01:37 PM


All times are GMT +1. The time now is 02:13 AM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"