LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #5   Report Post  
Posted to microsoft.public.excel.worksheet.functions
external usenet poster
 
Posts: 837
Default Logic Test ?

The math (and logic) is exactly right, given unavoidable approximations to
the original numbers.

Most terminating decimal fractions (including .2675 .2774 .2653 and .0011)
are nonterminating binary fractions that can only be approximated, much as
1/3 can only be approximated as a decimal number. You are seeing the binary
equivalent of
2/3 - 1/3 = 0.6667 - 0.3333 = 0.3334 < 0.3333

As documented in Excel's Help for "Excel specifications and limits" subtopic
"Calculation specifications", the approximations used may not be exact beyond
15 digits. You cancelled 3 of those 15 digits in the subtraction, revealing
residue of the original approximations.

You can use the D2D function at
http://groups.google.com/group/micro...06871cf92f8465
To see the actual decimal values of the approximations involved.

Jerry

"Zeno" wrote:

In Excel A1=1.2675-1.2664=.0011
B1=1.2664-1.2653=.0011
Ask A1=B1, result is False; why?



 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
MORE THAN ONE LOGIC TEST jan kay Excel Worksheet Functions 3 February 5th 06 01:41 AM
Need help on Logic test!!! jbf frylock Excel Discussion (Misc queries) 5 January 9th 06 09:43 PM
logic test toolman Excel Worksheet Functions 2 October 8th 05 04:28 AM
hpw do I logic test a cell then copy the row to diff. SS Debi Excel Worksheet Functions 4 October 5th 05 09:42 PM
Can wildcards be used in the Logic Test of an Excel "IF" function? Monte Excel Worksheet Functions 3 August 15th 05 01:25 PM


All times are GMT +1. The time now is 04:25 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"