View Single Post
  #5   Report Post  
Posted to microsoft.public.excel.misc
joeu2004[_2_] joeu2004[_2_] is offline
external usenet poster
 
Posts: 829
Default Formula to identify decimal versus whole numbers

wrote:
On Thursday, May 1, 2008 7:45:04 AM UTC-4, Dave Peterson wrote:
=if(a1=int(a1),"whole Number","not whole number")


Thank you!


I'm sure Dave will be happy to hear that 5(!) years later.

But FYI, in general, MOD(A1,1)=0 or INT(A1)-A1=0 is more reliable than
INT(A1)=A1.

If we type the number into A1, INT(A1)=A1 might suffice.

But if A1 is calculated (i.e. a formula), INT(A1)=A1 sometimes returns TRUE
incorrectly.

For example, enter the following formula into A1: =12.9999999999999 +
5E-14.

Excel displays 13.0000000000000 no matter how many decimal places we
specify. And INT(A1)=A1 returns TRUE.

But A1 is not really exactly 13.

Note that MATCH(INT(A1),A1,0) returns a #N/A error, indicating no match.

Moreover, MOD(A1,1)=0 and INT(A1)-A1=0 return FALSE, indicating that A1 is
not an exact integer.

The reason why INT(A1)=A1 mistakenly returns TRUE is complicated to explain.

First, INT(A1) does not always truncate the value, as we might expect. (In
contrast, compare with VBA Int(Range("A1")).) Instead, INT(A1) seems to
first round to 15 significant digits, then truncate.

Second, INT(A1)=A1 returns TRUE because Excel deems INT(A1) to be "close
enough" to A1, presumably an extension of the dubious heuristic poorly
described under the misleading title "Example When a Value Reaches Zero" at
http://support.microsoft.com/kb/78113.

The inconsistent implementation of that heuristic leads to seeming
contradiction like INT(A1)-A1 is exactly zero, but INT(A1)-A1-0 is not.
Presumably, that causes similar contractions like INT(A1)=A1 is TRUE, but
INT(A1)-A1=0 is FALSE.