View Single Post
  #13   Report Post  
Posted to microsoft.public.excel.misc
James Silverton[_2_] James Silverton[_2_] is offline
external usenet poster
 
Posts: 115
Default ROUNDING RESULT OF CALCULATION UP OR DOWN

joeu2004 wrote on Thu, 24 Jan 2008 10:11:40 -0800 (PST):

j On Jan 24, 5:28 am, "James Silverton"
j wrote:
?? Jerry wrote on Wed, 23 Jan 2008 20:27:37 -0800:
JWL For instance, =ROUND(10^14+0.5,0)
JWL incorrectly returns the value of 100000000000000
??
?? This is all very interesting but ultimately what does it
?? matter if ROUND sometimes goes in the wromg direction at
?? the limits of precision? It is hard for me to imagine a
?? real case where so many figures are necessary but I am
?? ready to be instructed.

j It isn't so much that anyone needs this much precision, but
j that these flaws cause problems in formulas that involve
j large numbers. The explanation is more precise than the
j solution requires.

j I presented this anomaly when I was having a problem with
j INT(). Actually, the problem is with MOD(x,y), which is
j well-know to an error with large numbers. I wanted to use x
j - y*INT(x/y) work around the problem. When that was
j misbehaving, I did a binary search and discovered the 10^14
j boundary and speculated the relationship to the number of
j significant digits that Excel will display.

Again without wishing to start a fight, can you give me an
example of a real calculation where the rounding error makes a
difference to a conclusion? I probably wouldn't use Excel if
there were such a case since it's not that hard to program in
double precision or the like.

James Silverton
Potomac, Maryland

E-mail, with obvious alterations:
not.jim.silverton.at.verizon.not