View Single Post
  #11   Report Post  
Posted to microsoft.public.excel.misc
Greg Lovern Greg Lovern is offline
external usenet poster
 
Posts: 224
Default Improve Excel accuracy over 15 digits

On Aug 27, 7:02*am, Jim Thomlinson <James_Thomlin...@owfg-Re-Move-
This-.com wrote:
Now that every one else has waded in I will ask the question... Why would you
need more than 15 digits of accuracy? With that many digits I can get the
distance from here to the sun to less than a millimeter. Just curious.



Hi Jim,

Most of my customers don't use xlPrecison for numbers that represent
any kind of physical measurement. Instead, they are mostly using
numbers to represent abstract, non-physical measurements. Many of them
are in finance.

Others are doing pure mathematics -- mostly number theory, where as
far as I can tell, the number doesn't represent anything other than
itself and the patterns of digits within it.

Others are doing things so unique I'm not sure how to categorize them.
There may be a few delusional cranks among them, but most of them come
across as highly intelligent and educated; certainly not less so than
the chorus of dogmatists who insist there is no valid use for more
than 15 significant digits of precision.

In some cases it's not only about the precision but also about very
large and very small numbers, far outside Excel's numeric range.

Some of them just want exact results (no binary conversion errors)
without dealing with the issues of Precision As Displayed or Round.
Generally those customers are well aware of what the options are.

BTW, here's an example of how precision can be a matter of life and
death:

During the Gulf War, a binary conversion error led to the deaths of 28
American soldiers (and around 100 injured) on February 25th, 1991 when
an American Patriot missile failed to intercept an Iraqi SCUD missile
headed toward their Army barracks. The error was caused by storing
time in 1/10 second increments, with binary conversion errors on
converting 0.1 to binary. Though each conversion error was tiny, the
error accumulated enough to make the Patriot's navigation software
miss the SCUD, which then reached its target.


Also BTW, I'm curious -- how does 15 significant digits of precision
get you the distance from here to the sun to less than a millimeter?
Isn't the average distance approximately 149,600,000,000,000
millimeters, which would let you get to the *nearest* millimeter in 15
significant digits of precision (or within one millimeter, assuming
truncation rather than rounding), but not to a fraction of a
millimeter? Not that it really matters, of course.


Greg Lovern
http://precisioncalc.com