Rounding errors can be annoying.

Hi. If it makes you feel better, even math programs don't return 0.' for
inexact input.

Excel:
=COS(PI()/2)
6.1257422745431E-17

Math Program:
Cos[Pi/2.]
6.123233995736766*^-17

Note the slightly more digits in the answer also then Excel's limited 15
digits.

This is a difference of ~~
2.50827..*^-20

I've always be curious as to what's going on under the hood.
Especially since the value of Pi in Excel has been reduced to 15 digits.
Instead of:
3.1415926535897932
Excel uses:
3.14159265358979
--
Dana DeLouis


"Gordon Arnaut" wrote in message
...
Harlan,

Thanks for the heads up about GAMMMALN.

Rounding errors can be annoying. On a density altitude calculator I worked
up, it comes up with a rounding error of about a foot for every couple of
thousand feet increase in altitude. More of an annoyance than an
impediment,
but still...

I spent a lot of time going through the formulas trying to get rid of it,
to
no avail.

Microsoft needs to do a better job with this product.


Regards,

Gordon.

"Harlan Grove" wrote:

"Rick Rothstein (MVP - VB)" wrote...
....
The cosine function returns the wrong answer for the cosine of 90
degrees.
The right answer is zero, of course, but Excel insists on returning a
very
small number, 6e-17.

Cosine works fine for all other angles -- both smaller and larger than
90
degrees.
....
Excel's trig functions don't work in Degrees, they work in Radians.
Multiply your Degrees by PI()/180 to convert them to Radians.

Testing is good. It often prevents seriously flawed responses.

The formula =COS(90) returns -0.44807361612917, NOT 6.xxE-17. However,
the
formula =COS(90*PI()/180) does return 6.1257422745431E-17.

As Sandy Mann already pointed out, this is due to rounding error. This is
one reason trig functions taking degrees (or grads) as arguments would
have
made more sense than trig functions taking radians since trig functions
taking degrees or grads could have special-cased particular integer
values
in order to return exact results.

This is common to all too many Excel functions. Even worse, GAMMALN
doesn't
return 0 when passed 1 or 2 as arguments. There's much less excuse for
that.