View Single Post
  #15   Report Post  
Posted to microsoft.public.excel.misc
Dana DeLouis Dana DeLouis is offline
external usenet poster
 
Posts: 947
Default Cosine Function Returns Wrong Answer!

=(PI()-884279719003555*2^-48)
which is the closest double precision approximation to the value of Pi,
even
though Excel will natively display no more than 15 digits of it.


Hi. This is always an interesting subject. Just gee wiz is that this also
returns 0.0...
=PI()-80143857/25510582

--
Dana DeLouis


"Jerry W. Lewis" wrote in message
...
Remember that what Excel uses and what it displays may be two different
things. The value Excel returns with PI() is 3.1415926535897931, as
verified
by
=(PI()-884279719003555*2^-48)
which is the closest double precision approximation to the value of Pi,
even
though Excel will natively display no more than 15 digits of it.

The exact value for cos(884279719003555*2^-49) is 6.1232339957367660E-17
(to
17 figures), which gets back to your question of what is going on under
the
hood? I would presume that cosine is accurately implemented in the math
coprocessor, and that both Excel and your math program use it ...

FYI, it takes at least 21 decimal digits to distinguish the arccos of
these
two cos values.

Jerry

"Dana DeLouis" wrote:

Rounding errors can be annoying.


Hi. If it makes you feel better, even math programs don't return 0.' for
inexact input.

Excel:
=COS(PI()/2)
6.1257422745431E-17

Math Program:
Cos[Pi/2.]
6.123233995736766*^-17

Note the slightly more digits in the answer also then Excel's limited 15
digits.

This is a difference of ~~
2.50827..*^-20

I've always be curious as to what's going on under the hood.
Especially since the value of Pi in Excel has been reduced to 15 digits.
Instead of:
3.1415926535897932
Excel uses:
3.14159265358979
--
Dana DeLouis