View Single Post
  #2   Report Post  
Posted to microsoft.public.excel.worksheet.functions
Bernard Liengme[_3_] Bernard Liengme[_3_] is offline
external usenet poster
 
Posts: 1,104
Default Cosine of 90 degrees

This is a common occurrence: expecting zero and getting very close to it. It
all has to do with IEEE 754 and the way Excel (and moth other computer apps)
convert decimal numbers (numbers using base 10) to binary numbers (numbers
using base 2).

Only a problem with real numbers (i.e. numbers with decimal places) not with
integers

How to avoid: round to 12 decimal places =ROUND(COS(RADIANS(90)),12)
You handheld calculator does this automatically, so you never see it with
that machine

For a full explanation:
Chip's clear explanation
http://www.cpearson.com/excel/rounding.htm

Floating-point arithmetic may give inaccurate results in Excel
http://support.microsoft.com/kb/78113/en-us

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/kb/42980

What Every Computer Scientist Should Know About Floating Point
http://docs.sun.com/source/806-3568/ncg_goldberg.html
http://www.cpearson.com/excel/rounding.htm

Visual Basic and Arithmetic Precision
http://support.microsoft.com/default...NoWebContent=1

Others:
http://support.microsoft.com/kb/214118

http://docs.sun.com/source/806-3568/ncg_goldberg.html

best wishes
--
Bernard V Liengme
Microsoft Excel MVP
http://people.stfx.ca/bliengme
remove caps from email


"mikelee101" <mikelee101athotmaildotcom wrote in message
...
Hello All,
I was just curious why I wasn't getting zero for the result of the cosine
of
a 90 degree angle. This formula

=COS(RADIANS(90))

returns

6.12574E-17

when I would have expected 0. Am I misusing a function in there
somewhere?
Any thoughts would be greatly appreciated.

Excel 07, XPPro SP3.

Thanks.
--
Mike Lee
McKinney,TX USA