Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.worksheet.functions
external usenet poster
 
Posts: 1,104
Default Cosine of 90 degrees

This is a common occurrence: expecting zero and getting very close to it. It
all has to do with IEEE 754 and the way Excel (and moth other computer apps)
convert decimal numbers (numbers using base 10) to binary numbers (numbers
using base 2).

Only a problem with real numbers (i.e. numbers with decimal places) not with
integers

How to avoid: round to 12 decimal places =ROUND(COS(RADIANS(90)),12)
You handheld calculator does this automatically, so you never see it with
that machine

For a full explanation:
Chip's clear explanation
http://www.cpearson.com/excel/rounding.htm

Floating-point arithmetic may give inaccurate results in Excel
http://support.microsoft.com/kb/78113/en-us

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/kb/42980

What Every Computer Scientist Should Know About Floating Point
http://docs.sun.com/source/806-3568/ncg_goldberg.html
http://www.cpearson.com/excel/rounding.htm

Visual Basic and Arithmetic Precision
http://support.microsoft.com/default...NoWebContent=1

Others:
http://support.microsoft.com/kb/214118

http://docs.sun.com/source/806-3568/ncg_goldberg.html

best wishes
--
Bernard V Liengme
Microsoft Excel MVP
http://people.stfx.ca/bliengme
remove caps from email


"mikelee101" <mikelee101athotmaildotcom wrote in message
...
Hello All,
I was just curious why I wasn't getting zero for the result of the cosine
of
a 90 degree angle. This formula

=COS(RADIANS(90))

returns

6.12574E-17

when I would have expected 0. Am I misusing a function in there
somewhere?
Any thoughts would be greatly appreciated.

Excel 07, XPPro SP3.

Thanks.
--
Mike Lee
McKinney,TX USA



  #2   Report Post  
Posted to microsoft.public.excel.worksheet.functions
external usenet poster
 
Posts: 38
Default Cosine of 90 degrees

On Sep 14, 7:53�pm, "Bernard Liengme" wrote:
<snip
How to avoid: round to 12 decimal places =ROUND(COS(RADIANS(90)),12)
You handheld calculator does this automatically, so you never see it with
that machine

<snip

12 decimal places would give you an accuracy of about 6 inches / 150mm
in measuring the distance between the earth & the sun - good enough
for government work.

Alan Lloyd
Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
How do you convert decimal degrees to degrees minutes seconds David Excel Discussion (Misc queries) 2 September 21st 06 09:38 PM
how can i convert degrees/minutes/seconds to decimal degrees? Chrissy Excel Discussion (Misc queries) 2 April 28th 06 12:14 PM
calculate from degrees to degrees/minutes/seconds on excel? C. Hollinger Excel Worksheet Functions 1 March 14th 06 08:57 PM
convert decimal degrees to degrees minutes seconds Chris r Excel Discussion (Misc queries) 2 December 6th 05 10:40 PM
converting from digital degrees to degrees minutes seconds Patricio Boric Excel Worksheet Functions 1 November 28th 04 07:43 PM


All times are GMT +1. The time now is 12:34 AM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"