View Single Post
  #4   Report Post  
Posted to microsoft.public.excel.worksheet.functions
MrShorty
 
Posts: n/a
Default drawing rect function graph in excel


Nothing wrong (other than that series of Y values will plot a line
essentially down the x axis rather than a square wave). Basically,
you've discovered that a computer can't represent an infinitely long
number (like PI) within the finite confines of its memory. In short,
the function PI() doesn't return exactly PI, but rather a 15 digit
approximation to PI. Then you put this approximation to PI into the
SIN function and you get an approximate value for the SIN(PI). You've
got to admit, 10^-16 is awful close to 0.

This "round-off error" gets discussed quite a bit in these forums. See
http://www.cpearson.com/excel/rounding.htm for a discussion as it
applies to Excel, the Microsoft knowledge base has a page or two
dedicated to round-off error and the IEEE standard. Since this problem
isn't unique to Excel, but applies to all computers in general, there
are probably a lot of places on the net and in numerical method or
programming texts where this issue is discussed.


--
MrShorty
------------------------------------------------------------------------
MrShorty's Profile: http://www.excelforum.com/member.php...o&userid=22181
View this thread: http://www.excelforum.com/showthread...hreadid=487122