Home |
Search |
Today's Posts |
#4
![]()
Posted to microsoft.public.excel.programming
|
|||
|
|||
![]()
Thanks both for replying to this thread - have fixed the problem in a
'better' way - I did a quick search on the web and issues with FP maths is more fundamental than I previously understood; rounding to 15dp IS a viable solution. Thanks again. "Rick Rothstein (MVP - VB)" wrote: Am having problems with the use of trig functions within in VBA. sin(worksheetfunction.pi/2) = 1 //this is ok cos(worksheetfunction.pi/2) = 6.12303176911189E-17 //this is causing me a real headache... it should and needs to equal 0. For reasons I wont go into I cannot fudge with the round() function. I have tried using decimal type to define PI, but have not succeeded (not sure that I am coding correctly, but VB 'truncates' anything over 15 sig. figures in the programming environment, so I can't increase accuracy by simple const definition. I don't think there is anything you can do about this (except to use some kind of rounding operation)... these slight and minor discrepencies with floating point numbers is a well-known problem... it is inherent in the system used to store them. If this is new ground for you, perhaps these links will help... INFO: Visual Basic and Arithmetic Precision http://support.microsoft.com/default...NoWebContent=1 (Complete) Tutorial to Understand IEEE Floating-Point Errors http://support.microsoft.com/default...NoWebContent=1 Just out of curiosity, why are you opposed to using the Round function with, say, 15 for the number of decimal places? Rick |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Trig Function seems odd. | Excel Worksheet Functions | |||
Trig functions | Excel Worksheet Functions | |||
Trig on drag and drop event? | Excel Programming | |||
Trig in degrees not radians | Excel Programming | |||
Data Types and inaccuracies with Trig functions in VBA | Excel Programming |