Home |
Search |
Today's Posts |
#1
![]()
Posted to microsoft.public.excel.programming
|
|||
|
|||
![]()
Hello;
The following two sample codes produce two completely different results !! SUB TEST1() ' ... my code... SUM = 0.0 FOR I = 1 TO 20000 SUM = SUM + A(I)*B(I)/C(I) NEXT RESULT1 = SUM ' ...my code... END SUB SUB TEST2() ' ... my code... SUM = 0.0 FOR I = 1 TO 20000 SUM = SUM + A(I)*B(I)/C(I) /1000. NEXT RESULT2 = SUM*1000. ' ...my code END SUB Variables: Result1, Result2, SUM, A,B,C are DOUBLE data types Values of A,B,C vary from very small to very large, in no specific order For a typical set of analytical data, Result1 = 0.9986, while Result2 = 4.2432, a considerable difference !! The different results are most likely associated with accumilated round-off errors known to be associated with floating-point arithmetic precission. I would very much appreciate your suggestion(s) on how to minimize this serious round-off error. Can one use a higher precission than DOUBLE ... either in VBA or FORTRAN ?? Thank you kindly. |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
problem with round down | Excel Discussion (Misc queries) | |||
Round() problem | Excel Worksheet Functions | |||
Problem with the round function | Excel Programming | |||
How do I ROUND() round off decimals of a column dataset? | Excel Worksheet Functions | |||
Round problem with sorting | Excel Programming |