Home |
Search |
Today's Posts |
|
#1
![]()
Posted to microsoft.public.excel.programming
|
|||
|
|||
![]()
Change the "As Single" declarations to "As Double"
Single precision only gives 7 decimal digit accuracy, where double precision gives 15 decimal digit accuracy. Since both use binary representations, figues beyond that point will usually be different than you expect, although Excel will not directly display them. In this case you got to see them because you stored a single precision value into a double precision variable where Excel would display figures 8-15. Jerry "Terry Detrie" wrote: OK, new question In the example below, I apply a custom function RR to each CalcCurve array element. The RR function has a rounding statement at the end RR = Round(RR * 100, 2). When I do a Watch on CalcCurve I see that each element is indeed rounded to two decimal places. The first four numbers in array a 5.17 5.24 5.29 5.32 When I populate the DestRng with CalcCurve, I get random rounding errors. In the worksheet those same four numbers a 5.170000076 5.239999771 5.289999962 5.320000172 Can someone enlighten me on what's going on and how to make Excel behave? Terry |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Create a collection of ranges | Excel Programming | |||
Working with ranges in arrays... or an introduction to arrays | Excel Programming | |||
Alternitive to Collection of Ranges | Excel Programming | |||
Collection vs. Array of Arrays, nszim | Excel Programming | |||
update collection of arrays | Excel Programming |