View Single Post
  #1   Report Post  
Posted to microsoft.public.excel.programming
Jerry W. Lewis Jerry W. Lewis is offline
external usenet poster
 
Posts: 837
Default Collection of Ranges and Arrays

Change the "As Single" declarations to "As Double"

Single precision only gives 7 decimal digit accuracy, where double precision
gives 15 decimal digit accuracy. Since both use binary representations,
figues beyond that point will usually be different than you expect, although
Excel will not directly display them. In this case you got to see them
because you stored a single precision value into a double precision variable
where Excel would display figures 8-15.

Jerry

"Terry Detrie" wrote:

OK, new question

In the example below, I apply a custom function RR to each CalcCurve array
element. The RR function has a rounding statement at the end RR = Round(RR
* 100, 2).

When I do a Watch on CalcCurve I see that each element is indeed rounded
to two decimal places. The first four numbers in array a
5.17 5.24 5.29 5.32

When I populate the DestRng with CalcCurve, I get random rounding errors.
In the worksheet those same four numbers a
5.170000076 5.239999771 5.289999962 5.320000172

Can someone enlighten me on what's going on and how to make Excel behave?

Terry