Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 837
Default Collection of Ranges and Arrays

Change the "As Single" declarations to "As Double"

Single precision only gives 7 decimal digit accuracy, where double precision
gives 15 decimal digit accuracy. Since both use binary representations,
figues beyond that point will usually be different than you expect, although
Excel will not directly display them. In this case you got to see them
because you stored a single precision value into a double precision variable
where Excel would display figures 8-15.

Jerry

"Terry Detrie" wrote:

OK, new question

In the example below, I apply a custom function RR to each CalcCurve array
element. The RR function has a rounding statement at the end RR = Round(RR
* 100, 2).

When I do a Watch on CalcCurve I see that each element is indeed rounded
to two decimal places. The first four numbers in array a
5.17 5.24 5.29 5.32

When I populate the DestRng with CalcCurve, I get random rounding errors.
In the worksheet those same four numbers a
5.170000076 5.239999771 5.289999962 5.320000172

Can someone enlighten me on what's going on and how to make Excel behave?

Terry

Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Create a collection of ranges Norman Jones Excel Programming 0 January 3rd 07 07:58 PM
Working with ranges in arrays... or an introduction to arrays Glen Excel Programming 5 September 10th 06 08:32 AM
Alternitive to Collection of Ranges Abode Excel Programming 1 May 4th 06 04:45 PM
Collection vs. Array of Arrays, nszim Neal Zimm[_2_] Excel Programming 13 February 6th 06 02:36 PM
update collection of arrays RB Smissaert Excel Programming 7 December 26th 05 09:28 AM


All times are GMT +1. The time now is 12:33 AM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"