LinkBack Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 11
Default Number precision problem

Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Number precision Kenneth the polynomial man Excel Discussion (Misc queries) 2 October 4th 06 07:37 AM
Number precision Jake Excel Discussion (Misc queries) 2 May 5th 06 11:24 PM
Precision displayed does not match precision in cell James Wilkerson Excel Discussion (Misc queries) 10 June 15th 05 02:40 PM
Precision problem with automated formula fill-in Erny Meyer Excel Worksheet Functions 1 February 4th 05 01:22 PM
32,767 Significant Digits of Precision, with Number Formatting Greg Lovern Excel Programming 1 July 9th 03 12:05 PM


All times are GMT +1. The time now is 10:58 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©2004-2025 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"