Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 11
Default Number precision problem

Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

  #2   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 226
Default Number precision problem

Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in message
...
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



  #3   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 11
Default Number precision problem

Juan:


It didn't work.



Thank you,
Adrian T



-----Original Message-----
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in

message
...
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
--------------------------------------------------------

-
Public Function RefundEarnedPremiumSchd(ByRef

arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
--------------------------------------------------------

-

In a module, I have a calling procedure to populate

cells
with above numbers:

--------------------------------------------------------

-
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
--------------------------------------------------------

-

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How

can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



.

  #4   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 226
Default Number precision problem

How did you use it ? should be something like

ActiveCell.Value = Round(arrRefundEarnedPremium(i), 4)


--
Regards

Juan Pablo González

"Adrian T" wrote in message
...
Juan:


It didn't work.



Thank you,
Adrian T



-----Original Message-----
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in

message
...
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
--------------------------------------------------------

-
Public Function RefundEarnedPremiumSchd(ByRef

arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
--------------------------------------------------------

-

In a module, I have a calling procedure to populate

cells
with above numbers:

--------------------------------------------------------

-
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
--------------------------------------------------------

-

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How

can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



.



  #5   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 11
Default Number precision problem

Juan:

That's exactly how I did it. The cells still show
unnecessary decimals.


Thank you,
Adrian T



-----Original Message-----
How did you use it ? should be something like

ActiveCell.Value = Round(arrRefundEarnedPremium(i), 4)


--
Regards

Juan Pablo González

"Adrian T" wrote in

message
...
Juan:


It didn't work.



Thank you,
Adrian T



-----Original Message-----
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in

message
.. .
Hi:


I have encountered a precision error. Hopefully,

someone
could answer this.


In a class (clsAssumptions), I have following:
-------------------------------------------------------

-
-
Public Function RefundEarnedPremiumSchd(ByRef

arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
-------------------------------------------------------

-
-

In a module, I have a calling procedure to populate

cells
with above numbers:

-------------------------------------------------------

-
-
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
-------------------------------------------------------

-
-

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How

can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



.



.



  #6   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 11
Default Number precision problem

Hi:


I thought you all might want to know the solution. The
CDec function solves my problem.


ActiveCell.Value = CDec(arrRefundEarnedPremium(i))




-----Original Message-----
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can

I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

.

  #7   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 690
Default Number precision problem

My opinion is that the use of "Currency" vs. your "Single" data type may be
better given the financial data you are working with.

Dim arrRefundEarnedPremium() As Currency


Dim arrRefundEarnedPremium() As Single



HTH
Dana DeLouis



"Adrian T" wrote in message
...
Hi:


I thought you all might want to know the solution. The
CDec function solves my problem.


ActiveCell.Value = CDec(arrRefundEarnedPremium(i))




-----Original Message-----
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can

I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

.



Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Number precision Kenneth the polynomial man Excel Discussion (Misc queries) 2 October 4th 06 07:37 AM
Number precision Jake Excel Discussion (Misc queries) 2 May 5th 06 11:24 PM
Precision displayed does not match precision in cell James Wilkerson Excel Discussion (Misc queries) 10 June 15th 05 02:40 PM
Precision problem with automated formula fill-in Erny Meyer Excel Worksheet Functions 1 February 4th 05 01:22 PM
32,767 Significant Digits of Precision, with Number Formatting Greg Lovern Excel Programming 1 July 9th 03 12:05 PM


All times are GMT +1. The time now is 04:51 AM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"