ExcelBanter

ExcelBanter (https://www.excelbanter.com/)
-   Excel Programming (https://www.excelbanter.com/excel-programming/)
-   -   Number precision problem (https://www.excelbanter.com/excel-programming/302714-number-precision-problem.html)

Adrian T[_2_]
Number precision problem
 
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

Juan Pablo González
Number precision problem
 
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in message
...
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

Adrian T[_2_]
Number precision problem
 
Juan:


It didn't work.



Thank you,
Adrian T



-----Original Message-----
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in
message
...
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
--------------------------------------------------------
-
Public Function RefundEarnedPremiumSchd(ByRef
arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
--------------------------------------------------------
-

In a module, I have a calling procedure to populate
cells
with above numbers:

--------------------------------------------------------
-
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
--------------------------------------------------------
-

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How
can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



.

Juan Pablo González
Number precision problem
 
How did you use it ? should be something like

ActiveCell.Value = Round(arrRefundEarnedPremium(i), 4)


--
Regards

Juan Pablo González

"Adrian T" wrote in message
...
Juan:


It didn't work.



Thank you,
Adrian T



-----Original Message-----
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in
message
...
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
--------------------------------------------------------
-
Public Function RefundEarnedPremiumSchd(ByRef
arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
--------------------------------------------------------
-

In a module, I have a calling procedure to populate
cells
with above numbers:

--------------------------------------------------------
-
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
--------------------------------------------------------
-

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How
can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



.

Adrian T[_2_]
Number precision problem
 
Juan:

That's exactly how I did it. The cells still show
unnecessary decimals.


Thank you,
Adrian T



-----Original Message-----
How did you use it ? should be something like

ActiveCell.Value = Round(arrRefundEarnedPremium(i), 4)


--
Regards

Juan Pablo González

"Adrian T" wrote in
message
...
Juan:


It didn't work.



Thank you,
Adrian T



-----Original Message-----
Use the round function with 4 or 5 decimals

--
Regards

Juan Pablo González

"Adrian T" wrote in
message
.. .
Hi:


I have encountered a precision error. Hopefully,
someone
could answer this.


In a class (clsAssumptions), I have following:
-------------------------------------------------------
-
-
Public Function RefundEarnedPremiumSchd(ByRef
arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
-------------------------------------------------------
-
-

In a module, I have a calling procedure to populate
cells
with above numbers:

-------------------------------------------------------
-
-
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
-------------------------------------------------------
-
-

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How
can I
eliminate extra decimals? I want 0.1, not
0.100000001490116.



.



.

Adrian T[_2_]
Number precision problem
 
Hi:


I thought you all might want to know the solution. The
CDec function solves my problem.


ActiveCell.Value = CDec(arrRefundEarnedPremium(i))




-----Original Message-----
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can
I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

.

Dana DeLouis[_3_]
Number precision problem
 
My opinion is that the use of "Currency" vs. your "Single" data type may be
better given the financial data you are working with.

Dim arrRefundEarnedPremium() As Currency


Dim arrRefundEarnedPremium() As Single


HTH
Dana DeLouis



"Adrian T" wrote in message
...
Hi:


I thought you all might want to know the solution. The
CDec function solves my problem.


ActiveCell.Value = CDec(arrRefundEarnedPremium(i))




-----Original Message-----
Hi:


I have encountered a precision error. Hopefully, someone
could answer this.


In a class (clsAssumptions), I have following:
---------------------------------------------------------
Public Function RefundEarnedPremiumSchd(ByRef arrSchedule
() As Single)
Dim i As Integer

'Return array of earned premium schedule to the calling
procedure
arrSchedule(1) = 0.1
arrSchedule(2) = 0.2
arrSchedule(3) = 0.175
arrSchedule(4) = 0.135
arrSchedule(5) = 0.11
arrSchedule(6) = 0.09
arrSchedule(7) = 0.07
arrSchedule(8) = 0.05
arrSchedule(9) = 0.035
arrSchedule(10) = 0.035

For i = 11 To 30
arrSchedule(i) = 0
Next i

End Function
---------------------------------------------------------

In a module, I have a calling procedure to populate cells
with above numbers:

---------------------------------------------------------
Public Sub popRefundEarnedPremiumSchd()
Dim Assumptions As clsAssumptions
Dim arrRefundEarnedPremium() As Single
Dim i As Integer
Set Assumptions = New clsAssumptions

ReDim arrRefundEarnedPremium(1 To 30)
Assumptions.RefundEarnedPremiumSchd
arrRefundEarnedPremium()
Range("A1")(1, 1).Activate
For i = 1 To 30
ActiveCell.Value = arrRefundEarnedPremium(i)
ActiveCell.Next.Activate
Next i

Set Assumptions = Nothing

End Sub
---------------------------------------------------------

When I go to the sheet, the results a

0.100000001490116 (input is 0.1)
0.200000002980232 (input is 0.2)
0.174999997019768 (input is 0.175)
0.135000005364418
0.109999999403954
0.090000003576279
0.070000000298023
0.050000000745058
0.035000000149012
0.035000000149012


QUESTIONs: Where are extra decimals coming from? How can
I
eliminate extra decimals? I want 0.1, not
0.100000001490116.

.


All times are GMT +1. The time now is 04:55 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com