Home |
Search |
Today's Posts |
#1
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
Hi:
I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: --------------------------------------------------------- Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function --------------------------------------------------------- In a module, I have a calling procedure to populate cells with above numbers: --------------------------------------------------------- Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub --------------------------------------------------------- When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. |
#2
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
Use the round function with 4 or 5 decimals
-- Regards Juan Pablo González "Adrian T" wrote in message ... Hi: I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: --------------------------------------------------------- Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function --------------------------------------------------------- In a module, I have a calling procedure to populate cells with above numbers: --------------------------------------------------------- Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub --------------------------------------------------------- When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. |
#3
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
Juan:
It didn't work. Thank you, Adrian T -----Original Message----- Use the round function with 4 or 5 decimals -- Regards Juan Pablo González "Adrian T" wrote in message ... Hi: I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: -------------------------------------------------------- - Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function -------------------------------------------------------- - In a module, I have a calling procedure to populate cells with above numbers: -------------------------------------------------------- - Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub -------------------------------------------------------- - When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. . |
#4
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
How did you use it ? should be something like
ActiveCell.Value = Round(arrRefundEarnedPremium(i), 4) -- Regards Juan Pablo González "Adrian T" wrote in message ... Juan: It didn't work. Thank you, Adrian T -----Original Message----- Use the round function with 4 or 5 decimals -- Regards Juan Pablo González "Adrian T" wrote in message ... Hi: I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: -------------------------------------------------------- - Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function -------------------------------------------------------- - In a module, I have a calling procedure to populate cells with above numbers: -------------------------------------------------------- - Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub -------------------------------------------------------- - When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. . |
#5
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
Juan:
That's exactly how I did it. The cells still show unnecessary decimals. Thank you, Adrian T -----Original Message----- How did you use it ? should be something like ActiveCell.Value = Round(arrRefundEarnedPremium(i), 4) -- Regards Juan Pablo González "Adrian T" wrote in message ... Juan: It didn't work. Thank you, Adrian T -----Original Message----- Use the round function with 4 or 5 decimals -- Regards Juan Pablo González "Adrian T" wrote in message .. . Hi: I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: ------------------------------------------------------- - - Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function ------------------------------------------------------- - - In a module, I have a calling procedure to populate cells with above numbers: ------------------------------------------------------- - - Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub ------------------------------------------------------- - - When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. . . |
#6
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
Hi:
I thought you all might want to know the solution. The CDec function solves my problem. ActiveCell.Value = CDec(arrRefundEarnedPremium(i)) -----Original Message----- Hi: I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: --------------------------------------------------------- Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function --------------------------------------------------------- In a module, I have a calling procedure to populate cells with above numbers: --------------------------------------------------------- Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub --------------------------------------------------------- When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. . |
#7
Posted to microsoft.public.excel.programming
|
|||
|
|||
Number precision problem
My opinion is that the use of "Currency" vs. your "Single" data type may be
better given the financial data you are working with. Dim arrRefundEarnedPremium() As Currency Dim arrRefundEarnedPremium() As Single HTH Dana DeLouis "Adrian T" wrote in message ... Hi: I thought you all might want to know the solution. The CDec function solves my problem. ActiveCell.Value = CDec(arrRefundEarnedPremium(i)) -----Original Message----- Hi: I have encountered a precision error. Hopefully, someone could answer this. In a class (clsAssumptions), I have following: --------------------------------------------------------- Public Function RefundEarnedPremiumSchd(ByRef arrSchedule () As Single) Dim i As Integer 'Return array of earned premium schedule to the calling procedure arrSchedule(1) = 0.1 arrSchedule(2) = 0.2 arrSchedule(3) = 0.175 arrSchedule(4) = 0.135 arrSchedule(5) = 0.11 arrSchedule(6) = 0.09 arrSchedule(7) = 0.07 arrSchedule(8) = 0.05 arrSchedule(9) = 0.035 arrSchedule(10) = 0.035 For i = 11 To 30 arrSchedule(i) = 0 Next i End Function --------------------------------------------------------- In a module, I have a calling procedure to populate cells with above numbers: --------------------------------------------------------- Public Sub popRefundEarnedPremiumSchd() Dim Assumptions As clsAssumptions Dim arrRefundEarnedPremium() As Single Dim i As Integer Set Assumptions = New clsAssumptions ReDim arrRefundEarnedPremium(1 To 30) Assumptions.RefundEarnedPremiumSchd arrRefundEarnedPremium() Range("A1")(1, 1).Activate For i = 1 To 30 ActiveCell.Value = arrRefundEarnedPremium(i) ActiveCell.Next.Activate Next i Set Assumptions = Nothing End Sub --------------------------------------------------------- When I go to the sheet, the results a 0.100000001490116 (input is 0.1) 0.200000002980232 (input is 0.2) 0.174999997019768 (input is 0.175) 0.135000005364418 0.109999999403954 0.090000003576279 0.070000000298023 0.050000000745058 0.035000000149012 0.035000000149012 QUESTIONs: Where are extra decimals coming from? How can I eliminate extra decimals? I want 0.1, not 0.100000001490116. . |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Number precision | Excel Discussion (Misc queries) | |||
Number precision | Excel Discussion (Misc queries) | |||
Precision displayed does not match precision in cell | Excel Discussion (Misc queries) | |||
Precision problem with automated formula fill-in | Excel Worksheet Functions | |||
32,767 Significant Digits of Precision, with Number Formatting | Excel Programming |