Precision in formulas?
 

I have a formula as follows:
C1=A1*B1 (assume A1=$482,933 and B1=0.9695)
My expected result is $468,203. However the actual result shown in C1 is
$468,181.
It appears as though the value used in B1 to derive the result is accurate
to a greater precision (i.e. 0.9695?????) than the 4 decimal places displayed
in cell B1.
Is there a way I can force the formula to use a specfic precision or some
other solution to my porblem?
TIA. Chris.

Precision in formulas?
 

C1=A1*ROUND(B1,4)

-OR-

<Tools<Options
Calculation Tab
Check "Precision as displayed"... This option will use the precision as
displayed in the cell to calculate on, instead of the actual value stored in
that cell.

Does that help?
--
Regards,
Dave


"Chris W" wrote:

I have a formula as follows:
C1=A1*B1 (assume A1=$482,933 and B1=0.9695)
My expected result is $468,203. However the actual result shown in C1 is
$468,181.
It appears as though the value used in B1 to derive the result is accurate
to a greater precision (i.e. 0.9695?????) than the 4 decimal places displayed
in cell B1.
Is there a way I can force the formula to use a specfic precision or some
other solution to my porblem?
TIA. Chris.

Precision in formulas?
 

Always difficult using binary for decimal calculations. Who remembers
BCD?

You will find your answer in a formula such as
=ROUND(A9*ROUND(B9,4),0)

but whether you use round , rounddown, int, floor, ceiling etc really
depends on your data

hope this helps


--
tony h
------------------------------------------------------------------------
tony h's Profile: http://www.excelforum.com/member.php...o&userid=21074
View this thread: http://www.excelforum.com/showthread...hreadid=500163

Precision in formulas?
 

Just what I was looking for. Thanks for your help!

"David Billigmeier" wrote:

C1=A1*ROUND(B1,4)

-OR-

<Tools<Options
Calculation Tab
Check "Precision as displayed"... This option will use the precision as
displayed in the cell to calculate on, instead of the actual value stored in
that cell.

Does that help?
--
Regards,
Dave


"Chris W" wrote:

I have a formula as follows:
C1=A1*B1 (assume A1=$482,933 and B1=0.9695)
My expected result is $468,203. However the actual result shown in C1 is
$468,181.
It appears as though the value used in B1 to derive the result is accurate
to a greater precision (i.e. 0.9695?????) than the 4 decimal places displayed
in cell B1.
Is there a way I can force the formula to use a specfic precision or some
other solution to my porblem?
TIA. Chris.

Precision in formulas?
 

Thanks for your help.

"tony h" wrote:


Always difficult using binary for decimal calculations. Who remembers
BCD?

You will find your answer in a formula such as
=ROUND(A9*ROUND(B9,4),0)

but whether you use round , rounddown, int, floor, ceiling etc really
depends on your data

hope this helps


--
tony h
------------------------------------------------------------------------
tony h's Profile: http://www.excelforum.com/member.php...o&userid=21074
View this thread: http://www.excelforum.com/showthread...hreadid=500163