Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 1
Default Format affects precision?

Apologies if this is well known, couldn't find any references to it.

It seems that setting a cell to currency format affects the precision
with which the cell's values are passed to VBA. For example:

Public Function TestDouble(dDouble As Double) As Double

TestDouble = dDouble

End Function

Then set cells:
A1: 0.123456789, formatted as currency, displayed to 15 decimal
points.
A2: =A1, displayed to 15 decimal points.

Precision as displayed shouldn't factor in, but is turned off just to
be sure.

TestDouble(A1) returns 0.1235
TestDouble(A2) returns 0.123456789
TestDouble(A1*1) returns 0.123456789 (!!)

Yikes. Is there any way to turn this "feature" off, so that formatting
a cell does not change the precision of calculations based on it? I
tried to find documentation for this behavior but didn't have any luck
there either.

Thanks...

  #2   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 698
Default Format affects precision?

Don't confuse a "format" with a Type Declaration.

When you format a number as Currency....the result is the original value,
displayed with a dollar sign ($) and only 2 decimal places displayed. The
rest of the original value still exists....it's just not displayed.

When you declare a variable as Currency (Dim x as Currency)...VBA rounds the
original value to 4 decimal places THEN stores the resulting value.

Does that help?
***********
Regards,
Ron

XL2003, WinXP


"Caveman" wrote:

Apologies if this is well known, couldn't find any references to it.

It seems that setting a cell to currency format affects the precision
with which the cell's values are passed to VBA. For example:

Public Function TestDouble(dDouble As Double) As Double

TestDouble = dDouble

End Function

Then set cells:
A1: 0.123456789, formatted as currency, displayed to 15 decimal
points.
A2: =A1, displayed to 15 decimal points.

Precision as displayed shouldn't factor in, but is turned off just to
be sure.

TestDouble(A1) returns 0.1235
TestDouble(A2) returns 0.123456789
TestDouble(A1*1) returns 0.123456789 (!!)

Yikes. Is there any way to turn this "feature" off, so that formatting
a cell does not change the precision of calculations based on it? I
tried to find documentation for this behavior but didn't have any luck
there either.

Thanks...


  #3   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 968
Default Format affects precision?

This happens because by default Excel/VBA converts an excel cell formatted
as currency to a VBA currency variable (and an excel cell formatted as date
to a VBA Date variable).

The confusing thing is the implicit conversions that are being done 'under
the covers' where you dont see them:
excel cell formatted as currency --VBA currency variable--VBA double
variable

You can prevent this by using the .Value2 property of a range object:

Public Function TestVar(dDouble As Range) As Double
' use an explicit range object

TestVar = dDouble.Value2
End Function

Public Function TestVar(dDouble As Variant) As Double
' use a variant that contains a range

TestVar = dDouble.Value2
End Function

Charles
______________________
Decision Models
FastExcel 2.3 now available
Name Manager 4.0 now available
www.DecisionModels.com


"Caveman" wrote in message
ps.com...
Apologies if this is well known, couldn't find any references to it.

It seems that setting a cell to currency format affects the precision
with which the cell's values are passed to VBA. For example:

Public Function TestDouble(dDouble As Double) As Double

TestDouble = dDouble

End Function

Then set cells:
A1: 0.123456789, formatted as currency, displayed to 15 decimal
points.
A2: =A1, displayed to 15 decimal points.

Precision as displayed shouldn't factor in, but is turned off just to
be sure.

TestDouble(A1) returns 0.1235
TestDouble(A2) returns 0.123456789
TestDouble(A1*1) returns 0.123456789 (!!)

Yikes. Is there any way to turn this "feature" off, so that formatting
a cell does not change the precision of calculations based on it? I
tried to find documentation for this behavior but didn't have any luck
there either.

Thanks...



Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Can I format one cell so that it affects another cell? David Excel Worksheet Functions 4 July 17th 09 02:52 AM
Sort affects formula Its-Just-Me New Users to Excel 9 November 3rd 08 08:31 PM
One workbook affects another MMcD Excel Discussion (Misc queries) 0 August 15th 07 10:04 PM
VBA compiled - affects conditional compile? [email protected] Excel Programming 1 January 4th 07 04:12 PM
Precision displayed does not match precision in cell James Wilkerson Excel Discussion (Misc queries) 10 June 15th 05 02:40 PM


All times are GMT +1. The time now is 11:28 PM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"