ExcelBanter

ExcelBanter (https://www.excelbanter.com/)
-   Excel Discussion (Misc queries) (https://www.excelbanter.com/excel-discussion-misc-queries/)
-   -   general question about variables (https://www.excelbanter.com/excel-discussion-misc-queries/89649-general-question-about-variables.html)

integreat

general question about variables
 

I can't for the life of me get this line of code to work:

deltaX = .01

When I go to the immediate window I find deltaX = 0

I can't assign a decimal value to that variable. I remember it was much
easier in quick basic to do this What am I doing wrong?

BTW the variable deltaX has been dimensioned as LONG

The good news is that I almost have a custom function that will allow
intergration by the trapezoidal method.


--
integreat
------------------------------------------------------------------------
integreat's Profile: http://www.excelforum.com/member.php...o&userid=34282
View this thread: http://www.excelforum.com/showthread...hreadid=543908


JMB

general question about variables
 
Long is an integer data type

Maybe Currency, Single, or Double would work better?


Single
(single-precision floating-point) 4 bytes -3.402823E38 to -1.401298E-45 for
negative values; 1.401298E-45 to 3.402823E38 for positive values

Double
(double-precision floating-point) 8 bytes -1.79769313486231E308 to
-4.94065645841247E-324 for negative values; 4.94065645841247E-324 to
1.79769313486232E308 for positive values

Currency
(scaled integer) 8 bytes -922,337,203,685,477.5808 to
922,337,203,685,477.5807

"integreat" wrote:


I can't for the life of me get this line of code to work:

deltaX = .01

When I go to the immediate window I find deltaX = 0

I can't assign a decimal value to that variable. I remember it was much
easier in quick basic to do this What am I doing wrong?

BTW the variable deltaX has been dimensioned as LONG

The good news is that I almost have a custom function that will allow
intergration by the trapezoidal method.


--
integreat
------------------------------------------------------------------------
integreat's Profile: http://www.excelforum.com/member.php...o&userid=34282
View this thread: http://www.excelforum.com/showthread...hreadid=543908



[email protected]

general question about variables
 
"integreat" wrote:
I can't for the life of me get this line of code to work:
deltaX = .01
When I go to the immediate window I find deltaX = 0
[....] BTW the variable deltaX has been dimensioned as LONG


That is precisely your mistake. Type long is an integer type.
When you assign a floating-point value to an integer type, the
value is rounding (apparently using banker's rounding! :-<) to
integer value. Either declare deltaX as a floating-point type,
e.g. double, or do not declare its type at all, which some
people consider "poor form".


All times are GMT +1. The time now is 08:53 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com