![]() |
general question about variables
I can't for the life of me get this line of code to work: deltaX = .01 When I go to the immediate window I find deltaX = 0 I can't assign a decimal value to that variable. I remember it was much easier in quick basic to do this What am I doing wrong? BTW the variable deltaX has been dimensioned as LONG The good news is that I almost have a custom function that will allow intergration by the trapezoidal method. -- integreat ------------------------------------------------------------------------ integreat's Profile: http://www.excelforum.com/member.php...o&userid=34282 View this thread: http://www.excelforum.com/showthread...hreadid=543908 |
general question about variables
Long is an integer data type
Maybe Currency, Single, or Double would work better? Single (single-precision floating-point) 4 bytes -3.402823E38 to -1.401298E-45 for negative values; 1.401298E-45 to 3.402823E38 for positive values Double (double-precision floating-point) 8 bytes -1.79769313486231E308 to -4.94065645841247E-324 for negative values; 4.94065645841247E-324 to 1.79769313486232E308 for positive values Currency (scaled integer) 8 bytes -922,337,203,685,477.5808 to 922,337,203,685,477.5807 "integreat" wrote: I can't for the life of me get this line of code to work: deltaX = .01 When I go to the immediate window I find deltaX = 0 I can't assign a decimal value to that variable. I remember it was much easier in quick basic to do this What am I doing wrong? BTW the variable deltaX has been dimensioned as LONG The good news is that I almost have a custom function that will allow intergration by the trapezoidal method. -- integreat ------------------------------------------------------------------------ integreat's Profile: http://www.excelforum.com/member.php...o&userid=34282 View this thread: http://www.excelforum.com/showthread...hreadid=543908 |
general question about variables
"integreat" wrote:
I can't for the life of me get this line of code to work: deltaX = .01 When I go to the immediate window I find deltaX = 0 [....] BTW the variable deltaX has been dimensioned as LONG That is precisely your mistake. Type long is an integer type. When you assign a floating-point value to an integer type, the value is rounding (apparently using banker's rounding! :-<) to integer value. Either declare deltaX as a floating-point type, e.g. double, or do not declare its type at all, which some people consider "poor form". |
All times are GMT +1. The time now is 08:53 PM. |
Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com