![]() |
Another numeric formatting anomaly
myMod(12346789-0.0000004,1) returns a binary representation which is exactly
0.999999597668647,76611328125. (The comma demarcates the first 15 significant digits.) But when the cell (A1) is formatted as Number with 15 decimal places, Excel displays 0.999999597668647 (!). In contrast, myCstr(A1) returns 0.999999597668648 as expected. These examples are for Excel 2003 11.5612.5606, part of MS Office Sm Busn Ed 2003 on MS Win XP SP3, and VB 6.3 version 9969, VBA Retail 6.4.8896, Forms3 11.0.5601. The UDFs a Function mymod(a As Double, b As Double) As Double mymod = a - b * Int(a / b) End Function Function myCstr(arg As Double) As String myCstr = CStr(arg) End Function |
All times are GMT +1. The time now is 02:52 AM. |
Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com