Does rounding work at all?
In VBA
Factor = 3
X = 1712.1465
SymArith = Fix(X * (10 ^ Factor) + 0.5 * Sgn(X)) / (10 ^ Factor)
assigns the value 1712.147 to SymArith. Therefore your x must be
<1712.1465. What do you get for 1712.1465-X? Assuming that X contains the
result of calculations that equals 1712.1465 to 15 figures, then you might be
happier with
SymArith = Fix(CStr(X) * (10 ^ Factor) + 0.5 * Sgn(X)) / (10 ^ Factor)
which will get rid of accumulated binary approximations from the previous
calculations that might cause discrepancies beyond the 15th figure. (Recall
that the vast majority of terminating decimal fractions are non-terminating
binary fractions that can only be approximated, much as 1/3 can only be
approximated in decimal).
Jerry
"Maury Markowitz" wrote:
Wow, this just gets weirder and weirder.
Ok, so I followed links from the page Tom noted, and found some source code
that MS posted for doing various types of roundings. First off they decided
to use different syntax than the Round function, so to round to the 3rd
decimal it's not 3, it's 1000. Genius!
Anyway I fix that and it still doesn't work. Ready for this one? Here's the
code...
SymArith = fix(X * (10 ^ Factor) + 0.5 * Sgn(X)) / (10 ^ Factor)
In this case X is 1712.1465, and I am attempting to get it to round the same
way SQL will, which will be 1712.147. Ok, so the inside of the Fix returns
1712147 -- so in other works if you Fix this you'll get exactly what I would
expect.
Except Fix returns 1712146. So the rounding still doesn't work. Can anyone
explain THIS?
I found that if I took the inside of the Fix "out" and cast it to a Long
then it works...
y = X * (10 ^ Factor) + 0.5 * Sgn(X)
SymArith = Fix(y) / (10 ^ Factor)
However the numbers in question are fairly big, and thus overflow the Long
data type.
Wow, this is really frustrating!
Maury
|