"Lynn McGuire" wrote:
double precision chptst
double precision divtwo
double precision top
double precision bottom
data top / 4195835.0D0 /
data bottom / 3145727.0D0 /
DIVTWO = top / bottom
CHPTST = (DIVTWO * bottom) - top In my user interface, the chptst
result is zero.
Under Excel VBA, the chptst result is 0.2851266E-09.

You are certainly looking at a floating-point anomaly; that is, sensitivity
to 64-bit rounding. But I do not believe it can be explained by the FPU
rounding mode alone.

And while I might be able to suggest a solution for this particular example,
in general, it is better to bullet-proof your arithmetic to correctly all
numerical examples and computations.

In general, you cannot expect 64-bit floating-point arithmetic to exactly
equal a mathematical solution. All computer arithmetic is limited by a
finite number of bits, whereas mathematics effectively relies on an infinite
representation of information.

In deference to the finite limitations of computer arithmetic in general,
and 64-bit floating-point in particular, it is prudent to explicitly round
any arithmetic result to the precision of accuracy that you require. That
is, any arithmetic result involving non-integer operands, division, or
integer operands or integer results greater than 2^53 in magnitude.

-----

To address your particular example....

We can simulate the two different results using VBA in a manner that sheds
some light on the problem. Consider the following two VBA procedures:

Sub testit1()
Const top As Double = 4195835#
Const bottom As Double = 3145727#
Dim chptst As Double
Dim divtwo As Double
divtwo = top / bottom
chptst = (divtwo * bottom) - top
MsgBox Format(chptst, "0.000E+00")
End Sub

Sub testit2()
Const top As Double = 4195835#
Const bottom As Double = 3145727#
Dim chptst As Double
Dim divtwo As Double
chptst = ((top / bottom) * bottom) - top
MsgBox Format(chptst, "0.000E+00")
End Sub

testit1 displays about 2.851E-10, whereas testit2 display 0.000E+00 --
exactly zero.

The difference is that in testit1, the 80-bit floating-point result of
top/bottom (the FPU of Intel CPUs use 80-bit floating-point internally) is
rounded to a 64-bit floating-point result stored into divtwo. Then VBA uses
the 64-bit divtwo in the computation of chptst.

But in testit2, VBA does all of the computation with 80-bit precision,
rounding to 64 bits only when storing the result into chptst.

I do not know anything about Microsoft C++ or how C++ DLLs might work when
called from VBA.

But based on your observations, I suspect that when the code is compiled and
linked in a C++ program, a better C++ compiler is used that optimizes the
computation of chptst to use the 80-bit result of top/bottom despite the
fact that you stored it into the 64-bit divtwo. However, when the DLL is
compiled and linked into VBA, obviously C++ is using the 64-bit divtwo, just
as testit1 does.

(Although I refer to "a better compiler" as if there are two, the difference
might actually be a difference in the behavior of __the__ so-called
"back-end compiler"; that is, a phase of the C++ compiler.)

Honestly, that does not make all that much sense to me based on my
experience with (Unix) compilers. But that conclusion seems to be supported
by my experiments below.

The more reasonable assumption is that the 80-bit rounding to 64-bit is
handled differently when the C++ DLL is called from VBA.

However, I cannot duplicate the results of testit2 even when I modify
testit1 in either of the following manners:

1. divtwo = top / bottom + 2^-52 ' add 1 to the least-significant bit
2. divtwo = top / bottom - 2^-52 ' sub 1 from the least-significant bit

Those modifications do have the intended effect, which we can see when we
look at the binary representation:

1. Original divtwo is 3FF55754,1C7C6B43.
2. Adding 2^-52, divtwo is 3FF55754,1C7C6B44.
3. Subtracting 2^-52, divtwo is 3FF55754,1C7C6B42.

But the testit1 results in chptst a

1. With original divtwo, chptst is about 2.851E-10.
2. With divtwo + 2^-52, chptst is about 9.836E-10.
3. With divtwo - 2^-52, chptst is about -4.134E-10.

Since none is exactly zero, as we see in testit2, I conclude that altering
the rounding to 64-bit alone does determine the result in testit1, but the
additional precision of 80-bit representation does.

However, arguably, that is only conjecture.


"Lynn McGuire" wrote:
I have tried resetting the math coprocessor in my
DLL with the following code but it is not working:
unsigned old87Status = 0;
unsigned new87ControlWord = 0;
unsigned new87ControlMask = 0;
unsigned new87result = 0;
old87Status = _status87 ();
if (old87Status != 0)
new87result = _control87 (new87ControlWord, new87ControlMask);

I presume that the intended purpose of this code is to change the FPU
rounding mode.

My conclusion above should explain why that does not work.

Unfortunately, I do not know how to set and read the FPU control word in
VBA. So I can offer a dispositive explanation.