Home |
Search |
Today's Posts |
#1
![]()
Posted to microsoft.public.excel,microsoft.public.excel.programming
|
|||
|
|||
![]()
I have a large Win32 DLL (10 MB) that is called from
my user interface (written in C++) or from VBA in MS Excel. In my user interface, the DLL runs in its own space and calculates correctly. Under Excel VBA, my DLL is having problems with double precision accuracy. The following test passes in my user interface but fails under my bad pentium test: double precision chptst double precision divtwo double precision top double precision bottom data top / 4195835.0D0 / data bottom / 3145727.0D0 / DIVTWO = top / bottom CHPTST = (DIVTWO * bottom) - top In my user interface, the chptst result is zero. Under Excel VBA, the chptst result is 0.2851266E-09. I have tried resetting the math coprocessor in my DLL with the following code but it is not working: unsigned old87Status = 0; unsigned new87ControlWord = 0; unsigned new87ControlMask = 0; unsigned new87result = 0; old87Status = _status87 (); if (old87Status != 0) new87result = _control87 (new87ControlWord, new87ControlMask); I have verified this behavior in both Excel 2003 and 2010. Does anyone have any ideas here ? Sincerely, Lynn McGuire |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Excel is messing with the math coprocessor | Excel Programming | |||
VSTO updating excel dlls | Excel Programming | |||
How are excel function names in dlls/xlas resolved ? | Excel Programming | |||
Referencing .Net DLLs | Excel Programming | |||
DLLs | Excel Programming |