Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a series of
numbers by 100, then multiplying that quotient by 100, and getting illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and calculate the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the result
through VBA is incorrectly calculated at several hundred times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so does the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen or so
denominator values (175,000 being one), and I get the same strange results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the equivalent of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not make a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and how did
you work around it?
Is the number 100 a magical or evil number for such calculations given my
discovery of the numerator being < = 98 and = 175 resulted in success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis


  #2   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100 by a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a series of
numbers by 100, then multiplying that quotient by 100, and getting

illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and calculate

the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the

result
through VBA is incorrectly calculated at several hundred times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so does

the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen or so
denominator values (175,000 being one), and I get the same strange results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the equivalent

of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not make

a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and how

did
you work around it?
Is the number 100 a magical or evil number for such calculations given my
discovery of the numerator being < = 98 and = 175 resulted in success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis




  #3   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 5,441
Default Calculation bug or is it me

Tom,

OK, I'll bite: what results are you getting? Examples are always good....

HTH,
Bernie
MS Excel MVP

"Tom Urtis" wrote in message
...
Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100 by a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a series

of
numbers by 100, then multiplying that quotient by 100, and getting

illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and calculate

the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the

result
through VBA is incorrectly calculated at several hundred times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or equal

to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so does

the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen or

so
denominator values (175,000 being one), and I get the same strange

results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the equivalent

of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not

make
a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and how

did
you work around it?
Is the number 100 a magical or evil number for such calculations given

my
discovery of the numerator being < = 98 and = 175 resulted in success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis






  #4   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 10
Default Calculation bug or is it me

5.17428571428571E-02 is the scientific notation for .0517428571428571, right?

"Tom Urtis" wrote:

Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100 by a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a series of
numbers by 100, then multiplying that quotient by 100, and getting

illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and calculate

the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the

result
through VBA is incorrectly calculated at several hundred times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so does

the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen or so
denominator values (175,000 being one), and I get the same strange results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the equivalent

of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not make

a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and how

did
you work around it?
Is the number 100 a magical or evil number for such calculations given my
discovery of the numerator being < = 98 and = 175 resulted in success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis





  #5   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

Hi Bernie - - did you not see my original message? On the lengthy side with
examples.


"Bernie Deitrick" <deitbe @ consumer dot org wrote in message
...
Tom,

OK, I'll bite: what results are you getting? Examples are always

good....

HTH,
Bernie
MS Excel MVP

"Tom Urtis" wrote in message
...
Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100 by a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a

series
of
numbers by 100, then multiplying that quotient by 100, and getting

illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and

calculate
the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the

result
through VBA is incorrectly calculated at several hundred times more

than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and

paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or

equal
to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so

does
the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen

or
so
denominator values (175,000 being one), and I get the same strange

results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the

equivalent
of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not

make
a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and

how
did
you work around it?
Is the number 100 a magical or evil number for such calculations given

my
discovery of the numerator being < = 98 and = 175 resulted in

success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis










  #6   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

Yep...missed that obvious operator, time for more coffee.
Thanks.

"Mister T" wrote in message
...
5.17428571428571E-02 is the scientific notation for .0517428571428571,

right?

"Tom Urtis" wrote:

Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100 by a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a

series of
numbers by 100, then multiplying that quotient by 100, and getting

illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and

calculate
the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the

result
through VBA is incorrectly calculated at several hundred times more

than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and

paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or

equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so

does
the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen

or so
denominator values (175,000 being one), and I get the same strange

results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the

equivalent
of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not

make
a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and

how
did
you work around it?
Is the number 100 a magical or evil number for such calculations given

my
discovery of the numerator being < = 98 and = 175 resulted in

success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis







  #7   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 5,441
Default Calculation bug or is it me

Tom,

I'm sorry, my newsreader has been acting up....

You're just getting the scientific version of the same number. To control
how a number is displayed, simply use the Format function:

MsgBox Format(Range("A1").Value / Range("B1").Value * Range("C1").Value,
"0.0000000000000")

HTH,
Bernie
MS Excel MVP

"Tom Urtis" wrote in message
...
Hi Bernie - - did you not see my original message? On the lengthy side

with
examples.


"Bernie Deitrick" <deitbe @ consumer dot org wrote in message
...
Tom,

OK, I'll bite: what results are you getting? Examples are always

good....

HTH,
Bernie
MS Excel MVP

"Tom Urtis" wrote in message
...
Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100 by

a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a

series
of
numbers by 100, then multiplying that quotient by 100, and getting
illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and

calculate
the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and

the
result
through VBA is incorrectly calculated at several hundred times more

than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and

paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or

equal
to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so

does
the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen

or
so
denominator values (175,000 being one), and I get the same strange

results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the

equivalent
of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not

make
a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen and

how
did
you work around it?
Is the number 100 a magical or evil number for such calculations

given
my
discovery of the numerator being < = 98 and = 175 resulted in

success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis










  #8   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 1
Default Calculation bug or is it me

0.0571428571428571 = 5.17428571428571E-02
They are one and the same number, are they not?

Loz

-----Original Message-----
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to

divide a series of
numbers by 100, then multiplying that quotient by 100,

and getting illogical
results.

Example:

If you take a pencil and paper, or any decent calculator,

and calculate the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation

programmatically and the result
through VBA is incorrectly calculated at several hundred

times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range

("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been

copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window,

entering
? Range("A1").Value / Range("B1").Value * Range

("C1").Value

If I use a numerator less than or equal to 98, or greater

than or equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static

100, and so does the
other 100 multiplicand.
I am developing a project that requires this calculation

for a dozen or so
denominator values (175,000 being one), and I get the

same strange results
for all denominator values (240,000; 333,701; 350,000 as

examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent

the equivalent of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the

formula did not make a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone

has seen and how did
you work around it?
Is the number 100 a magical or evil number for such

calculations given my
discovery of the numerator being < = 98 and = 175

resulted in success?
What's going on here?

Thanks you guys...any words of wisdom are greatly

appreciated.

Tom Urtis


.

  #9   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

No problem Bernie, thank you, and thanks again to Mister T, my eyes were too
bleary this morning where I interpreted the minus operator as a plus.
Squinting at too much football on weekend television I guess.


"Bernie Deitrick" <deitbe @ consumer dot org wrote in message
...
Tom,

I'm sorry, my newsreader has been acting up....

You're just getting the scientific version of the same number. To control
how a number is displayed, simply use the Format function:

MsgBox Format(Range("A1").Value / Range("B1").Value * Range("C1").Value,
"0.0000000000000")

HTH,
Bernie
MS Excel MVP

"Tom Urtis" wrote in message
...
Hi Bernie - - did you not see my original message? On the lengthy side

with
examples.


"Bernie Deitrick" <deitbe @ consumer dot org wrote in message
...
Tom,

OK, I'll bite: what results are you getting? Examples are always

good....

HTH,
Bernie
MS Excel MVP

"Tom Urtis" wrote in message
...
Edit, I should have written the first sentence as:
Using XL2K3 XP, no problems with my computer, trying to divide 100

by
a
number, then multiplying that quotient by 100, and getting illogical
results.



"Tom Urtis" wrote in message
...
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a

series
of
numbers by 100, then multiplying that quotient by 100, and getting
illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and

calculate
the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and

the
result
through VBA is incorrectly calculated at several hundred times

more
than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and

paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or

equal
to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so

does
the
other 100 multiplicand.
I am developing a project that requires this calculation for a

dozen
or
so
denominator values (175,000 being one), and I get the same strange
results
for all denominator values (240,000; 333,701; 350,000 as

examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the

equivalent
of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did

not
make
a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone has seen

and
how
did
you work around it?
Is the number 100 a magical or evil number for such calculations

given
my
discovery of the numerator being < = 98 and = 175 resulted in

success?
What's going on here?

Thanks you guys...any words of wisdom are greatly appreciated.

Tom Urtis












  #10   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 5,651
Default Calculation bug or is it me

On Mon, 27 Sep 2004 10:00:14 -0700, "Tom Urtis" wrote:

OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a series of
numbers by 100, then multiplying that quotient by 100, and getting illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and calculate the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the result
through VBA is incorrectly calculated at several hundred times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so does the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen or so
denominator values (175,000 being one), and I get the same strange results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the equivalent of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not make a
difference.

Questions:
Am I missing something obvious,



I think you are.

0.0571428571428571

and

5.17428571428571E-02

are the SAME value.

The second is scientific notation.


--ron


  #11   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

Yes I was missing the obvious Ron, as Mister T and Bernie and Loz had also
pointed out...thanks for the response.

Tom


"Ron Rosenfeld" wrote in message
...
On Mon, 27 Sep 2004 10:00:14 -0700, "Tom Urtis"

wrote:

OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to divide a series

of
numbers by 100, then multiplying that quotient by 100, and getting

illogical
results.

Example:

If you take a pencil and paper, or any decent calculator, and calculate

the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation programmatically and the

result
through VBA is incorrectly calculated at several hundred times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window, entering
? Range("A1").Value / Range("B1").Value * Range("C1").Value

If I use a numerator less than or equal to 98, or greater than or equal

to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static 100, and so does

the
other 100 multiplicand.
I am developing a project that requires this calculation for a dozen or

so
denominator values (175,000 being one), and I get the same strange

results
for all denominator values (240,000; 333,701; 350,000 as examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent the equivalent

of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the formula did not make

a
difference.

Questions:
Am I missing something obvious,



I think you are.

0.0571428571428571

and

5.17428571428571E-02

are the SAME value.

The second is scientific notation.


--ron



  #12   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 12
Default Calculation bug or is it me

Yes they are one and the same, my oversight as others also pointed out on
this Monday where I should start the day over again. Thanks for
responding; have a nice day.

Tom


"Loz" wrote in message
...
0.0571428571428571 = 5.17428571428571E-02
They are one and the same number, are they not?

Loz

-----Original Message-----
OK, what am I missing here.

Using XL2K3 XP, no problems with my computer, trying to

divide a series of
numbers by 100, then multiplying that quotient by 100,

and getting illogical
results.

Example:

If you take a pencil and paper, or any decent calculator,

and calculate the
formula
100 divided by 175,000 times 100
the result should be 0.0571428571428571

If you enter 100 in A1, 175,000 in B1, and 100 in C1
and in D1 enter the formula =A1/B1*C1
Then D1 returns the same result of 0.0571428571428571
if D1 is formatted for 16 decimal places.

The problem is I need to do this calculation

programmatically and the result
through VBA is incorrectly calculated at several hundred

times more than
what the correct result should be.

Example:
Given the above values in A1:C1, this codeline
MsgBox Range("A1").Value / Range("B1").Value * Range

("C1").Value
displays "5.17428571428571E-02"

and a Message Box will display that same result given
MsgBox Range("D1").Value
whether the formula resides in cell D1, or D1 has been

copied and paste
special'd for values.

The same bad result is obtained in the Immediate Window,

entering
? Range("A1").Value / Range("B1").Value * Range

("C1").Value

If I use a numerator less than or equal to 98, or greater

than or equal to
175, I get expected results from the calculation.
The problem is, the 100 numerator needs to be a static

100, and so does the
other 100 multiplicand.
I am developing a project that requires this calculation

for a dozen or so
denominator values (175,000 being one), and I get the

same strange results
for all denominator values (240,000; 333,701; 350,000 as

examples).

I tried altering the formula such as
MsgBox Range("D1").Value * 0.01
and a whole lot of other creative variations to represent

the equivalent of
the base formula, but still get the same bad result.
Assigning variables to the numeric components of the

formula did not make a
difference.

Questions:
Am I missing something obvious, or is this a bug anyone

has seen and how did
you work around it?
Is the number 100 a magical or evil number for such

calculations given my
discovery of the numerator being < = 98 and = 175

resulted in success?
What's going on here?

Thanks you guys...any words of wisdom are greatly

appreciated.

Tom Urtis


.



Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Multi threaded calculation (multi CPU) - impact on calculation spe Pascal[_2_] Excel Discussion (Misc queries) 1 December 3rd 08 10:46 AM
sum calculation shar Excel Worksheet Functions 2 March 24th 05 05:16 PM
How do I use a rounded calculation result in another calculation? vnsrod2000 Excel Worksheet Functions 1 January 26th 05 10:11 PM
How do I use a rounded calculation result in another calculation? vnsrod2000 Excel Worksheet Functions 1 January 26th 05 09:36 PM
range.calculation with UDF not working when calculation is set to automatic Brian Murphy Excel Programming 5 October 14th 03 07:02 PM


All times are GMT +1. The time now is 10:05 PM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"