#1   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 1
Default calculation accuracy

Hi friends,

I have a simple question about calculation accuracy (excel 2002) that drives
me crazy: just enter in
A1 value 388
B1 value 372.58
C1 value 15.42

and then in a new cell the formula = A1 - B1 - C1

One would expect 0 as result, instead gets 1.59872E-14

Do you know why this occur? There is a way to force the result to 0 anyway?

Thanks a lot for any help or suggestion on this.

Ciao

Flavio


  #2   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 8,651
Default calculation accuracy

Try to work out the exact binary representation for 372.58 and that will
give you a clue as to the reason. [Hint: there isn't an exact binary
representation for most decimal numbers, just as there isn't an exact
decimal representation for a number like =10/3. You're OK with 0.5 or 0.25,
but with 0.1 or 0.01]

Try =ROUND(A1-B1-C1,2) or rounding to whatever resolution you need.

More info at http://www.mcgimpsey.com/excel/pennyoff.html
--
David Biddulph

"flavio.c" wrote in message
...
Hi friends,

I have a simple question about calculation accuracy (excel 2002) that
drives me crazy: just enter in
A1 value 388
B1 value 372.58
C1 value 15.42

and then in a new cell the formula = A1 - B1 - C1

One would expect 0 as result, instead gets 1.59872E-14

Do you know why this occur? There is a way to force the result to 0
anyway?

Thanks a lot for any help or suggestion on this.

Ciao

Flavio



  #3   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 3,440
Default calculation accuracy

Hi David,

<More info at http://www.mcgimpsey.com/excel/pennyoff.html

Actually, that is another subject; the difference between what is displayed and what is used in calculations.

The problem of the OP is described he

http://support.microsoft.com/kb/78113

--
Kind regards,

Niek Otten
Microsoft MVP - Excel

"David Biddulph" <groups [at] biddulph.org.uk wrote in message ...
| Try to work out the exact binary representation for 372.58 and that will
| give you a clue as to the reason. [Hint: there isn't an exact binary
| representation for most decimal numbers, just as there isn't an exact
| decimal representation for a number like =10/3. You're OK with 0.5 or 0.25,
| but with 0.1 or 0.01]
|
| Try =ROUND(A1-B1-C1,2) or rounding to whatever resolution you need.
|
| More info at http://www.mcgimpsey.com/excel/pennyoff.html
| --
| David Biddulph
|
| "flavio.c" wrote in message
| ...
| Hi friends,
|
| I have a simple question about calculation accuracy (excel 2002) that
| drives me crazy: just enter in
| A1 value 388
| B1 value 372.58
| C1 value 15.42
|
| and then in a new cell the formula = A1 - B1 - C1
|
| One would expect 0 as result, instead gets 1.59872E-14
|
| Do you know why this occur? There is a way to force the result to 0
| anyway?
|
| Thanks a lot for any help or suggestion on this.
|
| Ciao
|
| Flavio
|
|
|


  #4   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 1
Default calculation accuracy

Thank you Niek, it is clear now.
I think I'll use 'Set precision as displayed', it works fine.
Ciao

Flavio

"Niek Otten" ha scritto nel messaggio
...
Hi David,

<More info at http://www.mcgimpsey.com/excel/pennyoff.html

Actually, that is another subject; the difference between what is
displayed and what is used in calculations.

The problem of the OP is described he

http://support.microsoft.com/kb/78113

--
Kind regards,

Niek Otten
Microsoft MVP - Excel

"David Biddulph" <groups [at] biddulph.org.uk wrote in message
...
| Try to work out the exact binary representation for 372.58 and that will
| give you a clue as to the reason. [Hint: there isn't an exact binary
| representation for most decimal numbers, just as there isn't an exact
| decimal representation for a number like =10/3. You're OK with 0.5 or
0.25,
| but with 0.1 or 0.01]
|
| Try =ROUND(A1-B1-C1,2) or rounding to whatever resolution you need.
|
| More info at http://www.mcgimpsey.com/excel/pennyoff.html
| --
| David Biddulph
|
| "flavio.c" wrote in message
| ...
| Hi friends,
|
| I have a simple question about calculation accuracy (excel 2002) that
| drives me crazy: just enter in
| A1 value 388
| B1 value 372.58
| C1 value 15.42
|
| and then in a new cell the formula = A1 - B1 - C1
|
| One would expect 0 as result, instead gets 1.59872E-14
|
| Do you know why this occur? There is a way to force the result to 0
| anyway?
|
| Thanks a lot for any help or suggestion on this.
|
| Ciao
|
| Flavio
|
|
|




Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
How do I improve Accuracy of trendlines? Tom21 Charts and Charting in Excel 3 August 1st 06 09:17 PM
Forecasting Accuracy Gregc. Charts and Charting in Excel 0 May 3rd 06 02:01 AM
Data Accuracy Nicole Excel Discussion (Misc queries) 0 March 28th 06 04:56 PM
CHecking accuracy schlempyasst Excel Discussion (Misc queries) 0 October 13th 05 08:15 PM
Accuracy Studies Ginny Excel Discussion (Misc queries) 1 April 1st 05 03:42 AM


All times are GMT +1. The time now is 01:31 PM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"