#1   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 26
Default Curious math

In Excel why does the sum of these three cells come to zero?
-1000000000000000000
1
1000000000000000000
Clearly a negative number take plus a positive number (regardless of the
size) should cancel each other out and leave the answer as one...
  #2   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 7,247
Default Curious math


"Illya Teideman" wrote in message
...
In Excel why does the sum of these three cells come to zero?
-1000000000000000000
1
1000000000000000000
Clearly a negative number take plus a positive number (regardless of the
size) should cancel each other out and leave the answer as one...


You are seeing a consequence of rounding when a number exceeds the
limitations of number precision. Excel, like nearly all software, uses what
are called IEEE Double Precision Floating Point numbers. This type of data
storage is limited to 15 digits of precision, which is the number of digits
both to the left and right of the decimal places that can be accurately
stored. When that limitation is reached, rounding occurs.

Your numbers are +/- 1 * 10^18, or 18 digits. Excel can't store that without
rounding. When you add

1000000000000000000 + 1,

the "+1" is rounded away because to store that level of precision, Excel
would need 18 digit precision, but Excel is limited to 15 digits. Change
the 1 to greater powers of 10 and you'll see various degrees of accuracy due
to rounding. For example, change it to 1000 then 10000 then 100000. You'll
see the accuracy get better until it is correct.

VBA has a data type called Decimal that has much higher precision. You will
get the correct result with code like

Dim D1 As Variant
Dim D2 As Variant
Dim D3 As Variant
Dim D4 As Variant

D1 = CDec("1E18")
D2 = CDec("-1E18")
D3 = CDec(1)
D4 = CDec(D1 + D2 + D3)
Debug.Print D4

You get the correct result here because a Decimal type can store numbers
with greater precision.

For a more in depth discussion of rounding, see
www.cpearson.com/Excel/Rounding.htm


--
Cordially,
Chip Pearson
Microsoft MVP - Excel
Pearson Software Consulting
www.cpearson.com
(email on the web site)


  #3   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 26
Default Curious math

Excellent chip you are a scholar a gentleman

"Chip Pearson" wrote:


"Illya Teideman" wrote in message
...
In Excel why does the sum of these three cells come to zero?
-1000000000000000000
1
1000000000000000000
Clearly a negative number take plus a positive number (regardless of the
size) should cancel each other out and leave the answer as one...


You are seeing a consequence of rounding when a number exceeds the
limitations of number precision. Excel, like nearly all software, uses what
are called IEEE Double Precision Floating Point numbers. This type of data
storage is limited to 15 digits of precision, which is the number of digits
both to the left and right of the decimal places that can be accurately
stored. When that limitation is reached, rounding occurs.

Your numbers are +/- 1 * 10^18, or 18 digits. Excel can't store that without
rounding. When you add

1000000000000000000 + 1,

the "+1" is rounded away because to store that level of precision, Excel
would need 18 digit precision, but Excel is limited to 15 digits. Change
the 1 to greater powers of 10 and you'll see various degrees of accuracy due
to rounding. For example, change it to 1000 then 10000 then 100000. You'll
see the accuracy get better until it is correct.

VBA has a data type called Decimal that has much higher precision. You will
get the correct result with code like

Dim D1 As Variant
Dim D2 As Variant
Dim D3 As Variant
Dim D4 As Variant

D1 = CDec("1E18")
D2 = CDec("-1E18")
D3 = CDec(1)
D4 = CDec(D1 + D2 + D3)
Debug.Print D4

You get the correct result here because a Decimal type can store numbers
with greater precision.

For a more in depth discussion of rounding, see
www.cpearson.com/Excel/Rounding.htm


--
Cordially,
Chip Pearson
Microsoft MVP - Excel
Pearson Software Consulting
www.cpearson.com
(email on the web site)


Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Curious about formula robert morris Excel Discussion (Misc queries) 4 July 16th 07 09:18 PM
Curious symbol in certain cells PT New Users to Excel 1 October 27th 06 05:31 AM
Just curious famdamly Excel Discussion (Misc queries) 2 December 7th 05 01:33 AM
Just curious RedChequer Excel Discussion (Misc queries) 3 March 10th 05 11:15 PM
Curious about SUMPRODUCT Wazooli Excel Worksheet Functions 5 December 20th 04 08:21 PM


All times are GMT +1. The time now is 08:14 AM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"