ExcelBanter

ExcelBanter (https://www.excelbanter.com/)
-   Excel Discussion (Misc queries) (https://www.excelbanter.com/excel-discussion-misc-queries/)
-   -   Data (https://www.excelbanter.com/excel-discussion-misc-queries/174017-data.html)

Jeff

Data
 
Hi,

When I program in C++ a variabl declared as a double

x = 2.0

comes out as 2.0000001 or 1.99999999

but in excel it is 2.0 exactly why is that?

Thanks

David Biddulph[_2_]

Data
 
You need to ask the question (in an appropriate place, which wouldn't be
this newsgroup) as to why your C++ program got it wrong, not why Excel got
it right (assuming that your number really was 2.0).

There are problems with storing some numbers exactly if using fixed point
decimal or fixed point binary, but 2.0 can easily be stored in either of
those formats. [As examples, 1/3 can't be expressed exactly in fixed point
decimal, and 1/10 can't in fixed point binary.]
Also, if you display a number with a limited number of decimal places, then
it may of course look like a different number from what it actually is, so
if a calculation comes to 2.0000001 or 1.99999999 and you ask Excel to
display to 1 decimal place it will look like 2.0. You'll also need to
remember that precision in Excel is limited to 15 significant figures.
--
David Biddulph

"Jeff" wrote in message
...
Hi,

When I program in C++ a variabl declared as a double

x = 2.0

comes out as 2.0000001 or 1.99999999

but in excel it is 2.0 exactly why is that?

Thanks





All times are GMT +1. The time now is 03:17 PM.

Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com