![]() |
Data
Hi,
When I program in C++ a variabl declared as a double x = 2.0 comes out as 2.0000001 or 1.99999999 but in excel it is 2.0 exactly why is that? Thanks |
Data
You need to ask the question (in an appropriate place, which wouldn't be
this newsgroup) as to why your C++ program got it wrong, not why Excel got it right (assuming that your number really was 2.0). There are problems with storing some numbers exactly if using fixed point decimal or fixed point binary, but 2.0 can easily be stored in either of those formats. [As examples, 1/3 can't be expressed exactly in fixed point decimal, and 1/10 can't in fixed point binary.] Also, if you display a number with a limited number of decimal places, then it may of course look like a different number from what it actually is, so if a calculation comes to 2.0000001 or 1.99999999 and you ask Excel to display to 1 decimal place it will look like 2.0. You'll also need to remember that precision in Excel is limited to 15 significant figures. -- David Biddulph "Jeff" wrote in message ... Hi, When I program in C++ a variabl declared as a double x = 2.0 comes out as 2.0000001 or 1.99999999 but in excel it is 2.0 exactly why is that? Thanks |
All times are GMT +1. The time now is 03:17 PM. |
Powered by vBulletin® Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
ExcelBanter.com