Home |
Search |
Today's Posts |
#1
Posted to microsoft.public.excel.misc
|
|||
|
|||
Data
Hi,
When I program in C++ a variabl declared as a double x = 2.0 comes out as 2.0000001 or 1.99999999 but in excel it is 2.0 exactly why is that? Thanks |
#2
Posted to microsoft.public.excel.misc
|
|||
|
|||
Data
You need to ask the question (in an appropriate place, which wouldn't be
this newsgroup) as to why your C++ program got it wrong, not why Excel got it right (assuming that your number really was 2.0). There are problems with storing some numbers exactly if using fixed point decimal or fixed point binary, but 2.0 can easily be stored in either of those formats. [As examples, 1/3 can't be expressed exactly in fixed point decimal, and 1/10 can't in fixed point binary.] Also, if you display a number with a limited number of decimal places, then it may of course look like a different number from what it actually is, so if a calculation comes to 2.0000001 or 1.99999999 and you ask Excel to display to 1 decimal place it will look like 2.0. You'll also need to remember that precision in Excel is limited to 15 significant figures. -- David Biddulph "Jeff" wrote in message ... Hi, When I program in C++ a variabl declared as a double x = 2.0 comes out as 2.0000001 or 1.99999999 but in excel it is 2.0 exactly why is that? Thanks |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
MULTIPLE DATA - How to insert new data into existing data.... | Excel Discussion (Misc queries) |