#1   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 921
Default Data

Hi,

When I program in C++ a variabl declared as a double

x = 2.0

comes out as 2.0000001 or 1.99999999

but in excel it is 2.0 exactly why is that?

Thanks
  #2   Report Post  
Posted to microsoft.public.excel.misc
external usenet poster
 
Posts: 8,651
Default Data

You need to ask the question (in an appropriate place, which wouldn't be
this newsgroup) as to why your C++ program got it wrong, not why Excel got
it right (assuming that your number really was 2.0).

There are problems with storing some numbers exactly if using fixed point
decimal or fixed point binary, but 2.0 can easily be stored in either of
those formats. [As examples, 1/3 can't be expressed exactly in fixed point
decimal, and 1/10 can't in fixed point binary.]
Also, if you display a number with a limited number of decimal places, then
it may of course look like a different number from what it actually is, so
if a calculation comes to 2.0000001 or 1.99999999 and you ask Excel to
display to 1 decimal place it will look like 2.0. You'll also need to
remember that precision in Excel is limited to 15 significant figures.
--
David Biddulph

"Jeff" wrote in message
...
Hi,

When I program in C++ a variabl declared as a double

x = 2.0

comes out as 2.0000001 or 1.99999999

but in excel it is 2.0 exactly why is that?

Thanks



Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
MULTIPLE DATA - How to insert new data into existing data.... Rodorodo Excel Discussion (Misc queries) 0 December 15th 06 11:50 PM


All times are GMT +1. The time now is 04:49 AM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"