Home |
Search |
Today's Posts |
|
#1
![]()
Posted to microsoft.public.excel.programming
|
|||
|
|||
![]()
Floating Point means decimal point.
A bit of a simplication. More than you want to know?: http://en.wikipedia.org/wiki/Floating_point A waste of time (but funny?) http://del.icio.us/url/53db119a950dc8c8b8dfbd087cf4680e Ignore this if it makes things more confusing: (This ignores negative numbers) (This uses small numbers just for explanation) If you have a 3-digit field, three options for storing numbers: 1) Integer: You can store 0 to 999. You can store only the integer part of 1.2 You can't store 12000 nor .00034 You can't store 12345 nor .00056789. 2) Fixed (decimal) Point, say 2 decimal places. You can store 0.00 to 9.99. 3) Floating Point: Allows you to store a very large range of numbers but at 2-digit accuracy instead of 3. The first 2 digits will be the "mantissa", the first 2 significant digits, which will range from .00 to .99 The 3rd digit will be 4 + the exponent of 10. The exponent can range from -4 to 5. The equation of the value is mantissa * 10 ^ (3rd digit - 4). ("^" means "to the power of") You will be able to store 0 to 99. store 1.2 and 34000 and .00056. store 999 and 12345 and .00056789, but not exactly. 1 stores as 105 (.10*10^1=1) 1.2 stores as 125 (.12*10^1=1.2) 10 stores as 106 (.10*10^2=10) 99 stores as 996 (.99*10^2=99) 100 stores as 107 (.10*10^3=100) 230 stores as 237 (.23*10^3=230) 1000 stores as 108 (.10*10^4=1000) 34000 stores as 349 (.34*10^5=34000) ..1 stores as 104 (.10*10^0=.1) ..00056 stores as 561 (.56*10^-3=.00056) 999 stores as 997 (.99*10^3=990) not exact 12345 stores as 129 (12*10^5=12000) not exact ..00056789 stores as 561 (.56*10^-3=.00056) not exact In binary it's a binary point instead of a decimal point, but the idea is the same. But .8 in binary is ..110011001100..(infinitely) as 1/3 in decimal is ..3333..(inifinitely); and there is rounding, so there are comparing issues in floating point. D-C Dave ----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==---- http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups ----= East and West-Coast Server Farms - Total Privacy via Encryption =---- |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Insert, position, and resize a picture w/ Macro | Excel Discussion (Misc queries) | |||
Resize Last Row to Fill a Page Width | Excel Discussion (Misc queries) | |||
resize row height and column height | Setting up and Configuration of Excel | |||
resize row and colomn height | Excel Programming | |||
get a picture width and height without inserting the picture | Excel Programming |