Home |
Search |
Today's Posts |
#8
![]()
Posted to microsoft.public.excel.misc
|
|||
|
|||
![]()
joeu2004 wrote on Thu, 24 Jan 2008 10:11:40 -0800 (PST):
j On Jan 24, 5:28 am, "James Silverton" j wrote: ?? Jerry wrote on Wed, 23 Jan 2008 20:27:37 -0800: JWL For instance, =ROUND(10^14+0.5,0) JWL incorrectly returns the value of 100000000000000 ?? ?? This is all very interesting but ultimately what does it ?? matter if ROUND sometimes goes in the wromg direction at ?? the limits of precision? It is hard for me to imagine a ?? real case where so many figures are necessary but I am ?? ready to be instructed. j It isn't so much that anyone needs this much precision, but j that these flaws cause problems in formulas that involve j large numbers. The explanation is more precise than the j solution requires. j I presented this anomaly when I was having a problem with j INT(). Actually, the problem is with MOD(x,y), which is j well-know to an error with large numbers. I wanted to use x j - y*INT(x/y) work around the problem. When that was j misbehaving, I did a binary search and discovered the 10^14 j boundary and speculated the relationship to the number of j significant digits that Excel will display. Again without wishing to start a fight, can you give me an example of a real calculation where the rounding error makes a difference to a conclusion? I probably wouldn't use Excel if there were such a case since it's not that hard to program in double precision or the like. James Silverton Potomac, Maryland E-mail, with obvious alterations: not.jim.silverton.at.verizon.not |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Forum | |||
Prevent EXCEL from rounding the result of a calculation | Excel Discussion (Misc queries) | |||
Rounding up the result of an IF function | Excel Worksheet Functions | |||
How do I use a rounded calculation result in another calculation? | Excel Worksheet Functions | |||
How do I use a rounded calculation result in another calculation? | Excel Worksheet Functions | |||
Rounding up the result | Excel Worksheet Functions |