Home |
Search |
Today's Posts |
#1
Posted to microsoft.public.excel.misc
|
|||
|
|||
I don´t view the value right after six decimal place in Excel
From the sixth decimal place of any number in Excel does not present the
accuracy of the system in Cobol. Would I like to do this in Excel? (have the same accuracy from the sixth home) If necessary I send the file in Excel showing differences in the accuracy of calculating the System in Cobol x Microsoft Excel 2003. Other e-mail: |
#2
Posted to microsoft.public.excel.misc
|
|||
|
|||
I don´t view the value right after six decimal place in Excel
I do not know which implementation of COBOL you refer to, and it has
many years (decades!) since I looked at the COBOL language. (I was never really a COBOL programmer, but I married one :-.) As I recall, COBOL has a data type that maintains fractional digits in decimal. So there might be differences between the COBOL representation and the Excel representation. (Excel maintains numeric values using the standard IEEE floating-point internal representation.) If that is the root cause of the differences that you see, I am not sure there is any way to guarantee getting the same results in COBOL and Excel. There are decimal arithmetic add-ons for Excel; do a Google search. I am not familiar with any of them; I cannot say whether or not they will match the COBOL results of arithmetic operations. ------ original posting ----- On Aug 5, 9:02*am, Claudio wrote: From the sixth decimal place of any number in Excel does not present the accuracy of the system in Cobol. Would I like to do this in Excel? (have the same accuracy from the sixth home) If necessary I send the file in Excel showing differences in the accuracy of calculating the System in Cobol x Microsoft Excel 2003. Other e-mail: |
#3
Posted to microsoft.public.excel.misc
|
|||
|
|||
I don´t view the value right after six decimal place in Excel
In theory, Excel has a precision of 15 decimal places.
If you have an example of what you think is a discrepancy, post here the details of the formula you are using, the values you are feeding into it, the value you got, and the value you expected. -- David Biddulph "Claudio" wrote in message ... From the sixth decimal place of any number in Excel does not present the accuracy of the system in Cobol. Would I like to do this in Excel? (have the same accuracy from the sixth home) If necessary I send the file in Excel showing differences in the accuracy of calculating the System in Cobol x Microsoft Excel 2003. Other e-mail: |
#4
Posted to microsoft.public.excel.misc
|
|||
|
|||
I don´t view the value right after six decimal place in Excel
PS....
On Aug 5, 9:02*am, Claudio wrote: From the sixth decimal place of any number in Excel does not present the accuracy of the system in Cobol. Would I like to do this in Excel? (have the same accuracy from the sixth home) This might help..... For every calculation, use the ROUND() function to set the result to the desired precision. You might want to do that around intermediate results (subexpressions) as well as final results. This ensures that the internal binary representation is exactly the same as if you had entered the result as a constant. That still might have the same effect as COBOL decimal arithmetic. But it should reduce the effects of "propagation error". Consider this example: enter =100*ROUND(2.00666666666667,2)-200 and compare with 1. They are not identical in Excel 2003. Now compare 1 with round(100*ROUND(2.00666666666667,2),0)-200. You might consider setting the option Tools Options Calculation Precision As Displayed. But first, I believe that affects only the final result, not intermediated calculations (subexpressions). And second, it might not have the desired effect when using values with different precisions. If you experiment with this option, it would be wise to make a copy of the workbook first because some changes cannot be undone by simply unsetting the option and recalculating. |
#5
Posted to microsoft.public.excel.misc
|
|||
|
|||
I don´t view the value right after six decimal place in Excel
The procedure informed does not work. I wish that I send the Schedules so you
can make the tests? If so, tell me your e-mail. Thank you. "joeu2004" wrote: PS.... On Aug 5, 9:02 am, Claudio wrote: From the sixth decimal place of any number in Excel does not present the accuracy of the system in Cobol. Would I like to do this in Excel? (have the same accuracy from the sixth home) This might help..... For every calculation, use the ROUND() function to set the result to the desired precision. You might want to do that around intermediate results (subexpressions) as well as final results. This ensures that the internal binary representation is exactly the same as if you had entered the result as a constant. That still might have the same effect as COBOL decimal arithmetic. But it should reduce the effects of "propagation error". Consider this example: enter =100*ROUND(2.00666666666667,2)-200 and compare with 1. They are not identical in Excel 2003. Now compare 1 with round(100*ROUND(2.00666666666667,2),0)-200. You might consider setting the option Tools Options Calculation Precision As Displayed. But first, I believe that affects only the final result, not intermediated calculations (subexpressions). And second, it might not have the desired effect when using values with different precisions. If you experiment with this option, it would be wise to make a copy of the workbook first because some changes cannot be undone by simply unsetting the option and recalculating. |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Subtracting two 2-decimal place numbers gives result 13-decimal places? | Excel Worksheet Functions | |||
Converting 2-place decimal value to floating point decimal number with leading zero | Excel Discussion (Misc queries) | |||
Set default in Excel so each number entered has same decimal place | Setting up and Configuration of Excel | |||
excel validation to 1 decimal place | New Users to Excel | |||
How do I override fixed decimal place settings in EXcel 2003? | Excel Worksheet Functions |