View Single Post
  #10   Report Post  
Jerry W. Lewis
 
Posts: n/a
Default

Additional thoughts:

Your validation should have shown the result of the calculation to match
an independent source of the answer, so that should be sufficient for
regulartors, particularly given FDA's current "risk-based" environment.

If your QAV people need more, you can note that the IEEE binary
representation (see the cpearson.com link of my original reply) for
5420.06727 is equivalent to 11639505833385/2147483648 and the IEEE
binary representation for 5420.06726999999 is equivalent to
5959426986693109/1099511627776; their difference is 11/1099511627776 or
1.00044417195022106170654296875E-11. You can then subtract
5420.06726999999 from the cell that is supposed to contain 5420.06727 to
show that Excel reports that difference as 1.00044417195022E-11 (Excel's
documented limit is 15 digits -- see Help for "Excel specifications and
limits" sutopic "Calculation specifications"). Therefore the value in
the cell is the correct representation of 5420.06727, even though it
displays as 5420.06726999999.

Jerry

Jerry W. Lewis wrote:

The referenced KB article says nothing about a patch, so AFAIK there is
none. If challenged, just cite the KB article and point out that even
if it were a numeric rather than a display issue, it would be around a
0.00000000002% error. Your data is not that accurate.

Jerry

James V. Wilkerson wrote:

As this may be the case, is there a "fix" for it. I can print the data
sheet showing the correct number, but when I print out the formulas it
changes the number to 5420.06726999999. I have to put this in a
notebook and have these two match so that when our client or the FDA
reviews it they won't have questions. I'm sure we could explain this
issue to them, but it would be easier if we didn't have to. Any
suggestions?