View Single Post
  #1   Report Post  
Posted to microsoft.public.excel.programming
Julie Julie is offline
external usenet poster
 
Posts: 150
Default Excel precision issues with float/single

If I have a float variable with a value of 9.9 (as seen in the debugger), and
I set it to an Excel cell, the cell value is 9.89999961853027. However, if I
have a double variable with the same value of 9.9, setting that to an Excel
cell gives me the correct value.

(Note that the cell, in it's default size, looks correct, but if you expand
the cell, or click on the value and look on the editor area, the value is
incorrect.)

Am I doing something wrong? How do I fix this? Thanks much!