On Dec 2, 10:32*am, Martin Brown
wrote:
<snip

There is a good reason for this. The mantissa of a double precision real
is only good to 15-16 decimal digits. Put those into a cell as real
double precision numbers and they will get rounded to the nearest value
that the machine can represent. You will have to declare Dec as Decimal
which should allow you up to 28 decimal places. If you want more then
you will have to use strings and/or find a multiple precision arithmetic
code in VBA to avoid reinventing too many wheels.

BTW Indexing into a string containing "0123456789ABCDEF" will save a
fair amount of coding.

Regards,
Martin Brown

Hi, Martin,

Yes, I was getting the idea that it was the parameter of the data
type, but a couple followup notes:
- I read else where that there's a bug that doesn't allow dimming as
Dec but instead to dim as Variant then use cdec, which I have already
done on the Dec variable.
- the overflow issue doesn't seem to be triggered by the Dec variable,
but rather by the value Dec is to be compared to, 20 ^ (i - 1) (where
i = 256!) --
obviously, that is a huuuuuuge number, so I'm not surprised, but
I'm not sure what the need of i being 256 is, so not sure I can change
it or what I can change it to, for that matter.

Bottomline is, I'm not really familiar with the underlying conversion
process/algorithm, so I'm rather tweaking in the dark.

Also, I'm not familar with the idea of "Indexing into a string
containing "0123456789ABCDEF" " -- it looks interesting -- any chance
you could point me towards more info on that?

Thanks again.