View Single Post
  #1   Report Post  
Posted to microsoft.public.excel.programming
duane[_74_] duane[_74_] is offline
external usenet poster
 
Posts: 1
Default Precision in Excle VBA


I was trying to have VBA to recurse on an assumed result, to
calculated one (I need to assume the answer to do some intermediat
calculations which ultimately end up with a final answer), in this cas
a temperature.

when I coded

recurse:
terror = tassume -tcalc
if terror <0 the terror = terror * -1
if terror <0.3 then goto done
tassume = tassume +0.5 * (tcalc - tassume)
go to recurse

the code did not recurse to within 0.5

but when I coded

recurse:
terror = (tassume *100 - tcalc*100)
if terror <0 the terror = terror * -1
if terror < 30 then goto done
tassume = (tassume*100 +0.5 * (tcalc - tassume)*100)/100
go to recurse

it worked fine

any thoughts??

--
duan

-----------------------------------------------------------------------
duane's Profile: http://www.excelforum.com/member.php...fo&userid=1162
View this thread: http://www.excelforum.com/showthread.php?threadid=56167