Problem with .navigate method
 

I am trying to read data from a website using the navigate method. The
website returns 40 data items of interest out of a total of 18,000 so I
have to read many times to get all 18,000 items.

The problem is that sometimes the data returned by Execel is incomplete
(only part of the html is returned). The problem does not appear to be the
website because only this happens when using Excel. Repeating the
identical attempt to read the data from the web will almost always give the
correct results. Rarely, I'll have to make 3 attempts to get it right.
Here is the code I am using:

Dim oie As Object
Dim i as Long
Dim Done as Boolean

Set oie = CreateObject("InternetExplorer.Application")
Done = False
i = 2
While Not Done
oie.Navigate "http://www.WebSiteOfInterest.com/" & ThisTimesReq(i)
oie.Visible = False
Do
Loop Until oie.ReadyState = 4
Range("A" & i) = oie.Document.body.outerhtml
If instr(Range("A" & i), "Data past what I'm looking for") = 0 then
i =i - 1
End If
i = i + 1
If SomeTestSaysWeAreDone then Done = True
Wend

The code above works only because , after each read, I look for html that
comes after all of the stuff I am interested in. If it is missing, I know
the read was incomplete and all of the info I need may not be there. By
setting i=i-1 another attempt is made to read the same data.

I am using Windows XP-Pro with all updates and Excel 2003 SP 2

QUESTIONS:
1. Is there something wrong with my code?
2. If not, is this a common Excel problem?
3. Is there another way to do this that is reliable so I don't have to
repeat failed attempts?

Many thanks for any help/