"GS" wrote in message ...
spilly39 expressed precisely :
I'd like to see the code you are using to read the lines
- shouldn't be more than a few lines of CODE (as opposed to DATA that
is!)

I've handled large log files, tho' not in the GB range

It's vital you do not read the whole file, but just a block at a time and
within that only one logical line at a time. My issue was that some data
files were exUnix (Lf) and other data ex Windoze (CrLf line terminated)
and VB Line Input statement needs CRLF data.

My solution was to write a class (UnixDosReader) with properties of
filename, size etc, and methods of fOpen, GetNextLine and fClose

I'm looking at my code now (not having used it since I retired)
I see I handled the file unblocking in VBA myself if it was a Unix file,
reading one block of data (of whatever size I chose) at a time.

What I did was to scan the 1st lump of the data at file open time to
determine if it was LF or CRLF data
If CRLF in block 1, then my GetNextLine method reads the file one line at
at time using std VB IO code
Line Input #pFileNo, dataLine

If it's Unix it gets a bit more complicated, with a Private Sub inside
the class:
ReadaBlock(s as String, Size as Integer) 'size is size of block buffer
which internally did
Get #pFile,,s '(read into s String argument)
You have to gwt it right at the end of the file, which doesn't fill your
standard blocksize.

Altogether there's 273 lines of code (of which the 1st 94 are solid
comments)

I could send the code - it's a bit long to post here

BUT I have one little nagging worry...
FileSize is typed as Long, which on my 32 bit machine only gets to 2GB. I
assume you're on a 64 bit box? Just checking..
You'll have to look carefully at any such Typing matters if you want to
use my code

spilly


Thanks for stepping in!
I ran into the same issues with reading larger files in all at once. Like
you I've never done GB-size files but have had to read files as large as
10MB in blocks as you describe. For me, however, this was a onetime
workaround and so I never bothered to refine it beyond that.

Your solution sounds exactly what this OP needs. I'd also be very
interested if you're willing to share.

--
Garry

Free usenet access at http://www.eternal-september.org
ClassicVB Users Regroup! comp.lang.basic.visual.misc

Hi Garry

That was quick!

I see no reason not to share.
In fact I'd be willing to post an attachment here if that's allowed, but I
doubt it is.

I'm not a big NG man, so I'm unsure of the Netiquette;
I'm really only subscribed (today!) becuase of another prob I've got
but when I saw this thread, it was a dead cert for my UnixDosReader class...
Oh sod it! I'll try an attachment anyway - it's got the usual chances:
Success or Failure!
Health warning:
I'm a totally self taught VBA afficionado.
I'm always impressed with my code when I write it
but never impressed when I read it again two years later

emails from me to GS (those are MY initials!!) at somehwere dot net ain't
likely to reach you.
However if you email me on spilly39 buttons (my other prob is to do with
ActiveX buttons), that's a valid but disposable email address, & I should
get your mail.

Listening out
spilly