Import data selectively from a CSV larger than 65000 rows
 

Hi Guys,
Over the past year I've been working on a reporting system that imports
data from 3 different files and does some magic with it. These files
are usually emailed to us daily by external parties we do business
with. The files contain month-to-date data, from the 1st of the month
up until the day before. We do not have control over the format and/or
structure of this data.

The problem I am now having is that the data has grown such that it
frequently surpasses the 65000 row excel limit. I looked for and found
code both on the Microsoft site and on these forums that truncates
large text files into multiple sheets, but these didn't work out for me
for different reasons (the Microsoft code is too slow, the other code I
found didn't work on my csv files).

In any case, since my reporting is processed daily, I typically only
need the previous day's data or a few days of data on the weekends,
etc.

My question/Request:
================
Does anyone have VBA code that would open a large csv file, scan column
(A) for a certain date, and import every row after and including that
date into a specific sheet.

One thing I should note is that the data starts at row 8 for the column
headers, not row 1.


Thanks a bunch!

Import data selectively from a CSV larger than 65000 rows
 

Take the larger file into MS-Access. If you need to do an overwrite of
existing records, be sure to provision a resolution for duplicate records,
e.g., ignore the import of a duplicate, or overwrite duplicates.

If you need to use Excel for some reason, then set up a subset of the larger
database in Access. Create a subset table, or a view table. Bring in only
what you need. Or, use the Data Query in Excel to only extract the records
you need.

P.
--
Overcome Sales Barriers
http://salesbarriers.typepad.com/


" wrote:

Hi Guys,
Over the past year I've been working on a reporting system that imports
data from 3 different files and does some magic with it. These files
are usually emailed to us daily by external parties we do business
with. The files contain month-to-date data, from the 1st of the month
up until the day before. We do not have control over the format and/or
structure of this data.

The problem I am now having is that the data has grown such that it
frequently surpasses the 65000 row excel limit. I looked for and found
code both on the Microsoft site and on these forums that truncates
large text files into multiple sheets, but these didn't work out for me
for different reasons (the Microsoft code is too slow, the other code I
found didn't work on my csv files).

In any case, since my reporting is processed daily, I typically only
need the previous day's data or a few days of data on the weekends,
etc.

My question/Request:
================
Does anyone have VBA code that would open a large csv file, scan column
(A) for a certain date, and import every row after and including that
date into a specific sheet.

One thing I should note is that the data starts at row 8 for the column
headers, not row 1.


Thanks a bunch!

Import data selectively from a CSV larger than 65000 rows
 

Thanks Kooster,
I've considered using Access, but would much rather do it with VBA if I
can. I download these 3 files every single day and do not relish the
thought of having to intervene using Access on a daily basis.

Thanks!


Kooster wrote:
Take the larger file into MS-Access. If you need to do an overwrite of
existing records, be sure to provision a resolution for duplicate records,
e.g., ignore the import of a duplicate, or overwrite duplicates.

If you need to use Excel for some reason, then set up a subset of the larger
database in Access. Create a subset table, or a view table. Bring in only
what you need. Or, use the Data Query in Excel to only extract the records
you need.

P.
--
Overcome Sales Barriers
http://salesbarriers.typepad.com/


" wrote:

Hi Guys,
Over the past year I've been working on a reporting system that imports
data from 3 different files and does some magic with it. These files
are usually emailed to us daily by external parties we do business
with. The files contain month-to-date data, from the 1st of the month
up until the day before. We do not have control over the format and/or
structure of this data.

The problem I am now having is that the data has grown such that it
frequently surpasses the 65000 row excel limit. I looked for and found
code both on the Microsoft site and on these forums that truncates
large text files into multiple sheets, but these didn't work out for me
for different reasons (the Microsoft code is too slow, the other code I
found didn't work on my csv files).

In any case, since my reporting is processed daily, I typically only
need the previous day's data or a few days of data on the weekends,
etc.

My question/Request:
================
Does anyone have VBA code that would open a large csv file, scan column
(A) for a certain date, and import every row after and including that
date into a specific sheet.

One thing I should note is that the data starts at row 8 for the column
headers, not row 1.


Thanks a bunch!