Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 1
Default Big dataset - batch process

Hi

I've got a huge (14,000,000+) dataset I need to process. At present,
this is sitting in an Access db in one table. I have
code ready to go to split it into around 39,000 separate tables and
export that as (if necessary) 39,000 separate Excel files.

What I need to do is perform a regression (data analysis toolpak) on
each separate file. To the best of my knowledge this is best
achieved in Excel not Access but if it's easier to have Excel connect
to the Access db to retrieve the data I have no problem with this.

The code to run the regression is simple enough -

Application.Run "ATPVBAEN.XLAM!Regress", ActiveSheet.Range("$E$2:$E
$1930") _
, ActiveSheet.Range("$D$2:$D$1930"), False, False, 95, "",
False, False _
, False, True, , True

but how do I find out which row contains the last row of data (it will
vary from file to file) to plug into that code?

But my main question is - is this the best approach? So far as I
know, you can't run regressions in Access so this is, as far as I can
tell, the best method isn't it?

Thx
  #2   Report Post  
Posted to microsoft.public.excel.programming
external usenet poster
 
Posts: 1,522
Default Big dataset - batch process

On Saturday, March 17, 2012 1:48:03 PM UTC-5, Sunstormrider wrote:
Hi

I've got a huge (14,000,000+) dataset I need to process. At present,
this is sitting in an Access db in one table. I have
code ready to go to split it into around 39,000 separate tables and
export that as (if necessary) 39,000 separate Excel files.

What I need to do is perform a regression (data analysis toolpak) on
each separate file. To the best of my knowledge this is best
achieved in Excel not Access but if it's easier to have Excel connect
to the Access db to retrieve the data I have no problem with this.

The code to run the regression is simple enough -

Application.Run "ATPVBAEN.XLAM!Regress", ActiveSheet.Range("$E$2:$E
$1930") _
, ActiveSheet.Range("$D$2:$D$1930"), False, False, 95, "",
False, False _
, False, True, , True

but how do I find out which row contains the last row of data (it will
vary from file to file) to plug into that code?

But my main question is - is this the best approach? So far as I
know, you can't run regressions in Access so this is, as far as I can
tell, the best method isn't it?

Thx

I have no experience with Access or regression but maybe this helps
===============
with activesheet
lr = .Cells.find("*", Cells(Rows.Count, Columns.Count) _
, , , xlByRows, xlPrevious).Row
.Range("$E$2:$E$" & lr") _
.Range("$D$2:$D$" & lr), False, False, 95, "", _
False, False, False, True, , True
end with
Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
batch file call from a macro - how ? and required batch format VB-rookie Excel Programming 4 September 6th 08 12:52 AM
batch process to covert file to excel file [email protected] Excel Programming 4 February 22nd 08 10:09 PM
assign dataset to matching dataset Michael Dirksen Excel Programming 0 June 16th 06 04:34 PM
Converting A Quarterly Dataset to Weekly Dataset Dan Thompson Excel Programming 5 November 25th 05 09:27 PM
Run a macro batch process marijo[_8_] Excel Programming 2 June 2nd 04 02:46 AM


All times are GMT +1. The time now is 03:56 PM.

Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 ExcelBanter.
The comments are property of their posters.
 

About Us

"It's about Microsoft Excel"