View Single Post
  #7   Report Post  
Posted to microsoft.public.excel.programming
NickHK NickHK is offline
external usenet poster
 
Posts: 4,391
Default Huge data structure in Excel

Milo,
Trying to work with data spread across a number of Excel files 100MB+ will
make your life difficult.
Don't be daunted about Access. There a number wizards to help you.
The easiest way to get you data into Access is FileExternal DataImport,
then follow the steps. ExcelAccess direct.
The first time you'll need to create a new table. Subsequently you dump the
other data into that table.
Obviously all the data has to be in a compatible format, but this is the
case even if you leave it in Excel, otherwise RIRO.

Once you have you data in Access, to analysis in Excel is a case of DataGet
External DataNew Database Query, or use Pivot Tables.

NickHK

"Milo" wrote in message
...
To all of you, thank you very much for your contributions. They are all
sensible.
However,
1. I do not have access to SQL server only Access.
2. I only did db design as subject years ago and never realy used Access,
although I might have to start now!
Currently all of this data is contained in yearly Excel workbooks in
multiple aircraft sheets in raw form (i.e. the VA values are not grouped

into
histogram bins. Some of these workbooks are 100Mb in size.
Thus I have to somehow transfer this data into csv files, bin them, and

then
load into Access?
3. This is why I wanted to stay in Excel if possible. But it seems by your
suggestions that this may not be feasible.

Regards Milo.

"Milo" wrote:

Hi,
I wonder if Excel is the right app for this. I'll try and be brief.

I have some aircraft data: latitude, logitude, height, time/date and a
crucial number called vertical acceleration in m/s (call it VA). I have

this
data for 40 aircraft spanning 10 years and so the number of these

records is
several million.

Now, I would like to set up a multidimensional structure where I can

view
histograms of VA for any height band (say 9000 to 10000 feet) for any
month/year for any aircraft for a particular latitude/longitude square

(say 2
x 2 degrees).
I was thinking in terms of multiple workbooks (one for each aircraft)

and
multiple sheets in each workbook, a sheet one for each month where each

sheet
would contain lat/lon height columns and rows would contain numbers of

VA
occurances from which the VA histograms could be built.
How fast would such a structure be if I wanted to sum over several years

for
a particular number of aircraft for a given height band, month and

location
and come up with a histogram?
Put simply is this feasible in Excel?
Any suggestions welcome, Milo.