Re: Expect to work with terrabytes.. now what?

In article <376FE86E.C46DEB60_at_gnncast.net>,   Daniel Murray <dmurray_at_gnncast.net> wrote:
> One thing you could possibly do is summarize the data if it is
possible. It
> will cut down on the number of entries, but implementing it can be a
little
> tricky.
>
> Ferdinand Swaters wrote:
>
> > Hi All,
> >
> > I am currently on a project where the data in my database is going
to run
> > into the 1-10 Terabyte range. I have no experience with these sizes
of data
> > in a DBMS, and I am wondering what new pitfalls I might run into.
> >
> > The system will do almost no updates on the data, most is storage
and then
> > delete after a while. There won't be many queries on the data, but
the
> > queries will be large. It is much like a data warehouse situation.
The DBMS
> > will be oracle.
> >
> > I don't have a good idea on what questions to ask first, because
I'm mainly
> > concerned about the things I might forget now, and run into when
the beast
> > is operational. One thing that I would like to know is if there is
a way I
> > could compress the data while it is in the database. That would
reduce
> > storage space required by a huge factor.
> >
> > Anyone has some experience they wish to share?
> >
> > advTHANKSance,
> >
> > Ferd.
>
>

The management of large quantities of data is something the group at CERN have reported in a number of papers. They have a number of similar requirements to yourself, but use an OODB. However, their experiences may still be of value to you. Try searching for RD45 at the CERN web site: www.cern.ch. One paper recent paper also appeared here:

http://www.disi.unige.it/conferences/oodbws99/programme.html

Good luck.

akmal

--
email: akmal(at)bigfoot(dot)com || http://www.bigfoot.com/~akmal/
"I need a job where my immense ego seems normal." - Dogbert
DISCLAIMER: I work for Computer Associates and my opinions are my own.


Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.