Re: Oracle Myths

Nuno Souto <nsouto_at_optushome.com.au.nospam> wrote in message news:<3d03f7ed$0$28008$afc38c87_at_news.optusnet.com.au>...
> In article <f369a0eb.0206091530.3c0920ab_at_posting.google.com>, you said
> (and I quote):
> >
> > I'd be shocked. That's growth of more than 10 TB a year. If that indeed is what
> > they do I'd be very interested to know how they manage to complete all the
> > merge/purge/transformation and then get the data loaded/updated in a 8-10 hour
> > window.
>
>
> Simple. Data load is via sqlloader from flat files. The flat files are
> created by the data capture computers.
> Use nolog, parallel loads. Into specific, empty partitions. About 8
> tables, all partitioned. Around 1 hour in the class of machine these
> guys are using. Sun E10K, full CPU complement. Humongous EMC disk farm.
> Indexing takes about 40 minutes, again heavy parallel used.
> Daily processing takes about 7 hours. Once that's done, data is ready
> for passing to other systems.
>

So it's sqlldr followed by some simple database processing. Makes more sense now. In a lot of shops merge and purge alone would take a few hours for 30GB data. The horsepower these guys have makes a difference too. Thanks for the information.

> This is what happens then: EMC bcv volumes are split together with the
> Sun E10K. Virtually a second computer is created, with its own instance.
> The original data is kept and worked on a little bit more, then
> partitions are created for the next business cycle. Once a week, older
> partitions are simply purged and dropped.
>
> The second computer (the one "created" by splitting the system) is then
> used for additional heavy batch processing, reporting and preparation for
> download into data warehouse. It only has to deal with the daily 30Gb
> plus some summary info. And it has potentially up to 24 hours ahead of
> it for this, until the next split.
>
> Once it is done, the system is "glued" back together again: the split
> instance is shutdown and dropped and the EMC bcv volumes are re-synched.
> This process takes about 1 hour to complete (EMCs are tremendously fast
> at re-synching these volumes). And they're ready for another daily
> cycle. With a few hours to spare. Amazing what you can do nowadays, eh?
> And it's only 8.0.6, imagine what 9i could do here?
Received on Mon Jun 10 2002 - 13:53:42 CDT