Re: Oracle Myths

In article <f369a0eb.0206091530.3c0920ab_at_posting.google.com>, you said (and I quote):
>
> Very entertaining. Further suggests that anything LMV, file systems, or things
> of that nature should be avoided if you want to achieve truly sequential I/O.

Start here:
http://www.bethel.edu/Majors/MathCS/os98smallgroup/files/unix1.html and pay particular attention to the bit that says: <QUOTE>
A file may be stored in several different blocks, seemingly randomly chosen.
</QUOTE>
Amazing, isn't it?

>
> I'd be shocked. That's growth of more than 10 TB a year. If that indeed is what
> they do I'd be very interested to know how they manage to complete all the
> merge/purge/transformation and then get the data loaded/updated in a 8-10 hour
> window.

Simple. Data load is via sqlloader from flat files. The flat files are created by the data capture computers. Use nolog, parallel loads. Into specific, empty partitions. About 8 tables, all partitioned. Around 1 hour in the class of machine these guys are using. Sun E10K, full CPU complement. Humongous EMC disk farm. Indexing takes about 40 minutes, again heavy parallel used. Daily processing takes about 7 hours. Once that's done, data is ready for passing to other systems.

This is what happens then: EMC bcv volumes are split together with the Sun E10K. Virtually a second computer is created, with its own instance. The original data is kept and worked on a little bit more, then partitions are created for the next business cycle. Once a week, older partitions are simply purged and dropped.

The second computer (the one "created" by splitting the system) is then used for additional heavy batch processing, reporting and preparation for download into data warehouse. It only has to deal with the daily 30Gb plus some summary info. And it has potentially up to 24 hours ahead of it for this, until the next split.

Once it is done, the system is "glued" back together again: the split instance is shutdown and dropped and the EMC bcv volumes are re-synched. This process takes about 1 hour to complete (EMCs are tremendously fast at re-synching these volumes). And they're ready for another daily cycle. With a few hours to spare. Amazing what you can do nowadays, eh? And it's only 8.0.6, imagine what 9i could do here?

-- 
Cheers
Nuno Souto
nsouto_at_optushome.com.au.nospam