Re: Web Services w/Oracle and lots of data - practical?

Roy Zolnoski wrote:

> I'm involved in a project where we are extracting data from multiple
> source systems and placing in a data warehouse. The central
> database/DW is Oracle 9i. The source systems are Oracle 8i and
> possibly other databases (unknown at this point, it's still early).
> I'd also like to prepare for non-Oracle databases.
>
> Data transfer would be once per day after an end of day process is run
> in the source system. Each system would transfer 150,000+ records to
> the DW each day. The records are from large, denormalized tables.
> The average row length from one system is ~370, others might be
> somewhat less or more.
>
> Are web services practical in this environment? Can web services be
> used to transfer large amounts of data like this? Any horror or
> success stories to share?
>
> Any help to this open ended request would be most appreciated.

It is NOT clear to me why you'd want to get Apache involved with this ETL operation. I would think that the use of messages queues (i.e. Oracle's AQ) to pass XML formated data would be an alternative.

AFAIK, with "web services" both data bases AND intervening network have to all be concurrently operational while the transfer completes. Both side have to be configured to gracefully recover should the transfer be interrupted while the transfer is in progress. With web services you'll have a number of single points of failure. Received on Fri Jul 09 2004 - 02:22:12 CEST