Re: Loading data continually

Hi Pete,

    In my last job we were uploading and downloading datafiles from forty sites
every night -- all night, and during part of the day as well. We found it best to
capture data as flat files and then run SQL*Loader to get the data into the data
bases.

    The governing factors in capturing data as flat files rather than having it come
directly into the database were:

1. Communications is not reliable. Retransmissions are common. Net data transfer rates can vary tremendously depending on the quality of service between your site and the remote sites.
2. When your user community (including your developers) forgets to reflect a change in your load programs/scripts you don't have to retransmit the data since you already have it sitting in a flat file.
3. You have the opportunity to split the workload between a database server and an inexpensive dedicated communications server such as a PC.
4. You have some flexibility in scheduling your database loads and can avoid/minimize impact on your local processing

regards
Jerry Gitomer

Pete_Thomas_at_inmarsat.org wrote in message <7lfon0$cn7$1_at_nnrp1.deja.com>...
>We are looking to load an Oracle DB using data from four remote machines
>that are in different countries, but would like to provide a continuous
>almost real time load.
>
>Our DB Server is running NT.
>
>Has anyone tried this type of thing ?
>What are the options & pros & cons of the following options ?
>
>- FTPing files back to our DB server, then continually loading
>but I guess we would need some kind of staging area ?
>
>- Using some messaging type software ? Like OrbixTalk ?
>
>- Setting up the remote machines as clients using SQL*Net ?
>
>In total we would be loading about 202,000 rows per hour
>or 3,380 per minute ?
>
>Is this feasible, what sort of Server specification could cope ?
>
>Also users would be querying data at the same time, but that data would
>normally be in a different partition.
>
>
>Sent via Deja.com http://www.deja.com/
>Share what you know. Learn what you don't.
Received on Thu Jul 01 1999 - 12:55:04 CDT