Re: SQLLDR: interactive commit?

Steffen Conrad <steffen.conrad_at_gmx.de> wrote in message news:<2h0si6F7ub1uU1_at_uni-berlin.de>...
> Hi all,
>
> I have a tool that decodes binary data from various files and parses it
> onto multiple SQLLDR-instances using direct path load.
>
> Overview:
>
> create n SQLLDR processes
> for 1..m files
> do
> decode file and deliver output to the n SQLLDRs
> (every SQLLDR gets separate data)
> done
> close SQLLDR processes
>
> Problem:
> After loading the contents of each file I have to assure, that the
> rows I put into the loader processes we formatted according to the
> control file. Therefore I need the same functionality as when i closed
> the SQLLDR process, which would result in an error in the exit code.
>
> Because I parse a very high number of small files to a high number of
> SQLLDR processes, I can't stop each loader process after each file.
>
> Example:
> For loading 260 files onto ~80 loader processes I needed ~45 minutes
> when starting and stopping the 80 loaders for each file. when reusing
> the 80 loaders it only takes 5 minutes for loading all 260 files. But
> when following this approach, I can't test, if a input file was
> formatted correclty for input into the loader processes.
>
> Question:
> What I am searching for, is to control the SQLLDr processes that they do
> a commit/save at end of each input file. Im starting the SQLLDR
> processes by popen in C, therefore the only possible way is to "send" an
> interactive commit statement into the input data stream of the according
> SQLLDR instances. Is this possible? I didnt found anything like this.
>
> Controlling the commit points of the SQLLDRs according to commandline
> parameters is not possible, because I cant count the rows/sizes of input
> data for each SQLLDR before starting it.
>
> Regards,
> Steffen
Our Super*SQL product has very similar functionality to SQL*Loader but commits on each record. I'm not sure that it would perform as well as the bulk method you are using but we could certainly customize the way error trapping took place. Take a look at www.sqlmagic.com and, if you think Super*SQL can solve your problem, send me a direct E-mail so we can agree on any changes required. Received on Thu May 20 2004 - 17:41:08 CEST