RE: Informatica Bulk Mode behavior

Good points.

One of the problems that might have to do with "data saves" is that the saved data is not available until the last "finish" is executed.

The option provided by Informatica to control the data availability is called "commit interval" not the save point and is the same one used for conventional inserts and with other database vendors.

There might be the need for partial data availability during the bulk load to achieve some multi-tasking (run other processes on the saved data), in this case some warning about the effect of <low commit interval + Oracle> would be useful in the documentation.

Regards,

Waleed

-----Original Message-----
From: Tim Gorman [mailto:tim_at_sagelogix.com] Sent: Tuesday, April 27, 2004 11:00 AM
To: oracle-l_at_freelists.org
Subject: Re: Informatica Bulk Mode behavior

Sounds like they're rationalizing the fact that they screwed up the coding... :-)

They should be using "OCIDirPathDataSave()" for data saves, not "OCIDirPathFinish()" followed by "OCIDirPathPrepare()". The amazing thing
is that they are aware of their screw-up! Their coding is analogous to closing and re-opening a frequently-executed cursor after each execution

--
unnecessary and foolish.

The fact that they coded incorrectly and that it is affecting space
management adversely, causing bulk loads to be perform unnecessarily
poorly
as well, is bad enough.  But the mis-information in blaming their faulty
coding on an "Oracle design limitation" is either dishonesty or
stupidity or
both.



on 4/27/04 6:52 AM, Thomas Jeff at jeff.thomas_at_thomson.net wrote:



> Tim, Waleed, we got this response from Informatica Support:

>=20


> We are using the OCI DPL (direct path load) in the following way:

>=20


> First, We get a block with "x" # of rows.

>=20


> Do following loop

>  Call ColArrayToStream for these "x" rows.

>  Load whatever we can (call DirPathLoadStream)

>  Do a DirPathStreamReset.

> while ColArrayToStream did not convert all the rows, repeat the above

> steps for the remainder rows.

>=20


> Every so often, depending on our "commit" interval, we will call

> DirPathFinish (and then reprepare).

>=20


> The 2 differences between our process and Oracle's demo program are:

> 1) the # of rows "x" may be different.  They may be optimizing the "x"

> that they pick.  We derive "x"=3D20

> from the buffer block size.

>=20


> 2) Oracle does 1 DirPathFinish at the end of the process, whereas we

do


> it on every commit. =3D20

>=20


> Thus, we could probably simulate exactly what Oracle's test program is

> doing by making sure that the=3D20

> # of rows in the block matches their test program and the target

commit


> interval is really high.

>=20


> So there seem to be 2 possible places where we may be causing more

> extents to be created than necessary. =3D20

>=20


> 1) The last load in the loop may be less than the amount of rows that

> can fit in a transfer block.  Not sure if this matters at all, but if

> the last load contains only 1 row, this may be a wasted allocated

> extent.

>=20


> 2) The DirPathFinish DEFINITELY seems to be causing a new extent to be

> created.

>=20


> We still have one X factor that we need to account for.  The CR

mentions


> that if the commit interval is high, we=3D20

> sometimes get the following error:

>=20


> ORA-01658: unable to create INITIAL extent for segment in table

space=3D20


> Our guess is that somehow we ran out of space to get an extent.

> (perhaps in the temp table space)  Perhaps=3D20

> this can be limited somehow if there is a way to turn off logging (if

it


> was, indeed, on), but I didn't see=3D20

> this option available.  Although we haven't tested this yet, we would

> think that the sample program would have=3D20

> similar issues, although it may take a little longer because they may

> not have wasted extents as listed in option=3D20

> 1) above.

>=20


> Solutions:

> The workaround fix is to play with the buffer block size and the

commit


> size.  If the buffer block size fits=3D20

> the correct amount of rows, we may have a somewhat optimal fix.  We

may


> want to consider logging certain=3D20

> information that may help the user pick a correct buffer size, but, of

> course, this can be kinda cryptic.


----------------------------------------------------------------
Please see the official ORACLE-L FAQ: http://www.orafaq.com
----------------------------------------------------------------
To unsubscribe send email to:  oracle-l-request_at_freelists.org
put 'unsubscribe' in the subject line.
--
Archives are at http://www.freelists.org/archives/oracle-l/
FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
-----------------------------------------------------------------
----------------------------------------------------------------
Please see the official ORACLE-L FAQ: http://www.orafaq.com
----------------------------------------------------------------
To unsubscribe send email to:  oracle-l-request_at_freelists.org
put 'unsubscribe' in the subject line.
--
Archives are at http://www.freelists.org/archives/oracle-l/
FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
-----------------------------------------------------------------