Home » SQL & PL/SQL » SQL & PL/SQL » BULK COLLECTIONS and CURRENT OF (Oracle 9i)
BULK COLLECTIONS and CURRENT OF [message #521829] Fri, 02 September 2011 05:26 Go to next message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
I have a driver table from which I need to update another table while, at the same time, record the fact that I have processed each record on the driver table.

The driver table will contain around 3.5 million records, therefore I intended to handle this using some bulk collections, with a LIMIT option so that I don't hit any memory problems.

I would also prefer to commit in batches, or at least handle exceptions using the SAVE EXCEPTION clause. The problem is I seem to be running into an error when trying to make the update to the driver table (the commented out code). With this in, I get the error:
ORA-01410: invalid ROWID
ORA-06512: at "CUST_MAIL_UPDATE", line 217
ORA-06512: at line 38

Can the CURRENT OF not work with the FORALL? What is my best approach here? If I use a FOR LOOP I lose my SAVE EXCEPTIONS exception handling.

The Procedure is as follows:
  -- declare some object structures to hold the retrieved data
  TYPE driver_rec IS RECORD (
    account_no         ext_driver.account_no%TYPE,
    update_action      ext_driver.update_action%TYPE,
    customers_rowid    ext_driver.customers_rowid%TYPE);

  TYPE driver_recs_tt IS TABLE OF driver_rec;

  -- cursor to get the records from the driver table
  CURSOR cur_get_driver_recs IS
   SELECT account_no,
          update_action,
          customers_rowid
     FROM ext_driver
    WHERE processed = 'N'
      FOR UPDATE OF processed;

  driver_records  driver_recs_tt;

  error_count NUMBER;
  CUST_BULK_UPDATE_FAIL EXCEPTION;
  PRAGMA EXCEPTION_INIT(CUST_BULK_UPDATE_FAIL, -24381);

BEGIN

  -- get the records from the driver table
  OPEN cur_get_driver_recs;
  LOOP
    -- process the records using Bulk Collections.
    FETCH cur_get_driver_recs BULK COLLECT INTO driver_records LIMIT 1000;

    FORALL i IN 1..driver_records.COUNT SAVE EXCEPTIONS

    UPDATE customers c
       SET mailing_option = driver_records(i).update_action
     WHERE c.rowid = driver_records(i).customers_rowid;

--    UPDATE ext_driver
--       SET processed = 'Y'
--     WHERE CURRENT OF cur_get_driver_recs;

    EXIT WHEN cur_get_driver_recs%NOTFOUND;

  END LOOP;

  COMMIT;

  EXCEPTION
    WHEN CUST_BULK_UPDATE_FAIL THEN
     error_count := SQL%BULK_EXCEPTIONS.COUNT;
     DBMS_OUTPUT.PUT_LINE('Number of updates that failed: ' || error_count);
     FOR i IN 1..error_count LOOP
        DBMS_OUTPUT.PUT_LINE('Error #' || i || ' occurred during '||
           'iteration #' || SQL%BULK_EXCEPTIONS(i).ERROR_INDEX);
            DBMS_OUTPUT.PUT_LINE('Error message is ' ||
            SQLERRM(-SQL%BULK_EXCEPTIONS(i).ERROR_CODE));
     END LOOP;
END;
Re: BULK COLLECTIONS and CURRENT OF [message #521836 is a reply to message #521829] Fri, 02 September 2011 05:41 Go to previous messageGo to next message
cookiemonster
Messages: 13967
Registered: September 2008
Location: Rainy Manchester
Senior Member
Current of is specific to cursor for loops. I really don't see why you need it though.
Re: BULK COLLECTIONS and CURRENT OF [message #521841 is a reply to message #521836] Fri, 02 September 2011 06:11 Go to previous messageGo to next message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
I need to update the driver table for each successful customers table update. In this case I am now actually not sure if the SAVE EXCEPTIONS gives me what I need. If the first update fails (on Customers table), the update to the driver table will not execute right?
Re: BULK COLLECTIONS and CURRENT OF [message #531014 is a reply to message #521841] Fri, 11 November 2011 15:00 Go to previous messageGo to next message
nelsond
Messages: 1
Registered: November 2011
Junior Member
Did you ever get a fix for this issue? I am experiencing the same issue and I have the update working, but it is slow compared to if I just insert the records into a table. Any help would be appreciated.
Thanks,
Dan
Re: BULK COLLECTIONS and CURRENT OF [message #531058 is a reply to message #531014] Sat, 12 November 2011 12:49 Go to previous messageGo to next message
cookiemonster
Messages: 13967
Registered: September 2008
Location: Rainy Manchester
Senior Member
A description of exactly what you are doing would make it easier to help.
Re: BULK COLLECTIONS and CURRENT OF [message #531266 is a reply to message #521829] Mon, 14 November 2011 17:21 Go to previous messageGo to next message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
Nelsond,

Yes, I realised my own errors! Wink

Firstly... Originally I wanted to BULK COLLECT and employ a LIMIT of 1000 with a FORALL (see the code in my original post). The problem with this approach though is that I make two updates, one to the target table but I also need to update the driver table showing that I have processed the record.

Re-reading the Oracle documentation I realised that I cannot have two separate DMLS in the same FORALL loop if you are BULK COLLECTING.

To fix this isssue I simply altered the code:

  OPEN cur_get_driver_recs;
  LOOP
    -- process the records using Bulk Collections.
    FETCH cur_get_driver_recs BULK COLLECT INTO driver_records LIMIT 1000;

    FORALL i IN 1..driver_records.COUNT SAVE EXCEPTIONS


to

  OPEN cur_get_driver_recs;
  LOOP
    FETCH cur_get_driver_recs BULK COLLECT INTO driver_records LIMIT 1000;
    EXIT WHEN driver_records.COUNT = 0;

    -- we have multiple DML statements so cannot use FORALL, instead use FOR LOOP
    FOR i IN 1..driver_records.COUNT
    LOOP  



However, this then gave me another problem... I got a fetch out of sequence error after processing the first 1000 rows. This was due to the cursor definition, cur_get_driver_recs , which I had declared 'FOR UPDATE of processed' on.

When I thought about it, this clause obviously opens a transaction in order to lock the processed table. The COMMIT I was using to attempt to commit every 1000 records was ending the transaction opened by the FOR UPDATE (As this transaction was outside of the inner loop of records I wanted to commit I was getting the fetch out of sequence error). Since I was the only process updating the PROCESSED table anyway, I simply removed the FOR UPDATE clause of the CURSOR.

In short, the new code looked like this (apologies for any small errors, I did this from memory using the original code in this thread so there may be some small typo... but you can see the FOR loop in operation with the BULK COLLECT.

  TYPE driver_rec IS RECORD (
    account_no         ext_driver.account_no%TYPE,
    update_action      ext_driver.update_action%TYPE,
    customers_rowid    ext_driver.customers_rowid%TYPE,
    driver_rowid       ext_driver.customers_rowid%TYPE);
    
  TYPE driver_recs_tt IS TABLE OF driver_rec;

  -- cursor to get the records from the driver table
  CURSOR cur_get_driver_recs IS
   SELECT account_no,
          update_action,
          customers_rowid,
          ROWID driver_rowid
     FROM ext_driver
    WHERE processed = 'N'
    
  driver_records  driver_recs_tt;    

BEGIN
  
  OPEN cur_get_driver_recs;
  LOOP
    FETCH cur_get_driver_recs BULK COLLECT INTO driver_records LIMIT 1000;
    EXIT WHEN driver_records.COUNT = 0;

    -- we have multiple DML statements so cannot use FORALL, instead use FOR LOOP
    FOR i IN 1..driver_records.COUNT
    LOOP  
 
      BEGIN
        UPDATE customers c
           SET mailing_option = driver_records(i).update_action
         WHERE a.rowid = driver_records(i).customers_rowid;
  
        -- now update the driver table
        UPDATE driver_table
           SET processed = 'Y'
         WHERE ROWID = driver_records(i).driver_rowid;

      EXCEPTION
        -- if either update fails then we get here. Result is no update to driver
        -- table and record will be processed the next time.
      WHEN OTHERS THEN
         dbms_output.put_line('Update failed for Acct: ' || driver_records(i).account_num ||
                              ' with Error: ' || SQLCODE || ' (' || SQLERRM || ')');
      END;
    END LOOP;

    dbms_output.put_line('Commiting ' || driver_records.COUNT);
    -- commit the batch
    COMMIT;

  END LOOP;  

  CLOSE cur_get_driver_recs;

END;

Re: BULK COLLECTIONS and CURRENT OF [message #531267 is a reply to message #531266] Mon, 14 November 2011 18:24 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
EXCEPTION handler is a bug waiting to bite you.
You'll never know when UPDATE fails.
If UPDATE fails this run, why will is succeed the next run?


remove/eliminate/delete the whole & complete EXCEPTION handler code
Re: BULK COLLECTIONS and CURRENT OF [message #531293 is a reply to message #531267] Tue, 15 November 2011 01:07 Go to previous messageGo to next message
Michel Cadot
Messages: 68767
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
... and read WHEN OTHERS wiki page.

Regards
Michel
Re: BULK COLLECTIONS and CURRENT OF [message #531310 is a reply to message #521829] Tue, 15 November 2011 03:11 Go to previous messageGo to next message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
Hi

Thanks BlackSwan / Michel for your comments. Just a question though.. On a bulk update (like my procedure), I do not really expect anything to go wrong! Wink

The reason for the WHEN OTHERS exception handling is more to allow the process to continue to update records after an error is encountered (I have a limited amount of time to get the work done within a batch schedule).

The procedure is called from a UNIX script which handles output to a log file in the file system, so anything that enters into the exception handling will be captured in the log.

Also, something that is not apparent in my original post is that it is not expected that the procedure can simply be run again (after failures) in order to correct them. The documentation I provided explains that intervention would be required on any failures.

However, given the above, the mechanism does allow the recording of failures and also the identification of records in the driver table that were not updated.

Would another approach be preferable?

Re: BULK COLLECTIONS and CURRENT OF [message #531311 is a reply to message #531266] Tue, 15 November 2011 03:19 Go to previous messageGo to next message
cookiemonster
Messages: 13967
Registered: September 2008
Location: Rainy Manchester
Senior Member
mjm22 wrote on Mon, 14 November 2011 23:21
Re-reading the Oracle documentation I realised that I cannot have two separate DMLS in the same FORALL loop if you are BULK COLLECTING.


While that is true you can do two seperate forall loops on the same array. Might be worth trying both approaches and seeing which is more efficient.
Re: BULK COLLECTIONS and CURRENT OF [message #531314 is a reply to message #531310] Tue, 15 November 2011 03:52 Go to previous message
Michel Cadot
Messages: 68767
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
Quote:
I do not really expect anything to go wrong!

Your assumption is wrong.
What about a memory shortage? A network, OS, memory error? And a thousand of other possible reasons your statement can fail.

Quote:
The reason for the WHEN OTHERS exception handling is more to allow the process to continue to update records after an error is encountered


And how will you know which ones have already been done and which have not? (maybe no output will be displayed or it can't be displayed or no one will read it or...)

Quote:
The procedure is called from a UNIX script which handles output to a log file in the file system, so anything that enters into the exception handling will be captured in the log.


And if the file system is full?

Regards
Michel
Previous Topic: Problem when exporting the .dat file
Next Topic: Formatting the TEXT in exported file
Goto Forum:
  


Current Time: Sat Aug 23 12:56:10 CDT 2025