Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Mailing Lists -> Oracle-L -> RE: RMAN recovery stuck - SOLVED!

RE: RMAN recovery stuck - SOLVED!

From: DENNIS WILLIAMS <DWILLIAMS_at_LIFETOUCH.COM>
Date: Tue, 27 Aug 2002 14:19:18 -0800
Message-ID: <F001.004C0D01.20020827141918@fatcity.com>

Thanks to everyone for your help. I finally was able to complete the RMAN disaster recovery! I am backing up to NFS-mounted disk, which is subsequently copied to tape. Oracle 8.1.6 on Compaq Tru64. Here is what I learned:
1. Even if you use the RMAN catalog to back up your database, you can recover the database from the control file alone. But I wouldn't set the parameter that clears RMAN data from the control file really low. 2. Just because you write RMAN backup files to an NFS-mounted drive doesn't mean RMAN will be able to recover them. That was the bottom line on my "stuck" symptoms. Eventually Oracle support "suggested" that I take NFS out of the picture. I moved the files to local disk and recovery worked fine. Since then my sys admin pushed harder with the system vendor, got further suggestions for things like IP threads, and I am recovering successfully from NFS.
3. You will need the archive log files from around the time of the backup. Evidently when RMAN is doing its online backup, it is counting on using the data in the redo logs that are being produced while the backup is in progress to complete the recovery. If you are using RMAN to back up your archive logs, this isn't an issue. In my case, it seemed to just be doubling the disk space needed for storing archive logs online. 4. Thanks to everyone for the date format suggestions. The one that finally worked was:
set until time "to_date('082620021229','mmddyyyyhh24mi')"; 5. Even after applying the archive logs, RMAN was still not able to complete the recovery. It was probably wanting the active redo logs from the time of the backup. I exited RMAN and fired up svrmgrl and issued "alter database open resetlogs" and I was home free. Recovery was complete. In a true disaster recovery situation where you only have the backup tape, I'm not going to quibble about the last transaction or so. Again, thanks to everyone for their help, and I hope others can learn something from my experiences.
Dennis Williams
DBA
Lifetouch, Inc.
dwilliams_at_lifetouch.com

-----Original Message-----
Sent: Friday, August 16, 2002 12:05 PM
To: 'ORACLE-L_at_fatcity.com'

Thanks Jim. This morning I did something close to what you describe. I just did

run {
allocate channel d1 type disk;
restore database;
}

Same result. It recovers the same set of files and hangs. At least this rules out a lot of things like the time format of time. Now I'm thinking maybe I should kill this RMAN session and issue it again. Since it reports successful recovery of these files, maybe it will go past them and recover more of the files. Thanks to you and everyone for their great suggestions. As long as I have ideas I can keep plugging away at this.

Dennis Williams
DBA
Lifetouch, Inc.
dwilliams_at_lifetouch.com

-----Original Message-----
Sent: Friday, August 16, 2002 9:39 AM
To: Multiple recipients of list ORACLE-L

Dennis,

Try this:

sql> startup mount;
sql> exit

    rman target sys/password nocatalog
then,      

run {
allocate channel d1 type disk;
restore database;
recover database until cancel;
alter database open resetlogs;
}

 Is you're controlfile coming from the RMAN backup set ora are you copying it from the production box?

I discovered the control file is backed-up during an RMAN Level 0 at the beginning of the process before the datafiles, therefore the backed-up control file doesn't know about the backup set being run ( at least this was the case during a database clone attempt).

...JIM... >>> DWILLIAMS_at_LIFETOUCH.COM 8/15/02 6:29:16 PM >>> Oops, I forgot to clarify that I have the production database in archivelog
mode, but the recovery database not in archivelog mode. Dennis Williams
DBA
Lifetouch, Inc.
dwilliams_at_lifetouch.com

-----Original Message-----
Sent: Thursday, August 15, 2002 5:08 PM
To: 'ORACLE-L_at_fatcity.com'

James

   I think you may have put your finger on a possible misconception of mine.
Here is my situation/understanding.

Any ideas would be appreciated. Thanks for everyone's patience while I flail
around with this.
Dennis Williams
DBA
Lifetouch, Inc.
dwilliams_at_lifetouch.com

-----Original Message-----
Sent: Thursday, August 15, 2002 4:08 PM
To: Multiple recipients of list ORACLE-L

Dennis,

This is just a wild guess and I'm probably wrong but I saw in you're original post this DB was not in archivelog mode, try putting it in archivelog mode and running the restore again???

...JIM... >>> DWILLIAMS_at_LIFETOUCH.COM 8/15/02 2:58:31 PM >>> Okay, I implemented everyone's comments and re-executed the RMAN recovery.
Here is what I did and the results.

  1. Action: Removed "alter database open resetlogs" from the run statement. Result: No change.
  2. Action: Added trace=1 to the allocate channel command. Result: No trace file is produced in udump.
  3. Action: Reviewed Note 145624.1 Result: Did not see the solution to my problem. Most of the suggestions seem appropriate to backup rather than recovery jobs.
  4. Action: Added log and debug trace statements to rman invocation line. Result: Produced log and trace file. The trace file at the point of the recovery hang contains the following:

krmxrpc: xc=5372006336 kpurpc2 returned 0 krmxrpc: xc=5372006336 RPC #10 completed immediately RMAN-08523: restoring datafile 00006 to /ora05/ams/data0501.dbf krmxrpc: xc=5372006336 kpurpc2 returned 0 krmxrpc: xc=5372006336 RPC #11 completed immediately RMAN-08523: restoring datafile 00017 to /ora05/ams/rbs01.dbf krmxrpc: xc=5372006336 kpurpc2 returned 0 krmxrpc: xc=5372006336 RPC #12 completed immediately RMAN-08523: restoring datafile 00025 to /ora05/ams/mls_data0401.dbf krmxrpc: xc=5372006336 kpurpc2 returned 3123 krmxrpc: xc=5372006336 starting longrunning RPC #13 to target: DBMS_BACKUP_RESTO
RE.RESTOREBACKUPPIECE
krmxr: xc=5372006336 started long running rpc krmxpoq: xc=5372006336, action="0000013 STARTED", col_l=15, ind=0, sid=13
krmxr: callback returned TRUE, skipping sleep krmxpoq: xc=5372006336, action="0000013 STARTED", col_l=15, ind=0, sid=13
krmxr: sleeping for 1 seconds
krmxpoq: xc=5372006336, action="0000013 STARTED", col_l=15, ind=0, sid=13
krmxr: sleeping for 2 seconds
krmxpoq: xc=5372006336, action="0000013 STARTED", col_l=15, ind=0, sid=13
krmxr: sleeping for 4 seconds
krmxpoq: xc=5372006336, action="0000013 STARTED", col_l=15, ind=0, sid=13
krmxr: sleeping for 8 seconds

And the trace continues with this statement.

Any suggestions would be appreciated, as are the suggestions to this point.
I'm about ready to think it is TAR time.

Dennis Williams
DBA
Lifetouch, Inc.
dwilliams_at_lifetouch.com

-----Original Message-----
Sent: Wednesday, August 14, 2002 2:23 PM To: 'ORACLE-L_at_fatcity.com'

I am trying to perform an RMAN disaster recovery task. While I use an RMAN
catalog to make backups, I am trying to recover using just the control file
information.
Oracle 8.1.6, Compaq/HP Tru64

I start RMAN with

     rman target sys/password nocatalog
then,

     startup mount

run {
set until time "to_date('08/11/2002 01:00:00','MM/DD/YYYY HH24:MI:SS')";
allocate channel d1 type disk;
restore database;
recover database;
alter database open resetlogs;
}

Everything appears normal for awhile. In the alert log RMAN tries to find
each file, doesn't find them. Then it successfully recovers 5 data files
(including system and rollback) and reports success in the alert log. Then .
. . nothing for hours. RMAN doesn't return an error. The RMAN shadow processes are still present but with no CPU consumption. Nothing is written
to the alert log.

     I check V$SESSION_WAIT, and the only entry for the RMAN shadow processes is one is SQL*Net message to client with seconds_in_wait = 0,
state = waited unknown time.

     In V$SYSTEM_EVENT, time_waited and average_wait are zero for all events. The following events have values of total_waits that are increasing:

                                   Increase in total_waits in
10-minutes
   rdbms ipc message               401
   pmon timer                       57
   control file parallel write      56
   SQL*Net message to client        24
   SQL*Net message from client      24
   virtual circuit status            5
   dispatch timer                    3
   smon timer                        1

Archiving is turned off.

I have attempted this recovery many times using different RMAN backup sets,
but the system always hangs at this point. Any ideas would be appreciated.

Dennis Williams
DBA
Lifetouch, Inc.
dwilliams_at_lifetouch.com    

-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.com 
-- 
Author: DENNIS WILLIAMS
  INET: DWILLIAMS_at_LIFETOUCH.COM 

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).
-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.com 
-- 
Author: James Howerton
  INET: jhowerton_at_uabmc.edu 

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).
-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.com 
-- 
Author: DENNIS WILLIAMS
  INET: DWILLIAMS_at_LIFETOUCH.COM 

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).
-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.com
-- 
Author: James Howerton
  INET: jhowerton_at_uabmc.edu

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).
-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.com
-- 
Author: DENNIS WILLIAMS
  INET: DWILLIAMS_at_LIFETOUCH.COM

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).
Received on Tue Aug 27 2002 - 17:19:18 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US