RE: Crash Recovery Procedure

From: Rosenthal, Jonas S. <JRosenth_at_oxhp.com>
Date: Fri, 23 Jun 2000 21:47:04 -0400
Message-Id: <10537.110351@fatcity.com>

Hi David,

This is a general answer. I know you specified you would like details, but you need to have the following in place in order to implement those details. Whole books have been written to answer your question.

If possible, you should have a test system to practice on which can accomdate your production database. You should be making sure that those backup tapes are reliable and that in the event of a disaster, they will restore correctly. The only way you can do that is by restoring on the test bed and testing your recovery procedures. You are playing with disaster if you don't. This is even more important from the point of view of "practice makes perfect" when it's 3am, your beeper goes off and the CIO expects you to perform like you've had a good nights sleep. If you know the procedures well, you won't have to worry about fumbling around with the manuals trying to figure it out while you are feeling sleep deprived. It's at those odd ours when instinct, not thinking, is the payoff.

2)Secondly, Unless you already have it already, I suggest you obtain the Oracle Press backup and recovery handbook by Rama Valpuri. Thats the bible. It has excellent discussion on backup and recovery scenarious and crash and dump analysis. I also recommend you take the Oracle seminars for crash dumps and backup and recovery. It's really good stuff. Read, read, read the Valpuri book and the Oracle manuals and other books as much as you can every day until you can't take it any longer.

3)Third, to help for those 3am recoveries, you and your sa's need to have a failure matrix on hand to help identify path forward procedures depending on what kinds of failures you have. The worst case scenario is be caught wondering what to do when failure strikes: The matrix should include recoveries of lost datafiles, lost redo logs, rollback segment files, system files, resetlog scenarious, control files, and even combinations. Each shop is different depending on how you run your business so each shop's matrix is different. You and your system/backup admins need to PLAY with these matrixed scenarious on your test bed until you are all blue in the face. There aren't any magic answers other than practice and being in the fire makes perfect, or at least a close facsimile.

4)If you are in a fix and you need to recover, GET SUPPORT ON THE LINE. Let them validate what you think you need to do, especially if you are not sure. Even when you are sure, they can be a good listening post to give you the extra confidence you need that what you are about to do at 3am when you are tired with limited instinct is the correct procedure.

If you tell me more about your system, I can provide you with a starting matrix...although the books already have matrixed many of the scenarious I would provide.

Hope that helps....

Jonas

-----Original Message-----
From: David Lee [mailto:dba_lee_at_hotmail.com] Sent: Friday, June 23, 2000 6:28 PM
To: Multiple recipients of list ORACLE-L Subject: Crash Recovery Procedure

Hi List,
I am looking for a recovery procedure so that when the server or database crashes we can overcome the panic mode and get thing back to normal. I know this is a general question because the procedure depends on backup strategies, the environment and availability requirements.

But in the meantime I would like to see details. Right now we are a 24 x 7 shop and we take hot backups every night to disk then to tape by the SA. The hardware has been very reliable so we have not had server failures yet. But I need to prepare for the worst. So I want to see what guru DBAs would do in the event of a disk failure or accidentally dropped a production table. One of the things I don't feel comfortable about is things are highly dependent on the SAs. We need to change that.

TIA David

Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com

-- 
Author: David Lee
  INET: dba_lee_at_hotmail.com

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may

Received on Fri Jun 23 2000 - 20:47:04 CDT