Oracle FAQ Your Portal to the Oracle Knowledge Grid

Home -> Community -> Mailing Lists -> Oracle-L -> Re: Re: RAC cluster down ...

Re: Re: RAC cluster down ...

From: Alex Gorbachev <>
Date: Tue, 13 Nov 2007 00:47:12 -0500
Message-ID: <>


Yes, you definitely need to know some more basics to be able to analyze and troubleshoot.

For now you need to know the following to get you our of crisis situation.

Your database name is powersv2. Database is the set of files on disk - datafiles, controlfiles and etc.
Every node has its own instance. Node oracle1 has instance powersv21 and node oracle2 has instance powersv22. Instance is the set of processes in memory that works with the database. In your RAC, you have two instances (one for each node) that are working with the same database (they shared the same database if you wish).

From what your showed me, it seems that instances are stuck in CRS in UNKNOWN state.
You should do the following to clear it up: crs_stop -f ora.powersv2.powersv21.inst
crs_stop -f ora.powersv2.powersv22.inst

Note that if your instances are already running - that will stop them. Btw, don't use crs_stop/start unless you are absolutely sure what you do. In CRS, you should use srvctl instead. So after you cleaned up as above, try to start them with srvctl:
srvctl start instance -d powersv2 -i powersv21 srvctl start instance -d powersv2 -i powersv22

*If that doesn't work* - start them manually. This will bring you back in service. You already started instance powersv21 according to the output you provided.

So what you need to do is to start those instances on every node: On oracle1 node, set your ORACLE_SID=powersv21 and start it using sqlplus. On oracle2 node, set your ORACLE_SID=powersv22 and start it using sqlplus.

After that you would need to investigate what's wrong with your CRS configuration.

By the way, for your other RAC database, intergen, you can do the same. By default, your instances will be the same as database name plus numeric index. I.e.:
oracle1 - intergen1
oracle2 - intergen2

Good luck!


Alex Gorbachev, Oracle DBA Brewer, The Pythian Group
BAAG party -
Received on Mon Nov 12 2007 - 23:47:12 CST

Original text of this message