NODE CANNOT JOIN THE CLUSTER AFTER reboot - Urgent!

From: Venkat srinivasan <srini.venkat.raman_at_gmail.com>
Date: Sun, 15 Nov 2009 11:38:51 +0530
Message-ID: <c0f7b5d60911142208te7c3d93rf4ce6fb417a547c1_at_mail.gmail.com>



Hi all,

We have 2 node rac on Solaris 10 & oracle 10.2.0.3. Due to power problem the rac nodes got rebooted. Node 1 is not able join the cluster. Node 2 is active and Asm & db is up and running.

I have done ocr check onthe both the nodes. It gives same output. cssd.log says "takeover aborted due to UNKNOWN nodes"

Status of Oracle Cluster Registry is as follows :

         Version                  :          2
         Total space (kbytes)     :     511764
         Used space (kbytes)      :       5252
         Available space (kbytes) :     506512
         ID                       : 1475358475
         Device/File Name         :
/dev/rdsk/c5t600A0B80002A2CE80000047A46A133BAd0s1
                                    Device/File integrity check succeeded
                                    Device/File not configured
         Cluster registry integrity check succeeded

The cssd.log from the bad node as follows

[ CSSD]2009-11-14 18:00:47.601 [9] >TRACE: clssnmConnComplete: connected to
node 2 (con 100822e90), state 3 birth 0, unique 1258196245/1258196245 prevConuni(0)

[ CSSD]2009-11-14 18:00:48.080 [16] >TRACE: clssnmRcfgMgrThread: Connection
complete

[ CSSD]2009-11-14 18:00:48.080 [14] >TRACE: clssnmPollingThread: Connection
complete

[ CSSD]2009-11-14 18:00:48.080 [15] >TRACE: clssnmSendingThread: Connection
complete

[ CSSD]2009-11-14 18:00:55.150 [16] >TRACE: clssnmRcfgMgrThread: Local Join

[ CSSD]2009-11-14 18:00:55.150 [16] >TRACE: clssnmLocalJoinEvent: set
node(2) inactive

[ CSSD]2009-11-14 18:00:55.150 [16] >WARNING: clssnmLocalJoinEvent: takeover
aborted due to UNKNOWN nodes

[ CSSD]2009-11-14 18:01:02.220 [16] >TRACE: clssnmRcfgMgrThread: Local Join

[ CSSD]2009-11-14 18:01:02.220 [16] >TRACE: clssnmLocalJoinEvent: set
node(2) inactive

[ CSSD]2009-11-14 18:01:02.220 [16] >WARNING: clssnmLocalJoinEvent: takeover
aborted due to UNKNOWN nodes

[ CSSD]2009-11-14 18:01:09.280 [16] >TRACE: clssnmRcfgMgrThread: Local Join

[ CSSD]2009-11-14 18:01:09.280 [16] >TRACE: clssnmLocalJoinEvent: set
node(2) inactive

Any pointers on this highly appreciated.

Thanks

Venkat

--
http://www.freelists.org/webpage/oracle-l
Received on Sun Nov 15 2009 - 00:08:51 CST

Original text of this message