Re: RAC VIP Node

From: Sanjay Mishra <>
Date: Tue, 3 Jun 2008 07:20:43 -0700 (PDT)
Message-ID: <>

I opened the ticket with Oracle and here is their reply
checked CRS log, which you provided, and found that VIP resource was locked:
2008-05-30 19:10:04.030: [ CRSRES][1]32Skipping locked resource
2008-05-30 19:10:04.031: [ CRSRES][1]32Skipping locked resource
2008-05-30 19:10:04.048: [ CRSMAIN][1]32CRSD locked during state recovery, please wait.
2008-05-30 19:10:04.080: [ CRSMAIN][1]32CRSD recovered, unlocked.

This issue is a known bug 6054978, which is described in a Note 6054978.8. The Note suggests applying Patch Set or you can apply a patch 6054978. 

The Strange thing is that I had created Dev, QA and even Production env with same exact Oracle and OS version but with no issues. This is integration Env and faced the issue and so not clear on the bug and even not able to see the patch on the site. I will keep updated as It can happen that somebody from the list can give more suggestion or approach

----- Original Message ----
From: Dan Norris <>
To: Sanjay Mishra <>
Sent: Sunday, June 1, 2008 9:48:04 PM
Subject: Re: RAC VIP Node


I forgot that this was just a patchset, but you did say that you invoked netca (which seems like an unusual action for a patchset) which could have modified your listener configuration. I would:
--Confirm that the current listener.ora is the same as the most recent listener.ora backup file (netca always makes backups). --actually start the correct listener manually on node1. The output below shows you started the listener named "listener" and the proper listener name is "listener_node1" as shown in the logging output you included below. On node1, do "lsnrctl start listener_node1" and see what happens. I'm going to guess it won't start based on the output below. If I'm right, the reason for the failure should be much easier to find, if not obvious. 
--If that still doesn't help, and since it seems that your VIP is not able to come online on node1, I would consider taking the whole cluster down and manually bringing up the IP alias (the VIP) on node1. Based on the incomplete information you've given, it's difficult to know if you have somehow encountered an IP address conflict. You shouldn't have, but I also wouldn't have expected to hear you running netca during a patchset, so...


Sanjay Mishra wrote: 
Ip address for Public interface is 30.135.65.X
Also it was fine after CLusterware and Oracle RAC software was installed and Patched.
The only point that I remember is that when I patched the RAC software to 10203 and the configuring Listener before going for ASM, I forgot to start nodeapps and invoked the netca.
Netca failed to start the Listener on Node1 
Name           Type           Target    State     Host
ora....e1.lsnr application    ONLINE    OFFLINE
ora....node1.gsd application    ONLINE    ONLINE    node1
ora....node1.ons application    ONLINE    ONLINE    node1 application    ONLINE    ONLINE    node2
ora....node2.lsnr application    ONLINE    ONLINE    node2
ora....node2.gsd application    ONLINE    ONLINE    node2
ora....node2.ons application    ONLINE    ONLINE    node2 application    ONLINE    ONLINE    node2
Yes u are right the listener is only issue and node1-vip is on Node2
----- Original Message ----
From: Dan Norris <>
Sent: Friday, May 30, 2008 4:27:37 PM
Subject: Re: RAC VIP Node

Looks like this listener is binding to "HOST=node1" instead of "HOST=node1-vip" as it should be. That wouldn't be causing the issue you reported, but should be noted anyway.

Tell me the IP addresses you're using for node1, node2 and node1-vip and node2-vip. You can also provide "crs_stat -p" and the same for to help debug. Sounds like maybe you're not using the right IP ranges?

It looks to me like the listener isn't the problem, just the symptom. The listener depends on the VIP and it seems that your VIP won't come online from the errors below. Hopefully, having the information above will help us (and maybe you) identify the reason for the problem.


Sanjay Mishra wrote: 
$lsnrctl start listener
LSNRCTL for IBM/AIX RISC System/6000: Version - Production on 30-MAY-2008 10:02:32
Copyright (c) 1991, 2006, Oracle.  All rights reserved.
Starting /u01/app/oracle/product/rdbms/10.2/bin/tnslsnr: please wait...
TNSLSNR for IBM/AIX RISC System/6000: Version - Production
System parameter file is /u01/app/oracle/product/rdbms/10.2/network/admin/listener.ora
Log messages written to /u01/app/oracle/product/rdbms/10.2/network/log/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=node1)(PORT=1521)))
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
Alias                     listener
Version                   TNSLSNR for IBM/AIX RISC System/6000: Version - Production
Start Date                30-MAY-2008 10:02:32
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      ON
Listener Parameter File   /u01/app/oracle/product/rdbms/10.2/network/admin/listener.ora
Listener Log File         /u01/app/oracle/product/rdbms/10.2/network/log/listener.log
Listening Endpoints Summary...
The listener supports no services
The command completed successfully

----- Original Message ----
From: Andrew Kerber <>
To: Sanjay Mishra <>
Sent: Friday, May 30, 2008 9:33:53 AM
Subject: Re: RAC VIP Node

When you tried lsnrctl start listener on node 1, what did that say?

On Fri, May 30, 2008 at 8:02 AM, Sanjay Mishra <> wrote:

This Log portion is from Node1 crsd.log. I did tried to mount the Listener on Node as shown below
/home/oracle$srvctl start listener -n Node1 parameters, or failed to bring up VIP (host=node1) parameters, or failed to bring up VIP (host=node1)
CRS-0215: Could not start resource 'ora.node1.LISTENER_NODE1.lsnr'.

Received on Tue Jun 03 2008 - 09:20:43 CDT

Original text of this message