Hemant K Chitale

Subscribe to Hemant K Chitale feed
I am an Oracle Database Specialist in Singapore.
get an rss feed of this blog at http://hemantoracledba.blogspot.com/feeds/posts/default?alt=rss
follow me on twitter : @HemantKChitale
Updated: 5 hours 54 min ago

12cR1 RAC Posts -- 11 : Managing Services in RAC (with PDB) (Video)

Sun, 2017-05-21 11:01
I have uploaded a video on managing services (add, start, check, remove) in a 12c RAC database with a Pluggable Database.
.
.
.


Categories: DBA Blogs

12c MultiTenant Posts -- 1 : Creating a PDB in a different directory

Sun, 2017-05-14 04:12
What if the current (default) location for Pluggable Databases is running out of space and, yet, you want to create an Oracle Managed Files Pluggable Database ?

First, I start with this configuration :

SQL> select con_id, file#, substr(name,1,56)
2 from v$datafile
3 order by 1,2;

CON_ID FILE#
---------- ----------
SUBSTR(NAME,1,56)
--------------------------------------------------------------------------------
1 1
/u01/app/oracle/oradata/orcl12c/system01.dbf

1 3
/u01/app/oracle/oradata/orcl12c/sysaux01.dbf

1 7
/u01/app/oracle/oradata/orcl12c/users01.dbf

1 15
/u01/app/oracle/oradata/orcl12c/undotbs2.dbf

2 5
/u01/app/oracle/oradata/orcl12c/pdbseed/system01.dbf

2 6
/u01/app/oracle/oradata/orcl12c/pdbseed/sysaux01.dbf

2 8
/u01/app/oracle/oradata/orcl12c/pdbseed/undotbs01.dbf

3 9
/u01/app/oracle/oradata/orcl12c/orcl/system01.dbf

3 10
/u01/app/oracle/oradata/orcl12c/orcl/sysaux01.dbf

3 11
/u01/app/oracle/oradata/orcl12c/orcl/undotbs01.dbf

3 12
/u01/app/oracle/oradata/orcl12c/orcl/users01.dbf

3 13
/u01/app/oracle/oradata/orcl12c/orcl/APEX_19913751733706

3 14
/u01/app/oracle/oradata/orcl12c/orcl/APEX_19931956603709


13 rows selected.

SQL>


Currently, I have one Pluggable Database (CON_ID=3) but none of the database files are OMF.  I don't have enough space in /u01 and want to create the new PDB in /u02 and also use OMF.  As long as I have /u02/oradata precreated by the system administrator, I can :

SQL> show parameter db_create_file

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest string
SQL> alter session set db_create_file_dest='/u02/oradata';

Session altered.

SQL> create pluggable database NEWPDB admin user newpdb_adm identified by newpdb_adm;

Pluggable database created.

SQL> alter pluggable database NEWPDB open;

Pluggable database altered.

SQL> select con_id, open_mode
2 from v$pdbs
3 where name = 'NEWPDB'
4 /

CON_ID OPEN_MODE
---------- ----------
4 READ WRITE

SQL>
SQL> select file#, name
2 from v$datafile
3 where con_id=4
4 order by file#
5 /

FILE#
----------
NAME
--------------------------------------------------------------------------------
16
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7
f8go_.dbf

17
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7
f8hf_.dbf

18
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dk
j7f8hg_.dbf


SQL>
SQL> show parameter db_unique_name

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_unique_name string orcl12c
SQL>


Note how the parent folder name "ORCL12C" is inherited from the DB_UNIQUE_NAME.  I can now proceed to setup this new PDB.  Later, I  can migrate it as an OMF PDB to another Container Database.
.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 10 : Video on Database Startup

Mon, 2017-05-08 09:38
I've created a short video on manually starting up a RAC database that has a PDB with two custom services.  If a database is shutdown before the cluster is shutdown, the cluster startup does not automatically startup the database instance, so I demonstrate a manual startup.

See the video at:  https://youtu.be/saFvo9QhYSI


.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 9 : Adding a Service to a PDB in RAC

Sat, 2017-04-29 11:02
My 2node RAC environment has 1 PDB.  Here I add (create) a new Service to the PDB.

Oracle recommends using srvctl instead of DBMS_SERVICE to add a Service.

Can I add a service without having the PDB OPEN ?

[oracle@collabn1 ~]$ srvctl start database -d RAC
[oracle@collabn1 ~]$ srvctl add service -db RAC -service MYSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT -pdb PDB
[oracle@collabn1 ~]$
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:16:01 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select con_id, name, open_mode from v$pdbs;

CON_ID NAME OPEN_MODE
---------- ------------------------------ ----------
2 PDB$SEED READ ONLY
3 PDB MOUNTED

SQL> alter pluggable database pdb open; -- command issued in both instances

Pluggable database altered.

SQL>
SQL> select con_id, pdb, name, creation_date, clb_goal
2 from v$services
3 where pdb='PDB'
4 /

CON_ID PDB
---------- ------------------------------
NAME CREATION_ CLB_G
---------------------------------------------------------------- --------- -----
3 PDB
pdb.racattack 29-DEC-16 LONG


SQL>


I add the Service to the TNSNAMES.ORA and try to connect to it.

MYSVC =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = MYSVC.racattack)
)
)

[oracle@collabn1 ~]$ tnsping MYSVC

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 29-APR-2017 23:22:43

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = MYSVC.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ sqlplus hemant/hemant@MYSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:22:51 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-12514: TNS:listener does not currently know of service requested in connect
descriptor


Enter user-name:


So, apparently, the service isn't running.   Shall I try re-adding the service ?

[oracle@collabn1 ~]$ srvctl add service -db RAC -service MYSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT -pdb PDB
PRKO-3117 : Service MYSVC already exists in database RAC
[oracle@collabn1 ~]$ srvctl remove service -db RAC -service MYSVC -pdb PDB
PRKO-2002 : Invalid command line option: -pdb
[oracle@collabn1 ~]$ srvctl remove service -db RAC -service MYSVC
[oracle@collabn1 ~]$ srvctl add service -db RAC -service MYSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT -pdb PDB
[oracle@collabn1 ~]$
[oracle@collabn1 ~]$ sqlplus hemant/hemant@MYSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:31:15 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-12514: TNS:listener does not currently know of service requested in connect
descriptor


Enter user-name:


Yet, it doesn't work !   How do I add and start service to a PDB ?  What's missing ?

[oracle@collabn1 ~]$ srvctl remove service -db RAC -service MYSVC
[oracle@collabn1 ~]$ srvctl start service -db RAC -service MYSVC
[oracle@collabn1 ~]$

SQL> select distinct name from v$services;

NAME
----------------------------------------------------------------
pdb.racattack
RAC.racattack
MYSVC
SYS$BACKGROUND
RAC_DGB
RACXDB
SYS$USERS


[oracle@collabn1 ~]$ sqlplus hemant/hemant@MYSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:43:01 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Sun Apr 16 2017 23:30:21 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA
MY_NEW_TABLE

SQL>


I had forgotten to *START* the service !   Let me go back and add another service with the PDB CLOSEd.

SQL> alter pluggable database pdb close immediate;  -- on both instances

Pluggable database altered.

SQL>

[oracle@collabn1 ~]$ srvctl add service -db RAC -pdb PDB -service NEWSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT
[oracle@collabn1 ~]$

SQL> alter pluggable database pdb open; -- on both instances

Pluggable database altered.

SQL>

NEWSVC =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = NEWSVC.racattack)
)
)

[oracle@collabn1 ~]$ srvctl start service -db RAC -service NEWSVC
[oracle@collabn1 ~]$ tnsping NEWSVC

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 29-APR-2017 23:54:38

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = NEWSVC.racattack)))
OK (0 msec)


[oracle@collabn1 ~]$ sqlplus hemant/hemant@NEWSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:55:25 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Sat Apr 29 2017 23:54:51 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA
MY_NEW_TABLE

SQL>


However, these entries in PDB_PLUG_IN_VIOLATIONS seem to be a bug :

SQL> alter pluggable database pdb close immediate;

Pluggable database altered.

SQL> alter pluggable database pdb open;

Pluggable database altered.

SQL> select * from pdb_plug_in_violations;

TIME
---------------------------------------------------------------------------
NAME
------------------------------
CAUSE TYPE
---------------------------------------------------------------- ---------
ERROR_NUMBER LINE
------------ ----------
MESSAGE
--------------------------------------------------------------------------------
STATUS
---------
ACTION
--------------------------------------------------------------------------------
29-APR-17 11.58.32.409572 PM
PDB
Service Name Conflict WARNING
0 1
Service name or network name of service MYSVC in the PDB is invalid or conflicts
with an existing service name or network name in the CDB.
PENDING
Drop the service and recreate it with an appropriate name.

29-APR-17 11.58.32.412142 PM
PDB
Service Name Conflict WARNING
0 2
Service name or network name of service NEWSVC in the PDB is invalid or conflict
s with an existing service name or network name in the CDB.
PENDING
Drop the service and recreate it with an appropriate name.


SQL>


So, remember, it is not sufficient to ADD a Service. You must also START the Service using srvctl.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8i : Switchback from SingleInstance to RAC

Sun, 2017-04-16 10:59
Earlier this week (10-April), I had done a Switchover from the 2node RAC database to a SingleInstance database.

It is time now to Switchback from SingleInstance to RAC.

First, I check the status of the two databases :

On STBY (the current Primary) :

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
READ WRITE PRIMARY

SQL> select instance_name, host_name from v$instance;

INSTANCE_NAME
----------------
HOST_NAME
----------------------------------------------------------------
STBY
oem132.racattack


SQL>


Next, I check on the RAC database instance RAC1 (the current Standby) :

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
MOUNTED PHYSICAL STANDBY

SQL> select instance_name, status, host_name from gv$instance;

INSTANCE_NAME STATUS
---------------- ------------
HOST_NAME
----------------------------------------------------------------
RAC1 MOUNTED
collabn1.racattack

RAC2 MOUNTED
collabn2.racattack


SQL>


I also confirm that only one of the two RAC instances is doing recovery (in 12.1 we have only 1 instance in RAC doing recovery) by verifying the (automatic) message in the alert log for RAC1 :

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT  NODELAY
Sun Apr 16 23:05:50 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Sun Apr 16 23:05:50 2017
MRP0 started with pid=52, OS id=16739
Sun Apr 16 23:05:50 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Sun Apr 16 23:05:55 2017
Started logmerger process
Sun Apr 16 23:05:56 2017
Managed Standby Recovery starting Real Time Apply
Sun Apr 16 23:06:10 2017
Parallel Media Recovery started with 2 slaves
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY


Now, I add some data to the PDB Pluggable Database currently running in STBY :

[oracle@oem132 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 16 23:29:37 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 10 2017 23:43:30 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select count(*) from my_data;

COUNT(*)
----------
300

SQL> create table my_new_table as select * from my_data;

Table created.

SQL>


I am now ready to switchover from STBY to RAC. However, I have a problem because I started the RAC database (and , therefore, DataGuard Broker) before I  started the SingleInstance node and database.  I have drcRAC1.log and drcRAC2.log both reporting :

04/16/2017 23:05:38
Failed to connect to remote database stby. Error is ORA-12543
Failed to send message to site stby. Error code is ORA-12543.
database rac unable to contact primary database for version check; status ORA-12543
completing bootstrap of this database


The fix is to have the Standby RAC database started *after* the singleInstance Primary and verify that Managed Recovery is restarted in RAC1 :

[oracle@collabn1 ~]$ srvctl stop database -d RAC
[oracle@collabn1 ~]$ srvctl start database -d RAC


ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Sun Apr 16 23:43:58 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Sun Apr 16 23:43:59 2017
MRP0 started with pid=53, OS id=2033
Sun Apr 16 23:43:59 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Sun Apr 16 23:44:03 2017
RFS[2]: Assigned to RFS process (PID:1922)
RFS[2]: Selected log 5 for thread 1 sequence 76 dbid 2519807290 branch 931825279
Sun Apr 16 23:44:04 2017
Started logmerger process
Sun Apr 16 23:44:07 2017
Managed Standby Recovery starting Real Time Apply
RFS[1]: Selected log 7 for thread 1 sequence 78 dbid 2519807290 branch 931825279
Sun Apr 16 23:44:20 2017
Archived Log entry 144 added for thread 1 sequence 77 ID 0x96bd0722 dest 1:
Sun Apr 16 23:44:25 2017
Archived Log entry 145 added for thread 1 sequence 76 ID 0x96bd0722 dest 1:
Sun Apr 16 23:44:27 2017
Parallel Media Recovery started with 2 slaves
Sun Apr 16 23:44:27 2017
Waiting for all non-current ORLs to be archived...
Sun Apr 16 23:44:27 2017
All non-current ORLs have been archived.
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY


Now, recheck the configuration from the SingleInstance node and then  Switchover to RAC :

[oracle@oem132 ~]$ dgmgrl
DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production

Copyright (c) 2000, 2013, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
DGMGRL> connect sys/racattack@STBY
Connected as SYSDBA.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
stby - Primary database
rac - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 16 seconds ago)

DGMGRL> switchover to rac;
Performing switchover NOW, please wait...
Operation requires a connection to instance "RAC1" on database "rac"
Connecting to instance "RAC1"...
Connected as SYSDBA.
New primary database "rac" is opening...
Operation requires start up of instance "STBY" on database "stby"
Starting instance "STBY"...
Unable to connect to database using (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=STBY_DGMGRL)(INSTANCE_NAME=STBY)(SERVER=DEDICATED)))
ORA-12514: TNS:listener does not currently know of service requested in connect descriptor

Failed.
Warning: You are no longer connected to ORACLE.

Please complete the following steps to finish switchover:
start up and mount instance "STBY" of database "stby"

DGMGRL>


The ORA-12514 error here is acceptable. I only need to startup STBY manually.

DGMGRL> exit
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 16 23:51:25 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 289409776 bytes
Database Buffers 541065216 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
MOUNTED PHYSICAL STANDBY

SQL> select instance_name, status, host_name from v$instance;

INSTANCE_NAME STATUS
---------------- ------------
HOST_NAME
----------------------------------------------------------------
STBY MOUNTED
oem132.racattack


SQL>


STBY has now reverted to being a Standby.

Let me check the RAC database instances

[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 16 23:55:28 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
READ WRITE PRIMARY

SQL> select instance_name, status, host_name from gv$instance;

INSTANCE_NAME STATUS
---------------- ------------
HOST_NAME
----------------------------------------------------------------
RAC2 OPEN
collabn2.racattack

RAC1 OPEN
collabn1.racattack


SQL>


Yes, RAC is now Primary with both instances OPEN.

Let me (now on RAC) verify the new table created and populated when STBY was the Primary. 

SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-65019: pluggable database PDB already open


SQL> alter session set container=PDB;

Session altered.

SQL> select table_name from dba_tables where owner = 'HEMANT';

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA
MY_NEW_TABLE

SQL> select count(*) from hemant.my_new_table;

COUNT(*)
----------
300

SQL>


Yes, the data in the Pluggable Database PDB has also come over to RAC.

So, on 10-April, I did a Switchover from RAC (2nodes) to STBY (SingleNode).  Today, I have done a Switchover from STBY to RAC.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8h : DataGuard Switchover (RAC to nonRAC) Messages

Tue, 2017-04-11 10:21
Yesterday, I demonstrated a DataGuard Switchover from a 2node RAC(ASM) database to a SingleInstance(FileSystem) Database.

These are the messages in the alert log and drc log files during the Switchover (Messages from the start of the Switchover to the subsequent restart and shutdown of all three instances.

First on node 1 which is running database instance RAC1. The drcRAC1.log and alert_RAC1.log :

04/10/2017 23:30:58
SWITCHOVER TO stby
Switchover to physical standby cannot be initiated from primary database; redirecting to the switchover target stby
using connect identifier: stby
04/10/2017 23:31:00
SWITCHOVER TO stby
04/10/2017 23:31:02
Notifying Oracle Clusterware to prepare primary database for switchover
04/10/2017 23:31:19
Target standby stby did not have Active Data Guard enabled at the time of switchover.
To maintain Active Data Guard license compliance Block Change Tracking will be disabled.
04/10/2017 23:31:20
Notifying Clusterware to restart this instance for Switchover
04/10/2017 23:31:23
Shutting down instance after CTL_SWITCH
04/10/2017 23:33:20
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
2017-04-10 23:33:20.858 DMON: Attach state object
2017-04-10 23:33:21.407 DMON: rfafoGetLocks reinitializing dubious PMYSHUT lock value block contents: sts=0, flags=0x0, spare1=0x0, spare2=0x0, cksm=0x0, rndm=0x0
2017-04-10 23:33:21.407 DMON: Broker state reconciled, version = 0, state = 00000000
2017-04-10 23:33:21.407 DMON: Broker State Initialized
2017-04-10 23:33:21.407 Version = 1
2017-04-10 23:33:21.407 State = 00000000
2017-04-10 23:33:21.407 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:33:21.422 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:33:24.580 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:33:24.580 DMON: Boot configuration (0.0.0), loading from "+DATA/RAC/dgbroker1.dat"
2017-04-10 23:33:24.856 DMON Registering service RAC_DGB with listener(s)
2017-04-10 23:33:24.857 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:33:24.902 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:33:25
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
Physical standby bootstrap requires Oracle Clusterware buildup
04/10/2017 23:33:39
Notifying Oracle Clusterware to buildup
04/10/2017 23:33:41
rac version check successfully completed
Broker configuration file is current on rac, completing initialization
Creating process RSM0
04/10/2017 23:42:33
Data Guard Broker shutting down
Data Guard Broker - notifying primary of shutdown
posting shutdown message to primary database 0x02001000
RSM0 successfully terminated
2017-04-10 23:42:36.437 >> DMON Process Shutdown <<



=============================================================================================



Mon Apr 10 23:30:59 2017
SWITCHOVER VERIFY: Send VERIFY request to switchover target stby
SWITCHOVER VERIFY COMPLETE
SWITCHOVER VERIFY: Send VERIFY request to switchover target stby
SWITCHOVER VERIFY COMPLETE
ALTER DATABASE SWITCHOVER TO 'stby'
Mon Apr 10 23:31:04 2017
Starting switchover [Process ID: 17791]
Mon Apr 10 23:31:04 2017
ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY [Process Id: 17791] (RAC1)
Waiting for target standby to receive all redo
Mon Apr 10 23:31:04 2017
Waiting for all non-current ORLs to be archived...
Mon Apr 10 23:31:04 2017
All non-current ORLs have been archived.
Mon Apr 10 23:31:04 2017
Waiting for all FAL entries to be archived...
Mon Apr 10 23:31:04 2017
All FAL entries have been archived.
Mon Apr 10 23:31:04 2017
Waiting for dest_id 2 to become synchronized...
Mon Apr 10 23:31:05 2017
Active, synchronized Physical Standby switchover target has been identified
Preventing updates and queries at the Primary
Generating and shipping final logs to target standby
Switchover End-Of-Redo Log thread 1 sequence 70 has been fixed
Switchover End-Of-Redo Log thread 2 sequence 39 has been fixed
Switchover: Primary highest seen SCN set to 0x0.0x3553b3
ARCH: Noswitch archival of thread 2, sequence 39
ARCH: End-Of-Redo Branch archival of thread 2 sequence 39
ARCH: LGWR is actively archiving destination LOG_ARCHIVE_DEST_2
ARCH: Standby redo logfile selected for thread 2 sequence 39 for destination LOG_ARCHIVE_DEST_2
ARCH: Noswitch archival of thread 1, sequence 70
ARCH: End-Of-Redo Branch archival of thread 1 sequence 70
ARCH: LGWR is actively archiving destination LOG_ARCHIVE_DEST_2
ARCH: Standby redo logfile selected for thread 1 sequence 70 for destination LOG_ARCHIVE_DEST_2
ARCH: Archiving is disabled due to current logfile archival
Primary will check for some target standby to have received all redo
Waiting for target standby to apply all redo
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rsm0_17791.trc
Converting the primary database to a new standby database
Clearing standby activation ID 2519803190 (0x96312536)
The primary database controlfile was created using the
'MAXLOGFILES 192' clause.
There is space for up to 188 standby redo logfiles
Use the following SQL commands on the standby database to create
standby redo logfiles that match the primary database:
ALTER DATABASE ADD STANDBY LOGFILE 'srl1.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl2.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl3.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl4.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl5.f' SIZE 52428800;
Archivelog for thread 1 sequence 70 required for standby recovery
Archivelog for thread 2 sequence 39 required for standby recovery
Offline data file 5 marked as online during convert to standby or switchover to standby.
Restore of backup may be required if the file is not physically accessible.
Offline data file 7 marked as online during convert to standby or switchover to standby.
Restore of backup may be required if the file is not physically accessible.
Switchover: Primary controlfile converted to standby controlfile succesfully.
Mon Apr 10 23:31:14 2017
Reconfiguration started (old inc 4, new inc 6)
List of instances (total 1) :
1
Dead instances (total 1) :
2
My inst 1
publish big name space - dead or down/up instance detected, invalidate domain 0
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:31:14 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:31:14 2017
Post SMON to start 1st pass IR
Mon Apr 10 23:31:14 2017
Switchover: Complete - Database shutdown required
Mon Apr 10 23:31:14 2017
Sending request(convert to primary database) to switchover target stby
Mon Apr 10 23:31:14 2017
minact-scn: Inst 1 is now the master inc#:4 mmon proc-id:17203 status:0x7
Mon Apr 10 23:31:14 2017
Process (ospid 17189) is suspended due to switchover to physical standby operation.
Mon Apr 10 23:31:14 2017
minact-scn status: grec-scn:0x0000.00000000 gmin-scn:0x0000.0033ccde gcalc-scn:0x0000.0033ccee
minact-scn: master found reconf/inst-rec before recscn scan old-inc#:4 new-inc#:4
Mon Apr 10 23:31:14 2017
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:31:14 2017
Reconfiguration complete (total time 0.6 secs)
Mon Apr 10 23:31:19 2017
Switchover complete. Database shutdown required
Completed: ALTER DATABASE SWITCHOVER TO 'stby'
Target standby stby did not have Active Data Guard enabled at the time of switchover.
To maintain Active Data Guard license compliance Block Change Tracking will be disabled.
Mon Apr 10 23:31:21 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rfs_6513.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rfs_6518.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:23 2017
Clusterware restarting instance for Data Guard Broker operation; shutting down instance now
DMON (ospid: 17306): terminating the instance
Mon Apr 10 23:31:24 2017
Instance terminated by DMON, pid = 17306
Mon Apr 10 23:31:39 2017
Starting ORACLE instance (normal) (OS id: 6761)
Mon Apr 10 23:31:40 2017
CLI notifier numLatches:3 maxDescs:519
Mon Apr 10 23:31:41 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:31:42 2017
**********************************************************************
Mon Apr 10 23:31:42 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Apr 10 23:31:42 2017
Per process system memlock (soft) limit = UNLIMITED
Mon Apr 10 23:31:42 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:31:42 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:31:42 2017
Supported system pagesize(s):
Mon Apr 10 23:31:42 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:31:42 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:31:42 2017
2048K 0 401 0 NONE
Mon Apr 10 23:31:42 2017
RECOMMENDATION:
Mon Apr 10 23:31:42 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:31:42 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.3.70, mac=08-00-27-6f-ef-ba, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=192.168.78.51, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=192.168.78.61, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:4' configured from GPnP for use as a public interface.
[name='eth0:4', type=1, ip=192.168.78.251, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:5' configured from GPnP for use as a public interface.
[name='eth0:5', type=1, ip=192.168.78.252, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Advanced Analytics
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: collabn1.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initRAC1.ora
System parameters with non-default values:
processes = 300
sga_max_size = 800M
spfile = "+DATA/RAC/spfilerac.ora"
control_files = "+DATA/RAC/CONTROLFILE/current.282.931825275"
control_files = "+FRA/RAC/CONTROLFILE/current.256.931825275"
db_block_size = 8192
db_cache_size = 100M
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="stby""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stby" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
log_archive_trace = 0
log_archive_config = "dg_config=(RAC,stby)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
archive_lag_target = 0
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+FRA"
db_recovery_file_dest_size= 4785M
standby_file_management = "MANUAL"
thread = 1
undo_tablespace = "UNDOTBS1"
instance_number = 1
remote_login_passwordfile= "EXCLUSIVE"
db_domain = "racattack"
dispatchers = "(PROTOCOL=TCP) (SERVICE=RACXDB)"
audit_file_dest = "/u01/app/oracle/admin/RAC/adump"
audit_trail = "DB"
db_name = "RAC"
open_cursors = 300
dg_broker_start = TRUE
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
diagnostic_dest = "/u01/app/oracle"
enable_pluggable_database= TRUE
Mon Apr 10 23:31:52 2017
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Mon Apr 10 23:31:52 2017
Cluster communication is configured to use the following interface(s) for this instance
169.254.3.70
cluster interconnect IPC version: Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
Starting background process PMON
Mon Apr 10 23:31:53 2017
PMON started with pid=2, OS id=6880
Starting background process PSP0
Starting background process VKTM
Mon Apr 10 23:31:53 2017
PSP0 started with pid=3, OS id=6882
Mon Apr 10 23:31:55 2017
VKTM started with pid=4, OS id=6892 at elevated (RT) priority
Starting background process GEN0
Mon Apr 10 23:31:55 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Starting background process MMAN
Mon Apr 10 23:31:55 2017
GEN0 started with pid=5, OS id=6896
Mon Apr 10 23:31:55 2017
MMAN started with pid=6, OS id=6898
Starting background process DIAG
Starting background process DBRM
Mon Apr 10 23:31:55 2017
DIAG started with pid=8, OS id=6910
Starting background process VKRM
Mon Apr 10 23:31:55 2017
DBRM started with pid=9, OS id=6912
Starting background process PING
Mon Apr 10 23:31:55 2017
VKRM started with pid=10, OS id=6914
Starting background process ACMS
Mon Apr 10 23:31:55 2017
PING started with pid=11, OS id=6916
Starting background process DIA0
Mon Apr 10 23:31:55 2017
ACMS started with pid=12, OS id=6918
Mon Apr 10 23:31:56 2017
DIA0 started with pid=13, OS id=6920
Starting background process LMON
Mon Apr 10 23:31:56 2017
LMON started with pid=14, OS id=6922
Starting background process LMD0
Starting background process LMS0
Mon Apr 10 23:31:56 2017
LMD0 started with pid=7, OS id=6929
Mon Apr 10 23:31:56 2017
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [1920 - 2560]
Mon Apr 10 23:31:56 2017
LMS0 started with pid=15, OS id=6931 at elevated (RT) priority
Starting background process RMS0
Starting background process LMHB
Mon Apr 10 23:31:56 2017
RMS0 started with pid=16, OS id=6935
Mon Apr 10 23:31:56 2017
LMHB started with pid=17, OS id=6937
Starting background process LCK1
Starting background process DBW0
Mon Apr 10 23:31:56 2017
LCK1 started with pid=18, OS id=6939
Starting background process LGWR
Mon Apr 10 23:31:56 2017
DBW0 started with pid=19, OS id=6941
Starting background process CKPT
Mon Apr 10 23:31:56 2017
LGWR started with pid=20, OS id=6943
Mon Apr 10 23:31:56 2017
CKPT started with pid=21, OS id=6945
Starting background process SMON
Mon Apr 10 23:31:56 2017
SMON started with pid=23, OS id=6949
Starting background process RECO
Starting background process LREG
Mon Apr 10 23:31:56 2017
RECO started with pid=25, OS id=6953
Starting background process PXMN
Mon Apr 10 23:31:56 2017
LREG started with pid=26, OS id=6955
Mon Apr 10 23:31:56 2017
PXMN started with pid=27, OS id=6957
Starting background process RBAL
Mon Apr 10 23:31:56 2017
RBAL started with pid=28, OS id=6959
Starting background process ASMB
Starting background process MMON
Mon Apr 10 23:31:56 2017
ASMB started with pid=29, OS id=6961
Starting background process MMNL
Mon Apr 10 23:31:56 2017
MMON started with pid=30, OS id=6963
Mon Apr 10 23:31:56 2017
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Mon Apr 10 23:31:56 2017
MMNL started with pid=31, OS id=6965
starting up 1 shared server(s) ...
Mon Apr 10 23:31:57 2017
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:839035202) (new connection)
NOTE: ASMB connected to ASM instance +ASM1 osid: 6971 (Flex mode; client id 0xffffffffffffffff)
Mon Apr 10 23:31:58 2017
lmon registered with NM - instance number 1 (internal mem no 0)
Mon Apr 10 23:31:58 2017
NOTE: initiating MARK startup
Starting background process MARK
Mon Apr 10 23:31:58 2017
MARK started with pid=35, OS id=6977
Mon Apr 10 23:31:58 2017
NOTE: MARK has subscribed
Mon Apr 10 23:31:58 2017
Reconfiguration started (old inc 0, new inc 2)
List of instances (total 1) :
1
My inst 1 (I'm a new instance)
Mon Apr 10 23:31:59 2017
Using default pga_aggregate_limit of 2048 MB
Mon Apr 10 23:32:18 2017
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:32:26 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:32:26 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:32:26 2017
Reconfiguration complete (total time 27.7 secs)
Starting background process LCK0
Mon Apr 10 23:32:29 2017
LCK0 started with pid=39, OS id=7084
Starting background process RSMN
Mon Apr 10 23:32:30 2017
RSMN started with pid=40, OS id=7094
Mon Apr 10 23:32:32 2017
Instance started by oraagent
Starting background process DMON
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:32:32 2017
DMON started with pid=41, OS id=7100
Mon Apr 10 23:32:34 2017
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))' SCOPE=MEMORY SID='RAC1';
Mon Apr 10 23:32:35 2017
ALTER SYSTEM SET remote_listener=' collabn-cluster-scan.racattack:1521' SCOPE=MEMORY SID='RAC1';
ALTER DATABASE MOUNT /* db agent *//* {1:46642:1193} */
Mon Apr 10 23:32:36 2017
This instance was first to mount
Mon Apr 10 23:32:38 2017
NOTE: ASMB mounting group 1 (DATA)
NOTE: Assigning number (1,0) to disk (/dev/asm-disk1)
NOTE: Assigning number (1,1) to disk (/dev/asm-disk2)
NOTE: Assigning number (1,2) to disk (/dev/asm-disk5)
SUCCESS: mounted group 1 (DATA)
NOTE: grp 1 disk 0: DATA_0000 path:/dev/asm-disk1
NOTE: grp 1 disk 1: DATA_0001 path:/dev/asm-disk2
NOTE: grp 1 disk 2: DATA_0002 path:/dev/asm-disk5
NOTE: ASMB mounting group 2 (FRA)
Mon Apr 10 23:32:39 2017
NOTE: dependency between database RAC and diskgroup resource ora.DATA.dg is established
Mon Apr 10 23:32:39 2017
NOTE: Assigning number (2,0) to disk (/dev/asm-disk3)
SUCCESS: mounted group 2 (FRA)
NOTE: grp 2 disk 0: FRA_0000 path:/dev/asm-disk3
Mon Apr 10 23:32:40 2017
NOTE: dependency between database RAC and diskgroup resource ora.FRA.dg is established
Mon Apr 10 23:32:46 2017
WARNING: STANDBY_FILE_MANAGEMENT initialization parameter is
not set to the value "AUTO".
This may cause recovery of the standby database to terminate
prior to applying all available redo data.
It may be necessary to use the ALTER DATABASE CREATE DATAFILE
command to add datafiles created on the primary database.
Starting background process TMON
ARCH: STARTING ARCH PROCESSES
Starting background process ARC0
Mon Apr 10 23:32:48 2017
TMON started with pid=44, OS id=7628
Mon Apr 10 23:32:48 2017
ARC0 started with pid=45, OS id=7630
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:32:48 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Starting background process ARC2
Mon Apr 10 23:32:48 2017
ARC1 started with pid=46, OS id=7632
Starting background process ARC3
Mon Apr 10 23:32:48 2017
ARC2 started with pid=47, OS id=7634
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:32:48 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC1: Thread not mounted
Mon Apr 10 23:32:48 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:32:48 2017
ARC3 started with pid=48, OS id=7636
Mon Apr 10 23:32:48 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
ARC0: Thread not mounted
Mon Apr 10 23:32:48 2017
ARC3: Thread not mounted
Mon Apr 10 23:32:49 2017
ARC2: Thread not mounted
Mon Apr 10 23:32:49 2017
Network Resource Management enabled for Process LGWR (pid 6943) for Exadata I/O
Successful mount of redo thread 1, with mount id 2528954836
Mon Apr 10 23:32:49 2017
Allocated 3981120 bytes in shared pool for flashback generation buffer
Starting background process RVWR
Mon Apr 10 23:32:49 2017
RVWR started with pid=49, OS id=7638
Physical Standby Database mounted.
Lost write protection disabled
Mon Apr 10 23:32:58 2017
Archiving previously deferred ORLs (RAC1)
ARCH: End-Of-Redo Branch archival of thread 1 sequence 70
Mon Apr 10 23:33:13 2017
Archived Log entry 136 added for thread 1 sequence 70 ID 0x96312536 dest 1:
ARCH: End-Of-Redo Branch archival of thread 2 sequence 39
Mon Apr 10 23:33:15 2017
Archived Log entry 137 added for thread 2 sequence 39 ID 0x96312536 dest 1:
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:46642:1193} */
Mon Apr 10 23:33:20 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:33:21 2017
INSV started with pid=42, OS id=7855
Starting background process NSV1
Mon Apr 10 23:33:25 2017
NSV1 started with pid=52, OS id=7869
Mon Apr 10 23:33:26 2017
Decreasing number of real time LMS from 1 to 0
Starting background process RSM0
Mon Apr 10 23:33:41 2017
RSM0 started with pid=55, OS id=7929
Mon Apr 10 23:33:41 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Network Resource Management enabled for Process (pid 7909) for Exadata I/O
Primary database is in MAXIMUM PERFORMANCE mode
Mon Apr 10 23:33:46 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:33:46 2017
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
Mon Apr 10 23:33:46 2017
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
RFS[1]: Assigned to RFS process (PID:7909)
RFS[1]: Selected log 5 for thread 1 sequence 72 dbid 2519807290 branch 931825279
Mon Apr 10 23:33:46 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:47 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='RAC1';
Mon Apr 10 23:33:47 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='RAC1';
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Mon Apr 10 23:33:50 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Mon Apr 10 23:33:50 2017
MRP0 started with pid=56, OS id=7988
Mon Apr 10 23:33:50 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Mon Apr 10 23:33:55 2017
Started logmerger process
Mon Apr 10 23:33:57 2017
Managed Standby Recovery starting Real Time Apply
Mon Apr 10 23:33:58 2017
RFS[2]: Assigned to RFS process (PID:8007)
RFS[2]: Opened log for thread 1 sequence 71 dbid 2519807290 branch 931825279
RFS[1]: Selected log 6 for thread 1 sequence 73 dbid 2519807290 branch 931825279
Mon Apr 10 23:33:59 2017
Reconfiguration started (old inc 2, new inc 4)
List of instances (total 2) :
1 2
New instances (total 1) :
2
My inst 1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:33:59 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:34:00 2017
Reconfiguration complete (total time 0.5 secs)
Mon Apr 10 23:34:15 2017
Archived Log entry 138 added for thread 1 sequence 72 ID 0x96bd0722 dest 1:
Mon Apr 10 23:34:16 2017
Archived Log entry 139 added for thread 1 sequence 71 rlc 931825279 ID 0x96bd0722 dest 2:
RFS[2]: Opened log for thread 2 sequence 40 dbid 2519807290 branch 931825279
Mon Apr 10 23:34:23 2017
Archived Log entry 140 added for thread 2 sequence 40 rlc 931825279 ID 0x0 dest 2:
Mon Apr 10 23:34:28 2017
Parallel Media Recovery started with 2 slaves
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Mon Apr 10 23:34:33 2017
Waiting for all non-current ORLs to be archived...
Mon Apr 10 23:34:33 2017
All non-current ORLs have been archived.
Clearing online redo logfile 1 +DATA/RAC/ONLINELOG/group_1.283.931825279

Clearing online log 1 of thread 1 sequence number 69
Mon Apr 10 23:34:50 2017
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 +DATA/RAC/ONLINELOG/group_2.284.931825283

Clearing online log 2 of thread 1 sequence number 70
Mon Apr 10 23:34:51 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:35:00 2017
Increasing number of real time LMS from 0 to 1
Mon Apr 10 23:35:09 2017
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 +DATA/RAC/ONLINELOG/group_3.290.931826413

Clearing online log 3 of thread 2 sequence number 39
Mon Apr 10 23:35:22 2017
Clearing online redo logfile 3 complete
Clearing online redo logfile 4 +DATA/RAC/ONLINELOG/group_4.291.931826417

Clearing online log 4 of thread 2 sequence number 38
Clearing online redo logfile 4 complete
Mon Apr 10 23:35:28 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_38.357.940980599
Mon Apr 10 23:35:28 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_67.355.940978999
Mon Apr 10 23:35:29 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_68.356.940980597
Mon Apr 10 23:35:31 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_69.358.940980603
Mon Apr 10 23:35:32 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_39.360.940980795
Mon Apr 10 23:35:35 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_70.359.940980783
Mon Apr 10 23:35:38 2017
Resetting standby activation ID 0 (0x0)
Mon Apr 10 23:35:39 2017
Media Recovery End-Of-Redo indicator encountered
Mon Apr 10 23:35:39 2017
Media Recovery Continuing
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_71.361.940980839
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_40.363.940980861
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_72.362.940980841
Mon Apr 10 23:35:51 2017
Media Recovery Waiting for thread 1 sequence 73 (in transit)
Mon Apr 10 23:35:51 2017
Recovery of Online Redo Log: Thread 1 Group 6 Seq 73 Reading mem 0
Mem# 0: +DATA/RAC/ONLINELOG/group_6.298.937936361
Mem# 1: +FRA/RAC/ONLINELOG/group_6.304.937936363
Mon Apr 10 23:41:26 2017
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
Mon Apr 10 23:41:28 2017
MRP0: Background Media Recovery cancelled with status 16037
Mon Apr 10 23:41:28 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_pr00_8020.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:41:28 2017
Managed Standby Recovery not using Real Time Apply
Mon Apr 10 23:41:29 2017
Recovery interrupted!
Recovered data files to a consistent state at change 3497232
Mon Apr 10 23:41:29 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 99
Mon Apr 10 23:41:29 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_pr00_8020.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:41:29 2017
MRP0: Background Media Recovery process shutdown (RAC1)
Mon Apr 10 23:41:29 2017
NOTE: Deferred communication with ASM instance
License high water mark = 10
Mon Apr 10 23:41:29 2017
All dispatchers and shared servers shutdown
ALTER DATABASE CLOSE NORMAL
Stopping Emon pool
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC0: Archival stopped
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC2: Relinquishing active heartbeat ARCH role
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC3: Archival stopped
Mon Apr 10 23:41:30 2017
ARC2: Archival stopped
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 5
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 14
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:37 2017
NOTE: De-assigning number (2,0) from disk (/dev/asm-disk3)
SUCCESS: diskgroup FRA was dismounted
NOTE: De-assigning number (1,0) from disk (/dev/asm-disk1)
NOTE: De-assigning number (1,1) from disk (/dev/asm-disk2)
NOTE: De-assigning number (1,2) from disk (/dev/asm-disk5)
SUCCESS: diskgroup DATA was dismounted
NOTE: Database dismounted; ASMB process exiting
NOTE: ASMB clearing idle groups before exit
Stopping background process MARK
Mon Apr 10 23:41:38 2017
NOTE: Shutting down MARK background process
Stopping background process RBAL
Mon Apr 10 23:41:56 2017
Reconfiguration started (old inc 4, new inc 6)
List of instances (total 1) :
1
Dead instances (total 1) :
2
My inst 1
publish big name space - dead or down/up instance detected, invalidate domain 0
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:41:56 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:41:56 2017
Reconfiguration complete (total time 0.2 secs)
Mon Apr 10 23:42:31 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:42:36 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:42:37 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:42:37 2017
Stopping background process VKTM
Mon Apr 10 23:42:42 2017
freeing rdom 0
Mon Apr 10 23:42:44 2017
Instance shutdown complete






Next on node2 running instance RAC2 :

04/10/2017 23:34:35
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
2017-04-10 23:34:35.584 DMON: Attach state object
2017-04-10 23:34:35.991 DMON: Broker State Initialized
2017-04-10 23:34:35.991 Version = 3
2017-04-10 23:34:35.991 State = 00000000
2017-04-10 23:34:35.992 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:34:35.993 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:34:39.031 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:34:39.130 DMON: Boot configuration (0.0.0), loading from "+DATA/RAC/dgbroker1.dat"
2017-04-10 23:34:39.192 DMON Registering service RAC_DGB with listener(s)
2017-04-10 23:34:39.192 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:34:39.193 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:34:39
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
04/10/2017 23:34:47
rac version check successfully completed
Broker configuration file is current on rac, completing initialization
Creating process RSM0
04/10/2017 23:41:41
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:41:44.988 >> DMON Process Shutdown <<



==========================================================================================================



Mon Apr 10 23:31:10 2017
Switchover in progress in another database instance - Database is shutdown automatically
LGWR (ospid: 3335): terminating the instance due to error 16456
Mon Apr 10 23:31:11 2017
Instance terminated by LGWR, pid = 3335
Mon Apr 10 23:33:50 2017
Starting ORACLE instance (normal) (OS id: 8233)
Mon Apr 10 23:33:51 2017
CLI notifier numLatches:3 maxDescs:519
Mon Apr 10 23:33:51 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:33:51 2017
**********************************************************************
Mon Apr 10 23:33:51 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Apr 10 23:33:51 2017
Per process system memlock (soft) limit = UNLIMITED
Mon Apr 10 23:33:51 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:33:51 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:33:51 2017
Supported system pagesize(s):
Mon Apr 10 23:33:51 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:33:51 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:33:51 2017
2048K 0 401 0 NONE
Mon Apr 10 23:33:51 2017
RECOMMENDATION:
Mon Apr 10 23:33:51 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:33:51 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.178.54, mac=08-00-27-c5-68-87, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=192.168.78.52, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=192.168.78.62, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:2' configured from GPnP for use as a public interface.
[name='eth0:2', type=1, ip=192.168.78.253, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Advanced Analytics
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: collabn2.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initRAC2.ora
System parameters with non-default values:
processes = 300
sga_max_size = 800M
spfile = "+DATA/RAC/spfilerac.ora"
control_files = "+DATA/RAC/CONTROLFILE/current.282.931825275"
control_files = "+FRA/RAC/CONTROLFILE/current.256.931825275"
db_block_size = 8192
db_cache_size = 100M
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="stby""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stby" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
log_archive_trace = 0
log_archive_config = "dg_config=(RAC,stby)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
archive_lag_target = 0
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+FRA"
db_recovery_file_dest_size= 4785M
standby_file_management = "MANUAL"
thread = 2
undo_tablespace = "UNDOTBS2"
instance_number = 2
remote_login_passwordfile= "EXCLUSIVE"
db_domain = "racattack"
dispatchers = "(PROTOCOL=TCP) (SERVICE=RACXDB)"
audit_file_dest = "/u01/app/oracle/admin/RAC/adump"
audit_trail = "DB"
db_name = "RAC"
open_cursors = 300
dg_broker_start = TRUE
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
diagnostic_dest = "/u01/app/oracle"
enable_pluggable_database= TRUE
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Mon Apr 10 23:33:55 2017
Cluster communication is configured to use the following interface(s) for this instance
169.254.178.54
cluster interconnect IPC version: Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
Starting background process PMON
Mon Apr 10 23:33:56 2017
PMON started with pid=2, OS id=8257
Starting background process PSP0
Mon Apr 10 23:33:56 2017
PSP0 started with pid=3, OS id=8259
Starting background process VKTM
Mon Apr 10 23:33:57 2017
VKTM started with pid=4, OS id=8261 at elevated (RT) priority
Starting background process GEN0
Mon Apr 10 23:33:57 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Starting background process MMAN
Mon Apr 10 23:33:57 2017
GEN0 started with pid=5, OS id=8265
Mon Apr 10 23:33:57 2017
MMAN started with pid=6, OS id=8267
Starting background process DIAG
Mon Apr 10 23:33:57 2017
Starting background process DBRM
DIAG started with pid=8, OS id=8271
Starting background process VKRM
Mon Apr 10 23:33:57 2017
DBRM started with pid=9, OS id=8273
Starting background process PING
Mon Apr 10 23:33:57 2017
VKRM started with pid=10, OS id=8275
Mon Apr 10 23:33:57 2017
PING started with pid=11, OS id=8277
Starting background process ACMS
Mon Apr 10 23:33:57 2017
Starting background process DIA0
ACMS started with pid=12, OS id=8279
Mon Apr 10 23:33:57 2017
Starting background process LMON
DIA0 started with pid=13, OS id=8281
Mon Apr 10 23:33:57 2017
Starting background process LMD0
LMON started with pid=14, OS id=8283
Mon Apr 10 23:33:57 2017
Starting background process LMS0
LMD0 started with pid=15, OS id=8285
Mon Apr 10 23:33:57 2017
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [1920 - 2560]
Starting background process RMS0
Mon Apr 10 23:33:57 2017
LMS0 started with pid=16, OS id=8287 at elevated (RT) priority
Mon Apr 10 23:33:57 2017
Starting background process LMHB
RMS0 started with pid=17, OS id=8291
Mon Apr 10 23:33:57 2017
Starting background process LCK1
LMHB started with pid=18, OS id=8293
Mon Apr 10 23:33:57 2017
Starting background process DBW0
LCK1 started with pid=19, OS id=8295
Mon Apr 10 23:33:57 2017
DBW0 started with pid=20, OS id=8297
Starting background process LGWR
Mon Apr 10 23:33:57 2017
Starting background process CKPT
LGWR started with pid=21, OS id=8299
Mon Apr 10 23:33:57 2017
CKPT started with pid=22, OS id=8301
Starting background process SMON
Mon Apr 10 23:33:57 2017
SMON started with pid=24, OS id=8305
Starting background process RECO
Mon Apr 10 23:33:57 2017
RECO started with pid=26, OS id=8309
Starting background process LREG
Mon Apr 10 23:33:57 2017
Starting background process PXMN
LREG started with pid=27, OS id=8311
Mon Apr 10 23:33:57 2017
Starting background process RBAL
PXMN started with pid=7, OS id=8313
Starting background process ASMB
Mon Apr 10 23:33:57 2017
RBAL started with pid=28, OS id=8315
Mon Apr 10 23:33:57 2017
ASMB started with pid=29, OS id=8317
Starting background process MMON
Starting background process MMNL
Mon Apr 10 23:33:57 2017
MMON started with pid=30, OS id=8319
Mon Apr 10 23:33:57 2017
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Mon Apr 10 23:33:57 2017
MMNL started with pid=31, OS id=8323
Mon Apr 10 23:33:57 2017
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:593534517) (new connection)
Mon Apr 10 23:33:57 2017
starting up 1 shared server(s) ...
Mon Apr 10 23:33:57 2017
NOTE: ASMB connected to ASM instance +ASM2 osid: 8321 (Flex mode; client id 0xffffffffffffffff)
NOTE: initiating MARK startup
Starting background process MARK
Mon Apr 10 23:33:57 2017
lmon registered with NM - instance number 2 (internal mem no 1)
Mon Apr 10 23:33:57 2017
MARK started with pid=34, OS id=8329
Mon Apr 10 23:33:57 2017
NOTE: MARK has subscribed
Mon Apr 10 23:33:59 2017
Reconfiguration started (old inc 0, new inc 4)
List of instances (total 2) :
1 2
My inst 2 (I'm a new instance)
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Mon Apr 10 23:33:59 2017
* domain 0 valid = 0 according to instance 1
Mon Apr 10 23:33:59 2017
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:33:59 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:33:59 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:34:00 2017
Reconfiguration complete (total time 0.8 secs)
Starting background process LCK0
Mon Apr 10 23:34:00 2017
LCK0 started with pid=38, OS id=8337
Mon Apr 10 23:34:00 2017
Using default pga_aggregate_limit of 2048 MB
Starting background process RSMN
Mon Apr 10 23:34:01 2017
RSMN started with pid=40, OS id=8351
Mon Apr 10 23:34:02 2017
Instance started by oraagent
Starting background process DMON
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:34:02 2017
DMON started with pid=41, OS id=8353
Mon Apr 10 23:34:03 2017
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.62)(PORT=1521))' SCOPE=MEMORY SID='RAC2';
Mon Apr 10 23:34:03 2017
ALTER SYSTEM SET remote_listener=' collabn-cluster-scan.racattack:1521' SCOPE=MEMORY SID='RAC2';
ALTER DATABASE MOUNT /* db agent *//* {1:46642:1276} */
Mon Apr 10 23:34:04 2017
NOTE: ASMB mounting group 1 (DATA)
NOTE: Assigning number (1,0) to disk (/dev/asm-disk1)
NOTE: Assigning number (1,1) to disk (/dev/asm-disk2)
NOTE: Assigning number (1,2) to disk (/dev/asm-disk5)
SUCCESS: mounted group 1 (DATA)
NOTE: grp 1 disk 0: DATA_0000 path:/dev/asm-disk1
NOTE: grp 1 disk 1: DATA_0001 path:/dev/asm-disk2
NOTE: grp 1 disk 2: DATA_0002 path:/dev/asm-disk5
Mon Apr 10 23:34:04 2017
NOTE: dependency between database RAC and diskgroup resource ora.DATA.dg is established
Mon Apr 10 23:34:04 2017
NOTE: ASMB mounting group 2 (FRA)
NOTE: Assigning number (2,0) to disk (/dev/asm-disk3)
SUCCESS: mounted group 2 (FRA)
NOTE: grp 2 disk 0: FRA_0000 path:/dev/asm-disk3
Mon Apr 10 23:34:04 2017
NOTE: dependency between database RAC and diskgroup resource ora.FRA.dg is established
Mon Apr 10 23:34:14 2017
WARNING: STANDBY_FILE_MANAGEMENT initialization parameter is
not set to the value "AUTO".
This may cause recovery of the standby database to terminate
prior to applying all available redo data.
It may be necessary to use the ALTER DATABASE CREATE DATAFILE
command to add datafiles created on the primary database.
Starting background process TMON
Mon Apr 10 23:34:16 2017
ARCH: STARTING ARCH PROCESSES
TMON started with pid=45, OS id=8374
Starting background process ARC0
Mon Apr 10 23:34:16 2017
ARC0 started with pid=46, OS id=8376
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:34:16 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Starting background process ARC2
Mon Apr 10 23:34:16 2017
ARC1 started with pid=47, OS id=8378
Starting background process ARC3
Mon Apr 10 23:34:16 2017
ARC2 started with pid=48, OS id=8380
Mon Apr 10 23:34:16 2017
ARC3 started with pid=49, OS id=8382
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:34:16 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC1: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:34:16 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:34:16 2017
ARC0: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC3: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC2: Thread not mounted
Mon Apr 10 23:34:17 2017
Network Resource Management enabled for Process LGWR (pid 8299) for Exadata I/O
Successful mount of redo thread 2, with mount id 2528954836
Mon Apr 10 23:34:22 2017
Allocated 3981120 bytes in shared pool for flashback generation buffer
Starting background process RVWR
Mon Apr 10 23:34:22 2017
RVWR started with pid=50, OS id=8385
Physical Standby Database mounted.
Lost write protection disabled
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:46642:1276} */
Mon Apr 10 23:34:35 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:34:36 2017
INSV started with pid=42, OS id=8424
Starting background process NSV1
Mon Apr 10 23:34:39 2017
NSV1 started with pid=51, OS id=8432
Starting background process RSM0
Mon Apr 10 23:34:47 2017
RSM0 started with pid=52, OS id=8439
Mon Apr 10 23:34:51 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:34:51 2017
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
Mon Apr 10 23:34:51 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:34:52 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='RAC2';
Mon Apr 10 23:34:52 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='RAC2';
Mon Apr 10 23:41:28 2017
Managed Standby Recovery not using Real Time Apply
Mon Apr 10 23:41:30 2017
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
Mon Apr 10 23:41:37 2017
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
License high water mark = 2
Mon Apr 10 23:41:40 2017
All dispatchers and shared servers shutdown
ALTER DATABASE CLOSE NORMAL
Stopping Emon pool
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:40 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:40 2017
ARCH shutting down
Mon Apr 10 23:41:40 2017
ARCH shutting down
Mon Apr 10 23:41:40 2017
ARC0: Archival stopped
Mon Apr 10 23:41:40 2017
ARC2: Relinquishing active heartbeat ARCH role
ARC2: Archival stopped
Mon Apr 10 23:41:40 2017
ARCH shutting down
ARC3: Archival stopped
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 5
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 15
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:41 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:41:43 2017
NOTE: Deferred communication with ASM instance
Mon Apr 10 23:41:44 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:41:45 2017
NOTE: force a map free for map id 2
Mon Apr 10 23:41:45 2017
Stopping background process VKTM
Mon Apr 10 23:41:45 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:46 2017
NOTE: Shutting down MARK background process
Mon Apr 10 23:41:53 2017
NOTE: ASMB releasing group resources due to shutdown
NOTE: ASMB clearing idle groups before exit
Stopping background process RBAL
Mon Apr 10 23:41:55 2017
freeing rdom 0
Mon Apr 10 23:41:57 2017
Instance shutdown complete





Next, the original Standby (SingleInstance) node : the drcSTBY.log and alert_STBY.log

04/10/2017 23:31:03
Notifying Oracle Clusterware to prepare target standby database for switchover
04/10/2017 23:33:49
Deferring associated archivelog destinations of sites permanently disabled due to Switchover
Notifying Oracle Clusterware to buildup primary database after switchover
Switchover Complete
New Primary Site is named: stby
04/10/2017 23:41:22
Notifying DMON of db close
Notifying RSM0 of db close
04/10/2017 23:41:27
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:41:30.761 >> DMON Process Shutdown <<
04/10/2017 23:41:55
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr1STBY.dat"
dg_broker_config_file2 = "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr2STBY.dat"
2017-04-10 23:41:55.856 DMON: Attach state object
2017-04-10 23:41:55.856 DMON: rfafoGetLocks reinitializing dubious PMYSHUT lock value block contents: sts=0, flags=0x0, spare1=0x0, spare2=0x0, cksm=0x0, rndm=0x0
2017-04-10 23:41:55.857 DMON: Broker state reconciled, version = 0, state = 00000000
2017-04-10 23:41:55.857 DMON: Broker State Initialized
2017-04-10 23:41:55.857 Version = 1
2017-04-10 23:41:55.857 State = 00000000
2017-04-10 23:41:55.857 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:41:55.857 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:41:58.902 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:41:58.902 DMON: Boot configuration (0.0.0), loading from "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr1STBY.dat"
2017-04-10 23:41:58.903 DMON Registering service STBY_DGB with listener(s)
2017-04-10 23:41:58.903 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:41:58.904 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:41:58
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
04/10/2017 23:42:11
Failed to connect to remote database rac. Error is ORA-12528
Failed to send message to site rac. Error code is ORA-12528.
Version Check Results:
Database rac returned ORA-12528
Creating process RSM0
04/10/2017 23:42:35
Processing shutdown notification from database rac, instance 1
Broker operation
Setting SHUTDOWN status for database rac
04/10/2017 23:45:20
Notifying DMON of db close
Notifying RSM0 of db close
04/10/2017 23:45:28
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:45:31.850 >> DMON Process Shutdown <<



=======================================================================================



Mon Apr 10 23:30:59 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY WARNING: no standby database is defined in LOG_ARCHIVE_DEST_n to protect this database if it is converted to a primary database
SWITCHOVER VERIFY COMPLETE
Mon Apr 10 23:31:00 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY WARNING: no standby database is defined in LOG_ARCHIVE_DEST_n to protect this database if it is converted to a primary database
SWITCHOVER VERIFY COMPLETE
Mon Apr 10 23:31:08 2017
RFS[3]: Assigned to RFS process (PID:30383)
RFS[3]: Selected log 7 for thread 2 sequence 39 dbid 2519807290 branch 931825279
Mon Apr 10 23:31:08 2017
Archived Log entry 44 added for thread 2 sequence 39 ID 0x96312536 dest 1:
Mon Apr 10 23:31:08 2017
RFS[4]: Assigned to RFS process (PID:30386)
RFS[4]: Selected log 5 for thread 1 sequence 70 dbid 2519807290 branch 931825279
Mon Apr 10 23:31:09 2017
Archived Log entry 45 added for thread 1 sequence 70 ID 0x96312536 dest 1:
Mon Apr 10 23:31:09 2017
Resetting standby activation ID 2519803190 (0x96312536)
Mon Apr 10 23:31:09 2017
Media Recovery End-Of-Redo indicator encountered
Mon Apr 10 23:31:09 2017
Media Recovery Continuing
Media Recovery Waiting for thread 2 sequence 40
Mon Apr 10 23:31:10 2017
RFS[5]: Assigned to RFS process (PID:29959)
RFS[5]: Possible network disconnect with primary database
Mon Apr 10 23:31:15 2017
SWITCHOVER: received request 'ALTER DTABASE COMMIT TO SWITCHOVER TO PRIMARY' from primary database.
Mon Apr 10 23:31:15 2017
ALTER DATABASE SWITCHOVER TO PRIMARY (STBY)
Maximum wait for role transition is 15 minutes.
Switchover: Media recovery is still active
Role Change: Canceling MRP - no more redo to apply
Mon Apr 10 23:31:15 2017
MRP0: Background Media Recovery cancelled with status 16037
Mon Apr 10 23:31:15 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_pr00_29969.trc:
ORA-16037: user requested cancel of managed recovery operation
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
Mon Apr 10 23:31:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_pr00_29969.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:31:16 2017
MRP0: Background Media Recovery process shutdown (STBY)
Mon Apr 10 23:31:17 2017
Role Change: Canceled MRP
Killing 1 processes (PIDS:30372) (all RFS) in order to disallow current and future RFS connections. Requested by OS process 30402
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_rmi_30402.trc
SwitchOver after complete recovery through change 3494835
Online logfile pre-clearing operation disabled by switchover
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413: Thread 2 Group 3 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417: Thread 2 Group 3 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417: Thread 2 Group 4 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421: Thread 2 Group 4 was previously cleared
Standby became primary SCN: 3494833
Switchover: Complete - Database mounted as primary
SWITCHOVER: completed request from primary database.
Mon Apr 10 23:31:21 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:31:21 2017
ALTER SYSTEM SET log_archive_dest_2='service="rac1"','ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="rac" net_timeout=30','valid_for=(online_logfile,all_roles)' SCOPE=BOTH;
Mon Apr 10 23:31:21 2017
ARC0: Becoming the 'no SRL' ARCH
Mon Apr 10 23:31:21 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
ALTER DATABASE OPEN
Mon Apr 10 23:31:21 2017
Data Guard Broker initializing...
Ping without log force is disabled
.
Mon Apr 10 23:31:21 2017
Assigning activation ID 2528970530 (0x96bd0722)
Mon Apr 10 23:31:21 2017
Thread 1 advanced to log sequence 72 (thread open)
Thread 1 opened at log sequence 72
Current log# 2 seq# 72 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283
Current log# 2 seq# 72 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287
Successful open of redo thread 1
Mon Apr 10 23:31:21 2017
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 10 23:31:21 2017
SMON: enabling cache recovery
Mon Apr 10 23:31:21 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
Mon Apr 10 23:31:21 2017
Archived Log entry 46 added for thread 1 sequence 71 ID 0x96bd0722 dest 1:
Mon Apr 10 23:31:21 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:21 2017
Error 16456 for archive log file 2 to 'rac1'
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc3_29915.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
FAL[server, ARC3]: Error 16456 creating remote archivelog file 'rac1'
ARC3: FAL archive failed with error 16456. See trace for details
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc3_29915.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Mon Apr 10 23:31:22 2017
ORACLE Instance STBY - Archival Error. Archiver continuing.
Mon Apr 10 23:31:24 2017
[29938] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:4159104 end:4161254 diff:2150 ms (2.2 seconds)
Dictionary check beginning
Dictionary check complete
Verifying minimum file header compatibility (11g) for tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Mon Apr 10 23:31:26 2017
SMON: enabling tx recovery
Starting background process SMCO
Mon Apr 10 23:31:26 2017
SMCO started with pid=33, OS id=30448
Mon Apr 10 23:31:28 2017
Database Characterset is AL32UTF8
Mon Apr 10 23:31:29 2017
Redo thread 2 internally disabled at seq 40 (CKPT)
Mon Apr 10 23:31:30 2017
ARC2: Archiving disabled thread 2 sequence 40
Mon Apr 10 23:31:30 2017
Archived Log entry 47 added for thread 2 sequence 40 ID 0x0 dest 1:
Mon Apr 10 23:31:31 2017
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process AQPC
Mon Apr 10 23:31:37 2017
AQPC started with pid=36, OS id=30543
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Mon Apr 10 23:31:39 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC4
Mon Apr 10 23:31:39 2017
ARC4 started with pid=37, OS id=30553
ARC4: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:32:00 2017
Pluggable database PDB$SEED dictionary check beginning
Pluggable Database PDB$SEED Dictionary check complete
Database Characterset for PDB$SEED is AL32UTF8
Mon Apr 10 23:32:17 2017
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:32:36 2017
Opening pdb PDB$SEED (2) with no Resource Manager plan active
Shutting down archive processes
Mon Apr 10 23:32:40 2017
ARCH shutting down
ARC4: Archival stopped
Mon Apr 10 23:32:45 2017
Pluggable database PDB dictionary check beginning
Pluggable Database PDB Dictionary check complete
Database Characterset for PDB is AL32UTF8
ALTER SYSTEM: Flushing buffer cache inst=0 container=3 local
Mon Apr 10 23:32:58 2017
Cannot start service pdb, reason=-1
Starting background process CJQ0
Mon Apr 10 23:33:13 2017
CJQ0 started with pid=37, OS id=31055
Completed: ALTER DATABASE OPEN
ALTER PLUGGABLE DATABASE ALL OPEN
Mon Apr 10 23:33:15 2017
Database Characterset for PDB is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:33:36 2017
Shared IO Pool defaulting to 36MB. Trying to get it from Buffer Cache for process 29792.
Mon Apr 10 23:33:37 2017
Opening pdb PDB (3) with no Resource Manager plan active
Mon Apr 10 23:33:38 2017
===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================
Mon Apr 10 23:33:38 2017
Pluggable database PDB opened read write
Completed: ALTER PLUGGABLE DATABASE ALL OPEN
Mon Apr 10 23:33:38 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='STBY';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='STBY';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET standby_file_management='MANUAL' SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET archive_lag_target=0 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_max_processes=4 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_min_succeed_dest=1 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:40 2017
ALTER SYSTEM SET db_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY' SCOPE=SPFILE;
Mon Apr 10 23:33:40 2017
ALTER SYSTEM SET log_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY' SCOPE=SPFILE;
Mon Apr 10 23:33:40 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
Mon Apr 10 23:33:41 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:41 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:41 2017
ALTER SYSTEM ARCHIVE LOG
Mon Apr 10 23:33:44 2017
Thread 1 advanced to log sequence 73 (LGWR switch)
Current log# 1 seq# 73 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279
Current log# 1 seq# 73 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281
Mon Apr 10 23:33:46 2017
db_recovery_file_dest_size of 16384 MB is 0.73% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Mon Apr 10 23:33:46 2017
TT00: Standby redo logfile selected for thread 1 sequence 72 for destination LOG_ARCHIVE_DEST_2
Mon Apr 10 23:33:46 2017
Archived Log entry 48 added for thread 1 sequence 72 ID 0x96bd0722 dest 1:
Mon Apr 10 23:33:59 2017
TT00: Standby redo logfile selected for thread 1 sequence 73 for destination LOG_ARCHIVE_DEST_2
Mon Apr 10 23:34:47 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:41:17 2017
Shutting down instance (immediate)
Stopping background process SMCO
Shutting down instance: further logons disabled
Stopping background process CJQ0
Stopping background process MMNL
Stopping background process MMON
License high water mark = 11
ALTER DATABASE CLOSE NORMAL
Mon Apr 10 23:41:21 2017
SMON: disabling tx recovery
Mon Apr 10 23:41:21 2017
Stopping Emon pool
Stopping Emon pool
Mon Apr 10 23:41:22 2017
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:24 2017
ARCH shutting down
Mon Apr 10 23:41:24 2017
ARC2: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
Mon Apr 10 23:41:24 2017
ARC3: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
ARC0: Relinquishing active heartbeat ARCH role
ARC0: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:25 2017
Thread 1 closed at log sequence 73
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:26 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:41:30 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:41:32 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:32 2017
Stopping background process VKTM
Mon Apr 10 23:41:35 2017
Instance shutdown complete
Mon Apr 10 23:41:44 2017
Starting ORACLE instance (normal) (OS id: 1281)
Mon Apr 10 23:41:44 2017
CLI notifier numLatches:3 maxDescs:354
Mon Apr 10 23:41:44 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:41:44 2017
**********************************************************************
Mon Apr 10 23:41:44 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Apr 10 23:41:44 2017
Per process system memlock (soft) limit = 128G
Mon Apr 10 23:41:44 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:41:44 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:41:44 2017
Supported system pagesize(s):
Mon Apr 10 23:41:44 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:41:44 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:41:44 2017
2048K 0 401 0 NONE
Mon Apr 10 23:41:44 2017
RECOMMENDATION:
Mon Apr 10 23:41:44 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:41:44 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
_ksb_restart_policy_times={0,60,120,240}
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
IMODE=BR
ILAT =35
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
NOTE: Using default ASM root directory ASM
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: oem132.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side spfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/spfileSTBY.ora
System parameters with non-default values:
_ksb_restart_policy_times= "0"
_ksb_restart_policy_times= "60"
_ksb_restart_policy_times= "120"
_ksb_restart_policy_times= "240"
sga_target = 800M
control_files = "/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl"
control_files = "/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl"
db_file_name_convert = "+DATA/RAC"
db_file_name_convert = "/u01/app/oracle/oradata/STBY"
db_file_name_convert = "+FRA/RAC"
db_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
log_file_name_convert = "+DATA/RAC"
log_file_name_convert = "/u01/app/oracle/oradata/STBY"
log_file_name_convert = "+FRA/RAC"
log_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="rac1""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="rac" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
fal_server = "rac1"
log_archive_trace = 0
log_archive_config = "dg_config=(rac)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
log_buffer = 5064K
archive_lag_target = 0
db_create_file_dest = "/u01/app/oracle/oradata"
db_recovery_file_dest = "/u01/app/oracle/fast_recovery_area"
db_recovery_file_dest_size= 16G
standby_file_management = "MANUAL"
_compression_compatibility= "12.1.0.2.0"
remote_login_passwordfile= "EXCLUSIVE"
connection_brokers = "((TYPE=DEDICATED)(BROKERS=1))"
connection_brokers = "((TYPE=EMON)(BROKERS=1))"
plsql_warnings = "DISABLE:ALL"
result_cache_max_size = 4M
_gby_vector_aggregation_enabled= TRUE
_optimizer_vector_transformation= TRUE
core_dump_dest = "/u01/app/oracle/diag/rdbms/stby/STBY/cdump"
db_name = "RAC"
db_unique_name = "STBY"
optimizer_mode = "ALL_ROWS"
_optimizer_mode_force = TRUE
_always_anti_join = "CHOOSE"
_optimizer_null_aware_antijoin= TRUE
_optimizer_partial_join_eval= TRUE
_partition_view_enabled = TRUE
_b_tree_bitmap_plans = TRUE
_cpu_to_io = 0
_optimizer_extended_cursor_sharing= "UDO"
_optimizer_extended_cursor_sharing_rel= "SIMPLE"
_optimizer_adaptive_cursor_sharing= TRUE
_optimizer_cost_model = "CHOOSE"
_optimizer_undo_cost_change= "12.1.0.2"
_optimizer_system_stats_usage= TRUE
_new_sort_cost_estimate = TRUE
_complex_view_merging = TRUE
_unnest_subquery = TRUE
_optimizer_unnest_scalar_sq= TRUE
_eliminate_common_subexpr= TRUE
_pred_move_around = TRUE
_convert_set_to_join = FALSE
_px_ual_serial_input = TRUE
_px_minus_intersect = TRUE
_remove_aggr_subquery = TRUE
_optimizer_push_down_distinct= 0
_optimizer_cost_based_transformation= "LINEAR"
_optimizer_squ_bottomup = TRUE
_push_join_predicate = TRUE
_push_join_union_view = TRUE
_push_join_union_view2 = TRUE
_fast_full_scan_enabled = TRUE
_optimizer_skip_scan_enabled= TRUE
_optimizer_join_sel_sanity_check= TRUE
_optim_enhance_nnull_detection= TRUE
_parallel_broadcast_enabled= TRUE
_always_semi_join = "CHOOSE"
_ordered_nested_loop = TRUE
_optimizer_max_permutations= 2000
_nlj_batching_enabled = 1
query_rewrite_enabled = "TRUE"
_mmv_query_rewrite_enabled= TRUE
_local_communication_costing_enabled= TRUE
_left_nested_loops_random= TRUE
_improved_row_length_enabled= TRUE
_index_join_enabled = TRUE
_enable_type_dep_selectivity= TRUE
_improved_outerjoin_card = TRUE
_use_column_stats_for_function= TRUE
_subquery_pruning_enabled= TRUE
_subquery_pruning_mv_enabled= FALSE
_or_expand_nvl_predicate = TRUE
_table_scan_cost_plus_one= TRUE
_cost_equality_semi_join = TRUE
_new_initial_join_orders = TRUE
_optim_peek_user_binds = TRUE
_minimal_stats_aggregation= TRUE
_gs_anti_semi_join_allowed= TRUE
_optim_new_default_join_sel= TRUE
optimizer_dynamic_sampling= 2
_pre_rewrite_push_pred = TRUE
_optimizer_new_join_card_computation= TRUE
_union_rewrite_for_gs = "YES_GSET_MVS"
_generalized_pruning_enabled= TRUE
_optim_adjust_for_part_skews= TRUE
_optimizer_compute_index_stats= TRUE
_optimizer_push_pred_cost_based= TRUE
_optimizer_extend_jppd_view_types= TRUE
_optimizer_filter_pred_pullup= TRUE
_optimizer_connect_by_cost_based= TRUE
_optimizer_connect_by_combine_sw= TRUE
_optimizer_connect_by_elim_dups= TRUE
_connect_by_use_union_all= "TRUE"
_right_outer_hash_enable = TRUE
_replace_virtual_columns = TRUE
skip_unusable_indexes = TRUE
_optimizer_correct_sq_selectivity= TRUE
_optimizer_dim_subq_join_sel= TRUE
_query_rewrite_setopgrw_enable= TRUE
_optimizer_join_order_control= 3
_bloom_filter_enabled = TRUE
_bloom_folding_enabled = TRUE
_bloom_serial_filter = "ON"
_optimizer_join_elimination_enabled= TRUE
_gby_hash_aggregation_enabled= TRUE
_globalindex_pnum_filter_enabled= TRUE
_sql_model_unfold_forloops= "RUN_TIME"
_optimizer_cost_hjsmj_multimatch= TRUE
_optimizer_transitivity_retain= TRUE
_px_pwg_enabled = TRUE
_optimizer_cbqt_no_size_restriction= TRUE
_optimizer_enhanced_filter_push= TRUE
_optimizer_rownum_pred_based_fkr= TRUE
_optimizer_better_inlist_costing= "ALL"
_optimizer_or_expansion = "DEPTH"
_optimizer_outer_to_anti_enabled= TRUE
_optimizer_order_by_elimination_enabled= TRUE
_optimizer_star_tran_in_with_clause= TRUE
_optimizer_sortmerge_join_inequality= TRUE
_selfjoin_mv_duplicates = TRUE
_dimension_skip_null = TRUE
_optimizer_complex_pred_selectivity= TRUE
_bloom_pruning_enabled = TRUE
_first_k_rows_dynamic_proration= TRUE
_optimizer_distinct_elimination= TRUE
_optimizer_multi_level_push_pred= TRUE
_optimizer_group_by_placement= TRUE
_optimizer_distinct_placement= TRUE
_optimizer_coalesce_subqueries= TRUE
_optimizer_enable_density_improvements= TRUE
_optimizer_rownum_bind_default= 10
_optimizer_improve_selectivity= TRUE
_optimizer_native_full_outer_join= "FORCE"
_optimizer_ansi_join_lateral_enhance= TRUE
_optimizer_multi_table_outerjoin= TRUE
_optimizer_null_accepting_semijoin= TRUE
_optimizer_ansi_rearchitecture= TRUE
_optimizer_aggr_groupby_elim= TRUE
_optimizer_enable_extended_stats= TRUE
_pivot_implementation_method= "CHOOSE"
_optimizer_interleave_jppd= TRUE
_optimizer_fkr_index_cost_bias= 10
_optimizer_extended_stats_usage_control= 192
_optimizer_fast_pred_transitivity= TRUE
_optimizer_fast_access_pred_analysis= TRUE
_optimizer_unnest_disjunctive_subq= TRUE
_optimizer_unnest_corr_set_subq= TRUE
_optimizer_distinct_agg_transform= TRUE
_aggregation_optimization_settings= 0
_optimizer_eliminate_filtering_join= TRUE
_optimizer_join_factorization= TRUE
_optimizer_use_cbqt_star_transformation= TRUE
_optimizer_table_expansion= TRUE
_and_pruning_enabled = TRUE
_optimizer_use_feedback = TRUE
_optimizer_gather_feedback= TRUE
_optimizer_try_st_before_jppd= TRUE
_px_partition_scan_enabled= TRUE
_optimizer_false_filter_pred_pullup= TRUE
_optimizer_enable_table_lookup_by_nl= TRUE
_optimizer_cube_join_enabled= TRUE
_optimizer_outer_join_to_inner= TRUE
_optimizer_hybrid_fpwj_enabled= TRUE
_px_object_sampling_enabled= TRUE
_px_concurrent = TRUE
_px_replication_enabled = TRUE
_optimizer_full_outer_join_to_outer= TRUE
_px_filter_parallelized = TRUE
_px_filter_skew_handling = TRUE
_px_groupby_pushdown = "FORCE"
_px_parallelize_expression= TRUE
_optimizer_gather_stats_on_load= TRUE
_optimizer_batch_table_access_by_rowid= TRUE
_px_wif_dfo_declumping = "CHOOSE"
_px_wif_extend_distribution_keys= TRUE
_px_join_skew_handling = TRUE
_px_adaptive_dist_method = "CHOOSE"
_px_partial_rollup_pushdown= "ADAPTIVE"
_optimizer_dsdir_usage_control= 126
_px_cpu_autodop_enabled = TRUE
_px_single_server_enabled= TRUE
_optimizer_use_gtt_session_stats= TRUE
_optimizer_adaptive_plans= TRUE
_optimizer_strans_adaptive_pruning= TRUE
_optimizer_proc_rate_level= "BASIC"
_optimizer_use_histograms= TRUE
_adaptive_window_consolidator_enabled= TRUE
_optimizer_cluster_by_rowid= TRUE
_optimizer_cluster_by_rowid_control= 129
_distinct_agg_optimization_gsets= "CHOOSE"
_px_scalable_invdist = TRUE
_optimizer_reduce_groupby_key= TRUE
_optimizer_cluster_by_rowid_batched= TRUE
_optimizer_inmemory_table_expansion= TRUE
_optimizer_inmemory_gen_pushable_preds= TRUE
_optimizer_inmemory_autodop= TRUE
_optimizer_inmemory_access_path= TRUE
_px_external_table_default_stats= TRUE
_optimizer_nlj_hj_adaptive_join= TRUE
_optimizer_inmemory_bloom_filter= TRUE
_optimizer_inmemory_cluster_aware_dop= TRUE
_optimizer_inmemory_minmax_pruning= TRUE
dg_broker_start = TRUE
_diag_adr_trace_dest = "/u01/app/oracle/diag/rdbms/stby/STBY/trace"
enable_pluggable_database= TRUE
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Starting background process PMON
Mon Apr 10 23:41:46 2017
PMON started with pid=2, OS id=1315
Starting background process PSP0
Mon Apr 10 23:41:46 2017
PSP0 started with pid=3, OS id=1317
Starting background process VKTM
Starting background process GEN0
Mon Apr 10 23:41:47 2017
VKTM started with pid=4, OS id=1321 at elevated (RT) priority
Mon Apr 10 23:41:47 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Mon Apr 10 23:41:47 2017
GEN0 started with pid=5, OS id=1327
Starting background process MMAN
Mon Apr 10 23:41:47 2017
MMAN started with pid=6, OS id=1329
Starting background process DIAG
Starting background process DBRM
Mon Apr 10 23:41:47 2017
DIAG started with pid=8, OS id=1333
Starting background process VKRM
Mon Apr 10 23:41:47 2017
DBRM started with pid=9, OS id=1335
Starting background process DIA0
Mon Apr 10 23:41:47 2017
VKRM started with pid=10, OS id=1337
Starting background process DBW0
Mon Apr 10 23:41:47 2017
DIA0 started with pid=11, OS id=1339
Starting background process LGWR
Mon Apr 10 23:41:47 2017
DBW0 started with pid=12, OS id=1342
Starting background process CKPT
Mon Apr 10 23:41:47 2017
LGWR started with pid=13, OS id=1344
Mon Apr 10 23:41:47 2017
CKPT started with pid=14, OS id=1347
Starting background process SMON
Mon Apr 10 23:41:47 2017
SMON started with pid=16, OS id=1355
Starting background process RECO
Starting background process LREG
Mon Apr 10 23:41:47 2017
RECO started with pid=18, OS id=1359
Starting background process PXMN
Mon Apr 10 23:41:47 2017
LREG started with pid=19, OS id=1361
Starting background process MMON
Mon Apr 10 23:41:47 2017
PXMN started with pid=20, OS id=1363
Starting background process MMNL
Mon Apr 10 23:41:47 2017
MMON started with pid=21, OS id=1365
Starting background process DMON
Mon Apr 10 23:41:47 2017
MMNL started with pid=22, OS id=1367
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:41:47 2017
DMON started with pid=23, OS id=1369
Mon Apr 10 23:41:47 2017
ALTER DATABASE MOUNT
Mon Apr 10 23:41:50 2017
Using default pga_aggregate_limit of 2048 MB
Mon Apr 10 23:41:52 2017
Successful mount of redo thread 1, with mount id 2528976638
Mon Apr 10 23:41:52 2017
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE MOUNT
Mon Apr 10 23:41:55 2017
ALTER DATABASE OPEN
Mon Apr 10 23:41:55 2017
Data Guard Broker initializing...
Mon Apr 10 23:41:55 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:41:55 2017
INSV started with pid=24, OS id=1402
Mon Apr 10 23:41:56 2017
Data Guard Broker initialization complete
Data Guard: verifying database primary role...
Starting background process NSV0
Mon Apr 10 23:41:58 2017
NSV0 started with pid=25, OS id=1410


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Starting background process RSM0
Mon Apr 10 23:42:11 2017
RSM0 started with pid=26, OS id=1452
Mon Apr 10 23:42:12 2017
Data Guard: version check completed
Data Guard: primary role verified
Ping without log force is disabled
.
Starting background process TMON
Mon Apr 10 23:42:12 2017
LGWR: STARTING ARCH PROCESSES
Starting background process ARC0
Mon Apr 10 23:42:12 2017
TMON started with pid=27, OS id=1456
Mon Apr 10 23:42:12 2017
ARC0 started with pid=28, OS id=1460
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:42:12 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Mon Apr 10 23:42:12 2017
ARC1 started with pid=29, OS id=1462
Starting background process ARC2
Starting background process ARC3
Mon Apr 10 23:42:13 2017
ARC2 started with pid=30, OS id=1464
Mon Apr 10 23:42:13 2017
ARC3 started with pid=31, OS id=1466
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:42:13 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
Mon Apr 10 23:42:13 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:42:13 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:42:13 2017
Thread 1 advanced to log sequence 74 (thread open)
Thread 1 opened at log sequence 74
Current log# 2 seq# 74 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283
Current log# 2 seq# 74 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287
Successful open of redo thread 1
Mon Apr 10 23:42:13 2017
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 10 23:42:13 2017
SMON: enabling cache recovery
Mon Apr 10 23:42:13 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Error 12528 for archive log file 2 to 'rac1'
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:13 2017
Archived Log entry 52 added for thread 1 sequence 73 ID 0x96bd0722 dest 1:


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc0_1460.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
FAL[server, ARC0]: Error 12528 creating remote archivelog file 'rac1'
ARC0: FAL archive failed with error 12528. See trace for details
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc0_1460.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Mon Apr 10 23:42:13 2017
ORACLE Instance STBY - Archival Error. Archiver continuing.
Mon Apr 10 23:42:14 2017
[1400] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:4809484 end:4810074 diff:590 ms (0.6 seconds)
Verifying minimum file header compatibility (11g) for tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Mon Apr 10 23:42:14 2017
SMON: enabling tx recovery
Starting background process SMCO
Mon Apr 10 23:42:14 2017
SMCO started with pid=34, OS id=1474
Mon Apr 10 23:42:14 2017
Database Characterset is AL32UTF8
No Resource Manager plan active
Mon Apr 10 23:42:16 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:42:16 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Mon Apr 10 23:42:16 2017
Error 12528 received logging on to the standby
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc2_1464.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
PING[ARC2]: Heartbeat failed to connect to standby 'rac1'. Error is 12528.
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:16 2017
Error 12528 for archive log file 2 to 'rac1'
Mon Apr 10 23:42:16 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Mon Apr 10 23:42:16 2017
Error 12528 received logging on to the standby
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc2_1464.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
PING[ARC2]: Heartbeat failed to connect to standby 'rac1'. Error is 12528.
Mon Apr 10 23:42:17 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='STBY';
Mon Apr 10 23:42:17 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:17 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:17 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='STBY';
Mon Apr 10 23:42:17 2017
replication_dependency_tracking turned off (no async multimaster replication found)
Mon Apr 10 23:42:18 2017
Thread 1 advanced to log sequence 75 (LGWR switch)
Current log# 1 seq# 75 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279
Current log# 1 seq# 75 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281
Mon Apr 10 23:42:18 2017
Archived Log entry 53 added for thread 1 sequence 74 ID 0x96bd0722 dest 1:
Starting background process AQPC
Mon Apr 10 23:42:18 2017
AQPC started with pid=37, OS id=1489
Mon Apr 10 23:42:21 2017
Database Characterset for PDB$SEED is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Opening pdb PDB$SEED (2) with no Resource Manager plan active
Mon Apr 10 23:42:35 2017
ALTER SYSTEM SET log_archive_dest_state_2='RESET' SCOPE=BOTH;
Starting background process CJQ0
Completed: ALTER DATABASE OPEN
Mon Apr 10 23:42:39 2017
CJQ0 started with pid=50, OS id=1904
Mon Apr 10 23:42:58 2017
Shared IO Pool defaulting to 36MB. Trying to get it from Buffer Cache for process 1365.
===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================
Mon Apr 10 23:42:59 2017
db_recovery_file_dest_size of 16384 MB is 0.84% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
alter pluggable database pdb open
Mon Apr 10 23:43:00 2017
Database Characterset for PDB is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:43:15 2017
Opening pdb PDB (3) with no Resource Manager plan active
Pluggable database PDB opened read write
Completed: alter pluggable database pdb open
Mon Apr 10 23:45:07 2017
alter pluggable database pdb close
ALTER SYSTEM: Flushing buffer cache inst=0 container=3 local
Mon Apr 10 23:45:12 2017
Pluggable database PDB closed
Completed: alter pluggable database pdb close
Shutting down instance (immediate)
Stopping background process SMCO
Shutting down instance: further logons disabled
Stopping background process CJQ0
Killed process oracle@oem132.racattack (Q002) with pid is 48, OS pid 1537
Stopping background process MMNL
Stopping background process MMON
License high water mark = 9
ALTER DATABASE CLOSE NORMAL
Mon Apr 10 23:45:20 2017
SMON: disabling tx recovery
Mon Apr 10 23:45:20 2017
Stopping Emon pool
Stopping Emon pool
Mon Apr 10 23:45:20 2017
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:45:24 2017
ARCH shutting down
Mon Apr 10 23:45:24 2017
ARCH shutting down
ARC2: Archival stopped
Mon Apr 10 23:45:24 2017
ARC3: Archival stopped
Mon Apr 10 23:45:24 2017
ARCH shutting down
Mon Apr 10 23:45:24 2017
Mon Apr 10 23:45:24 2017
ARC1: Archival stoppedARCH shutting down

Mon Apr 10 23:45:24 2017
ARC0: Archival stopped
Mon Apr 10 23:45:25 2017
Thread 1 closed at log sequence 75
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:45:27 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:45:31 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:45:33 2017
Mon Apr 10 23:45:33 2017
ARCH: Archival disabled due to shutdown: 1089
Stopping background process VKTM
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:45:41 2017
Instance shutdown complete


Ignore the Fatal NI and TNS errrors.  I plan to address them later.

.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8g : Switchover from RAC Primary to SingleInstance Standby

Mon, 2017-04-10 10:40
Continuing this series of posts where I have a SingleInstance/FileSystem Standby database for a RAC/ASM database ...


Checking the status of the configuration :

DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 28 seconds ago)

DGMGRL>


SQL> select thread#, max(sequence#)
2 from v$archived_log
3 group by thread#
4 order by 1
5 /

THREAD# MAX(SEQUENCE#)
---------- --------------
1 66
2 36

SQL> alter system archive log current;

System altered.


RFS[2]: Selected log 8 for thread 2 sequence 38 dbid 2519807290 branch 931825279
Mon Apr 10 23:03:17 2017
Archived Log entry 39 added for thread 2 sequence 37 ID 0x96312536 dest 1:
Mon Apr 10 23:03:18 2017
Media Recovery Waiting for thread 2 sequence 38 (in transit)
Mon Apr 10 23:03:18 2017
Recovery of Online Redo Log: Thread 2 Group 8 Seq 38 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_8.300.937936389
Mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_8.306.937936389
RFS[1]: Selected log 6 for thread 1 sequence 68 dbid 2519807290 branch 931825279
Mon Apr 10 23:03:23 2017
Archived Log entry 40 added for thread 1 sequence 67 ID 0x96312536 dest 1:
Mon Apr 10 23:03:23 2017
Media Recovery Waiting for thread 1 sequence 68 (in transit)
Mon Apr 10 23:03:23 2017
Recovery of Online Redo Log: Thread 1 Group 6 Seq 68 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_6.298.937936361
Mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_6.304.937936363


The dgmgrl status above is from node1 (collabn1) of the RAC (Primary)database.
The query on v$archived_log is from node2 (collabn2), instance RAC2 of the RAC (Primary) database.
The listing of messages in the alert log are from the Standby database.

Let me add some rows to the data I have in the PDB in the RAC database.

[oracle@collabn1 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 10 23:06:31 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 03 2017 22:49:41 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select count(*) from my_data;

COUNT(*)
----------
100

SQL> insert into my_data select rownum+100, to_char(rownum+100)
2 from dual
3 connect by level < 201;

200 rows created.

SQL> select count(*) from my_data;

COUNT(*)
----------
300

SQL> commit;

Commit complete.

SQL>


Let me try a SWITCHOVER now.

DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 102 seconds ago)

DGMGRL> switchover to stby;
Performing switchover NOW, please wait...
Operation requires a connection to instance "STBY" on database "stby"
Connecting to instance "STBY"...
Connected as SYSDBA.
New primary database "stby" is opening...
Oracle Clusterware is restarting database "rac" ...
Switchover succeeded, new primary is "stby"
DGMGRL>


Querying on the new "standby" that is RAC :

[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 10 23:36:12 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
MOUNTED PHYSICAL STANDBY

SQL> select instance_name, host_name from gv$instance;

INSTANCE_NAME
----------------
HOST_NAME
----------------------------------------------------------------
RAC1
collabn1.racattack

RAC2
collabn2.racattack


SQL>


While, the old Standby :

[oracle@oem132 trace]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 10 23:37:34 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysdba

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select open_mode, databas_role from v$database;
select open_mode, databas_role from v$database
*
ERROR at line 1:
ORA-00904: "DATABAS_ROLE": invalid identifier


SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
READ WRITE PRIMARY

SQL> select instance_name, host_name from gv$instance;

INSTANCE_NAME
----------------
HOST_NAME
----------------------------------------------------------------
STBY
oem132.racattack


SQL>


So, host "oem132.racattack" now has the Primary database in Read Write mode. The instance name is STBY because I chose that to be the instance name through the testing.  (I could have chosen BOSTON and CHICAGO as is used in some Oracle examples / documentation).

Let me verify my data on STBY :

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
Database mounted.
Database opened.
SQL> alter pluggable database pdb open;

Pluggable database altered.

SQL> connect hemant/hemant@PDB
Connected.
SQL> select count(*) from my_data;

COUNT(*)
----------
300

SQL>


The rows inserted at the RAC Primary are now in the Pluggable Database PDB at STBY.

.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 8f : Accessing data in the PDB in the Standby

Sun, 2017-04-09 03:03
Apparently, the error :
< br />
alter pluggable database pdb open
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.

in my previous post was a spurious error.

I am able to access the PDB in the Standby.  Thus :

[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [STBY] ? STBY
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 9 15:57:27 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL> rem here I notice the alert log showing :
SQL> rem ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
SQL> rem Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
SQL> recover managed standby database cancel;
Media recovery complete.
SQL> rem I cancel the automatic recovery as I do not want to run Active Data Guard
SQL> alter database open read only;

Database altered.

SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.


SQL>
SQL> select con_id, name, open_mode, open_Time from v$pdbs;

CON_ID NAME OPEN_MODE
---------- ------------------------------ ----------
OPEN_TIME
---------------------------------------------------------------------------
2 PDB$SEED READ ONLY
09-APR-17 04.00.48.232 PM +08:00

3 PDB READ ONLY
09-APR-17 04.00.59.105 PM +08:00


SQL>
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 9 16:02:02 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 03 2017 22:49:41 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA

SQL> select count(*) from my_data;

COUNT(*)
----------
100

SQL> show con_id;

CON_ID
------------------------------
3
SQL>


Thus, the PDB on the Standby *does* OPEN READ ONLY and I can see the data that had been populated from the Primary.
.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 8e : Redo Shipping and Apply (RAC to nonRAC)

Mon, 2017-04-03 10:22
Continuing the series of posts on the RAC/ASM to SingleInstance/FileSystem  DataGuard configuration ...

With both instances RAC1 and RAC2 running, I have redo shipping from both instances to the single instance STBY.
(In my test server with 3 VMs, it takes a number of minutes to have all 3 instances running and communicating with each other ... so there are spurious ORA and TNS errors until the 3 instances have stabilized. {in fact, it would be best to start the Standby before starting the RAC Primary instances}. I am posting messages after the stabilization).

Thus, on RAC 1 :
Mon Apr 03 22:43:51 2017
Archived Log entry 97 added for thread 1 sequence 59 ID 0x96312536 dest 1:
Mon Apr 03 22:43:52 2017
TT00: Standby redo logfile selected for thread 1 sequence 60 for destination LOG_ARCHIVE_DEST_2

And on RAC 2:
Mon Apr 03 22:44:12 2017
Thread 2 cannot allocate new log, sequence 32
Checkpoint not complete
Current log# 3 seq# 31 mem# 0: +DATA/RAC/ONLINELOG/group_3.290.931826413
Current log# 3 seq# 31 mem# 1: +FRA/RAC/ONLINELOG/group_3.259.931826417
Mon Apr 03 22:44:19 2017
Thread 2 advanced to log sequence 32 (LGWR switch)
Current log# 4 seq# 32 mem# 0: +DATA/RAC/ONLINELOG/group_4.291.931826417
Current log# 4 seq# 32 mem# 1: +FRA/RAC/ONLINELOG/group_4.260.931826421
Mon Apr 03 22:44:20 2017
Archived Log entry 99 added for thread 2 sequence 31 ID 0x96312536 dest 1:
Mon Apr 03 22:44:21 2017
TT00: Standby redo logfile selected for thread 2 sequence 32 for destination LOG_ARCHIVE_DEST_2

And on STBY :
Mon Apr 03 22:43:13 2017
Media Recovery Waiting for thread 2 sequence 31 (in transit)
RFS[2]: Selected log 5 for thread 1 sequence 60 dbid 2519807290 branch 931825279
Mon Apr 03 22:43:52 2017
Archived Log entry 25 added for thread 1 sequence 59 ID 0x96312536 dest 1:
Mon Apr 03 22:44:20 2017
Archived Log entry 26 added for thread 2 sequence 31 rlc 931825279 ID 0x96312536 dest 2:
Mon Apr 03 22:44:20 2017
Media Recovery Log /u01/app/oracle/fast_recovery_area/STBY/archivelog/2017_04_03/o1_mf_2_31_dg4qr2qw_.arc
RFS[1]: Selected log 7 for thread 2 sequence 32 dbid 2519807290 branch 931825279
Mon Apr 03 22:44:21 2017
Media Recovery Log /u01/app/oracle/fast_recovery_area/STBY/archivelog/2017_04_03/o1_mf_1_59_dg4qx70d_.arc
Resize operation completed for file# 3, old size 768000K, new size 778240K
Media Recovery Waiting for thread 1 sequence 60 (in transit)
Mon Apr 03 22:44:24 2017
Recovery of Online Redo Log: Thread 1 Group 5 Seq 60 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_5.292.937936339
Mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_5.303.937936343

Thus, Thread1 (instance RAC1) is currently at Sequence#60, Thread2 (instanc RAC2) is currently at Sequence=32.  The STBY alert log shows that it is receiving Redo for both Threads.

Let me login to the PDB in the RAC database and create some data.

[oracle@collabn1 ~]$ tnsping PDB

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 03-APR-2017 22:49:34

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = PDB.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 22:49:39 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 03 2017 22:48:56 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> show con_id

CON_ID
------------------------------
3
SQL> create table my_data (id_col number, data_col varchar2(15));

Table created.

SQL> insert into my_data select rownum, 'Row:' || to_char(rownum)
2 from dual connect by level < 101;

100 rows created.

SQL> commit;

Commit complete.

SQL>


So, user HEMANT in the Pluggable Database PDB has a table with 100 rows.

I open the Standby Database Read Only.

[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [oracle] ? STBY
The Oracle base has been set to /u01/app/oracle
[oracle@oem132 ~]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 22:53:05 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysdba

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> recover managed standby database cancel;
Media recovery complete.
SQL> alter database open;

Database altered.

SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.


SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-65019: pluggable database PDB already open

SQL> alter pluggable database pdb close;

Pluggable database altered.


So, there is an error opening the PDB in the Standby database.  I can't find a note about it on MoS.  This will need more research. Let my try with a Common User in CDB$ROOT.

Restart the Standby.
SQL> shutdown
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 23:14:54 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysdba
Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL> recover managed standby database using current logfile disconnect from session;
Media recovery complete.
SQL>


Create a Common User with data in the RAC Primary.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 23:17:00 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> create user C##HKCCOMMON identified by hemant;

User created.

SQL> grant connect, resource, dba to c##HKCCOMMON;

Grant succeeded.

SQL> connect C##HKCCOMMON/hemant
Connected.
SQL> create table root_my_table (id_col number, data_col varchar2(15));

Table created.

SQL> insert into root_my_table select rownum, 'AA' || to_char(rownum)
2 from dual connect by level < 101;

100 rows created.

SQL> commit;

Commit complete.

SQL>


Now, open the Standby Read Only and verify the ROOT_MY_TABLE.

SQL> recover managed standby database cancel;
Media recovery complete.

SQL> alter database open read only;

Database altered.

SQL> connect c##HKCCOMMON/hemant
Connected.
SQL> select count(*) from root_my_table;

COUNT(*)
----------
100

SQL>


Yes, the CDB ROOT user and data have gone over to the Standby !

.
.


Categories: DBA Blogs

12cR1 RAC Posts -- 8d : Registering the two databases in DataGuard Broker

Mon, 2017-03-27 10:19
Continuing my series on setting up a non-RAC, FileSystem Standby for a RAC, ASM Database....

On the Standby:

[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [oracle] ? STBY
The Oracle base has been set to /u01/app/oracle
[oracle@oem132 ~]$ lsnrctl start

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:24:54

Copyright (c) 1991, 2014, Oracle. All rights reserved.

Starting /u01/app/oracle/product/12.1.0/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 12.1.0.2.0 - Production
System parameter file is /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Log messages written to /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oem132.racattack)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date 27-MAR-2017 22:24:58
Uptime 0 days 0 hr. 0 min. 2 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
Services Summary...
Service "STBY" has 1 instance(s).
Instance "STBY", status UNKNOWN, has 2 handler(s) for this service...
The command completed successfully
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Mar 27 22:25:07 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL>
SQL> show parameter dg_broker

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
dg_broker_config_file1 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr1STBY.dat
dg_broker_config_file2 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr2STBY.dat
dg_broker_start boolean FALSE
SQL>
SQL> alter system set dg_broker_Start=TRUE;

System altered.

SQL>


On the first instance of the Primary :

SQL> show parameter dg_broker

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
dg_broker_config_file1 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr1RAC.dat
dg_broker_config_file2 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr2RAC.dat
dg_broker_start boolean FALSE
SQL> alter system set dg_broker_config_file1='+DATA/RAC/dgbroker1.dat' sid='*' scope=BOTH;

System altered.

SQL> alter system set dg_broker_config_file2='+FRA/RAC/dgbroker2.dat' sid='*' scope=BOTH;

System altered.

SQL> alter system set dg_broker_start=TRUE;

System altered.

SQL>

[oracle@collabn1 ~]$ tnsping RAC1

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:37:41

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = RAC.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ sqlplus sys/racattack@STBY

SQL*Plus: Release 12.1.0.2.0 Production on Mon Mar 27 22:37:51 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-01033: ORACLE initialization or shutdown in progress
Process ID: 0
Session ID: 0 Serial number: 0


Enter user-name: sys/racattack@STBY as SYSDBA

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@collabn1 ~]$ tnsping RAC1

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:38:07

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = RAC.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ tnsping STBY

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:38:10

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = oem132)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = STBY)))
OK (0 msec)
[oracle@collabn1 ~]$ dgmgrl
DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production

Copyright (c) 2000, 2013, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
DGMGRL> connect sys/racattack@RAC1
Connected as SYSDBA.
DGMGRL> create configuration RAC as primary database is RAC connect identifier is RAC1;
Configuration "rac" created with primary database "rac"
DGMGRL> add database stby as connect identifier is stby maintained as physical;
Database "stby" added
DGMGRL> enable configuration;
Enabled.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database
Error: ORA-16664: unable to receive the result from a database

Fast-Start Failover: DISABLED

Configuration Status:
ERROR (status updated 7 seconds ago)

DGMGRL>


Errors in the STBY DataGuard Broker Logfile drcSTBY.log :

03/27/2017 22:45:12
Warning: Property 'StandbyFileManagement' has inconsistent values:METADATA='MANUAL', SPFILE='', DATABASE='MANUAL'
Warning: Property 'ArchiveLagTarget' has inconsistent values:METADATA='0', SPFILE='', DATABASE='0'
Warning: Property 'LogArchiveMaxProcesses' has inconsistent values:METADATA='4', SPFILE='', DATABASE='4'
Warning: Property 'LogArchiveMinSucceedDest' has inconsistent values:METADATA='1', SPFILE='', DATABASE='1'
SPFILE is missing value for property 'LogArchiveTrace' with sid='STBY'
Warning: Property 'LogArchiveTrace' has inconsistent values:METADATA='0', SPFILE='(missing)', DATABASE='0'
SPFILE is missing value for property 'LogArchiveFormat' with sid='STBY'
Warning: Property 'LogArchiveFormat' has inconsistent values:METADATA='%t_%s_%r.dbf', SPFILE='(missing)', DATABASE='%t_%s_%r.dbf'
Failed to send message to site rac. Error code is ORA-16501.

Errors in the RAC1 DataGuard Broker Logfile drcRAC1.log :

Site stby returned ORA-16664.
Data Guard Broker Status Summary:
Type Name Severity Status
Configuration rac Warning ORA-16607
Primary Database rac Success ORA-00000
Physical Standby Database stby Error ORA-16664


Actions taken on the Standby :

SQL> show parameter standby

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
standby_archive_dest string ?/dbs/arch
standby_file_management string MANUAL
SQL> alter system set standby_file_management='AUTO' scope=BOTH;

System altered.

SQL> alter system set archive_lag_target=0 scope=BOTH;

System altered.

SQL> alte system set log_archive_max_processes=4 scope=BOTH;
SP2-0734: unknown command beginning "alte syste..." - rest of line ignored.
SQL> alter system set log_archive_max_processes=4 scope=BOTH;

System altered.

SQL> alter system set log_archive_min_succeed_dest=1 scope=BOTH;

System altered.

SQL> alter system set log_archive_trace=0 scope=BOTH;

System altered.

SQL> alter system set log_archive_format='%t_%s_%r.dbf' scope=BOTH;
alter system set log_archive_format='%t_%s_%r.dbf' scope=BOTH
*
ERROR at line 1:
ORA-02095: specified initialization parameter cannot be modified


SQL>


I also had to edit the TNSNAMES.ORA on the STBY server for the correct *VIP* entry (not the physical hostname) for the RAC1 instance.

RAC1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.78.61)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = RAC.racattack)
)
)

RAC1_DGB =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.78.61)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = RAC_DGMGRL)
)
)


Thereafter, it worked.  Note : You sometimes have to wait a while to enter the SHOW CONFIGURATION, the first result may show an error.  So I entered it twice.

DGMGRL> disable configuration;
Disabled.
DGMGRL> enable configuration;
Enabled.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database
Error: ORA-16664: unable to receive the result from a database

Fast-Start Failover: DISABLED

Configuration Status:
ERROR (status updated 125 seconds ago)

DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 14 seconds ago)

DGMGRL>


These are the drcSTBY.log entries when the TNSNAMES.ORA for RAC1 was incorrect on the Standby Server :

Failed to connect to remote database rac. Error is ORA-12541
Failed to send message to site rac. Error code is ORA-12541.


This is the drcSTBY.log entry on success :

03/27/2017 23:00:51
Apply Instance for Database stby set to STBY


And the entry in drcRAC1.log on success :

03/27/2017 23:00:10
DISABLE CONFIGURATION
03/27/2017 23:00:46
ENABLE CONFIGURATION
03/27/2017 23:00:57
EDIT DATABASE stby SET PROPERTY ActualApplyInstance = STBY
03/27/2017 23:01:00
Apply Instance for database stby is STBY


Setting Maximum Performance mode (command issued on the *Primary*) :

SQL> alter database set standby to maximize performance;

Database altered.

SQL>

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxPerformance;
Succeeded.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 34 seconds ago)

DGMGRL>
DGMGRL> show configuration verbose

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Properties:
FastStartFailoverThreshold = '30'
OperationTimeout = '30'
TraceLevel = 'USER'
FastStartFailoverLagLimit = '30'
CommunicationTimeout = '180'
ObserverReconnect = '0'
FastStartFailoverAutoReinstate = 'TRUE'
FastStartFailoverPmyShutdown = 'TRUE'
BystandersFollowRoleChange = 'ALL'
ObserverOverride = 'FALSE'
ExternalDestination1 = ''
ExternalDestination2 = ''
PrimaryLostWriteAction = 'CONTINUE'

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS

DGMGRL>


Will be testing Redo Transmission and Apply later.
.
.
.

Categories: DBA Blogs

1.5million PageViews

Mon, 2017-03-13 10:01
My Oracle Blog now has had 1.5million PageViews.


The 1million PageViews mark was hit in March 2015.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8c : Ignorable "Errors" during the DUPLICATE

Mon, 2017-03-13 09:56
In yesterday's post, I had shown a DUPLICATE DATABASE from RAC-ASM to SingleInstance-FileSystem.

During the course of the DUPLICATE DATABASE run, the Standby alert log seemingly reported errors.  I chose to ignore the "errors" as I know the DUPLICATE was running successfully.

For your reference, here are some of the good messages :

Sun Mar 12 23:17:55 2017
ALTER SYSTEM SET control_files='/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl','/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl' COMMENT='Set by RMAN' SCOPE=SPFILE;
Sun Mar 12 23:17:59 2017
ALTER SYSTEM SET control_files='/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl','/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl' COMMENT='Set by RMAN' SCOPE=SPFILE;

Sun Mar 12 23:21:41 2017
Switch of datafile 1 complete to datafile copy
checkpoint is 3315193
Switch of datafile 3 complete to datafile copy
checkpoint is 3315211
Switch of datafile 4 complete to datafile copy
checkpoint is 3315270
Switch of datafile 5 complete to datafile copy
checkpoint is 1754712
Switch of datafile 6 complete to datafile copy
checkpoint is 3315280
Switch of datafile 7 complete to datafile copy
checkpoint is 1754712
Switch of datafile 8 complete to datafile copy
checkpoint is 3315276
Switch of datafile 9 complete to datafile copy
checkpoint is 2978862
Switch of datafile 10 complete to datafile copy
checkpoint is 2978862
Switch of datafile 11 complete to datafile copy
checkpoint is 2978862


And here are the errors (about the online redo logs) continuously being reported.  These are messages which I ignored because the Standby has no online redo logs yet.

Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 3 of thread 2
ORA-00312: online log 3 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 3 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 3 of thread 2
ORA-00312: online log 3 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 3 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 4 of thread 2
ORA-00312: online log 4 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 4 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 4 of thread 2
ORA-00312: online log 4 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 4 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 5 of thread 0
ORA-00312: online log 5 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_5.303.937936343'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 5 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_5.292.937936339'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 5 of thread 0
ORA-00312: online log 5 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_5.303.937936343'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 5 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_5.292.937936339'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 6 of thread 0
ORA-00312: online log 6 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_6.304.937936363'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 6 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_6.298.937936361'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 6 of thread 0
ORA-00312: online log 6 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_6.304.937936363'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 6 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_6.298.937936361'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 7 of thread 0
ORA-00312: online log 7 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_7.305.937936377'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 7 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_7.299.937936375'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 7 of thread 0
ORA-00312: online log 7 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_7.305.937936377'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 7 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_7.299.937936375'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 8 of thread 0
ORA-00312: online log 8 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_8.306.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 8 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_8.300.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 8 of thread 0
ORA-00312: online log 8 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_8.306.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 8 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_8.300.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 9 of thread 0
ORA-00312: online log 9 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_9.307.937936405'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 9 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_9.301.937936403'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 9 of thread 0
ORA-00312: online log 9 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_9.307.937936405'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 9 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_9.301.937936403'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3

alter database clear logfile group 1
Clearing online log 1 of thread 1 sequence number 41
Sun Mar 12 23:21:42 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_ora_16474.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3



It is also interesting to see how CONTROL_FILES, DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT are reported in the alert log :

  
control_files = "/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl"
control_files = "/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl"
db_file_name_convert = "+DATA/RAC"
db_file_name_convert = "/u01/app/oracle/oradata/STBY"
db_file_name_convert = "+FRA/RAC"
db_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
log_file_name_convert = "+DATA/RAC"
log_file_name_convert = "/u01/app/oracle/oradata/STBY"
log_file_name_convert = "+FRA/RAC"
log_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"

Each component of the entry in the init parameter file is reported on a separate line in the alert log.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8b : DUPLICATE DATABASE FOR STANDBY

Sun, 2017-03-12 10:27
Continuing the previous post, where I have done the pre-setup...
(A reminder : This is a SingleInstance Standby on FileSystem for a RAC Database with a PDB on ASM)


On the SingleInstance Node (for the Standby database)

Precreate the required folders for the database (and redo and control) files (I later realised that the "onlinelog" folders for the SEED and PDB are not required)

[oracle@oem132 ~]$ cd /u01/app/oracle/oradata
[oracle@oem132 oradata]$ mkdir STBY
[oracle@oem132 oradata]$ cd STBY
[oracle@oem132 STBY]$ mkdir datafile
[oracle@oem132 STBY]$ mkdir onlinelog
[oracle@oem132 STBY]$ mkdir 44bbc69ce8f552aee053334ea8c07365
[oracle@oem132 STBY]$ mkdir fd9ac20f64d244d7e043b6a9e80a2f2f
[oracle@oem132 STBY]$ cd 44*
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ mkdir datafile tempfile
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ cd ../fd*
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$ mkdir datafile tempfile
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$
[oracle@oem132 STBY]$
[oracle@oem132 STBY]$ cd /u01/app/oracle/fast_recovery_area
[oracle@oem132 fast_recovery_area]$ mkdir STBY
[oracle@oem132 fast_recovery_area]$ cd STBY
[oracle@oem132 STBY]$ mkdir onlinelog
[oracle@oem132 STBY]$ mkdir 44bbc69ce8f552aee053334ea8c07365
[oracle@oem132 STBY]$ mkdir fd9ac20f64d244d7e043b6a9e80a2f2f
[oracle@oem132 STBY]$ cd 44*
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ mkdir onlinelog
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ cd ../fd*
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$ mkdir onlinelog
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$
[oracle@oem132 STBY]$

Note that the names of the two folders for the SEED and the PDB are in *lower case*.

Next start the listener for STBY.

[oracle@oem132 STBY]$ cd
[oracle@oem132 ~]$ tail -1 /etc/oratab
STBY:/u01/app/oracle/product/12.1.0/dbhome_1:N
[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [oracle] ? STBY
The Oracle base has been set to /u01/app/oracle
[oracle@oem132 ~]$ ls -l $ORACLE_HOME/dbs/*STBY*
-rw-r--r-- 1 oracle oinstall 508 Mar 6 22:11 /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initSTBY.ora
-rw-r----- 1 oracle oinstall 7680 Mar 6 22:13 /u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwSTBY
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Mar 12 23:03:43 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ lsnrctl start

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 12-MAR-2017 23:04:28

Copyright (c) 1991, 2014, Oracle. All rights reserved.

Starting /u01/app/oracle/product/12.1.0/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 12.1.0.2.0 - Production
System parameter file is /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Log messages written to /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oem132.racattack)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date 12-MAR-2017 23:04:28
Uptime 0 days 0 hr. 0 min. 12 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
Services Summary...
Service "STBY" has 1 instance(s).
Instance "STBY", status UNKNOWN, has 2 handler(s) for this service...
The command completed successfully
[oracle@oem132 ~]$

I am now ready to issue the DUPLICATE DATABASE command from the RAC Primary.

[oracle@collabn1 ~]$ . oraenv
ORACLE_SID = [RAC1] ? RAC1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@collabn1 ~]$ rman

Recovery Manager: Release 12.1.0.2.0 - Production on Sun Mar 12 23:17:27 2017

Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.

RMAN> connect target sys/racattack

connected to target database: RAC (DBID=2519807290)

RMAN> connect auxiliary sys/racattack@STBY

connected to auxiliary database: RAC (not mounted)

RMAN> DUPLICATE TARGET DATABASE FOR STANDBY FROM ACTIVE DATABASE ;

Starting Duplicate Db at 12-MAR-17
using target database control file instead of recovery catalog
allocated channel: ORA_AUX_DISK_1
channel ORA_AUX_DISK_1: SID=12 device type=DISK

contents of Memory Script:
{
backup as copy reuse
targetfile '+DATA/RAC/PASSWORD/pwdrac.277.931824933' auxiliary format
'/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwSTBY' ;
}
executing Memory Script

Starting backup at 12-MAR-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=273 instance=RAC1 device type=DISK
Finished backup at 12-MAR-17

contents of Memory Script:
{
sql clone "alter system set control_files =
''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment=
''Set by RMAN'' scope=spfile";
backup as copy current controlfile for standby auxiliary format '/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl';
restore clone primary controlfile to '/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl' from
'/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl';
sql clone "alter system set control_files =
''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment=
''Set by RMAN'' scope=spfile";
shutdown clone immediate;
startup clone nomount;
}
executing Memory Script

sql statement: alter system set control_files = ''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment= ''Set by RMAN'' scope=spfile

Starting backup at 12-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
copying standby control file
output file name=/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl tag=TAG20170312T231755
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 12-MAR-17

Starting restore at 12-MAR-17
using channel ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: copied control file copy
Finished restore at 12-MAR-17

sql statement: alter system set control_files = ''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment= ''Set by RMAN'' scope=spfile

Oracle instance shut down

connected to auxiliary database (not started)
Oracle instance started

Total System Global Area 838860800 bytes

Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes

contents of Memory Script:
{
sql clone 'alter database mount standby database';
}
executing Memory Script

sql statement: alter database mount standby database

contents of Memory Script:
{
set newname for tempfile 1 to
"/u01/app/oracle/oradata/STBY/tempfile/temp.285.931825311";
set newname for tempfile 2 to
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/pdbseed_temp012016-12-29_12-23-03-am.dbf";
set newname for tempfile 3 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/tempfile/temp.295.931827153";
switch clone tempfile all;
set newname for datafile 1 to
"/u01/app/oracle/oradata/STBY/datafile/system.279.931825083";
set newname for datafile 3 to
"/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019";
set newname for datafile 4 to
"/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149";
set newname for datafile 5 to
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323";
set newname for datafile 6 to
"/u01/app/oracle/oradata/STBY/datafile/users.280.931825149";
set newname for datafile 7 to
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323";
set newname for datafile 8 to
"/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143";
set newname for datafile 9 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089";
set newname for datafile 10 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089";
set newname for datafile 11 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189";
backup as copy reuse
datafile 1 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/system.279.931825083" datafile
3 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019" datafile
4 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149" datafile
5 auxiliary format
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323" datafile
6 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/users.280.931825149" datafile
7 auxiliary format
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323" datafile
8 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143" datafile
9 auxiliary format
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089" datafile
10 auxiliary format
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089" datafile
11 auxiliary format
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189" ;
sql 'alter system archive log current';
}
executing Memory Script

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

renamed tempfile 1 to /u01/app/oracle/oradata/STBY/tempfile/temp.285.931825311 in control file
renamed tempfile 2 to /u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/pdbseed_temp012016-12-29_12-23-03-am.dbf in control file
renamed tempfile 3 to /u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/tempfile/temp.295.931827153 in control file

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

Starting backup at 12-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile file number=00001 name=+DATA/RAC/DATAFILE/system.279.931825083
output file name=/u01/app/oracle/oradata/STBY/datafile/system.279.931825083 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:45
channel ORA_DISK_1: starting datafile copy
input datafile file number=00003 name=+DATA/RAC/DATAFILE/sysaux.278.931825019
output file name=/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00007 name=+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/sysaux.286.931825323
output file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00010 name=+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/sysaux.294.931827089
output file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:25
channel ORA_DISK_1: starting datafile copy
input datafile file number=00009 name=+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/system.293.931827089
output file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00005 name=+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/system.287.931825323
output file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00004 name=+DATA/RAC/DATAFILE/undotbs1.281.931825149
output file name=/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00008 name=+DATA/RAC/DATAFILE/undotbs2.289.931826143
output file name=/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting datafile copy
input datafile file number=00006 name=+DATA/RAC/DATAFILE/users.280.931825149
output file name=/u01/app/oracle/oradata/STBY/datafile/users.280.931825149 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
channel ORA_DISK_1: starting datafile copy
input datafile file number=00011 name=+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/users.296.931827189
output file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 12-MAR-17

sql statement: alter system archive log current

contents of Memory Script:
{
switch clone datafile all;
}
executing Memory Script

datafile 1 switched to datafile copy
input datafile copy RECID=3 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/system.279.931825083
datafile 3 switched to datafile copy
input datafile copy RECID=4 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019
datafile 4 switched to datafile copy
input datafile copy RECID=5 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149
datafile 5 switched to datafile copy
input datafile copy RECID=6 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323
datafile 6 switched to datafile copy
input datafile copy RECID=7 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/users.280.931825149
datafile 7 switched to datafile copy
input datafile copy RECID=8 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323
datafile 8 switched to datafile copy
input datafile copy RECID=9 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143
datafile 9 switched to datafile copy
input datafile copy RECID=10 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089
datafile 10 switched to datafile copy
input datafile copy RECID=11 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089
datafile 11 switched to datafile copy
input datafile copy RECID=12 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189
Finished Duplicate Db at 12-MAR-17

RMAN>

The DUPLICATE ... FOR STANDBY has completed.  Let me just run two verification checks.

On the Standby :

[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Mar 12 23:24:05 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY

SQL> select open_mode from v$database;

OPEN_MODE
--------------------
MOUNTED

SQL>
SQL> select file#, name from v$datafile order by 1;

FILE#
----------
NAME
--------------------------------------------------------------------------------
1
/u01/app/oracle/oradata/STBY/datafile/system.279.931825083

3
/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019

4
/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149

5
/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.28
7.931825323

6
/u01/app/oracle/oradata/STBY/datafile/users.280.931825149

7
/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.28
6.931825323

8
/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143

9
/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.29
3.931827089

10
/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.29
4.931827089

11
/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296
.931827189


10 rows selected.

SQL>
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ cd $ORACLE_HOME/dbs
[oracle@oem132 dbs]$ ls -ltr *STBY*
-rw-r--r-- 1 oracle oinstall 508 Mar 6 22:11 initSTBY.ora
-rw-r----- 1 oracle oinstall 24 Mar 12 23:07 lkSTBY
-rw-r----- 1 oracle oinstall 7680 Mar 12 23:17 orapwSTBY
-rw-rw---- 1 oracle oinstall 1544 Mar 12 23:18 hc_STBY.dat
-rw-r----- 1 oracle oinstall 17920 Mar 12 23:18 spfileSTBY.ora
[oracle@oem132 dbs]$

On the Primary :

RMAN> exit


Recovery Manager complete.
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Mar 12 23:26:17 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PRIMARY

SQL> select open_mode from v$database;

OPEN_MODE
--------------------
READ WRITE

SQL> set pages600
SQL> select file#, name from v$datafile order by 1;

FILE#
----------
NAME
--------------------------------------------------------------------------------
1
+DATA/RAC/DATAFILE/system.279.931825083

3
+DATA/RAC/DATAFILE/sysaux.278.931825019

4
+DATA/RAC/DATAFILE/undotbs1.281.931825149

5
+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/system.287.931825323

6
+DATA/RAC/DATAFILE/users.280.931825149

7
+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/sysaux.286.931825323

8
+DATA/RAC/DATAFILE/undotbs2.289.931826143

9
+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/system.293.931827089

10
+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/sysaux.294.931827089

11
+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/users.296.931827189


10 rows selected.

SQL>

All datafiles (CDB, SEED and 1 PDB) are replicated.  Note how the folder names for the SEED and PDB are in lower-case on the Standby.  Since the source database is RAC, it has two Undo Tablespaces.  (This is 12.1 so I do not have the 12.2 feature of Local Undo in PDBs)

The next few posts will be on the DataGuard Broker and Redo Shipping and Applly.
(Should I also do a SWITCHOVER ?)
.
.
.


Categories: DBA Blogs

12cR1 RAC Posts -- 8a : Setting up SingleInstance DG Standby for RAC

Wed, 2017-03-08 09:08
This is the first of a small series of subposts on setting up SingleInstance DataGuard Standby for my 12cR1 RAC environment.

Primary Database : 12cR1 MultiTenant RAC : 2 nodes, database on ASM
Standby Database : 12cR1 MultiTenant SingleInstance : 1 node, database on FileSystem

The "difficulties" are :
a.  The Primary Database is MultiTenant with 1 PDB besides the SEED
b.  The Primary Database is on ASM but the Standby is to be built on FileSystem
c.  The Primary is a 2-node Cluster while the Standby is a SingleInstance installatin

This post is on the initial setup.

RAC Nodes
collabn1.racattack   192.168.78.51
collabn2.racattack   192.168.78.52

SingleInstance Node 
oem132.racattack     192.168.78.100


Database Changes :
SQL> alter database force logging;

Database altered.

SQL> alter database flashback on;

Database altered.

SQL>
SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL>
SQL>select  group# , status, type, member from v$logfile order by 1;


Listener on Standby Server :
LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = oem132.racattack)(PORT = 1521))
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
    )
  )

SID_LIST_LISTENER=
(SID_LIST =
  (SID_DESC =
    (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1)
    (SID_NAME = STBY)
   )
  (SID_DESC =
    (GLOBAL_NAME = STBY_DGMGRL)
    (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1)
    (SID_NAME = STBY)
   )
)


tnsnames.ora on first RAC Node :
STBY =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = oem132)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = STBY)
    )
  )

STBY_DGB =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = oem132)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = STBY_DGMGRL)
    )
  )


STBY Instance Parameter File :
compatible=12.1.0.2.0
db_name=RAC
db_unique_name=STBY
enable_pluggable_database=TRUE
sga_target=800M
db_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY'
log_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY'
db_create_file_dest='/u01/app/oracle/oradata'
db_recovery_file_dest='/u01/app/oracle/fast_recovery_area'
db_recovery_file_dest_size=16G
remote_login_passwordfile=EXCLUSIVE


Copying the Password File :
ASMCMD> pwcopy +DATA/RAC/PASSWORD/pwdrac.277.931824933 /var/tmp/RACPasswordFile
copying +DATA/RAC/PASSWORD/pwdrac.277.931824933 -> /var/tmp/RACPasswordFile
ASMCMD> exit
scp /var/tmp/RACPasswordFile oracle@oem132:/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwSTBY


to be continued in post 8b on the filesystem directories for the STBY instance and the execution of the DUPLICATE DATABASE command.

Categories: DBA Blogs

12cR1 RAC Posts -- 7 : OCR Commands

Mon, 2017-03-06 03:11
[Yes, I know that 12.2 is now available for download but it will be some time before I have a running 12.2 RAC environment]

Some OCR / OLR Commands :

The OCR is the Cluster Registry.  We also have an OLR that is the Local Registry which is created on a local filesystem.

We can check the consistency of the Registry with ocrcheck.  Note the difference between using oracle (or grid) and using root to run the check.  oracle can't check the OLR and can't do a logical consistency check of the OCR -- both require to be run as root.

[root@collabn1 ~]# su - oracle
[oracle@collabn1 ~]$ . oraenv
ORACLE_SID = [oracle] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[oracle@collabn1 ~]$ ocrcheck -local
PROTL-602: Failed to retrieve data from the local registry
PROCL-26: Error while accessing the physical storage Operating System error [Permission denied] [13]
[oracle@collabn1 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1676
Available space (kbytes) : 407892
ID : 827167720
Device/File Name : +OCRVOTE
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check bypassed due to non-privileged user

[oracle@collabn1 ~]$ su
Password:
[root@collabn1 oracle]# ocrcheck -local
Status of Oracle Local Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1036
Available space (kbytes) : 408532
ID : 1014277103
Device/File Name : /u01/app/12.1.0/grid/cdata/collabn1.olr
Device/File integrity check succeeded

Local registry integrity check succeeded

Logical corruption check succeeded

[root@collabn1 oracle]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1676
Available space (kbytes) : 407892
ID : 827167720
Device/File Name : +OCRVOTE
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@collabn1 oracle]#


Oracle automates backups of the OCR (but not the OLR !).  Below, the -showbackuploc shows the location of backups.

[root@collabn1 oracle]# ocrconfig -showbackuploc
The Oracle Cluster Registry backup location is [/u01/app/12.1.0/grid/cdata/]
[root@collabn1 oracle]# ls -lt /u01/app/12.1.0/grid/cdata
total 1272
-rw-------. 1 root oinstall 503484416 Mar 6 17:03 collabn1.olr
drwxrwxr-x. 2 oracle oinstall 4096 Jan 16 14:12 collabn-cluster
drwxr-xr-x. 2 oracle oinstall 4096 Dec 19 15:06 collabn1
drwxr-xr-x. 2 oracle oinstall 4096 Dec 19 14:37 localhost
[root@collabn1 oracle]# ls -lt /u01/app/12.1.0/grid/cdata/collabn1
total 820
-rw-r--r--. 1 root root 839680 Dec 19 15:06 backup_20161219_150615.olr
[root@collabn1 oracle]# ocrconfig -showbackup

collabn1 2017/01/16 14:09:40 /u01/app/12.1.0/grid/cdata/collabn-cluster/backup00.ocr 0

collabn1 2017/01/16 14:09:40 /u01/app/12.1.0/grid/cdata/collabn-cluster/day.ocr 0

collabn1 2017/01/16 14:09:40 /u01/app/12.1.0/grid/cdata/collabn-cluster/week.ocr 0

collabn2 2016/12/19 15:47:24 /u01/app/12.1.0/grid/cdata/collabn-cluster/backup_20161219_154724.ocr 0

collabn2 2016/12/19 15:47:16 /u01/app/12.1.0/grid/cdata/collabn-cluster/backup_20161219_154716.ocr 0
[root@collabn1 oracle]#


All recent (4-Hourly, Daily, Weekly) of the OCR are on the "master" node -- collabn1 -- which comes up first in my cluster.    The 19-Dec backups (of the OCR and OLR) are when I started setting up the Cluster.  Note that there are no subsequent (automated) OLR backups.
Note : There are no 4-Hourly/Daily/Weekly backups since 16-Jan because I haven't had my cluster running for long enough for those backups to kick in.

[root@collabn1 oracle]# ocrconfig -local -manualbackup

collabn1 2017/03/06 17:11:29 /u01/app/12.1.0/grid/cdata/collabn1/backup_20170306_171129.olr 0

collabn1 2016/12/19 15:06:15 /u01/app/12.1.0/grid/cdata/collabn1/backup_20161219_150615.olr 0
[root@collabn1 oracle]# ocrconfig -manualbackup

collabn1 2017/03/06 17:12:21 /u01/app/12.1.0/grid/cdata/collabn-cluster/backup_20170306_171221.ocr 0

collabn2 2016/12/19 15:47:24 /u01/app/12.1.0/grid/cdata/collabn-cluster/backup_20161219_154724.ocr 0

collabn2 2016/12/19 15:47:16 /u01/app/12.1.0/grid/cdata/collabn-cluster/backup_20161219_154716.ocr 0
[root@collabn1 oracle]#


I can run manual backups (the -local is for the OLR) as shown above.

It is important to include these backups in the backup strategy for the filesystem(s) that hold the Grid Infrastructure and RDBMS installations (binaries, configuration files, trace files etc).
.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 6 :Running the Cluster Verification Utility

Mon, 2017-02-20 03:45
With a successful RAC Cluster, running the Cluster Verification Utility

[oracle@collabn1 ~]$ cluvfy stage -post crsinst -n collabn1,collabn2

Performing post-checks for cluster services setup

Checking node reachability...
Node reachability check passed from node "collabn1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity using interfaces on subnet "192.168.78.0"
Node connectivity passed for subnet "192.168.78.0" with node(s) collabn2,collabn1
TCP connectivity check passed for subnet "192.168.78.0"


Check: Node connectivity using interfaces on subnet "172.16.100.0"
Node connectivity passed for subnet "172.16.100.0" with node(s) collabn1,collabn2
TCP connectivity check passed for subnet "172.16.100.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.78.0".
Subnet mask consistency check passed for subnet "172.16.100.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "172.16.100.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "172.16.100.0" for multicast communication with multicast group "224.0.0.251" passed.

Check of multicast communication passed.

Checking whether the ASM filter driver is active and consistent on all nodes
ASM filter driver library is not installed on any of the cluster nodes.
ASM filter driver configuration was found consistent across all the cluster nodes.
Time zone consistency check passed

Checking Cluster manager integrity...


Checking CSS daemon...
Oracle Cluster Synchronization Services appear to be online.

Cluster manager integrity check passed


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations

Default user file creation mask check passed

Checking cluster integrity...


Cluster integrity check passed


Checking OCR integrity...

Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations


Checking daemon liveness...
Liveness check passed for "CRS daemon"

Checking OCR config file "/etc/oracle/ocr.loc"...

OCR config file "/etc/oracle/ocr.loc" check successful


Disk group for ocr location "+OCRVOTE/collabn-cluster/OCRFILE/registry.255.934671619" is available on all the nodes


Checking OCR backup location "/u01/app/12.1.0/grid/cdata/collabn-cluster"

OCR backup location "/u01/app/12.1.0/grid/cdata/collabn-cluster" check passed

NOTE:
This check does not verify the integrity of the OCR contents. Execute 'ocrcheck' as a privileged user to verify the contents of OCR.

OCR integrity check passed

Checking CRS integrity...

Clusterware version consistency passed.

CRS integrity check passed

Checking node application existence...

Checking existence of VIP node application (required)
VIP node application check passed

Checking existence of NETWORK node application (required)
NETWORK node application check passed

Checking existence of ONS node application (optional)
ONS node application check passed


Checking Single Client Access Name (SCAN)...

Checking TCP connectivity to SCAN listeners...
TCP connectivity to SCAN listeners exists on all cluster nodes

Checking name resolution setup for "collabn-cluster-scan.racattack"...

Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed


Checking SCAN IP addresses...
Check of SCAN IP addresses passed

Verification of SCAN VIP and listener setup passed

Checking OLR integrity...
Check of existence of OLR configuration file "/etc/oracle/olr.loc" passed
Check of attributes of OLR configuration file "/etc/oracle/olr.loc" passed

WARNING:
This check does not verify the integrity of the OLR contents. Execute 'ocrcheck -local' as a privileged user to verify the contents of OLR.

OLR integrity check passed

Checking Oracle Cluster Voting Disk configuration...

Oracle Cluster Voting Disk configuration check passed

User "oracle" is not part of "root" group. Check passed
Oracle Clusterware is installed on all nodes.
CTSS resource check passed
Query of CTSS for time offset passed

CTSS is in Active state. Proceeding with check of clock time offsets on all nodes...
Check of clock time offsets passed


Oracle Cluster Time Synchronization Services check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Post-check for cluster services setup was successful.
[oracle@collabn1 ~]$
[oracle@collabn1 ~]$ su
Password:
[root@collabn1 oracle]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1688
Available space (kbytes) : 407880
ID : 827167720
Device/File Name : +OCRVOTE
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@collabn1 oracle]#
[root@collabn1 oracle]# ocrcheck -local
Status of Oracle Local Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1028
Available space (kbytes) : 408540
ID : 1014277103
Device/File Name : /u01/app/12.1.0/grid/cdata/collabn1.olr
Device/File integrity check succeeded

Local registry integrity check succeeded

Logical corruption check succeeded

[root@collabn1 oracle]#



[root@collabn2 oracle]# ocrcheck -local
Status of Oracle Local Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 796
Available space (kbytes) : 408772
ID : 1896774486
Device/File Name : /u01/app/12.1.0/grid/cdata/collabn2.olr
Device/File integrity check succeeded

Local registry integrity check succeeded

Logical corruption check succeeded

[root@collabn2 oracle]#


The "cluvfy stage -post crsinst" command as the Grid Infrastructure user checks the status of all components after the installation and configuration of the Cluster.  Note : This does not and cannot check any database (RAC or non-RAC) that is created.

The "ocrcheck" commands by root are to check the integrity of the Cluster Registry and Local Registry.  (You would run "ocrcheck -local" on each node of the Cluster).

.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 5 : Relocating OCR and VoteDisk

Mon, 2017-01-30 09:09
Most default installation guides for Grid Infrastructure will have you creating a DATA Disk Group during the install.  This results in the OCR, the Vote Disk and MGMTDB being created in the same DATA Disk Group.

The proper installation is to have OCR and Vote Disk separated from DATA.  If you create a different Disk Group during the installation, you would have all three components (MGMTDB is new in 12c, did not exist in 11g) and you can have these components properly placed in a non-DATA Disk Group that you create as the default.

But what if they have already been located in DATA ?  Can you relocate them to another Disk Group ?



I start with creating a new Disk Group called OCRVOTE on an existing unused ASM Disk.

[root@collabn1 ~]# su - oracle
[oracle@collabn1 ~]$ . oraenv
ORACLE_SID = [oracle] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[oracle@collabn1 ~]$ sqlplus '/ as sysasm'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Jan 30 22:34:38 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL>
SQL> set pages600
SQL> col name format a12
SQL> col path format a16
SQL> select name, path, state, group_number, disk_number
2 from v$asm_disk
3 order by 4,5,1
4 /

NAME PATH STATE GROUP_NUMBER DISK_NUMBER
------------ ---------------- -------- ------------ -----------
/dev/asm-disk4 NORMAL 0 0
DATA_0000 /dev/asm-disk1 NORMAL 1 0
DATA_0001 /dev/asm-disk2 NORMAL 1 1
DATA_0002 /dev/asm-disk5 NORMAL 1 2
FRA_0000 /dev/asm-disk3 NORMAL 2 0

SQL>
SQL> create diskgroup OCRVOTE external redundancy disk '/dev/asm-disk4';

Diskgroup created.

SQL>
SQL> select name, path, state, group_number, disk_number
2 from v$asm_disk
3 order by 4,5,1
4 /

NAME PATH STATE GROUP_NUMBER DISK_NUMBER
------------ ---------------- -------- ------------ -----------
DATA_0000 /dev/asm-disk1 NORMAL 1 0
DATA_0001 /dev/asm-disk2 NORMAL 1 1
DATA_0002 /dev/asm-disk5 NORMAL 1 2
FRA_0000 /dev/asm-disk3 NORMAL 2 0
OCRVOTE_0000 /dev/asm-disk4 NORMAL 3 0

SQL>
SQL> alter diskgroup OCRVOTE set attribute 'COMPATIBLE.ASM' = '12.1';

Diskgroup altered.

SQL>


I have identified the ASM Disk on /dev/asm-disk4 as available and created a DiskGroup on it.  I have to mount the DiskGroup on the second node as well.  (If I don't mount the DiskGroup on the second node collabn2, I get errors
"PROT-30: The Oracle Cluster Registry location to be added is not usable
PROC-50: The Oracle Cluster Registry location to be added is inaccessible on nodes collabn2."
when attempting to add the OCR on the DiskGroup).

[oracle@collabn2 trace]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Jan 30 22:52:18 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysasm

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> alter diskgroup OCRVOTE mount;

Diskgroup altered.

SQL> set pages60
SQL> col name format a12
SQL> col path format a16
SQL> select name, path, state, group_number, disk_number
2 from v$asm_disk
3 order by 4,5,1
4 /

NAME PATH STATE GROUP_NUMBER DISK_NUMBER
------------ ---------------- -------- ------------ -----------
DATA_0000 /dev/asm-disk1 NORMAL 1 0
DATA_0001 /dev/asm-disk2 NORMAL 1 1
DATA_0002 /dev/asm-disk5 NORMAL 1 2
FRA_0000 /dev/asm-disk3 NORMAL 2 0
OCRVOTE_0000 /dev/asm-disk4 NORMAL 3 0

SQL>


Next, I plan to relocate the OCR which is currently in the DATA DiskGroup.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[oracle@collabn1 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1688
Available space (kbytes) : 407880
ID : 827167720
Device/File Name : +DATA
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check bypassed due to non-privileged user

[oracle@collabn1 ~]$
[oracle@collabn1 ~]$ su
Password:
[root@collabn1 oracle]# ocrconfig -add +OCRVOTE
[root@collabn1 oracle]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1684
Available space (kbytes) : 407884
ID : 827167720
Device/File Name : +DATA
Device/File integrity check succeeded
Device/File Name : +OCRVOTE
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@collabn1 oracle]# ocrconfig -delete +DATA
[root@collabn1 oracle]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 1684
Available space (kbytes) : 407884
ID : 827167720
Device/File Name : +OCRVOTE
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@collabn1 oracle]#


I first used OCRCHECK ("ocrcheck" at the commandline) to list the existing OCR location.  I then used root and OCRCONFIG -ADD to add +OCRVOTE as a location and verified it with OCRCHECK.  I then used OCRCONFIG -DELETE ("ocrconfig -add" and "ocrconfig -delete") to delete the old location.

I could have used "ocrconfig -replace" to replace the OCR location but I prefer ADD and DELETE.



How do I relocate the VoteDisk ?  Since I am using a single DiskGroup and External Redundancy with only 1 Failure Group, I have only 1 VoteDisk.

[root@collabn1 oracle]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 96fbcb40bfeb4ff7bf18881adcfef149 (/dev/asm-disk1) [DATA]
Located 1 voting disk(s).
[root@collabn1 oracle]#
[root@collabn1 oracle]# crsctl replace votedisk +OCRVOTE
Successful addition of voting disk a58b8b9d58064fb8bf6df0b3ee701e32.
Successful deletion of voting disk 96fbcb40bfeb4ff7bf18881adcfef149.
Successfully replaced voting disk group with +OCRVOTE.
CRS-4266: Voting file(s) successfully replaced
[root@collabn1 oracle]#
[root@collabn1 oracle]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE a58b8b9d58064fb8bf6df0b3ee701e32 (/dev/asm-disk4) [OCRVOTE]
Located 1 voting disk(s).
[root@collabn1 oracle]#


With ASM as  the location for VoteDisk, I cannot use CRSCTL ADD VOTEDISK  but have to use CRSCTL REPLACE VOTEDISK.

So, I have added a new DiskGroup called OCRVOTE and relocated both OCR and VoteDisk to this new DiskGroup (on disk /dev/asm-disk4).

Can I relocate +MGMTDB ?  I am open to suggestions.

.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 4 : Adding a Disk of a different size

Sat, 2017-01-21 10:44
How does 12.1.0.2 ASM handle adding a disk of a different size to an existing DiskGroup ?

I currently have 4 disks of 5GB each in 2 DiskGroups

[oracle@collabn1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Sat Jan 21 23:48:00 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> select group_number, disk_number, name, state, total_mb
2 from v$asm_disk
3 order by 1,2,3
4 /

GROUP_NUMBER DISK_NUMBER NAME STATE TOTAL_MB
------------ ----------- ------------------------------ -------- ----------
0 0 NORMAL 0
1 0 DATA_0000 NORMAL 5114
1 1 DATA_0001 NORMAL 5114
2 0 FRA_0000 NORMAL 5114

SQL>
SQL> select group_number, name
2 from v$asm_diskgroup
3 order by 1
4 /

GROUP_NUMBER NAME
------------ ------------------------------
1 DATA
2 FRA

SQL>


The DATA DiskGroup has 2 disks of 5GB each and the FRA DiskGroup has 1 disk of 5GB.  One disk (identified as DiskNumber=0) is not yet assigned.

What happens if I try to expand the DATA DiskGroup with a Disk of 10GB ?

[root@collabn1 dev]# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xff8b0ab7.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help): p

Disk /dev/sdf: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xff8b0ab7

Device Boot Start End Blocks Id System

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1566, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1566, default 1566):
Using default value 1566

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@collabn1 dev]#
[root@collabn1 dev]# /sbin/scsi_id -g -u -d /dev/sdf
1ATA_VBOX_HARDDISK_VB535deca9-9a295efe
[root@collabn1 dev]#
[root@collabn1 dev]# /sbin/scsi_id -g -u -d /dev/sdf
1ATA_VBOX_HARDDISK_VB535deca9-9a295efe
[root@collabn1 dev]# cd /etc/udev/rules.d
[root@collabn1 rules.d]# vi 99-oracle-asmdevices.rules
[root@collabn1 rules.d]# tail -1 99-oracle-asmdevices.rules
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB535deca9-9a295efe", NAME="asm-disk5", OWNER="oracle", GROUP="dba", MODE="0660"
[root@collabn1 rules.d]#
[root@collabn1 rules.d]# /sbin/partprobe /dev/sdf1
[root@collabn1 rules.d]# /sbin/udevadm test /block/sdb/sdf1
run_command: calling: test
udevadm_test: version 147
This program is for debugging only, it does not run any program,
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

parse_file: reading '/lib/udev/rules.d/10-console.rules' as rules file
parse_file: reading '/lib/udev/rules.d/10-dm.rules' as rules file
parse_file: reading '/lib/udev/rules.d/11-dm-lvm.rules' as rules file
parse_file: reading '/lib/udev/rules.d/13-dm-disk.rules' as rules file
parse_file: reading '/lib/udev/rules.d/40-isdn.rules' as rules file
parse_file: reading '/lib/udev/rules.d/40-redhat.rules' as rules file
parse_file: reading '/lib/udev/rules.d/42-qemu-usb.rules' as rules file
parse_file: reading '/lib/udev/rules.d/50-firmware.rules' as rules file
parse_file: reading '/lib/udev/rules.d/50-udev-default.rules' as rules file
parse_file: reading '/etc/udev/rules.d/55-usm.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-alias-kmsg.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-cdrom_id.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-fprint-autosuspend.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-net.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-pcmcia.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-alsa.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-input.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-serial.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-storage-tape.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-storage.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-v4l.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-raw.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-vboxadd.rules' as rules file
parse_file: reading '/lib/udev/rules.d/61-mobile-action.rules' as rules file
parse_file: reading '/lib/udev/rules.d/61-option-modem-modeswitch.rules' as rules file
parse_file: reading '/lib/udev/rules.d/61-persistent-storage-edd.rules' as rules file
parse_file: reading '/lib/udev/rules.d/64-device-mapper.rules' as rules file
parse_file: reading '/lib/udev/rules.d/64-md-raid.rules' as rules file
parse_file: reading '/lib/udev/rules.d/65-md-incremental.rules' as rules file
parse_file: reading '/lib/udev/rules.d/69-dm-lvm-metad.rules' as rules file
parse_file: reading '/lib/udev/rules.d/70-acl.rules' as rules file
parse_file: reading '/lib/udev/rules.d/70-cups-libusb.rules' as rules file
parse_file: reading '/lib/udev/rules.d/70-hid2hci.rules' as rules file
parse_file: reading '/etc/udev/rules.d/70-persistent-cd.rules' as rules file
parse_file: reading '/etc/udev/rules.d/70-persistent-net.rules' as rules file
parse_file: reading '/lib/udev/rules.d/71-biosdevname.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-cd-aliases-generator.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-net-description.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-persistent-net-generator.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-tty-description.rules' as rules file
parse_file: reading '/lib/udev/rules.d/78-sound-card.rules' as rules file
parse_file: reading '/lib/udev/rules.d/79-fstab_import.rules' as rules file
parse_file: reading '/lib/udev/rules.d/80-drivers.rules' as rules file
parse_file: reading '/lib/udev/rules.d/80-iosched.rules' as rules file
parse_file: reading '/lib/udev/rules.d/80-mpath-iosched.rules' as rules file
parse_file: reading '/lib/udev/rules.d/85-regulatory.rules' as rules file
parse_file: reading '/lib/udev/rules.d/88-clock.rules' as rules file
parse_file: reading '/lib/udev/rules.d/89-microcode.rules' as rules file
parse_file: reading '/etc/udev/rules.d/90-alsa.rules' as rules file
parse_file: reading '/lib/udev/rules.d/90-btrfs.rules' as rules file
parse_file: reading '/etc/udev/rules.d/90-hal.rules' as rules file
parse_file: reading '/lib/udev/rules.d/91-drm-modeset.rules' as rules file
parse_file: reading '/lib/udev/rules.d/95-dm-notify.rules' as rules file
parse_file: reading '/lib/udev/rules.d/95-keyboard-force-release.rules' as rules file
parse_file: reading '/lib/udev/rules.d/95-keymap.rules' as rules file
parse_file: reading '/lib/udev/rules.d/95-udev-late.rules' as rules file
parse_file: reading '/etc/udev/rules.d/98-kexec.rules' as rules file
parse_file: reading '/etc/udev/rules.d/99-oracle-asmdevices.rules' as rules file
parse_file: reading '/dev/.udev/rules.d/99-root.rules' as rules file
udev_rules_new: rules use 32448 bytes tokens (2704 * 12 bytes), 19085 bytes buffer
udev_rules_new: temporary index used 19500 bytes (975 * 20 bytes)
unable to open device '/sys/block/sdb/sdf1'
[root@collabn1 rules.d]#
[root@collabn1 rules.d]# /sbin/udevadm control --reload-rules
[root@collabn1 rules.d]# /sbin/start_udev
Starting udev: [ OK ]
[root@collabn1 rules.d]#
[root@collabn1 rules.d]# ls -l /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Jan 22 00:07 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 33 Jan 22 00:07 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Jan 22 00:07 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Jan 22 00:05 /dev/asm-disk4
brw-rw----. 1 oracle dba 8, 81 Jan 22 00:05 /dev/asm-disk5


So I now have asm-disk5 as the new ASM Disk.  Let my try to add this disk.

SQL> set pages600
SQL> select group_number, disk_number, name, path, total_mb
2 from v$asm_disk
3 order by 1,2
4 /

GROUP_NUMBER DISK_NUMBER NAME
------------ ----------- ------------------------------
PATH
--------------------------------------------------------------------------------
TOTAL_MB
----------
0 0
/dev/asm-disk5
0

0 1
/dev/asm-disk4
0

1 0 DATA_0000
/dev/asm-disk1
5114

1 1 DATA_0001
/dev/asm-disk2
5114

2 0 FRA_0000
/dev/asm-disk3
5114


SQL>
SQL> alter diskgroup data add disk '/dev/asm-disk5';

Diskgroup altered.

SQL>
SQL> alter diskgroup data add disk '/dev/asm-disk5';

Diskgroup altered.

SQL> select group_number, name, total_mb
2 from v$asm_diskgroup
3 order by 1,2
4 /

GROUP_NUMBER NAME TOTAL_MB
------------ ------------------------------ ----------
1 DATA 22512
2 FRA 5114

SQL>
SQL> select group_number, name, type
2 from v$asm_diskgroup
3 order by 1,2
4 /

GROUP_NUMBER NAME TYPE
------------ ------------------------------ ------
1 DATA EXTERN
2 FRA EXTERN

SQL>
SQL> select group_number, name, compatibility, database_compatibility
2 from v$asm_diskgroup
3 order by 1
4 /

GROUP_NUMBER NAME
------------ ------------------------------
COMPATIBILITY
------------------------------------------------------------
DATABASE_COMPATIBILITY
------------------------------------------------------------
1 DATA
12.1.0.0.0
10.1.0.0.0

2 FRA
12.1.0.0.0
10.1.0.0.0


SQL>
SQL> select group_number, disk_number, name, path, total_mb
2 from v$asm_disk
3 order by 1,2,3
4 /

GROUP_NUMBER DISK_NUMBER NAME
------------ ----------- ------------------------------
PATH
--------------------------------------------------------------------------------
TOTAL_MB
----------
0 0
/dev/asm-disk4
0

1 0 DATA_0000
/dev/asm-disk1
5114

1 1 DATA_0001
/dev/asm-disk2
5114

1 2 DATA_0002
/dev/asm-disk5
12284

2 0 FRA_0000
/dev/asm-disk3
5114


SQL>


According to Oracle Support Document 1938950.1, adding a disk of a different size to an existing DiskGroup fails with an error ORA-15410 in 12.1.0.2.  However, that seems to apply to NORMAL or HIGH Redundancy and COMPATIBLE.ASM 12.1.0.2.   Here, I have EXTERNAL Redundancy and COMPATIBLE.ASM 12.1.0.0.0

Do I recommend Disks of different sizes ?  Absolutely *not* in Production.  This is a "play" environment in Virtual Machines on my desktop that I can destroy and recreate anytime.  I can monitor disk usage as well.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 3 : Convert PolicyManaged DB back to AdminManaged

Sun, 2017-01-15 20:35
In the previous post on 12cR1 RAC, I had converted my AdminManaged Database to PolicyManaged.

Here, I convert it back to AdminManaged.

First, I verify that the database is shutdown (note that I have only 1 node of the cluster currently up and running, I don't need both nodes and instances up).

[oracle@collabn1 ~]$ srvctl status database -d RAC
Instance RAC_1 is running on node collabn1
[oracle@collabn1 ~]$ srvctl stop database -d RAC
[oracle@collabn1 ~]$ ps -fuoracle |grep smon
oracle 3422 1 0 09:49 ? 00:00:00 asm_smon_+ASM1
oracle 4882 1 0 09:50 ? 00:00:00 mdb_smon_-MGMTDB
oracle 16889 9821 0 10:08 pts/0 00:00:00 grep smon
[oracle@collabn1 ~]$


Next, I remove the database from the Cluster Registry.

[oracle@collabn1 ~]$ srvctl config database -d RAC
Database unique name: RAC
Database name:
Oracle home: /u01/app/oracle/product/12.1.0/dbhome_1
Oracle user: oracle
Spfile:
Password file:
Domain: racattack
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: MyPool
Disk Groups: DATA,FRA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group:
Database instances:
Configured nodes:
Database is policy managed
[oracle@collabn1 ~]$ srvctl remove database -d RAC
Remove the database RAC? (y/[n]) y
[oracle@collabn1 ~]$ srvctl config database -d RAC
PRCD-1120 : The resource for database RAC could not be found.
PRCR-1001 : Resource ora.rac.db does not exist
[oracle@collabn1 ~]$


I then remove the defined Server Pool that I used for this database.

[oracle@collabn1 ~]$ srvctl config srvpool
Server pool name: Free
Importance: 0, Min: 0, Max: -1
Category:
Candidate server names:
Server pool name: Generic
Importance: 0, Min: 0, Max: -1
Category:
Candidate server names:
Server pool name: MyPool
Importance: 100, Min: 1, Max: 2
Category:
Candidate server names: collabn1,collabn2
[oracle@collabn1 ~]$ srvctl remove srvpool -serverpool MyPool
[oracle@collabn1 ~]$ srvctl config srvpool
Server pool name: Free
Importance: 0, Min: 0, Max: -1
Category:
Candidate server names:
Server pool name: Generic
Importance: 0, Min: 0, Max: -1
Category:
Candidate server names:
[oracle@collabn1 ~]$


I then add the database back into the Cluster Registry.

[oracle@collabn1 ~]$ srvctl add database -d RAC \
> -oraclehome /u01/app/oracle/product/12.1.0/dbhome_1 \
> -pwfile +DATA/RAC/PASSWORD/pwdrac.277.931824933
[oracle@collabn1 ~]$ srvctl config database -d RAC
Database unique name: RAC
Database name:
Oracle home: /u01/app/oracle/product/12.1.0/dbhome_1
Oracle user: oracle
Spfile:
Password file: +DATA/RAC/PASSWORD/pwdrac.277.931824933
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups:
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group:
Database instances:
Configured nodes:
Database is administrator managed
[oracle@collabn1 ~]$


I start the second node of the cluster before I configure the instances (Note : I have the $ORACLE_HOME/dbs pfiles created in advance).

[oracle@collabn1 ~]$ srvctl add instance -d RAC -i RAC1 -n collabn1
[oracle@collabn1 ~]$ srvctl add instance -d RAC -i RAC2 -n collabn2
[oracle@collabn1 ~]$ srvctl config database -d RAC
Database unique name: RAC
Database name:
Oracle home: /u01/app/oracle/product/12.1.0/dbhome_1
Oracle user: oracle
Spfile:
Password file: +DATA/RAC/PASSWORD/pwdrac.277.931824933
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups:
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group:
Database instances: RAC1,RAC2
Configured nodes: collabn1,collabn2
Database is administrator managed
[oracle@collabn1 ~]$


I am now ready to start the database (instances).

[oracle@collabn1 ~]$ srvctl start database -d RAC
[oracle@collabn1 ~]$ srvctl status database -d RAC
Instance RAC1 is running on node collabn1
Instance RAC2 is running on node collabn2
[oracle@collabn1 ~]$ ps -fuoracle |grep smon
oracle 3422 1 0 09:49 ? 00:00:00 asm_smon_+ASM1
oracle 4882 1 0 09:50 ? 00:00:00 mdb_smon_-MGMTDB
oracle 25431 1 0 10:30 ? 00:00:00 ora_smon_RAC1
oracle 27533 9821 0 10:33 pts/0 00:00:00 grep smon
[oracle@collabn1 ~]$
[root@collabn2 ~]# ps -fuoracle |grep smon
oracle 3460 1 0 10:19 ? 00:00:00 asm_smon_+ASM2
oracle 9561 1 0 10:30 ? 00:00:00 ora_smon_RAC2
[root@collabn2 ~]#
[oracle@collabn1 ~]$ env |grep SID
ORACLE_SID=RAC1
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Jan 16 10:34:15 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select * from v$containers;

CON_ID DBID CON_UID GUID
---------- ---------- ---------- --------------------------------
NAME OPEN_MODE RES
------------------------------ ---------- ---
OPEN_TIME
---------------------------------------------------------------------------
CREATE_SCN TOTAL_SIZE BLOCK_SIZE RECOVERY SNAPSHOT_PARENT_CON_ID
---------- ---------- ---------- -------- ----------------------
1 2519807290 1 FD9AC20F64D344D7E043B6A9E80A2F2F
CDB$ROOT READ WRITE NO
16-JAN-17 10.31.34.014 AM +08:00
0 0 8192 ENABLED 0


CON_ID DBID CON_UID GUID
---------- ---------- ---------- --------------------------------
NAME OPEN_MODE RES
------------------------------ ---------- ---
OPEN_TIME
---------------------------------------------------------------------------
CREATE_SCN TOTAL_SIZE BLOCK_SIZE RECOVERY SNAPSHOT_PARENT_CON_ID
---------- ---------- ---------- -------- ----------------------
2 2061548092 2061548092 44BB5E17F41A2618E053334EA8C006B9
PDB$SEED READ ONLY NO
16-JAN-17 10.31.34.859 AM +08:00
1594413 859832320 8192 ENABLED 0


CON_ID DBID CON_UID GUID
---------- ---------- ---------- --------------------------------
NAME OPEN_MODE RES
------------------------------ ---------- ---
OPEN_TIME
---------------------------------------------------------------------------
CREATE_SCN TOTAL_SIZE BLOCK_SIZE RECOVERY SNAPSHOT_PARENT_CON_ID
---------- ---------- ---------- -------- ----------------------
3 1857084550 1857084550 44BBC69CE8F552AEE053334EA8C07365
PDB MOUNTED

1755977 0 8192 ENABLED 0


SQL> alter pluggable database PDB open;

Pluggable database altered.

SQL>
SQL> select con_id, name, open_mode from v$pdbs;

CON_ID NAME OPEN_MODE
---------- ------------------------------ ----------
2 PDB$SEED READ ONLY
3 PDB READ WRITE

SQL>


Thus, I converted my PolicyManaged database to AdministratorManaged.
.
.
.

Categories: DBA Blogs

Copying a Tablespace from NonCDB to a PDB (using TTS)

Sun, 2017-01-15 02:54
A Tablespace can be "transported"  from a NonCDB to a PDB as a way of copying the Tablespace.  Here I work with ASM as well.

First in the NonCDB :

[oracle@ora12102 Desktop]$ . oraenv
ORACLE_SID = [oracle] ? NONCDB
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@ora12102 Desktop]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:03:43 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 402653184 bytes
Fixed Size 2924928 bytes
Variable Size 260050560 bytes
Database Buffers 134217728 bytes
Redo Buffers 5459968 bytes
Database mounted.
Database opened.
SQL> select file_name, bytes/1048576
2 from dba_data_files
3 where tablespace_name = 'EXAMPLE'
4 /

FILE_NAME
--------------------------------------------------------------------------------
BYTES/1048576
-------------
+DATA/NONCDB/DATAFILE/example.266.896482777
1243.75


SQL>
[oracle@ora12102 Desktop]$ expdp hemant/hemant \
> directory=data_pump_dir dumpfile=EXAMPLE_TTS.dmp \
> transport_tablespaces=EXAMPLE transport_full_check=Y

Export: Release 12.1.0.2.0 - Production on Sun Jan 15 16:08:27 2017

Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.

Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
Starting "HEMANT"."SYS_EXPORT_TRANSPORTABLE_01": hemant/******** directory=data_pump_dir dumpfile=EXAMPLE_TTS.dmp transport_tablespaces=EXAMPLE transport_full_check=Y
ORA-39123: Data Pump transportable tablespace job aborted
ORA-39185: The transportable tablespace failure list is

ORA-29335: tablespace 'EXAMPLE' is not read only
Job "HEMANT"."SYS_EXPORT_TRANSPORTABLE_01" stopped due to fatal error at Sun Jan 15 16:08:41 2017 elapsed 0 00:00:06

[oracle@ora12102 Desktop]$


The tablespace has to be set READ ONLY before we can use export to transport it (also, it should be READ ONLY while the data files are being copied.

[oracle@ora12102 Desktop]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:09:10 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

SQL> alter tablespace example read only;

Tablespace altered.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
[oracle@ora12102 Desktop]$ expdp hemant/hemant \
> directory=data_pump_dir dumpfile=EXAMPLE_TTS.dmp \
> transport_tablespaces=EXAMPLE transport_full_check=Y

Export: Release 12.1.0.2.0 - Production on Sun Jan 15 16:09:58 2017

Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.

Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
Starting "HEMANT"."SYS_EXPORT_TRANSPORTABLE_01": hemant/******** directory=data_pump_dir dumpfile=EXAMPLE_TTS.dmp transport_tablespaces=EXAMPLE transport_full_check=Y
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TYPE/TYPE_SPEC
Processing object type TRANSPORTABLE_EXPORT/TYPE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type TRANSPORTABLE_EXPORT/TYPE/TYPE_BODY
Processing object type TRANSPORTABLE_EXPORT/PROCACT_INSTANCE
Processing object type TRANSPORTABLE_EXPORT/XMLSCHEMA/XMLSCHEMA
Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type TRANSPORTABLE_EXPORT/INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/COMMENT
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/REF_CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/INDEX/BITMAP_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/INDEX/STATISTICS/BITMAP_INDEX/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/TRIGGER
Processing object type TRANSPORTABLE_EXPORT/TABLE_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/STATISTICS/MARKER
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/TABLE
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/SECONDARY_TABLE/INDEX
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/SECONDARY_TABLE/CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/MATERIALIZED_VIEW
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PROCACT_INSTANCE
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PROCDEPOBJ
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
Master table "HEMANT"."SYS_EXPORT_TRANSPORTABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for HEMANT.SYS_EXPORT_TRANSPORTABLE_01 is:
/u01/app/oracle/admin/NONCDB/dpdump/EXAMPLE_TTS.dmp
******************************************************************************
Datafiles required for transportable tablespace EXAMPLE:
+DATA/NONCDB/DATAFILE/example.266.896482777
Job "HEMANT"."SYS_EXPORT_TRANSPORTABLE_01" successfully completed at Sun Jan 15 16:12:48 2017 elapsed 0 00:02:46

[oracle@ora12102 Desktop]$


Now, I need to copy the datafile (while the tablespace is READ ONLY).

[oracle@ora12102 Desktop]$ su - grid
Password:
[grid@ora12102 ~]$ asmcmd
ASMCMD> cp +DATA/NONCDB/DATAFILE/example.266.896482777 /tmp/example.dbf
copying +DATA/NONCDB/DATAFILE/example.266.896482777 -> /tmp/example.dbf
ASMCMD>
ASMCMD> exit
[grid@ora12102 ~]$ exit
logout
[oracle@ora12102 Desktop]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:16:24 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

SQL> alter tablespace example read write ;

Tablespace altered.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
[oracle@ora12102 Desktop]$


I must now identify the target location for the datafile in the CDB database.

[oracle@ora12102 Desktop]$ . oraenv
ORACLE_SID = [NONCDB] ? CDB1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@ora12102 Desktop]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:17:44 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 1644167168 bytes
Fixed Size 2925024 bytes
Variable Size 973082144 bytes
Database Buffers 654311424 bytes
Redo Buffers 13848576 bytes
Database mounted.
Database opened.
SQL> alter pluggable database pdb1 open;

Pluggable database altered.

SQL>
SQL> alter session set container=PDB1;

Session altered.

SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
+DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/system.284.914408541
+DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/sysaux.285.914408541
+DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/users.287.914408663
+DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/hemant.288.914713623

SQL>


Now that I have identiied the default location for all PDB1 files, I need to use ASMCMD to copy the datafile.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
[oracle@ora12102 Desktop]$ su - grid
Password:
[grid@ora12102 ~]$
[grid@ora12102 ~]$ asmcmd
ASMCMD> cp /tmp/example.dbf +DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/
copying /tmp/example.dbf -> +DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/example.dbf
ASMCMD> cd +DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE
ASMCMD> ls
HEMANT.288.914713623
SYSAUX.285.914408541
SYSTEM.284.914408541
USERS.287.914408663
example.dbf
ASMCMD> exit
[grid@ora12102 ~]$


Now, I need to import the tablespace with the datafile. Before that, I need to setup the user that will do the import and all the users of the target tablespace.

[grid@ora12102 ~]$ exit
logout
[oracle@ora12102 Desktop]$
[oracle@ora12102 Desktop]$ sqlplus system/oracle@PDB1

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:28:16 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-28002: the password will expire within 7 days


Last Successful login time: Sun Jan 15 2017 16:27:18 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

SQL> grant select_catalog_role, imp_full_database to hemant;

Grant succeeded.

SQL> select privilege from dba_sys_privs where grantee = 'HEMANT';

PRIVILEGE
----------------------------------------
CREATE TABLE
CREATE SESSION

SQL> select granted_role from dba_role_privs where grantee = 'HEMANT';

GRANTED_ROLE
--------------------------------------------------------------------------------
SELECT_CATALOG_ROLE
IMP_FULL_DATABASE

SQL>


As with the Export, I am using a non-DBA user for the import.  I also have to setup the users and their grants.

[oracle@ora12102 Desktop]$ sqlplus sys/oracle@PDB1 as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:33:17 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

SQL> create directory imp_from_noncdb as '/u01/app/oracle/admin/NONCDB/dpdump';

Directory created.

SQL> grant read, write on directory imp_from_noncdb to hemant;

Grant succeeded.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
[oracle@ora12102 Desktop]$
[oracle@ora12102 Desktop]$ sqlplus system/oracle@PDB1

SQL*Plus: Release 12.1.0.2.0 Production on Sun Jan 15 16:41:36 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-28002: the password will expire within 7 days


Last Successful login time: Sun Jan 15 2017 16:28:16 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

SQL> create user HR identified by HR ;
create user IX identified by IX ;
create user OE identified by OE ;
create user PM identified by PM ;
create user SH identified by SH ;

User created.

SQL>
User created.

SQL>
User created.

SQL>
User created.

SQL>
User created.

SQL> SQL>
SQL> @grants_to_EXAMPLE
SQL> spool grants_to_EXAMPLE
SQL>
SQL> grant ALTER SESSION to HR ;

Grant succeeded.

SQL> grant ALTER SESSION to IX ;

Grant succeeded.

SQL> grant ALTER SESSION to SH ;

Grant succeeded.

SQL> grant CREATE CLUSTER to IX ;

Grant succeeded.

SQL> grant CREATE CLUSTER to SH ;

Grant succeeded.

SQL> grant CREATE DATABASE LINK to HR ;

Grant succeeded.

SQL> grant CREATE DATABASE LINK to IX ;

Grant succeeded.

SQL> grant CREATE DATABASE LINK to OE ;

Grant succeeded.

SQL> grant CREATE DATABASE LINK to SH ;

Grant succeeded.

SQL> grant CREATE DIMENSION to SH ;

Grant succeeded.

SQL> grant CREATE INDEXTYPE to IX ;

Grant succeeded.

SQL> grant CREATE MATERIALIZED VIEW to OE ;

Grant succeeded.

SQL> grant CREATE MATERIALIZED VIEW to SH ;

Grant succeeded.

SQL> grant CREATE OPERATOR to IX ;

Grant succeeded.

SQL> grant CREATE PROCEDURE to HR ;

Grant succeeded.

SQL> grant CREATE PROCEDURE to IX ;

Grant succeeded.

SQL> grant CREATE RULE to IX ;

Grant succeeded.

SQL> grant CREATE RULE SET to IX ;

Grant succeeded.

SQL> grant CREATE SEQUENCE to HR ;

Grant succeeded.

SQL> grant CREATE SEQUENCE to IX ;

Grant succeeded.

SQL> grant CREATE SEQUENCE to SH ;

Grant succeeded.

SQL> grant CREATE SESSION to HR ;

Grant succeeded.

SQL> grant CREATE SESSION to IX ;

Grant succeeded.

SQL> grant CREATE SESSION to OE ;

Grant succeeded.

SQL> grant CREATE SESSION to SH ;

Grant succeeded.

SQL> grant CREATE SYNONYM to HR ;

Grant succeeded.

SQL> grant CREATE SYNONYM to IX ;

Grant succeeded.

SQL> grant CREATE SYNONYM to OE ;

Grant succeeded.

SQL> grant CREATE SYNONYM to SH ;

Grant succeeded.

SQL> grant CREATE TABLE to IX ;

Grant succeeded.

SQL> grant CREATE TABLE to SH ;

Grant succeeded.

SQL> grant CREATE TRIGGER to IX ;

Grant succeeded.

SQL> grant CREATE TYPE to IX ;

Grant succeeded.

SQL> grant CREATE VIEW to HR ;

Grant succeeded.

SQL> grant CREATE VIEW to IX ;

Grant succeeded.

SQL> grant CREATE VIEW to OE ;

Grant succeeded.

SQL> grant CREATE VIEW to SH ;

Grant succeeded.

SQL> grant QUERY REWRITE to OE ;

Grant succeeded.

SQL> grant QUERY REWRITE to SH ;

Grant succeeded.

SQL> grant SELECT ANY DICTIONARY to IX ;

Grant succeeded.

SQL> grant UNLIMITED TABLESPACE to HR ;

Grant succeeded.

SQL> grant UNLIMITED TABLESPACE to IX ;

Grant succeeded.

SQL> grant UNLIMITED TABLESPACE to OE ;

Grant succeeded.

SQL> grant UNLIMITED TABLESPACE to PM ;

Grant succeeded.

SQL> grant UNLIMITED TABLESPACE to SH ;

Grant succeeded.

SQL>
SQL> spool off
SQL> @roles_to_EXAMPLE
SQL> set echo on
SQL> spool roles_to_EXAMPLE
SQL>
SQL> grant AQ_ADMINISTRATOR_ROLE to IX ;

Grant succeeded.

SQL> grant AQ_USER_ROLE to IX ;

Grant succeeded.

SQL> grant CONNECT to IX ;

Grant succeeded.

SQL> grant CONNECT to PM ;

Grant succeeded.

SQL> grant RESOURCE to HR ;

Grant succeeded.

SQL> grant RESOURCE to IX ;

Grant succeeded.

SQL> grant RESOURCE to OE ;

Grant succeeded.

SQL> grant RESOURCE to PM ;

Grant succeeded.

SQL> grant RESOURCE to SH ;

Grant succeeded.

SQL> grant SELECT_CATALOG_ROLE to IX ;

Grant succeeded.

SQL> grant SELECT_CATALOG_ROLE to SH ;

Grant succeeded.

SQL> grant XDBADMIN to OE ;

Grant succeeded.

SQL>
SQL> spool off
SQL>


I am now ready to import the tablespace and datafile.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
[oracle@ora12102 Desktop]$ impdp hemant/hemant@PDB1 \
> dumpfile=EXAMPLE_TTS.dmp directory=imp_from_noncdb \
> transport_datafiles=+DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/example.dbf

Import: Release 12.1.0.2.0 - Production on Sun Jan 15 16:50:16 2017

Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.

UDI-28002: operation generated ORACLE error 28002
ORA-28002: the password will expire within 7 days

Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
Master table "HEMANT"."SYS_IMPORT_TRANSPORTABLE_01" successfully loaded/unloaded
Source time zone is +00:00 and target time zone is -07:00.
Starting "HEMANT"."SYS_IMPORT_TRANSPORTABLE_01": hemant/********@PDB1 dumpfile=EXAMPLE_TTS.dmp directory=imp_from_noncdb transport_datafiles=+DATA/CDB1/35208E5B92306007E0530F02000A969A/DATAFILE/example.dbf
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TYPE/TYPE_SPEC
Processing object type TRANSPORTABLE_EXPORT/TYPE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type TRANSPORTABLE_EXPORT/TYPE/TYPE_BODY
Processing object type TRANSPORTABLE_EXPORT/PROCACT_INSTANCE
Processing object type TRANSPORTABLE_EXPORT/XMLSCHEMA/XMLSCHEMA
Processing object type TRANSPORTABLE_EXPORT/TABLE
ORA-39360: Table "OE"."ORDERS" was skipped due to transportable import and TSLTZ issues resulting from time zone mismatch.
ORA-39151: Table "OE"."PURCHASEORDER" exists. All dependent metadata and data will be skipped due to table_exists_action of skip
Processing object type TRANSPORTABLE_EXPORT/GRANT/OWNER_GRANT/OBJECT_GRANT
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."PROMOTIONS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."PRODUCT_DESCRIPTIONS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."PRODUCT_INFORMATION" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."INVENTORIES" TO "BI"
ORA-39112: Dependent object type OBJECT_GRANT:"OE" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type OBJECT_GRANT:"OE" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."ORDER_ITEMS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."WAREHOUSES" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "OE"."CUSTOMERS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."COSTS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."SALES" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."COUNTRIES" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."CUSTOMERS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."PROMOTIONS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."CHANNELS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."PRODUCTS" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."TIMES" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."FWEEK_PSCAT_SALES_MV" TO "BI"
ORA-39083: Object type OBJECT_GRANT failed to create with error:
ORA-01917: user or role 'BI' does not exist
Failing sql is:
GRANT SELECT ON "SH"."CAL_MONTH_SALES_MV" TO "BI"
Processing object type TRANSPORTABLE_EXPORT/INDEX/INDEX
ORA-39112: Dependent object type INDEX:"OE"."ORD_SALES_REP_IX" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type INDEX:"OE"."ORD_ORDER_DATE_IX" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type INDEX:"OE"."ORD_CUSTOMER_IX" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type INDEX:"OE"."ORDER_PK" skipped, base object type TABLE:"OE"."ORDERS" creation failed
Processing object type TRANSPORTABLE_EXPORT/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/CONSTRAINT
ORA-39112: Dependent object type CONSTRAINT:"OE"."ORDER_MODE_LOV" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type CONSTRAINT:"OE"."ORDER_TOTAL_MIN" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type CONSTRAINT:"OE"."ORDER_PK" skipped, base object type TABLE:"OE"."ORDERS" creation failed
Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/COMMENT
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type COMMENT skipped, base object type TABLE:"OE"."ORDERS" creation failed
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/REF_CONSTRAINT
ORA-39083: Object type REF_CONSTRAINT:"OE"."ORDER_ITEMS_ORDER_ID_FK" failed to create with error:
ORA-00942: table or view does not exist
Failing sql is:
ALTER TABLE "OE"."ORDER_ITEMS" ADD CONSTRAINT "ORDER_ITEMS_ORDER_ID_FK" FOREIGN KEY ("ORDER_ID") REFERENCES "OE"."ORDERS" ("ORDER_ID") ON DELETE CASCADE ENABLE NOVALIDATE
ORA-39112: Dependent object type REF_CONSTRAINT:"OE"."ORDERS_SALES_REP_FK" skipped, base object type TABLE:"OE"."ORDERS" creation failed
ORA-39112: Dependent object type REF_CONSTRAINT:"OE"."ORDERS_CUSTOMER_ID_FK" skipped, base object type TABLE:"OE"."ORDERS" creation failed
Processing object type TRANSPORTABLE_EXPORT/INDEX/BITMAP_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/INDEX/STATISTICS/BITMAP_INDEX/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/TRIGGER
Processing object type TRANSPORTABLE_EXPORT/TABLE_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/STATISTICS/MARKER
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/TABLE
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/SECONDARY_TABLE/INDEX
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/SECONDARY_TABLE/CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/DOMAIN_INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/MATERIALIZED_VIEW
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PROCACT_INSTANCE
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PROCDEPOBJ
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
ORA-39082: Object type TRIGGER:"HR"."SECURE_EMPLOYEES" created with compilation warnings
ORA-39082: Object type TRIGGER:"HR"."UPDATE_JOB_HISTORY" created with compilation warnings
Job "HEMANT"."SYS_IMPORT_TRANSPORTABLE_01" completed with 41 error(s) at Sun Jan 15 16:51:40 2017 elapsed 0 00:01:23

[oracle@ora12102 Desktop]$


The key error is the failure on the ORDERS table creation because of a TimeZone mismatch !   So, there is a lesson to be learnt !

.
.
.

Categories: DBA Blogs

Pages