Hemant K Chitale

Subscribe to Hemant K Chitale feed
I am an Oracle Database Specialist in Singapore.
get an rss feed of this blog at http://hemantoracledba.blogspot.com/feeds/posts/default?alt=rss
follow me on twitter : @HemantKChitale
Updated: 14 hours 19 min ago

12c MultiTenant Posts -- 6 : Partial (aka Subset) Cloning of PDB

Mon, 2017-07-10 09:27
Note : This is a 12.2 feature.

Normally, if you clone a PDB, you'd get a full copy with all the tablespaces.  Now, in 12.2, you can exclude non-essential tablespaces by specifying USER TABLESPACES -- those that you want cloned.  (SYSTEM, SYSAUX and Local UNDO will certainly be cloned).

Let me start with the "NEWPDB" PDB (that I've used in previous examples) that has one more schema and tablespace:

$sqlplus system/oracle@NEWPDB

SQL*Plus: Release 12.2.0.1.0 Production on Mon Jul 10 21:52:55 2017

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Last Successful login time: Mon Jul 10 2017 11:04:00 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> select tablespace_name from dba_tablespaces order by 1;

TABLESPACE_NAME
------------------------------
HRDATA
MYDATA
SYSAUX
SYSTEM
TEMP
UNDOTBS1

6 rows selected.

SQL> col owner format a8
SQL> col segment_name format a30
SQL> col tablespace_name format a8
SQL> select owner, segment_name, tablespace_name
2 from dba_segments
3 where tablespace_name like '%DATA'
4 order by 1,2
5 /

OWNER SEGMENT_NAME TABLESPA
-------- ------------------------------ --------
HEMANT BIN$UVb24iaCIE/gUwEAAH/WaQ==$0 MYDATA
HEMANT BIN$UVb24iaIIE/gUwEAAH/WaQ==$0 MYDATA
HEMANT HKC_STORE_FILE MYDATA
HEMANT I MYDATA
HEMANT OBJ_LIST MYDATA
HEMANT SYS_IL0000073525C00003$$ MYDATA
HEMANT SYS_IL0000073532C00003$$ MYDATA
HEMANT SYS_IL0000073535C00003$$ MYDATA
HEMANT SYS_LOB0000073525C00003$$ MYDATA
HEMANT SYS_LOB0000073532C00003$$ MYDATA
HEMANT SYS_LOB0000073535C00003$$ MYDATA
HEMANT T MYDATA
HR EMPLOYEES HRDATA

13 rows selected.

SQL>
SQL> select * from hr.employees;

EMPLOYEE_ID FIRST_NAME LAST_NAME
----------- ------------------------------ ------------------------------
HIRE_DATE DEPARTMENT_ID SALARY EMAIL_ID
--------- ------------- ---------- ---------------------------------------------
1 Hemant Chitale
06-JUL-17 1 15000 hemant@mydomain.com


SQL>


Besides, the HEMANT objects in the MYDATA tablespace, I now have HR owning an EMPLOYEES table in the HRDATA tablespace.

Now, I want to clone the NEWPDB tablespace but want to exclude HR data.

First, I set a target location for the datafiles.

$sqlplus '/ as sysdba'

SQL*Plus: Release 12.2.0.1.0 Production on Mon Jul 10 21:57:33 2017

Copyright (c) 1982, 2016, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> show parameter db_create_file

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest string
SQL> alter session set db_create_file_dest='/u02/oradata';

Session altered.

SQL>


Next, I create my Partial (or SubSet) Clone PDB:

SQL> create pluggable database NONHR from NEWPDB user_tablespaces=('MYDATA');

Pluggable database created.

SQL>
SQL> select con_id, file#, name
2 from v$datafile
3 order by 1,2
4 /

CON_ID FILE#
---------- ----------
NAME
--------------------------------------------------------------------------------
1 1
/u01/app/oracle/oradata/orcl12c/system01.dbf

1 3
/u01/app/oracle/oradata/orcl12c/sysaux01.dbf

1 7
/u01/app/oracle/oradata/orcl12c/users01.dbf

1 15
/u01/app/oracle/oradata/orcl12c/undotbs2.dbf

2 5
/u01/app/oracle/oradata/orcl12c/pdbseed/system01.dbf

2 6
/u01/app/oracle/oradata/orcl12c/pdbseed/sysaux01.dbf

2 8
/u01/app/oracle/oradata/orcl12c/pdbseed/undotbs01.dbf

3 9
/u01/app/oracle/oradata/orcl12c/orcl/system01.dbf

3 10
/u01/app/oracle/oradata/orcl12c/orcl/sysaux01.dbf

3 11
/u01/app/oracle/oradata/orcl12c/orcl/undotbs01.dbf

3 12
/u01/app/oracle/oradata/orcl12c/orcl/users01.dbf

3 13
/u01/app/oracle/oradata/orcl12c/orcl/APEX_1991375173370654.dbf

3 14
/u01/app/oracle/oradata/orcl12c/orcl/APEX_1993195660370985.dbf

4 16
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSTEM_FNO-16_0as7a8di

4 17
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSAUX_FNO-17_09s7a8d2

4 18
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-UNDOTBS1_FNO-18_0bs7a8e1

4 19
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-MYDATA_FNO-19_0cs7a8e4

4 20
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-HRDATA_FNO-20_0ds7a8e5

5 21
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_system_dp72
3vp5_.dbf

5 22
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_sysaux_dp72
3vsz_.dbf

5 23
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_undotbs1_dp
723vt1_.dbf

5 24
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_mydata_dp72
3vt3_.dbf


22 rows selected.

SQL>
SQL> select con_id, name, open_mode
2 from v$pdbs
3 order by 1
4 /

CON_ID
----------
NAME
--------------------------------------------------------------------------------
OPEN_MODE
----------
2
PDB$SEED
READ ONLY

3
ORCL
READ WRITE

4
NEWPDB
READ WRITE

5
NONHR
MOUNTED


SQL>
SQL> alter pluggable database nonhr open;

Pluggable database altered.

SQL>


I can identify the new PDB "NONHR" as CON_ID=5.
Note that in the CREATE PLUGGABLE DATABASE command with the USER_TABLESPACES clause, I can also specify either of COPY, NOCOPY, MOVE, NO DATA or even SNAPSHOT COPY.  This is the simplest Subset Clone that is a Copy with Data.

Let's create the TNSNAMES.ORA entry for NONHR:

NONHR =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = nonhr)
)
)


Let's now connect to NONHR and confirm its contents.

SQL> connect system/oracle@NONHR
Connected.
SQL> show con_id

CON_ID
------------------------------
5
SQL> show con_name

CON_NAME
------------------------------
NONHR

SQL> select tablespace_name
2 from dba_tablespaces
3 order by 1
4 /

TABLESPACE_NAME
------------------------------
HRDATA
MYDATA
SYSAUX
SYSTEM
TEMP
UNDOTBS1

6 rows selected.

SQL> select file_name from dba_data_files
2 where tablespace_name = 'HRDATA'
3 /

no rows selected

SQL> select owner, segment_name, segment_type
2 from dba_segments
3 where tablespace_name = 'HRDATA'
4 /

no rows selected

SQL>
SQL> select tablespace_name, file_name
2 from dba_data_files
3 order by 1
4 /

TABLESPACE_NAME
------------------------------
FILE_NAME
--------------------------------------------------------------------------------
MYDATA
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_mydata_dp72
3vt3_.dbf

SYSAUX
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_sysaux_dp72
3vsz_.dbf

SYSTEM
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_system_dp72
3vp5_.dbf

UNDOTBS1
/u02/oradata/ORCL12C/53F8012866211264E0530100007FD493/datafile/o1_mf_undotbs1_dp
723vt1_.dbf


SQL>
SQL> select segment_name, segment_type
2 from dba_segments
3 where owner = 'HR'
4 /

no rows selected

SQL> select username
2 from dba_users
3 where username = 'HR'
4 /

USERNAME
--------------------------------------------------------------------------------
HR

SQL>
SQL> select object_name, object_type
2 from dba_objects
3 where owner = 'HR'
4 /

OBJECT_NAME
--------------------------------------------------------------------------------
OBJECT_TYPE
-----------------------
EMPLOYEES
TABLE


SQL>
SQL> select owner, segment_name
2 from dba_segments
3 where tablespace_name = 'MYDATA'
4 /

OWNER SEGMENT_NAME
-------- ------------------------------
HEMANT BIN$UVb24iaCIE/gUwEAAH/WaQ==$0
HEMANT BIN$UVb24iaIIE/gUwEAAH/WaQ==$0
HEMANT HKC_STORE_FILE
HEMANT I
HEMANT OBJ_LIST
HEMANT SYS_IL0000073525C00003$$
HEMANT SYS_IL0000073532C00003$$
HEMANT SYS_IL0000073535C00003$$
HEMANT SYS_LOB0000073525C00003$$
HEMANT SYS_LOB0000073532C00003$$
HEMANT SYS_LOB0000073535C00003$$
HEMANT T

12 rows selected.

SQL>
SQL> select count(*) from hemant.obj_list;

COUNT(*)
----------
145282

SQL>


So, what has been copied to the NONHR PDB?  The HRDATA Tablespace, but not the DataFile.  The HR User and Table (definition only, no data), but not the Segment.
However, for the MYDATA Tablespace that was identified as a USER_TABLESPACE in the CREATE PLUGGABLE DATABASE statement, the Tablespace, Datafile, User, Table and Segment have all been copied.

Therefore, NONHR does not have the HR data!   I can drop the User and Tablespace.

SQL> drop tablespace hrdata including contents;

Tablespace dropped.

SQL> drop user hr;

User dropped.

SQL>


However, HR is still present in NEWPDB where NONHR was cloned from:

SQL> connect system/oracle@NEWPDB
Connected.
SQL> select owner, segment_name
2 from dba_segments
3 where tablespace_name = 'HRDATA'
4 /

OWNER SEGMENT_NAME
-------- ------------------------------
HR EMPLOYEES

SQL> select * from hr.employees;

EMPLOYEE_ID FIRST_NAME LAST_NAME
----------- ------------------------------ ------------------------------
HIRE_DATE DEPARTMENT_ID SALARY EMAIL_ID
--------- ------------- ---------- ---------------------------------------------
1 Hemant Chitale
06-JUL-17 1 15000 hemant@mydomain.com


SQL> show con_id

CON_ID
------------------------------
4
SQL> show con_name

CON_NAME
------------------------------
NEWPDB
SQL>
SQL> select tablespace_name, file_name
2 from dba_data_files
3 order by 1
4 /

TABLESPACE_NAME
------------------------------
FILE_NAME
--------------------------------------------------------------------------------
HRDATA
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-HRDATA_FNO-20_0ds7a8e5

MYDATA
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-MYDATA_FNO-19_0cs7a8e4

SYSAUX
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSAUX_FNO-17_09s7a8d2

SYSTEM
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSTEM_FNO-16_0as7a8di

UNDOTBS1
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-UNDOTBS1_FNO-18_0bs7a8e1


SQL>


So, 12.2 introduces the ability to create a clone PDB database that is a SubSet (i.e. selected User Tablespaces data) of an existing PDB.

(Note : NEWPDB is in /u03 where it was moved from /u02 earlier as a Relocated Database while NONHR is in /u02 where it was created with OMF based on DB_CREATE_FILE_DEST).
.
.
.

Categories: DBA Blogs

New Video on 11g Nested Loop (enhancement)

Sat, 2017-07-08 10:41
I've published a new Video on the Optimizer Nested Loop Enhancement in 11g :  11g Nested Loop

.
.
.
Categories: DBA Blogs

12c MultiTenant Posts -- 5 : Flashback a PDB

Sun, 2017-07-02 10:31
12.2 allows FLASHBACK DATABASE for a Pluggable Database.

Note that ALTER DATABASE FLASHBACK ON is not enabled by default in a newly created database -- you must issue this command at the CDB level to enable Flashback for all the Pluggable Databases.

SQL> connect hemant/hemant@NEWPDB
Connected.
SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
OBJ_LIST
HKC_STORE_FILE
T

SQL> select count(*) from obj_list;

COUNT(*)
----------
145282

SQL> select count(*) from hkc_store_file;

COUNT(*)
----------
1

SQL> truncate table obj_list;

Table truncated.

SQL> drop table hkc_store_file;

Table dropped.

SQL>


Now, let me Flashback the PDB.

SQL> connect / as sysdba
Connected.
SQL> alter pluggable database newpdb close;

Pluggable database altered.

SQL> flashback pluggable database newpdb to timestamp sysdate-3/1440;

Flashback complete.

SQL> alter pluggable database newpdb open;
alter pluggable database newpdb open
*
ERROR at line 1:
ORA-01113: file 19 needs media recovery
ORA-01110: data file 19:
'/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-MYDATA_FNO-19_0cs7a8e4'


SQL> alter pluggable database newpdb open resetlogs;

Pluggable database altered.

SQL>


Let me test the data.

SQL> connect hemant/hemant@NEWPDB
Connected.
SQL> select count(*) from obj_list;

COUNT(*)
----------
145282

SQL> select count(*) from hkc_store_file;

COUNT(*)
----------
1

SQL>


Yes, the FLASHBACK DATABASE is successful.

What are the pre-requisites ?

1.  ALTER DATABASE FLASHBACK ON  at the CDB
2.  ARCHIVELOG at the CDB
3.  LOCAL UNDO enabled -- highly recommended else a subsequent Point In Time Recovery of the CDB may prevent OPENing the PDB
4.  OPEN RESETLOGS for the PDB

.
.
.

Categories: DBA Blogs

12c MultiTenant Posts -- 4 : Relocate Individual PDB using RMAN

Tue, 2017-06-20 22:29
Given the current location of a PDB :

SQL> alter session set container=NEWPDB;

Session altered.

SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7
f8go_.dbf

/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7
f8hf_.dbf

/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dk
j7f8hg_.dbf

/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbc
p0wz_.dbf

FILE_NAME
--------------------------------------------------------------------------------


SQL>


I can use RMAN to relocate it. First I take an Image Copy Backup to the new location

RMAN> backup as copy pluggable database newpdb format '/u03/oradata/NEWPDB/%U';

Starting backup at 20-JUN-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK
channel ORA_DISK_1: starting datafile copy
input datafile file number=00017 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7f8hf_.dbf
output file name=/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSAUX_FNO-17_09s7a8d2 tag=TAG20170620T231338 RECID=4 STAMP=947200428
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:16
channel ORA_DISK_1: starting datafile copy
input datafile file number=00016 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7f8go_.dbf
output file name=/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSTEM_FNO-16_0as7a8di tag=TAG20170620T231338 RECID=5 STAMP=947200441
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00018 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dkj7f8hg_.dbf
output file name=/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-UNDOTBS1_FNO-18_0bs7a8e1 tag=TAG20170620T231338 RECID=6 STAMP=947200451
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting datafile copy
input datafile file number=00019 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbcp0wz_.dbf
output file name=/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-MYDATA_FNO-19_0cs7a8e4 tag=TAG20170620T231338 RECID=7 STAMP=947200454
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
Finished backup at 20-JUN-17

Starting Control File and SPFILE Autobackup at 20-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/autobackup/2017_06_20/o1_mf_s_947200455_dnms48pp_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 20-JUN-17

RMAN>


Then I switch the database file pointers to the new location.

RMAN> alter pluggable database newpdb close;

Statement processed

RMAN> switch pluggable database newpdb to copy;

datafile 16 switched to datafile copy "/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSTEM_FNO-16_0as7a8di"
datafile 17 switched to datafile copy "/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSAUX_FNO-17_09s7a8d2"
datafile 18 switched to datafile copy "/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-UNDOTBS1_FNO-18_0bs7a8e1"
datafile 19 switched to datafile copy "/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-MYDATA_FNO-19_0cs7a8e4"

RMAN> recover pluggable database newpdb;

Starting recover at 20-JUN-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=42 device type=DISK

starting media recovery
media recovery complete, elapsed time: 00:00:00

Finished recover at 20-JUN-17

RMAN> alter pluggable database newpdb open;

Statement processed

RMAN>


I can now verify the new location for the database files.

 
SQL> alter session set container=NEWPDB;

Session altered.

SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSTEM_FNO-16_0as7a8di
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-SYSAUX_FNO-17_09s7a8d2
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-UNDOTBS1_FNO-18_0bs7a8e1
/u03/oradata/NEWPDB/data_D-ORCL12C_I-768045447_TS-MYDATA_FNO-19_0cs7a8e4

SQL>


The datafiles at the old location still remain and can be manually deleted later as they are no longer part of the database.  Note that those datafiles are still registered by RMAN as COPY

RMAN> list copy of pluggable database newpdb;

using target database control file instead of recovery catalog
List of Datafile Copies
=======================

Key File S Completion Time Ckp SCN Ckp Time Sparse
------- ---- - --------------- ---------- --------------- ------
8 16 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7f8go_.dbf
Container ID: 4, PDB Name: NEWPDB

9 17 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7f8hf_.dbf
Container ID: 4, PDB Name: NEWPDB

10 18 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dkj7f8hg_.dbf
Container ID: 4, PDB Name: NEWPDB

11 19 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbcp0wz_.dbf
Container ID: 4, PDB Name: NEWPDB


RMAN>


If I manually delete the old location files, I'd need to also delete them from the RMAN Registry.  Alternatively, I can directly delete them from RMAN.

RMAN> delete copy of pluggable database newpdb;

allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=278 device type=DISK
List of Datafile Copies
=======================

Key File S Completion Time Ckp SCN Ckp Time Sparse
------- ---- - --------------- ---------- --------------- ------
8 16 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7f8go_.dbf
Container ID: 4, PDB Name: NEWPDB

9 17 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7f8hf_.dbf
Container ID: 4, PDB Name: NEWPDB

10 18 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dkj7f8hg_.dbf
Container ID: 4, PDB Name: NEWPDB

11 19 A 20-JUN-17 2188274 20-JUN-17 NO
Name: /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbcp0wz_.dbf
Container ID: 4, PDB Name: NEWPDB


Do you really want to delete the above objects (enter YES or NO)? YES
deleted datafile copy
datafile copy file name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7f8go_.dbf RECID=8 STAMP=947200522
deleted datafile copy
datafile copy file name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7f8hf_.dbf RECID=9 STAMP=947200522
deleted datafile copy
datafile copy file name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dkj7f8hg_.dbf RECID=10 STAMP=947200522
deleted datafile copy
datafile copy file name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbcp0wz_.dbf RECID=11 STAMP=947200522
Deleted 4 objects


RMAN>


I still need to relocate the TEMP Tablespace Tempfile.

SQL> alter session set container=NEWPDB;

Session altered.

SQL> select file_name from dba_temp_files;

FILE_NAME
--------------------------------------------------------------------------------
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_temp_dkj7f8
hg_.dbf


SQL> alter tablespace temp add tempfile '/u03/oradata/NEWPDB/temp01.dbf' size 100M;

Tablespace altered.

SQL>
SQL> alter tablespace temp drop tempfile '/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_temp_dkj7f8hg_.dbf';

Tablespace altered.

SQL> select file_name from dba_temp_files;

FILE_NAME
--------------------------------------------------------------------------------
/u03/oradata/NEWPDB/temp01.dbf

SQL>


So, I used the same Image Copy method we'd use in 11g databases to relocate a 12c Pluggable Database when within the same server and storage.

To verify that I can access the PDB in the new location :

SQL> connect hemant/hemant@NEWPDB
Connected.
SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
OBJ_LIST
HKC_STORE_FILE
T

SQL> insert into obj_list select * from obj_list;

72641 rows created.

SQL> commit;

Commit complete.

SQL>


.
.

Categories: DBA Blogs

12c MultiTenant Posts -- 3 : Restore Individual PB

Wed, 2017-06-07 05:36
Restoring the single PDB in a Container Database.

$rman target /

Recovery Manager: Release 12.2.0.1.0 - Production on Wed Jun 7 06:34:47 2017

Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.

connected to target database: ORCL12C (DBID=768045447)

RMAN> restore pluggable database newpdb;

Starting restore at 07-JUN-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=30 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00016 to /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7f8go_.dbf
channel ORA_DISK_1: restoring datafile 00017 to /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7f8hf_.dbf
channel ORA_DISK_1: restoring datafile 00018 to /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dkj7f8hg_.dbf
channel ORA_DISK_1: restoring datafile 00019 to /u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbcp0wz_.dbf
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/4F793A6D323D1344E0530100007FABC7/backupset/2017_06_05/o1_mf_nnndf_TAG20170605T061347_dmbcpvjr_.bkp
channel ORA_DISK_1: piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/4F793A6D323D1344E0530100007FABC7/backupset/2017_06_05/o1_mf_nnndf_TAG20170605T061347_dmbcpvjr_.bkp tag=TAG20170605T061347
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:25
Finished restore at 07-JUN-17

RMAN> recover pluggable database newpdb;

Starting recover at 07-JUN-17
using channel ORA_DISK_1

starting media recovery

archived log for thread 1 with sequence 16 is already on disk as file /u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/archivelog/2017_06_06/o1_mf_1_16_dmgyksjy_.arc
archived log for thread 1 with sequence 17 is already on disk as file /u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/archivelog/2017_06_07/o1_mf_1_17_dmhj6nbk_.arc
channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=1 sequence=15
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/backupset/2017_06_05/o1_mf_annnn_TAG20170605T061438_dmbcrgml_.bkp
channel ORA_DISK_1: piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/backupset/2017_06_05/o1_mf_annnn_TAG20170605T061438_dmbcrgml_.bkp tag=TAG20170605T061438
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
archived log file name=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/archivelog/2017_06_07/o1_mf_1_15_dmhoqlq3_.arc thread=1 sequence=15
channel default: deleting archived log(s)
archived log file name=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/archivelog/2017_06_07/o1_mf_1_15_dmhoqlq3_.arc RECID=4 STAMP=946017330
media recovery complete, elapsed time: 00:00:11
Finished recover at 07-JUN-17

RMAN> alter pluggable database newpdb open;

Statement processed

RMAN>


I had connected to the root to restore the PDB.
.
.

.
Categories: DBA Blogs

12c MultiTenant Posts -- 2 : Backup individual PDB

Mon, 2017-06-05 05:16
In the 12c MultiTenant architecture, you can backup (and restore) individual PDBs.  However, you must also backup the CDB (Root) !  You cannot restore a PDB as a "standalone" database --- it has to be restored back to the CDB.

SQL> startup mount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 8798312 bytes
Variable Size 343936920 bytes
Database Buffers 478150656 bytes
Redo Buffers 7974912 bytes
Database mounted.
SQL> alter database archivelog;

Database altered.

SQL> alter database open;

Database altered.

SQL> alter pluggable database newpdb open;

Pluggable database altered.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
$
$rman target /

Recovery Manager: Release 12.2.0.1.0 - Production on Mon Jun 5 06:10:17 2017

Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.

connected to target database: ORCL12C (DBID=768045447)

RMAN> backup database root;

Starting backup at 05-JUN-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=270 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/u01/app/oracle/oradata/orcl12c/system01.dbf
input datafile file number=00003 name=/u01/app/oracle/oradata/orcl12c/sysaux01.dbf
input datafile file number=00015 name=/u01/app/oracle/oradata/orcl12c/undotbs2.dbf
input datafile file number=00007 name=/u01/app/oracle/oradata/orcl12c/users01.dbf
channel ORA_DISK_1: starting piece 1 at 05-JUN-17
channel ORA_DISK_1: finished piece 1 at 05-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/backupset/2017_06_05/o1_mf_nnndf_TAG20170605T061033_dmbcjttq_.bkp tag=TAG20170605T061033 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:45
Finished backup at 05-JUN-17

Starting Control File and SPFILE Autobackup at 05-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/autobackup/2017_06_05/o1_mf_s_945843079_dmbcl8fm_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 05-JUN-17

RMAN>
RMAN> backup pluggable database newpdb;

Starting backup at 05-JUN-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00017 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7f8hf_.dbf
input datafile file number=00016 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7f8go_.dbf
input datafile file number=00018 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dkj7f8hg_.dbf
input datafile file number=00019 name=/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_mydata_dmbcp0wz_.dbf
channel ORA_DISK_1: starting piece 1 at 05-JUN-17
channel ORA_DISK_1: finished piece 1 at 05-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/4F793A6D323D1344E0530100007FABC7/backupset/2017_06_05/o1_mf_nnndf_TAG20170605T061347_dmbcpvjr_.bkp tag=TAG20170605T061347 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:25
Finished backup at 05-JUN-17

Starting Control File and SPFILE Autobackup at 05-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/autobackup/2017_06_05/o1_mf_s_945843252_dmbcqo6t_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 05-JUN-17

RMAN>
RMAN> backup archivelog all delete input;

Starting backup at 05-JUN-17
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=15 RECID=1 STAMP=945843278
channel ORA_DISK_1: starting piece 1 at 05-JUN-17
channel ORA_DISK_1: finished piece 1 at 05-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/backupset/2017_06_05/o1_mf_annnn_TAG20170605T061438_dmbcrgml_.bkp tag=TAG20170605T061438 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
channel ORA_DISK_1: deleting archived log(s)
archived log file name=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/archivelog/2017_06_05/o1_mf_1_15_dmbcrfq1_.arc RECID=1 STAMP=945843278
Finished backup at 05-JUN-17

Starting Control File and SPFILE Autobackup at 05-JUN-17
piece handle=/u01/app/oracle/fast_recovery_area/orcl12c/ORCL12C/autobackup/2017_06_05/o1_mf_s_945843280_dmbcrjdg_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 05-JUN-17

RMAN>


Thus, a PDB in a MultiTenant environment can be backed-up but the Root must always be backed up.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 11 : Managing Services in RAC (with PDB) (Video)

Sun, 2017-05-21 11:01
I have uploaded a video on managing services (add, start, check, remove) in a 12c RAC database with a Pluggable Database.
.
.
.


Categories: DBA Blogs

12c MultiTenant Posts -- 1 : Creating a PDB in a different directory

Sun, 2017-05-14 04:12
What if the current (default) location for Pluggable Databases is running out of space and, yet, you want to create an Oracle Managed Files Pluggable Database ?

First, I start with this configuration :

SQL> select con_id, file#, substr(name,1,56)
2 from v$datafile
3 order by 1,2;

CON_ID FILE#
---------- ----------
SUBSTR(NAME,1,56)
--------------------------------------------------------------------------------
1 1
/u01/app/oracle/oradata/orcl12c/system01.dbf

1 3
/u01/app/oracle/oradata/orcl12c/sysaux01.dbf

1 7
/u01/app/oracle/oradata/orcl12c/users01.dbf

1 15
/u01/app/oracle/oradata/orcl12c/undotbs2.dbf

2 5
/u01/app/oracle/oradata/orcl12c/pdbseed/system01.dbf

2 6
/u01/app/oracle/oradata/orcl12c/pdbseed/sysaux01.dbf

2 8
/u01/app/oracle/oradata/orcl12c/pdbseed/undotbs01.dbf

3 9
/u01/app/oracle/oradata/orcl12c/orcl/system01.dbf

3 10
/u01/app/oracle/oradata/orcl12c/orcl/sysaux01.dbf

3 11
/u01/app/oracle/oradata/orcl12c/orcl/undotbs01.dbf

3 12
/u01/app/oracle/oradata/orcl12c/orcl/users01.dbf

3 13
/u01/app/oracle/oradata/orcl12c/orcl/APEX_19913751733706

3 14
/u01/app/oracle/oradata/orcl12c/orcl/APEX_19931956603709


13 rows selected.

SQL>


Currently, I have one Pluggable Database (CON_ID=3) but none of the database files are OMF.  I don't have enough space in /u01 and want to create the new PDB in /u02 and also use OMF.  As long as I have /u02/oradata precreated by the system administrator, I can :

SQL> show parameter db_create_file

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest string
SQL> alter session set db_create_file_dest='/u02/oradata';

Session altered.

SQL> create pluggable database NEWPDB admin user newpdb_adm identified by newpdb_adm;

Pluggable database created.

SQL> alter pluggable database NEWPDB open;

Pluggable database altered.

SQL> select con_id, open_mode
2 from v$pdbs
3 where name = 'NEWPDB'
4 /

CON_ID OPEN_MODE
---------- ----------
4 READ WRITE

SQL>
SQL> select file#, name
2 from v$datafile
3 where con_id=4
4 order by file#
5 /

FILE#
----------
NAME
--------------------------------------------------------------------------------
16
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_system_dkj7
f8go_.dbf

17
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_sysaux_dkj7
f8hf_.dbf

18
/u02/oradata/ORCL12C/4F793A6D323D1344E0530100007FABC7/datafile/o1_mf_undotbs1_dk
j7f8hg_.dbf


SQL>
SQL> show parameter db_unique_name

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_unique_name string orcl12c
SQL>


Note how the parent folder name "ORCL12C" is inherited from the DB_UNIQUE_NAME.  I can now proceed to setup this new PDB.  Later, I  can migrate it as an OMF PDB to another Container Database.
.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 10 : Video on Database Startup

Mon, 2017-05-08 09:38
I've created a short video on manually starting up a RAC database that has a PDB with two custom services.  If a database is shutdown before the cluster is shutdown, the cluster startup does not automatically startup the database instance, so I demonstrate a manual startup.

See the video at:  https://youtu.be/saFvo9QhYSI


.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 9 : Adding a Service to a PDB in RAC

Sat, 2017-04-29 11:02
My 2node RAC environment has 1 PDB.  Here I add (create) a new Service to the PDB.

Oracle recommends using srvctl instead of DBMS_SERVICE to add a Service.

Can I add a service without having the PDB OPEN ?

[oracle@collabn1 ~]$ srvctl start database -d RAC
[oracle@collabn1 ~]$ srvctl add service -db RAC -service MYSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT -pdb PDB
[oracle@collabn1 ~]$
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:16:01 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select con_id, name, open_mode from v$pdbs;

CON_ID NAME OPEN_MODE
---------- ------------------------------ ----------
2 PDB$SEED READ ONLY
3 PDB MOUNTED

SQL> alter pluggable database pdb open; -- command issued in both instances

Pluggable database altered.

SQL>
SQL> select con_id, pdb, name, creation_date, clb_goal
2 from v$services
3 where pdb='PDB'
4 /

CON_ID PDB
---------- ------------------------------
NAME CREATION_ CLB_G
---------------------------------------------------------------- --------- -----
3 PDB
pdb.racattack 29-DEC-16 LONG


SQL>


I add the Service to the TNSNAMES.ORA and try to connect to it.

MYSVC =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = MYSVC.racattack)
)
)

[oracle@collabn1 ~]$ tnsping MYSVC

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 29-APR-2017 23:22:43

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = MYSVC.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ sqlplus hemant/hemant@MYSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:22:51 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-12514: TNS:listener does not currently know of service requested in connect
descriptor


Enter user-name:


So, apparently, the service isn't running.   Shall I try re-adding the service ?

[oracle@collabn1 ~]$ srvctl add service -db RAC -service MYSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT -pdb PDB
PRKO-3117 : Service MYSVC already exists in database RAC
[oracle@collabn1 ~]$ srvctl remove service -db RAC -service MYSVC -pdb PDB
PRKO-2002 : Invalid command line option: -pdb
[oracle@collabn1 ~]$ srvctl remove service -db RAC -service MYSVC
[oracle@collabn1 ~]$ srvctl add service -db RAC -service MYSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT -pdb PDB
[oracle@collabn1 ~]$
[oracle@collabn1 ~]$ sqlplus hemant/hemant@MYSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:31:15 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-12514: TNS:listener does not currently know of service requested in connect
descriptor


Enter user-name:


Yet, it doesn't work !   How do I add and start service to a PDB ?  What's missing ?

[oracle@collabn1 ~]$ srvctl remove service -db RAC -service MYSVC
[oracle@collabn1 ~]$ srvctl start service -db RAC -service MYSVC
[oracle@collabn1 ~]$

SQL> select distinct name from v$services;

NAME
----------------------------------------------------------------
pdb.racattack
RAC.racattack
MYSVC
SYS$BACKGROUND
RAC_DGB
RACXDB
SYS$USERS


[oracle@collabn1 ~]$ sqlplus hemant/hemant@MYSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:43:01 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Sun Apr 16 2017 23:30:21 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA
MY_NEW_TABLE

SQL>


I had forgotten to *START* the service !   Let me go back and add another service with the PDB CLOSEd.

SQL> alter pluggable database pdb close immediate;  -- on both instances

Pluggable database altered.

SQL>

[oracle@collabn1 ~]$ srvctl add service -db RAC -pdb PDB -service NEWSVC -preferred RAC1,RAC2 \
> -tafpolicy BASIC -clbgoal SHORT
[oracle@collabn1 ~]$

SQL> alter pluggable database pdb open; -- on both instances

Pluggable database altered.

SQL>

NEWSVC =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = NEWSVC.racattack)
)
)

[oracle@collabn1 ~]$ srvctl start service -db RAC -service NEWSVC
[oracle@collabn1 ~]$ tnsping NEWSVC

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 29-APR-2017 23:54:38

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = NEWSVC.racattack)))
OK (0 msec)


[oracle@collabn1 ~]$ sqlplus hemant/hemant@NEWSVC

SQL*Plus: Release 12.1.0.2.0 Production on Sat Apr 29 23:55:25 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Sat Apr 29 2017 23:54:51 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA
MY_NEW_TABLE

SQL>


However, these entries in PDB_PLUG_IN_VIOLATIONS seem to be a bug :

SQL> alter pluggable database pdb close immediate;

Pluggable database altered.

SQL> alter pluggable database pdb open;

Pluggable database altered.

SQL> select * from pdb_plug_in_violations;

TIME
---------------------------------------------------------------------------
NAME
------------------------------
CAUSE TYPE
---------------------------------------------------------------- ---------
ERROR_NUMBER LINE
------------ ----------
MESSAGE
--------------------------------------------------------------------------------
STATUS
---------
ACTION
--------------------------------------------------------------------------------
29-APR-17 11.58.32.409572 PM
PDB
Service Name Conflict WARNING
0 1
Service name or network name of service MYSVC in the PDB is invalid or conflicts
with an existing service name or network name in the CDB.
PENDING
Drop the service and recreate it with an appropriate name.

29-APR-17 11.58.32.412142 PM
PDB
Service Name Conflict WARNING
0 2
Service name or network name of service NEWSVC in the PDB is invalid or conflict
s with an existing service name or network name in the CDB.
PENDING
Drop the service and recreate it with an appropriate name.


SQL>


So, remember, it is not sufficient to ADD a Service. You must also START the Service using srvctl.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8i : Switchback from SingleInstance to RAC

Sun, 2017-04-16 10:59
Earlier this week (10-April), I had done a Switchover from the 2node RAC database to a SingleInstance database.

It is time now to Switchback from SingleInstance to RAC.

First, I check the status of the two databases :

On STBY (the current Primary) :

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
READ WRITE PRIMARY

SQL> select instance_name, host_name from v$instance;

INSTANCE_NAME
----------------
HOST_NAME
----------------------------------------------------------------
STBY
oem132.racattack


SQL>


Next, I check on the RAC database instance RAC1 (the current Standby) :

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
MOUNTED PHYSICAL STANDBY

SQL> select instance_name, status, host_name from gv$instance;

INSTANCE_NAME STATUS
---------------- ------------
HOST_NAME
----------------------------------------------------------------
RAC1 MOUNTED
collabn1.racattack

RAC2 MOUNTED
collabn2.racattack


SQL>


I also confirm that only one of the two RAC instances is doing recovery (in 12.1 we have only 1 instance in RAC doing recovery) by verifying the (automatic) message in the alert log for RAC1 :

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT  NODELAY
Sun Apr 16 23:05:50 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Sun Apr 16 23:05:50 2017
MRP0 started with pid=52, OS id=16739
Sun Apr 16 23:05:50 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Sun Apr 16 23:05:55 2017
Started logmerger process
Sun Apr 16 23:05:56 2017
Managed Standby Recovery starting Real Time Apply
Sun Apr 16 23:06:10 2017
Parallel Media Recovery started with 2 slaves
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY


Now, I add some data to the PDB Pluggable Database currently running in STBY :

[oracle@oem132 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 16 23:29:37 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 10 2017 23:43:30 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select count(*) from my_data;

COUNT(*)
----------
300

SQL> create table my_new_table as select * from my_data;

Table created.

SQL>


I am now ready to switchover from STBY to RAC. However, I have a problem because I started the RAC database (and , therefore, DataGuard Broker) before I  started the SingleInstance node and database.  I have drcRAC1.log and drcRAC2.log both reporting :

04/16/2017 23:05:38
Failed to connect to remote database stby. Error is ORA-12543
Failed to send message to site stby. Error code is ORA-12543.
database rac unable to contact primary database for version check; status ORA-12543
completing bootstrap of this database


The fix is to have the Standby RAC database started *after* the singleInstance Primary and verify that Managed Recovery is restarted in RAC1 :

[oracle@collabn1 ~]$ srvctl stop database -d RAC
[oracle@collabn1 ~]$ srvctl start database -d RAC


ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Sun Apr 16 23:43:58 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Sun Apr 16 23:43:59 2017
MRP0 started with pid=53, OS id=2033
Sun Apr 16 23:43:59 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Sun Apr 16 23:44:03 2017
RFS[2]: Assigned to RFS process (PID:1922)
RFS[2]: Selected log 5 for thread 1 sequence 76 dbid 2519807290 branch 931825279
Sun Apr 16 23:44:04 2017
Started logmerger process
Sun Apr 16 23:44:07 2017
Managed Standby Recovery starting Real Time Apply
RFS[1]: Selected log 7 for thread 1 sequence 78 dbid 2519807290 branch 931825279
Sun Apr 16 23:44:20 2017
Archived Log entry 144 added for thread 1 sequence 77 ID 0x96bd0722 dest 1:
Sun Apr 16 23:44:25 2017
Archived Log entry 145 added for thread 1 sequence 76 ID 0x96bd0722 dest 1:
Sun Apr 16 23:44:27 2017
Parallel Media Recovery started with 2 slaves
Sun Apr 16 23:44:27 2017
Waiting for all non-current ORLs to be archived...
Sun Apr 16 23:44:27 2017
All non-current ORLs have been archived.
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY


Now, recheck the configuration from the SingleInstance node and then  Switchover to RAC :

[oracle@oem132 ~]$ dgmgrl
DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production

Copyright (c) 2000, 2013, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
DGMGRL> connect sys/racattack@STBY
Connected as SYSDBA.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
stby - Primary database
rac - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 16 seconds ago)

DGMGRL> switchover to rac;
Performing switchover NOW, please wait...
Operation requires a connection to instance "RAC1" on database "rac"
Connecting to instance "RAC1"...
Connected as SYSDBA.
New primary database "rac" is opening...
Operation requires start up of instance "STBY" on database "stby"
Starting instance "STBY"...
Unable to connect to database using (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=STBY_DGMGRL)(INSTANCE_NAME=STBY)(SERVER=DEDICATED)))
ORA-12514: TNS:listener does not currently know of service requested in connect descriptor

Failed.
Warning: You are no longer connected to ORACLE.

Please complete the following steps to finish switchover:
start up and mount instance "STBY" of database "stby"

DGMGRL>


The ORA-12514 error here is acceptable. I only need to startup STBY manually.

DGMGRL> exit
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 16 23:51:25 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 289409776 bytes
Database Buffers 541065216 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
MOUNTED PHYSICAL STANDBY

SQL> select instance_name, status, host_name from v$instance;

INSTANCE_NAME STATUS
---------------- ------------
HOST_NAME
----------------------------------------------------------------
STBY MOUNTED
oem132.racattack


SQL>


STBY has now reverted to being a Standby.

Let me check the RAC database instances

[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 16 23:55:28 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
READ WRITE PRIMARY

SQL> select instance_name, status, host_name from gv$instance;

INSTANCE_NAME STATUS
---------------- ------------
HOST_NAME
----------------------------------------------------------------
RAC2 OPEN
collabn2.racattack

RAC1 OPEN
collabn1.racattack


SQL>


Yes, RAC is now Primary with both instances OPEN.

Let me (now on RAC) verify the new table created and populated when STBY was the Primary. 

SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-65019: pluggable database PDB already open


SQL> alter session set container=PDB;

Session altered.

SQL> select table_name from dba_tables where owner = 'HEMANT';

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA
MY_NEW_TABLE

SQL> select count(*) from hemant.my_new_table;

COUNT(*)
----------
300

SQL>


Yes, the data in the Pluggable Database PDB has also come over to RAC.

So, on 10-April, I did a Switchover from RAC (2nodes) to STBY (SingleNode).  Today, I have done a Switchover from STBY to RAC.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8h : DataGuard Switchover (RAC to nonRAC) Messages

Tue, 2017-04-11 10:21
Yesterday, I demonstrated a DataGuard Switchover from a 2node RAC(ASM) database to a SingleInstance(FileSystem) Database.

These are the messages in the alert log and drc log files during the Switchover (Messages from the start of the Switchover to the subsequent restart and shutdown of all three instances.

First on node 1 which is running database instance RAC1. The drcRAC1.log and alert_RAC1.log :

04/10/2017 23:30:58
SWITCHOVER TO stby
Switchover to physical standby cannot be initiated from primary database; redirecting to the switchover target stby
using connect identifier: stby
04/10/2017 23:31:00
SWITCHOVER TO stby
04/10/2017 23:31:02
Notifying Oracle Clusterware to prepare primary database for switchover
04/10/2017 23:31:19
Target standby stby did not have Active Data Guard enabled at the time of switchover.
To maintain Active Data Guard license compliance Block Change Tracking will be disabled.
04/10/2017 23:31:20
Notifying Clusterware to restart this instance for Switchover
04/10/2017 23:31:23
Shutting down instance after CTL_SWITCH
04/10/2017 23:33:20
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
2017-04-10 23:33:20.858 DMON: Attach state object
2017-04-10 23:33:21.407 DMON: rfafoGetLocks reinitializing dubious PMYSHUT lock value block contents: sts=0, flags=0x0, spare1=0x0, spare2=0x0, cksm=0x0, rndm=0x0
2017-04-10 23:33:21.407 DMON: Broker state reconciled, version = 0, state = 00000000
2017-04-10 23:33:21.407 DMON: Broker State Initialized
2017-04-10 23:33:21.407 Version = 1
2017-04-10 23:33:21.407 State = 00000000
2017-04-10 23:33:21.407 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:33:21.422 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:33:24.580 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:33:24.580 DMON: Boot configuration (0.0.0), loading from "+DATA/RAC/dgbroker1.dat"
2017-04-10 23:33:24.856 DMON Registering service RAC_DGB with listener(s)
2017-04-10 23:33:24.857 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:33:24.902 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:33:25
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
Physical standby bootstrap requires Oracle Clusterware buildup
04/10/2017 23:33:39
Notifying Oracle Clusterware to buildup
04/10/2017 23:33:41
rac version check successfully completed
Broker configuration file is current on rac, completing initialization
Creating process RSM0
04/10/2017 23:42:33
Data Guard Broker shutting down
Data Guard Broker - notifying primary of shutdown
posting shutdown message to primary database 0x02001000
RSM0 successfully terminated
2017-04-10 23:42:36.437 >> DMON Process Shutdown <<



=============================================================================================



Mon Apr 10 23:30:59 2017
SWITCHOVER VERIFY: Send VERIFY request to switchover target stby
SWITCHOVER VERIFY COMPLETE
SWITCHOVER VERIFY: Send VERIFY request to switchover target stby
SWITCHOVER VERIFY COMPLETE
ALTER DATABASE SWITCHOVER TO 'stby'
Mon Apr 10 23:31:04 2017
Starting switchover [Process ID: 17791]
Mon Apr 10 23:31:04 2017
ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY [Process Id: 17791] (RAC1)
Waiting for target standby to receive all redo
Mon Apr 10 23:31:04 2017
Waiting for all non-current ORLs to be archived...
Mon Apr 10 23:31:04 2017
All non-current ORLs have been archived.
Mon Apr 10 23:31:04 2017
Waiting for all FAL entries to be archived...
Mon Apr 10 23:31:04 2017
All FAL entries have been archived.
Mon Apr 10 23:31:04 2017
Waiting for dest_id 2 to become synchronized...
Mon Apr 10 23:31:05 2017
Active, synchronized Physical Standby switchover target has been identified
Preventing updates and queries at the Primary
Generating and shipping final logs to target standby
Switchover End-Of-Redo Log thread 1 sequence 70 has been fixed
Switchover End-Of-Redo Log thread 2 sequence 39 has been fixed
Switchover: Primary highest seen SCN set to 0x0.0x3553b3
ARCH: Noswitch archival of thread 2, sequence 39
ARCH: End-Of-Redo Branch archival of thread 2 sequence 39
ARCH: LGWR is actively archiving destination LOG_ARCHIVE_DEST_2
ARCH: Standby redo logfile selected for thread 2 sequence 39 for destination LOG_ARCHIVE_DEST_2
ARCH: Noswitch archival of thread 1, sequence 70
ARCH: End-Of-Redo Branch archival of thread 1 sequence 70
ARCH: LGWR is actively archiving destination LOG_ARCHIVE_DEST_2
ARCH: Standby redo logfile selected for thread 1 sequence 70 for destination LOG_ARCHIVE_DEST_2
ARCH: Archiving is disabled due to current logfile archival
Primary will check for some target standby to have received all redo
Waiting for target standby to apply all redo
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rsm0_17791.trc
Converting the primary database to a new standby database
Clearing standby activation ID 2519803190 (0x96312536)
The primary database controlfile was created using the
'MAXLOGFILES 192' clause.
There is space for up to 188 standby redo logfiles
Use the following SQL commands on the standby database to create
standby redo logfiles that match the primary database:
ALTER DATABASE ADD STANDBY LOGFILE 'srl1.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl2.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl3.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl4.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl5.f' SIZE 52428800;
Archivelog for thread 1 sequence 70 required for standby recovery
Archivelog for thread 2 sequence 39 required for standby recovery
Offline data file 5 marked as online during convert to standby or switchover to standby.
Restore of backup may be required if the file is not physically accessible.
Offline data file 7 marked as online during convert to standby or switchover to standby.
Restore of backup may be required if the file is not physically accessible.
Switchover: Primary controlfile converted to standby controlfile succesfully.
Mon Apr 10 23:31:14 2017
Reconfiguration started (old inc 4, new inc 6)
List of instances (total 1) :
1
Dead instances (total 1) :
2
My inst 1
publish big name space - dead or down/up instance detected, invalidate domain 0
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:31:14 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:31:14 2017
Post SMON to start 1st pass IR
Mon Apr 10 23:31:14 2017
Switchover: Complete - Database shutdown required
Mon Apr 10 23:31:14 2017
Sending request(convert to primary database) to switchover target stby
Mon Apr 10 23:31:14 2017
minact-scn: Inst 1 is now the master inc#:4 mmon proc-id:17203 status:0x7
Mon Apr 10 23:31:14 2017
Process (ospid 17189) is suspended due to switchover to physical standby operation.
Mon Apr 10 23:31:14 2017
minact-scn status: grec-scn:0x0000.00000000 gmin-scn:0x0000.0033ccde gcalc-scn:0x0000.0033ccee
minact-scn: master found reconf/inst-rec before recscn scan old-inc#:4 new-inc#:4
Mon Apr 10 23:31:14 2017
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:31:14 2017
Reconfiguration complete (total time 0.6 secs)
Mon Apr 10 23:31:19 2017
Switchover complete. Database shutdown required
Completed: ALTER DATABASE SWITCHOVER TO 'stby'
Target standby stby did not have Active Data Guard enabled at the time of switchover.
To maintain Active Data Guard license compliance Block Change Tracking will be disabled.
Mon Apr 10 23:31:21 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rfs_6513.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_rfs_6518.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:23 2017
Clusterware restarting instance for Data Guard Broker operation; shutting down instance now
DMON (ospid: 17306): terminating the instance
Mon Apr 10 23:31:24 2017
Instance terminated by DMON, pid = 17306
Mon Apr 10 23:31:39 2017
Starting ORACLE instance (normal) (OS id: 6761)
Mon Apr 10 23:31:40 2017
CLI notifier numLatches:3 maxDescs:519
Mon Apr 10 23:31:41 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:31:42 2017
**********************************************************************
Mon Apr 10 23:31:42 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Apr 10 23:31:42 2017
Per process system memlock (soft) limit = UNLIMITED
Mon Apr 10 23:31:42 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:31:42 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:31:42 2017
Supported system pagesize(s):
Mon Apr 10 23:31:42 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:31:42 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:31:42 2017
2048K 0 401 0 NONE
Mon Apr 10 23:31:42 2017
RECOMMENDATION:
Mon Apr 10 23:31:42 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:31:42 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.3.70, mac=08-00-27-6f-ef-ba, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=192.168.78.51, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=192.168.78.61, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:4' configured from GPnP for use as a public interface.
[name='eth0:4', type=1, ip=192.168.78.251, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:5' configured from GPnP for use as a public interface.
[name='eth0:5', type=1, ip=192.168.78.252, mac=08-00-27-44-0e-4c, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Advanced Analytics
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: collabn1.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initRAC1.ora
System parameters with non-default values:
processes = 300
sga_max_size = 800M
spfile = "+DATA/RAC/spfilerac.ora"
control_files = "+DATA/RAC/CONTROLFILE/current.282.931825275"
control_files = "+FRA/RAC/CONTROLFILE/current.256.931825275"
db_block_size = 8192
db_cache_size = 100M
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="stby""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stby" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
log_archive_trace = 0
log_archive_config = "dg_config=(RAC,stby)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
archive_lag_target = 0
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+FRA"
db_recovery_file_dest_size= 4785M
standby_file_management = "MANUAL"
thread = 1
undo_tablespace = "UNDOTBS1"
instance_number = 1
remote_login_passwordfile= "EXCLUSIVE"
db_domain = "racattack"
dispatchers = "(PROTOCOL=TCP) (SERVICE=RACXDB)"
audit_file_dest = "/u01/app/oracle/admin/RAC/adump"
audit_trail = "DB"
db_name = "RAC"
open_cursors = 300
dg_broker_start = TRUE
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
diagnostic_dest = "/u01/app/oracle"
enable_pluggable_database= TRUE
Mon Apr 10 23:31:52 2017
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Mon Apr 10 23:31:52 2017
Cluster communication is configured to use the following interface(s) for this instance
169.254.3.70
cluster interconnect IPC version: Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
Starting background process PMON
Mon Apr 10 23:31:53 2017
PMON started with pid=2, OS id=6880
Starting background process PSP0
Starting background process VKTM
Mon Apr 10 23:31:53 2017
PSP0 started with pid=3, OS id=6882
Mon Apr 10 23:31:55 2017
VKTM started with pid=4, OS id=6892 at elevated (RT) priority
Starting background process GEN0
Mon Apr 10 23:31:55 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Starting background process MMAN
Mon Apr 10 23:31:55 2017
GEN0 started with pid=5, OS id=6896
Mon Apr 10 23:31:55 2017
MMAN started with pid=6, OS id=6898
Starting background process DIAG
Starting background process DBRM
Mon Apr 10 23:31:55 2017
DIAG started with pid=8, OS id=6910
Starting background process VKRM
Mon Apr 10 23:31:55 2017
DBRM started with pid=9, OS id=6912
Starting background process PING
Mon Apr 10 23:31:55 2017
VKRM started with pid=10, OS id=6914
Starting background process ACMS
Mon Apr 10 23:31:55 2017
PING started with pid=11, OS id=6916
Starting background process DIA0
Mon Apr 10 23:31:55 2017
ACMS started with pid=12, OS id=6918
Mon Apr 10 23:31:56 2017
DIA0 started with pid=13, OS id=6920
Starting background process LMON
Mon Apr 10 23:31:56 2017
LMON started with pid=14, OS id=6922
Starting background process LMD0
Starting background process LMS0
Mon Apr 10 23:31:56 2017
LMD0 started with pid=7, OS id=6929
Mon Apr 10 23:31:56 2017
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [1920 - 2560]
Mon Apr 10 23:31:56 2017
LMS0 started with pid=15, OS id=6931 at elevated (RT) priority
Starting background process RMS0
Starting background process LMHB
Mon Apr 10 23:31:56 2017
RMS0 started with pid=16, OS id=6935
Mon Apr 10 23:31:56 2017
LMHB started with pid=17, OS id=6937
Starting background process LCK1
Starting background process DBW0
Mon Apr 10 23:31:56 2017
LCK1 started with pid=18, OS id=6939
Starting background process LGWR
Mon Apr 10 23:31:56 2017
DBW0 started with pid=19, OS id=6941
Starting background process CKPT
Mon Apr 10 23:31:56 2017
LGWR started with pid=20, OS id=6943
Mon Apr 10 23:31:56 2017
CKPT started with pid=21, OS id=6945
Starting background process SMON
Mon Apr 10 23:31:56 2017
SMON started with pid=23, OS id=6949
Starting background process RECO
Starting background process LREG
Mon Apr 10 23:31:56 2017
RECO started with pid=25, OS id=6953
Starting background process PXMN
Mon Apr 10 23:31:56 2017
LREG started with pid=26, OS id=6955
Mon Apr 10 23:31:56 2017
PXMN started with pid=27, OS id=6957
Starting background process RBAL
Mon Apr 10 23:31:56 2017
RBAL started with pid=28, OS id=6959
Starting background process ASMB
Starting background process MMON
Mon Apr 10 23:31:56 2017
ASMB started with pid=29, OS id=6961
Starting background process MMNL
Mon Apr 10 23:31:56 2017
MMON started with pid=30, OS id=6963
Mon Apr 10 23:31:56 2017
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Mon Apr 10 23:31:56 2017
MMNL started with pid=31, OS id=6965
starting up 1 shared server(s) ...
Mon Apr 10 23:31:57 2017
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:839035202) (new connection)
NOTE: ASMB connected to ASM instance +ASM1 osid: 6971 (Flex mode; client id 0xffffffffffffffff)
Mon Apr 10 23:31:58 2017
lmon registered with NM - instance number 1 (internal mem no 0)
Mon Apr 10 23:31:58 2017
NOTE: initiating MARK startup
Starting background process MARK
Mon Apr 10 23:31:58 2017
MARK started with pid=35, OS id=6977
Mon Apr 10 23:31:58 2017
NOTE: MARK has subscribed
Mon Apr 10 23:31:58 2017
Reconfiguration started (old inc 0, new inc 2)
List of instances (total 1) :
1
My inst 1 (I'm a new instance)
Mon Apr 10 23:31:59 2017
Using default pga_aggregate_limit of 2048 MB
Mon Apr 10 23:32:18 2017
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:32:26 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:32:26 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:32:26 2017
Reconfiguration complete (total time 27.7 secs)
Starting background process LCK0
Mon Apr 10 23:32:29 2017
LCK0 started with pid=39, OS id=7084
Starting background process RSMN
Mon Apr 10 23:32:30 2017
RSMN started with pid=40, OS id=7094
Mon Apr 10 23:32:32 2017
Instance started by oraagent
Starting background process DMON
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:32:32 2017
DMON started with pid=41, OS id=7100
Mon Apr 10 23:32:34 2017
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))' SCOPE=MEMORY SID='RAC1';
Mon Apr 10 23:32:35 2017
ALTER SYSTEM SET remote_listener=' collabn-cluster-scan.racattack:1521' SCOPE=MEMORY SID='RAC1';
ALTER DATABASE MOUNT /* db agent *//* {1:46642:1193} */
Mon Apr 10 23:32:36 2017
This instance was first to mount
Mon Apr 10 23:32:38 2017
NOTE: ASMB mounting group 1 (DATA)
NOTE: Assigning number (1,0) to disk (/dev/asm-disk1)
NOTE: Assigning number (1,1) to disk (/dev/asm-disk2)
NOTE: Assigning number (1,2) to disk (/dev/asm-disk5)
SUCCESS: mounted group 1 (DATA)
NOTE: grp 1 disk 0: DATA_0000 path:/dev/asm-disk1
NOTE: grp 1 disk 1: DATA_0001 path:/dev/asm-disk2
NOTE: grp 1 disk 2: DATA_0002 path:/dev/asm-disk5
NOTE: ASMB mounting group 2 (FRA)
Mon Apr 10 23:32:39 2017
NOTE: dependency between database RAC and diskgroup resource ora.DATA.dg is established
Mon Apr 10 23:32:39 2017
NOTE: Assigning number (2,0) to disk (/dev/asm-disk3)
SUCCESS: mounted group 2 (FRA)
NOTE: grp 2 disk 0: FRA_0000 path:/dev/asm-disk3
Mon Apr 10 23:32:40 2017
NOTE: dependency between database RAC and diskgroup resource ora.FRA.dg is established
Mon Apr 10 23:32:46 2017
WARNING: STANDBY_FILE_MANAGEMENT initialization parameter is
not set to the value "AUTO".
This may cause recovery of the standby database to terminate
prior to applying all available redo data.
It may be necessary to use the ALTER DATABASE CREATE DATAFILE
command to add datafiles created on the primary database.
Starting background process TMON
ARCH: STARTING ARCH PROCESSES
Starting background process ARC0
Mon Apr 10 23:32:48 2017
TMON started with pid=44, OS id=7628
Mon Apr 10 23:32:48 2017
ARC0 started with pid=45, OS id=7630
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:32:48 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Starting background process ARC2
Mon Apr 10 23:32:48 2017
ARC1 started with pid=46, OS id=7632
Starting background process ARC3
Mon Apr 10 23:32:48 2017
ARC2 started with pid=47, OS id=7634
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:32:48 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC1: Thread not mounted
Mon Apr 10 23:32:48 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:32:48 2017
ARC3 started with pid=48, OS id=7636
Mon Apr 10 23:32:48 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
ARC0: Thread not mounted
Mon Apr 10 23:32:48 2017
ARC3: Thread not mounted
Mon Apr 10 23:32:49 2017
ARC2: Thread not mounted
Mon Apr 10 23:32:49 2017
Network Resource Management enabled for Process LGWR (pid 6943) for Exadata I/O
Successful mount of redo thread 1, with mount id 2528954836
Mon Apr 10 23:32:49 2017
Allocated 3981120 bytes in shared pool for flashback generation buffer
Starting background process RVWR
Mon Apr 10 23:32:49 2017
RVWR started with pid=49, OS id=7638
Physical Standby Database mounted.
Lost write protection disabled
Mon Apr 10 23:32:58 2017
Archiving previously deferred ORLs (RAC1)
ARCH: End-Of-Redo Branch archival of thread 1 sequence 70
Mon Apr 10 23:33:13 2017
Archived Log entry 136 added for thread 1 sequence 70 ID 0x96312536 dest 1:
ARCH: End-Of-Redo Branch archival of thread 2 sequence 39
Mon Apr 10 23:33:15 2017
Archived Log entry 137 added for thread 2 sequence 39 ID 0x96312536 dest 1:
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:46642:1193} */
Mon Apr 10 23:33:20 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:33:21 2017
INSV started with pid=42, OS id=7855
Starting background process NSV1
Mon Apr 10 23:33:25 2017
NSV1 started with pid=52, OS id=7869
Mon Apr 10 23:33:26 2017
Decreasing number of real time LMS from 1 to 0
Starting background process RSM0
Mon Apr 10 23:33:41 2017
RSM0 started with pid=55, OS id=7929
Mon Apr 10 23:33:41 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Network Resource Management enabled for Process (pid 7909) for Exadata I/O
Primary database is in MAXIMUM PERFORMANCE mode
Mon Apr 10 23:33:46 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:33:46 2017
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
Mon Apr 10 23:33:46 2017
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
RFS[1]: Assigned to RFS process (PID:7909)
RFS[1]: Selected log 5 for thread 1 sequence 72 dbid 2519807290 branch 931825279
Mon Apr 10 23:33:46 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:47 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='RAC1';
Mon Apr 10 23:33:47 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='RAC1';
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Mon Apr 10 23:33:50 2017
Attempt to start background Managed Standby Recovery process (RAC1)
Starting background process MRP0
Mon Apr 10 23:33:50 2017
MRP0 started with pid=56, OS id=7988
Mon Apr 10 23:33:50 2017
MRP0: Background Managed Standby Recovery process started (RAC1)
Mon Apr 10 23:33:55 2017
Started logmerger process
Mon Apr 10 23:33:57 2017
Managed Standby Recovery starting Real Time Apply
Mon Apr 10 23:33:58 2017
RFS[2]: Assigned to RFS process (PID:8007)
RFS[2]: Opened log for thread 1 sequence 71 dbid 2519807290 branch 931825279
RFS[1]: Selected log 6 for thread 1 sequence 73 dbid 2519807290 branch 931825279
Mon Apr 10 23:33:59 2017
Reconfiguration started (old inc 2, new inc 4)
List of instances (total 2) :
1 2
New instances (total 1) :
2
My inst 1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:33:59 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:34:00 2017
Reconfiguration complete (total time 0.5 secs)
Mon Apr 10 23:34:15 2017
Archived Log entry 138 added for thread 1 sequence 72 ID 0x96bd0722 dest 1:
Mon Apr 10 23:34:16 2017
Archived Log entry 139 added for thread 1 sequence 71 rlc 931825279 ID 0x96bd0722 dest 2:
RFS[2]: Opened log for thread 2 sequence 40 dbid 2519807290 branch 931825279
Mon Apr 10 23:34:23 2017
Archived Log entry 140 added for thread 2 sequence 40 rlc 931825279 ID 0x0 dest 2:
Mon Apr 10 23:34:28 2017
Parallel Media Recovery started with 2 slaves
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
Mon Apr 10 23:34:33 2017
Waiting for all non-current ORLs to be archived...
Mon Apr 10 23:34:33 2017
All non-current ORLs have been archived.
Clearing online redo logfile 1 +DATA/RAC/ONLINELOG/group_1.283.931825279

Clearing online log 1 of thread 1 sequence number 69
Mon Apr 10 23:34:50 2017
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 +DATA/RAC/ONLINELOG/group_2.284.931825283

Clearing online log 2 of thread 1 sequence number 70
Mon Apr 10 23:34:51 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:35:00 2017
Increasing number of real time LMS from 0 to 1
Mon Apr 10 23:35:09 2017
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 +DATA/RAC/ONLINELOG/group_3.290.931826413

Clearing online log 3 of thread 2 sequence number 39
Mon Apr 10 23:35:22 2017
Clearing online redo logfile 3 complete
Clearing online redo logfile 4 +DATA/RAC/ONLINELOG/group_4.291.931826417

Clearing online log 4 of thread 2 sequence number 38
Clearing online redo logfile 4 complete
Mon Apr 10 23:35:28 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_38.357.940980599
Mon Apr 10 23:35:28 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_67.355.940978999
Mon Apr 10 23:35:29 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_68.356.940980597
Mon Apr 10 23:35:31 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_69.358.940980603
Mon Apr 10 23:35:32 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_39.360.940980795
Mon Apr 10 23:35:35 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_70.359.940980783
Mon Apr 10 23:35:38 2017
Resetting standby activation ID 0 (0x0)
Mon Apr 10 23:35:39 2017
Media Recovery End-Of-Redo indicator encountered
Mon Apr 10 23:35:39 2017
Media Recovery Continuing
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_71.361.940980839
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_2_seq_40.363.940980861
Mon Apr 10 23:35:39 2017
Media Recovery Log +FRA/RAC/ARCHIVELOG/2017_04_10/thread_1_seq_72.362.940980841
Mon Apr 10 23:35:51 2017
Media Recovery Waiting for thread 1 sequence 73 (in transit)
Mon Apr 10 23:35:51 2017
Recovery of Online Redo Log: Thread 1 Group 6 Seq 73 Reading mem 0
Mem# 0: +DATA/RAC/ONLINELOG/group_6.298.937936361
Mem# 1: +FRA/RAC/ONLINELOG/group_6.304.937936363
Mon Apr 10 23:41:26 2017
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
Mon Apr 10 23:41:28 2017
MRP0: Background Media Recovery cancelled with status 16037
Mon Apr 10 23:41:28 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_pr00_8020.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:41:28 2017
Managed Standby Recovery not using Real Time Apply
Mon Apr 10 23:41:29 2017
Recovery interrupted!
Recovered data files to a consistent state at change 3497232
Mon Apr 10 23:41:29 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 99
Mon Apr 10 23:41:29 2017
Errors in file /u01/app/oracle/diag/rdbms/rac/RAC1/trace/RAC1_pr00_8020.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:41:29 2017
MRP0: Background Media Recovery process shutdown (RAC1)
Mon Apr 10 23:41:29 2017
NOTE: Deferred communication with ASM instance
License high water mark = 10
Mon Apr 10 23:41:29 2017
All dispatchers and shared servers shutdown
ALTER DATABASE CLOSE NORMAL
Stopping Emon pool
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC0: Archival stopped
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC2: Relinquishing active heartbeat ARCH role
Mon Apr 10 23:41:30 2017
ARCH shutting down
ARC3: Archival stopped
Mon Apr 10 23:41:30 2017
ARC2: Archival stopped
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 5
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 14
Mon Apr 10 23:41:31 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:37 2017
NOTE: De-assigning number (2,0) from disk (/dev/asm-disk3)
SUCCESS: diskgroup FRA was dismounted
NOTE: De-assigning number (1,0) from disk (/dev/asm-disk1)
NOTE: De-assigning number (1,1) from disk (/dev/asm-disk2)
NOTE: De-assigning number (1,2) from disk (/dev/asm-disk5)
SUCCESS: diskgroup DATA was dismounted
NOTE: Database dismounted; ASMB process exiting
NOTE: ASMB clearing idle groups before exit
Stopping background process MARK
Mon Apr 10 23:41:38 2017
NOTE: Shutting down MARK background process
Stopping background process RBAL
Mon Apr 10 23:41:56 2017
Reconfiguration started (old inc 4, new inc 6)
List of instances (total 1) :
1
Dead instances (total 1) :
2
My inst 1
publish big name space - dead or down/up instance detected, invalidate domain 0
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:41:56 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:41:56 2017
Reconfiguration complete (total time 0.2 secs)
Mon Apr 10 23:42:31 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:42:36 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:42:37 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:42:37 2017
Stopping background process VKTM
Mon Apr 10 23:42:42 2017
freeing rdom 0
Mon Apr 10 23:42:44 2017
Instance shutdown complete






Next on node2 running instance RAC2 :

04/10/2017 23:34:35
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
2017-04-10 23:34:35.584 DMON: Attach state object
2017-04-10 23:34:35.991 DMON: Broker State Initialized
2017-04-10 23:34:35.991 Version = 3
2017-04-10 23:34:35.991 State = 00000000
2017-04-10 23:34:35.992 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:34:35.993 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:34:39.031 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:34:39.130 DMON: Boot configuration (0.0.0), loading from "+DATA/RAC/dgbroker1.dat"
2017-04-10 23:34:39.192 DMON Registering service RAC_DGB with listener(s)
2017-04-10 23:34:39.192 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:34:39.193 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:34:39
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
04/10/2017 23:34:47
rac version check successfully completed
Broker configuration file is current on rac, completing initialization
Creating process RSM0
04/10/2017 23:41:41
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:41:44.988 >> DMON Process Shutdown <<



==========================================================================================================



Mon Apr 10 23:31:10 2017
Switchover in progress in another database instance - Database is shutdown automatically
LGWR (ospid: 3335): terminating the instance due to error 16456
Mon Apr 10 23:31:11 2017
Instance terminated by LGWR, pid = 3335
Mon Apr 10 23:33:50 2017
Starting ORACLE instance (normal) (OS id: 8233)
Mon Apr 10 23:33:51 2017
CLI notifier numLatches:3 maxDescs:519
Mon Apr 10 23:33:51 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:33:51 2017
**********************************************************************
Mon Apr 10 23:33:51 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Apr 10 23:33:51 2017
Per process system memlock (soft) limit = UNLIMITED
Mon Apr 10 23:33:51 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:33:51 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:33:51 2017
Supported system pagesize(s):
Mon Apr 10 23:33:51 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:33:51 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:33:51 2017
2048K 0 401 0 NONE
Mon Apr 10 23:33:51 2017
RECOMMENDATION:
Mon Apr 10 23:33:51 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:33:51 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.178.54, mac=08-00-27-c5-68-87, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=192.168.78.52, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=192.168.78.62, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:2' configured from GPnP for use as a public interface.
[name='eth0:2', type=1, ip=192.168.78.253, mac=08-00-27-1e-02-8d, net=192.168.78.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Advanced Analytics
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: collabn2.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initRAC2.ora
System parameters with non-default values:
processes = 300
sga_max_size = 800M
spfile = "+DATA/RAC/spfilerac.ora"
control_files = "+DATA/RAC/CONTROLFILE/current.282.931825275"
control_files = "+FRA/RAC/CONTROLFILE/current.256.931825275"
db_block_size = 8192
db_cache_size = 100M
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="stby""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stby" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
log_archive_trace = 0
log_archive_config = "dg_config=(RAC,stby)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
archive_lag_target = 0
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+FRA"
db_recovery_file_dest_size= 4785M
standby_file_management = "MANUAL"
thread = 2
undo_tablespace = "UNDOTBS2"
instance_number = 2
remote_login_passwordfile= "EXCLUSIVE"
db_domain = "racattack"
dispatchers = "(PROTOCOL=TCP) (SERVICE=RACXDB)"
audit_file_dest = "/u01/app/oracle/admin/RAC/adump"
audit_trail = "DB"
db_name = "RAC"
open_cursors = 300
dg_broker_start = TRUE
dg_broker_config_file1 = "+DATA/RAC/dgbroker1.dat"
dg_broker_config_file2 = "+FRA/RAC/dgbroker2.dat"
diagnostic_dest = "/u01/app/oracle"
enable_pluggable_database= TRUE
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Mon Apr 10 23:33:55 2017
Cluster communication is configured to use the following interface(s) for this instance
169.254.178.54
cluster interconnect IPC version: Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
Starting background process PMON
Mon Apr 10 23:33:56 2017
PMON started with pid=2, OS id=8257
Starting background process PSP0
Mon Apr 10 23:33:56 2017
PSP0 started with pid=3, OS id=8259
Starting background process VKTM
Mon Apr 10 23:33:57 2017
VKTM started with pid=4, OS id=8261 at elevated (RT) priority
Starting background process GEN0
Mon Apr 10 23:33:57 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Starting background process MMAN
Mon Apr 10 23:33:57 2017
GEN0 started with pid=5, OS id=8265
Mon Apr 10 23:33:57 2017
MMAN started with pid=6, OS id=8267
Starting background process DIAG
Mon Apr 10 23:33:57 2017
Starting background process DBRM
DIAG started with pid=8, OS id=8271
Starting background process VKRM
Mon Apr 10 23:33:57 2017
DBRM started with pid=9, OS id=8273
Starting background process PING
Mon Apr 10 23:33:57 2017
VKRM started with pid=10, OS id=8275
Mon Apr 10 23:33:57 2017
PING started with pid=11, OS id=8277
Starting background process ACMS
Mon Apr 10 23:33:57 2017
Starting background process DIA0
ACMS started with pid=12, OS id=8279
Mon Apr 10 23:33:57 2017
Starting background process LMON
DIA0 started with pid=13, OS id=8281
Mon Apr 10 23:33:57 2017
Starting background process LMD0
LMON started with pid=14, OS id=8283
Mon Apr 10 23:33:57 2017
Starting background process LMS0
LMD0 started with pid=15, OS id=8285
Mon Apr 10 23:33:57 2017
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [1920 - 2560]
Starting background process RMS0
Mon Apr 10 23:33:57 2017
LMS0 started with pid=16, OS id=8287 at elevated (RT) priority
Mon Apr 10 23:33:57 2017
Starting background process LMHB
RMS0 started with pid=17, OS id=8291
Mon Apr 10 23:33:57 2017
Starting background process LCK1
LMHB started with pid=18, OS id=8293
Mon Apr 10 23:33:57 2017
Starting background process DBW0
LCK1 started with pid=19, OS id=8295
Mon Apr 10 23:33:57 2017
DBW0 started with pid=20, OS id=8297
Starting background process LGWR
Mon Apr 10 23:33:57 2017
Starting background process CKPT
LGWR started with pid=21, OS id=8299
Mon Apr 10 23:33:57 2017
CKPT started with pid=22, OS id=8301
Starting background process SMON
Mon Apr 10 23:33:57 2017
SMON started with pid=24, OS id=8305
Starting background process RECO
Mon Apr 10 23:33:57 2017
RECO started with pid=26, OS id=8309
Starting background process LREG
Mon Apr 10 23:33:57 2017
Starting background process PXMN
LREG started with pid=27, OS id=8311
Mon Apr 10 23:33:57 2017
Starting background process RBAL
PXMN started with pid=7, OS id=8313
Starting background process ASMB
Mon Apr 10 23:33:57 2017
RBAL started with pid=28, OS id=8315
Mon Apr 10 23:33:57 2017
ASMB started with pid=29, OS id=8317
Starting background process MMON
Starting background process MMNL
Mon Apr 10 23:33:57 2017
MMON started with pid=30, OS id=8319
Mon Apr 10 23:33:57 2017
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Mon Apr 10 23:33:57 2017
MMNL started with pid=31, OS id=8323
Mon Apr 10 23:33:57 2017
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:593534517) (new connection)
Mon Apr 10 23:33:57 2017
starting up 1 shared server(s) ...
Mon Apr 10 23:33:57 2017
NOTE: ASMB connected to ASM instance +ASM2 osid: 8321 (Flex mode; client id 0xffffffffffffffff)
NOTE: initiating MARK startup
Starting background process MARK
Mon Apr 10 23:33:57 2017
lmon registered with NM - instance number 2 (internal mem no 1)
Mon Apr 10 23:33:57 2017
MARK started with pid=34, OS id=8329
Mon Apr 10 23:33:57 2017
NOTE: MARK has subscribed
Mon Apr 10 23:33:59 2017
Reconfiguration started (old inc 0, new inc 4)
List of instances (total 2) :
1 2
My inst 2 (I'm a new instance)
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Mon Apr 10 23:33:59 2017
* domain 0 valid = 0 according to instance 1
Mon Apr 10 23:33:59 2017
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Apr 10 23:33:59 2017
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Mon Apr 10 23:33:59 2017
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Fix write in gcs resources
Mon Apr 10 23:34:00 2017
Reconfiguration complete (total time 0.8 secs)
Starting background process LCK0
Mon Apr 10 23:34:00 2017
LCK0 started with pid=38, OS id=8337
Mon Apr 10 23:34:00 2017
Using default pga_aggregate_limit of 2048 MB
Starting background process RSMN
Mon Apr 10 23:34:01 2017
RSMN started with pid=40, OS id=8351
Mon Apr 10 23:34:02 2017
Instance started by oraagent
Starting background process DMON
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:34:02 2017
DMON started with pid=41, OS id=8353
Mon Apr 10 23:34:03 2017
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.62)(PORT=1521))' SCOPE=MEMORY SID='RAC2';
Mon Apr 10 23:34:03 2017
ALTER SYSTEM SET remote_listener=' collabn-cluster-scan.racattack:1521' SCOPE=MEMORY SID='RAC2';
ALTER DATABASE MOUNT /* db agent *//* {1:46642:1276} */
Mon Apr 10 23:34:04 2017
NOTE: ASMB mounting group 1 (DATA)
NOTE: Assigning number (1,0) to disk (/dev/asm-disk1)
NOTE: Assigning number (1,1) to disk (/dev/asm-disk2)
NOTE: Assigning number (1,2) to disk (/dev/asm-disk5)
SUCCESS: mounted group 1 (DATA)
NOTE: grp 1 disk 0: DATA_0000 path:/dev/asm-disk1
NOTE: grp 1 disk 1: DATA_0001 path:/dev/asm-disk2
NOTE: grp 1 disk 2: DATA_0002 path:/dev/asm-disk5
Mon Apr 10 23:34:04 2017
NOTE: dependency between database RAC and diskgroup resource ora.DATA.dg is established
Mon Apr 10 23:34:04 2017
NOTE: ASMB mounting group 2 (FRA)
NOTE: Assigning number (2,0) to disk (/dev/asm-disk3)
SUCCESS: mounted group 2 (FRA)
NOTE: grp 2 disk 0: FRA_0000 path:/dev/asm-disk3
Mon Apr 10 23:34:04 2017
NOTE: dependency between database RAC and diskgroup resource ora.FRA.dg is established
Mon Apr 10 23:34:14 2017
WARNING: STANDBY_FILE_MANAGEMENT initialization parameter is
not set to the value "AUTO".
This may cause recovery of the standby database to terminate
prior to applying all available redo data.
It may be necessary to use the ALTER DATABASE CREATE DATAFILE
command to add datafiles created on the primary database.
Starting background process TMON
Mon Apr 10 23:34:16 2017
ARCH: STARTING ARCH PROCESSES
TMON started with pid=45, OS id=8374
Starting background process ARC0
Mon Apr 10 23:34:16 2017
ARC0 started with pid=46, OS id=8376
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:34:16 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Starting background process ARC2
Mon Apr 10 23:34:16 2017
ARC1 started with pid=47, OS id=8378
Starting background process ARC3
Mon Apr 10 23:34:16 2017
ARC2 started with pid=48, OS id=8380
Mon Apr 10 23:34:16 2017
ARC3 started with pid=49, OS id=8382
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:34:16 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC1: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:34:16 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:34:16 2017
ARC0: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC3: Thread not mounted
Mon Apr 10 23:34:16 2017
ARC2: Thread not mounted
Mon Apr 10 23:34:17 2017
Network Resource Management enabled for Process LGWR (pid 8299) for Exadata I/O
Successful mount of redo thread 2, with mount id 2528954836
Mon Apr 10 23:34:22 2017
Allocated 3981120 bytes in shared pool for flashback generation buffer
Starting background process RVWR
Mon Apr 10 23:34:22 2017
RVWR started with pid=50, OS id=8385
Physical Standby Database mounted.
Lost write protection disabled
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:46642:1276} */
Mon Apr 10 23:34:35 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:34:36 2017
INSV started with pid=42, OS id=8424
Starting background process NSV1
Mon Apr 10 23:34:39 2017
NSV1 started with pid=51, OS id=8432
Starting background process RSM0
Mon Apr 10 23:34:47 2017
RSM0 started with pid=52, OS id=8439
Mon Apr 10 23:34:51 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:34:51 2017
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
Mon Apr 10 23:34:51 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:34:52 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='RAC2';
Mon Apr 10 23:34:52 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='RAC2';
Mon Apr 10 23:41:28 2017
Managed Standby Recovery not using Real Time Apply
Mon Apr 10 23:41:30 2017
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
Mon Apr 10 23:41:37 2017
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
License high water mark = 2
Mon Apr 10 23:41:40 2017
All dispatchers and shared servers shutdown
ALTER DATABASE CLOSE NORMAL
Stopping Emon pool
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:40 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:40 2017
ARCH shutting down
Mon Apr 10 23:41:40 2017
ARCH shutting down
Mon Apr 10 23:41:40 2017
ARC0: Archival stopped
Mon Apr 10 23:41:40 2017
ARC2: Relinquishing active heartbeat ARCH role
ARC2: Archival stopped
Mon Apr 10 23:41:40 2017
ARCH shutting down
ARC3: Archival stopped
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 5
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 15
Mon Apr 10 23:41:41 2017
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:41 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:41:43 2017
NOTE: Deferred communication with ASM instance
Mon Apr 10 23:41:44 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:41:45 2017
NOTE: force a map free for map id 2
Mon Apr 10 23:41:45 2017
Stopping background process VKTM
Mon Apr 10 23:41:45 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:46 2017
NOTE: Shutting down MARK background process
Mon Apr 10 23:41:53 2017
NOTE: ASMB releasing group resources due to shutdown
NOTE: ASMB clearing idle groups before exit
Stopping background process RBAL
Mon Apr 10 23:41:55 2017
freeing rdom 0
Mon Apr 10 23:41:57 2017
Instance shutdown complete





Next, the original Standby (SingleInstance) node : the drcSTBY.log and alert_STBY.log

04/10/2017 23:31:03
Notifying Oracle Clusterware to prepare target standby database for switchover
04/10/2017 23:33:49
Deferring associated archivelog destinations of sites permanently disabled due to Switchover
Notifying Oracle Clusterware to buildup primary database after switchover
Switchover Complete
New Primary Site is named: stby
04/10/2017 23:41:22
Notifying DMON of db close
Notifying RSM0 of db close
04/10/2017 23:41:27
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:41:30.761 >> DMON Process Shutdown <<
04/10/2017 23:41:55
>> Starting Data Guard Broker bootstrap <<
Broker Configuration File Locations:
dg_broker_config_file1 = "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr1STBY.dat"
dg_broker_config_file2 = "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr2STBY.dat"
2017-04-10 23:41:55.856 DMON: Attach state object
2017-04-10 23:41:55.856 DMON: rfafoGetLocks reinitializing dubious PMYSHUT lock value block contents: sts=0, flags=0x0, spare1=0x0, spare2=0x0, cksm=0x0, rndm=0x0
2017-04-10 23:41:55.857 DMON: Broker state reconciled, version = 0, state = 00000000
2017-04-10 23:41:55.857 DMON: Broker State Initialized
2017-04-10 23:41:55.857 Version = 1
2017-04-10 23:41:55.857 State = 00000000
2017-04-10 23:41:55.857 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 2
2017-04-10 23:41:55.857 7fffffff 0 DMON: Entered rfm_get_chief_lock() for CTL_BOOTSTRAP, reason 0
2017-04-10 23:41:58.902 7fffffff 0 DMON: start task execution: broker initialization
2017-04-10 23:41:58.902 DMON: Boot configuration (0.0.0), loading from "/u01/app/oracle/product/12.1.0/dbhome_1/dbs/dr1STBY.dat"
2017-04-10 23:41:58.903 DMON Registering service STBY_DGB with listener(s)
2017-04-10 23:41:58.903 DMON: Executing SQL [ALTER SYSTEM REGISTER]
2017-04-10 23:41:58.904 SQL [ALTER SYSTEM REGISTER] Executed successfully
04/10/2017 23:41:58
Broker Configuration: "rac"
Protection Mode: Maximum Performance
Fast-Start Failover (FSFO): Disabled, flags=0x0, version=0
Primary Database: stby (0x02010000)
04/10/2017 23:42:11
Failed to connect to remote database rac. Error is ORA-12528
Failed to send message to site rac. Error code is ORA-12528.
Version Check Results:
Database rac returned ORA-12528
Creating process RSM0
04/10/2017 23:42:35
Processing shutdown notification from database rac, instance 1
Broker operation
Setting SHUTDOWN status for database rac
04/10/2017 23:45:20
Notifying DMON of db close
Notifying RSM0 of db close
04/10/2017 23:45:28
Data Guard Broker shutting down
RSM0 successfully terminated
2017-04-10 23:45:31.850 >> DMON Process Shutdown <<



=======================================================================================



Mon Apr 10 23:30:59 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY WARNING: no standby database is defined in LOG_ARCHIVE_DEST_n to protect this database if it is converted to a primary database
SWITCHOVER VERIFY COMPLETE
Mon Apr 10 23:31:00 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY WARNING: no standby database is defined in LOG_ARCHIVE_DEST_n to protect this database if it is converted to a primary database
SWITCHOVER VERIFY COMPLETE
Mon Apr 10 23:31:08 2017
RFS[3]: Assigned to RFS process (PID:30383)
RFS[3]: Selected log 7 for thread 2 sequence 39 dbid 2519807290 branch 931825279
Mon Apr 10 23:31:08 2017
Archived Log entry 44 added for thread 2 sequence 39 ID 0x96312536 dest 1:
Mon Apr 10 23:31:08 2017
RFS[4]: Assigned to RFS process (PID:30386)
RFS[4]: Selected log 5 for thread 1 sequence 70 dbid 2519807290 branch 931825279
Mon Apr 10 23:31:09 2017
Archived Log entry 45 added for thread 1 sequence 70 ID 0x96312536 dest 1:
Mon Apr 10 23:31:09 2017
Resetting standby activation ID 2519803190 (0x96312536)
Mon Apr 10 23:31:09 2017
Media Recovery End-Of-Redo indicator encountered
Mon Apr 10 23:31:09 2017
Media Recovery Continuing
Media Recovery Waiting for thread 2 sequence 40
Mon Apr 10 23:31:10 2017
RFS[5]: Assigned to RFS process (PID:29959)
RFS[5]: Possible network disconnect with primary database
Mon Apr 10 23:31:15 2017
SWITCHOVER: received request 'ALTER DTABASE COMMIT TO SWITCHOVER TO PRIMARY' from primary database.
Mon Apr 10 23:31:15 2017
ALTER DATABASE SWITCHOVER TO PRIMARY (STBY)
Maximum wait for role transition is 15 minutes.
Switchover: Media recovery is still active
Role Change: Canceling MRP - no more redo to apply
Mon Apr 10 23:31:15 2017
MRP0: Background Media Recovery cancelled with status 16037
Mon Apr 10 23:31:15 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_pr00_29969.trc:
ORA-16037: user requested cancel of managed recovery operation
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
Mon Apr 10 23:31:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_pr00_29969.trc:
ORA-16037: user requested cancel of managed recovery operation
Mon Apr 10 23:31:16 2017
MRP0: Background Media Recovery process shutdown (STBY)
Mon Apr 10 23:31:17 2017
Role Change: Canceled MRP
Killing 1 processes (PIDS:30372) (all RFS) in order to disallow current and future RFS connections. Requested by OS process 30402
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_rmi_30402.trc
SwitchOver after complete recovery through change 3494835
Online logfile pre-clearing operation disabled by switchover
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413: Thread 2 Group 3 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417: Thread 2 Group 3 was previously cleared
Online log /u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417: Thread 2 Group 4 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421: Thread 2 Group 4 was previously cleared
Standby became primary SCN: 3494833
Switchover: Complete - Database mounted as primary
SWITCHOVER: completed request from primary database.
Mon Apr 10 23:31:21 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:31:21 2017
ALTER SYSTEM SET log_archive_dest_2='service="rac1"','ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="rac" net_timeout=30','valid_for=(online_logfile,all_roles)' SCOPE=BOTH;
Mon Apr 10 23:31:21 2017
ARC0: Becoming the 'no SRL' ARCH
Mon Apr 10 23:31:21 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
ALTER DATABASE OPEN
Mon Apr 10 23:31:21 2017
Data Guard Broker initializing...
Ping without log force is disabled
.
Mon Apr 10 23:31:21 2017
Assigning activation ID 2528970530 (0x96bd0722)
Mon Apr 10 23:31:21 2017
Thread 1 advanced to log sequence 72 (thread open)
Thread 1 opened at log sequence 72
Current log# 2 seq# 72 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283
Current log# 2 seq# 72 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287
Successful open of redo thread 1
Mon Apr 10 23:31:21 2017
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 10 23:31:21 2017
SMON: enabling cache recovery
Mon Apr 10 23:31:21 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
Mon Apr 10 23:31:21 2017
Archived Log entry 46 added for thread 1 sequence 71 ID 0x96bd0722 dest 1:
Mon Apr 10 23:31:21 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:21 2017
Error 16456 for archive log file 2 to 'rac1'
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc3_29915.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_30433.trc:
ORA-16456: switchover to standby in progress or completed
Mon Apr 10 23:31:22 2017
FAL[server, ARC3]: Error 16456 creating remote archivelog file 'rac1'
ARC3: FAL archive failed with error 16456. See trace for details
Mon Apr 10 23:31:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc3_29915.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Mon Apr 10 23:31:22 2017
ORACLE Instance STBY - Archival Error. Archiver continuing.
Mon Apr 10 23:31:24 2017
[29938] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:4159104 end:4161254 diff:2150 ms (2.2 seconds)
Dictionary check beginning
Dictionary check complete
Verifying minimum file header compatibility (11g) for tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Mon Apr 10 23:31:26 2017
SMON: enabling tx recovery
Starting background process SMCO
Mon Apr 10 23:31:26 2017
SMCO started with pid=33, OS id=30448
Mon Apr 10 23:31:28 2017
Database Characterset is AL32UTF8
Mon Apr 10 23:31:29 2017
Redo thread 2 internally disabled at seq 40 (CKPT)
Mon Apr 10 23:31:30 2017
ARC2: Archiving disabled thread 2 sequence 40
Mon Apr 10 23:31:30 2017
Archived Log entry 47 added for thread 2 sequence 40 ID 0x0 dest 1:
Mon Apr 10 23:31:31 2017
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process AQPC
Mon Apr 10 23:31:37 2017
AQPC started with pid=36, OS id=30543
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Mon Apr 10 23:31:39 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC4
Mon Apr 10 23:31:39 2017
ARC4 started with pid=37, OS id=30553
ARC4: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:32:00 2017
Pluggable database PDB$SEED dictionary check beginning
Pluggable Database PDB$SEED Dictionary check complete
Database Characterset for PDB$SEED is AL32UTF8
Mon Apr 10 23:32:17 2017
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:32:36 2017
Opening pdb PDB$SEED (2) with no Resource Manager plan active
Shutting down archive processes
Mon Apr 10 23:32:40 2017
ARCH shutting down
ARC4: Archival stopped
Mon Apr 10 23:32:45 2017
Pluggable database PDB dictionary check beginning
Pluggable Database PDB Dictionary check complete
Database Characterset for PDB is AL32UTF8
ALTER SYSTEM: Flushing buffer cache inst=0 container=3 local
Mon Apr 10 23:32:58 2017
Cannot start service pdb, reason=-1
Starting background process CJQ0
Mon Apr 10 23:33:13 2017
CJQ0 started with pid=37, OS id=31055
Completed: ALTER DATABASE OPEN
ALTER PLUGGABLE DATABASE ALL OPEN
Mon Apr 10 23:33:15 2017
Database Characterset for PDB is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:33:36 2017
Shared IO Pool defaulting to 36MB. Trying to get it from Buffer Cache for process 29792.
Mon Apr 10 23:33:37 2017
Opening pdb PDB (3) with no Resource Manager plan active
Mon Apr 10 23:33:38 2017
===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================
Mon Apr 10 23:33:38 2017
Pluggable database PDB opened read write
Completed: ALTER PLUGGABLE DATABASE ALL OPEN
Mon Apr 10 23:33:38 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='STBY';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='STBY';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET standby_file_management='MANUAL' SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET archive_lag_target=0 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_max_processes=4 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:39 2017
ALTER SYSTEM SET log_archive_min_succeed_dest=1 SCOPE=BOTH SID='*';
Mon Apr 10 23:33:40 2017
ALTER SYSTEM SET db_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY' SCOPE=SPFILE;
Mon Apr 10 23:33:40 2017
ALTER SYSTEM SET log_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY' SCOPE=SPFILE;
Mon Apr 10 23:33:40 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
Mon Apr 10 23:33:41 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:41 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:33:41 2017
ALTER SYSTEM ARCHIVE LOG
Mon Apr 10 23:33:44 2017
Thread 1 advanced to log sequence 73 (LGWR switch)
Current log# 1 seq# 73 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279
Current log# 1 seq# 73 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281
Mon Apr 10 23:33:46 2017
db_recovery_file_dest_size of 16384 MB is 0.73% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Mon Apr 10 23:33:46 2017
TT00: Standby redo logfile selected for thread 1 sequence 72 for destination LOG_ARCHIVE_DEST_2
Mon Apr 10 23:33:46 2017
Archived Log entry 48 added for thread 1 sequence 72 ID 0x96bd0722 dest 1:
Mon Apr 10 23:33:59 2017
TT00: Standby redo logfile selected for thread 1 sequence 73 for destination LOG_ARCHIVE_DEST_2
Mon Apr 10 23:34:47 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;
Mon Apr 10 23:41:17 2017
Shutting down instance (immediate)
Stopping background process SMCO
Shutting down instance: further logons disabled
Stopping background process CJQ0
Stopping background process MMNL
Stopping background process MMON
License high water mark = 11
ALTER DATABASE CLOSE NORMAL
Mon Apr 10 23:41:21 2017
SMON: disabling tx recovery
Mon Apr 10 23:41:21 2017
Stopping Emon pool
Stopping Emon pool
Mon Apr 10 23:41:22 2017
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:24 2017
ARCH shutting down
Mon Apr 10 23:41:24 2017
ARC2: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
Mon Apr 10 23:41:24 2017
ARC3: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
ARC0: Relinquishing active heartbeat ARCH role
ARC0: Archival stopped
Mon Apr 10 23:41:24 2017
ARCH shutting down
ARC1: Archival stopped
Mon Apr 10 23:41:25 2017
Thread 1 closed at log sequence 73
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:41:26 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:41:30 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:41:32 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:41:32 2017
Stopping background process VKTM
Mon Apr 10 23:41:35 2017
Instance shutdown complete
Mon Apr 10 23:41:44 2017
Starting ORACLE instance (normal) (OS id: 1281)
Mon Apr 10 23:41:44 2017
CLI notifier numLatches:3 maxDescs:354
Mon Apr 10 23:41:44 2017
Due to limited space in shared pool (need 6094848 bytes, have 4194112 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:41:44 2017
**********************************************************************
Mon Apr 10 23:41:44 2017
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Apr 10 23:41:44 2017
Per process system memlock (soft) limit = 128G
Mon Apr 10 23:41:44 2017
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 802M
Mon Apr 10 23:41:44 2017
Available system pagesizes:
4K, 2048K
Mon Apr 10 23:41:44 2017
Supported system pagesize(s):
Mon Apr 10 23:41:44 2017
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
Mon Apr 10 23:41:44 2017
4K Configured 4 204804 NONE
Mon Apr 10 23:41:44 2017
2048K 0 401 0 NONE
Mon Apr 10 23:41:44 2017
RECOMMENDATION:
Mon Apr 10 23:41:44 2017
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
Mon Apr 10 23:41:44 2017
**********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 2
Number of processor cores in the system is 2
Number of processor sockets in the system is 1
_ksb_restart_policy_times={0,60,120,240}
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
IMODE=BR
ILAT =35
LICENSE_MAX_USERS = 0
SYS auditing is enabled
NOTE: remote asm mode is local (mode 0x1; from cluster type)
NOTE: Using default ASM root directory ASM
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1
System name: Linux
Node name: oem132.racattack
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Using parameter settings in server-side spfile /u01/app/oracle/product/12.1.0/dbhome_1/dbs/spfileSTBY.ora
System parameters with non-default values:
_ksb_restart_policy_times= "0"
_ksb_restart_policy_times= "60"
_ksb_restart_policy_times= "120"
_ksb_restart_policy_times= "240"
sga_target = 800M
control_files = "/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl"
control_files = "/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl"
db_file_name_convert = "+DATA/RAC"
db_file_name_convert = "/u01/app/oracle/oradata/STBY"
db_file_name_convert = "+FRA/RAC"
db_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
log_file_name_convert = "+DATA/RAC"
log_file_name_convert = "/u01/app/oracle/oradata/STBY"
log_file_name_convert = "+FRA/RAC"
log_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
compatible = "12.1.0.2.0"
log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
log_archive_dest_2 = "service="rac1""
log_archive_dest_2 = "ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="rac" net_timeout=30"
log_archive_dest_2 = "valid_for=(online_logfile,all_roles)"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
fal_server = "rac1"
log_archive_trace = 0
log_archive_config = "dg_config=(rac)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 4
log_buffer = 5064K
archive_lag_target = 0
db_create_file_dest = "/u01/app/oracle/oradata"
db_recovery_file_dest = "/u01/app/oracle/fast_recovery_area"
db_recovery_file_dest_size= 16G
standby_file_management = "MANUAL"
_compression_compatibility= "12.1.0.2.0"
remote_login_passwordfile= "EXCLUSIVE"
connection_brokers = "((TYPE=DEDICATED)(BROKERS=1))"
connection_brokers = "((TYPE=EMON)(BROKERS=1))"
plsql_warnings = "DISABLE:ALL"
result_cache_max_size = 4M
_gby_vector_aggregation_enabled= TRUE
_optimizer_vector_transformation= TRUE
core_dump_dest = "/u01/app/oracle/diag/rdbms/stby/STBY/cdump"
db_name = "RAC"
db_unique_name = "STBY"
optimizer_mode = "ALL_ROWS"
_optimizer_mode_force = TRUE
_always_anti_join = "CHOOSE"
_optimizer_null_aware_antijoin= TRUE
_optimizer_partial_join_eval= TRUE
_partition_view_enabled = TRUE
_b_tree_bitmap_plans = TRUE
_cpu_to_io = 0
_optimizer_extended_cursor_sharing= "UDO"
_optimizer_extended_cursor_sharing_rel= "SIMPLE"
_optimizer_adaptive_cursor_sharing= TRUE
_optimizer_cost_model = "CHOOSE"
_optimizer_undo_cost_change= "12.1.0.2"
_optimizer_system_stats_usage= TRUE
_new_sort_cost_estimate = TRUE
_complex_view_merging = TRUE
_unnest_subquery = TRUE
_optimizer_unnest_scalar_sq= TRUE
_eliminate_common_subexpr= TRUE
_pred_move_around = TRUE
_convert_set_to_join = FALSE
_px_ual_serial_input = TRUE
_px_minus_intersect = TRUE
_remove_aggr_subquery = TRUE
_optimizer_push_down_distinct= 0
_optimizer_cost_based_transformation= "LINEAR"
_optimizer_squ_bottomup = TRUE
_push_join_predicate = TRUE
_push_join_union_view = TRUE
_push_join_union_view2 = TRUE
_fast_full_scan_enabled = TRUE
_optimizer_skip_scan_enabled= TRUE
_optimizer_join_sel_sanity_check= TRUE
_optim_enhance_nnull_detection= TRUE
_parallel_broadcast_enabled= TRUE
_always_semi_join = "CHOOSE"
_ordered_nested_loop = TRUE
_optimizer_max_permutations= 2000
_nlj_batching_enabled = 1
query_rewrite_enabled = "TRUE"
_mmv_query_rewrite_enabled= TRUE
_local_communication_costing_enabled= TRUE
_left_nested_loops_random= TRUE
_improved_row_length_enabled= TRUE
_index_join_enabled = TRUE
_enable_type_dep_selectivity= TRUE
_improved_outerjoin_card = TRUE
_use_column_stats_for_function= TRUE
_subquery_pruning_enabled= TRUE
_subquery_pruning_mv_enabled= FALSE
_or_expand_nvl_predicate = TRUE
_table_scan_cost_plus_one= TRUE
_cost_equality_semi_join = TRUE
_new_initial_join_orders = TRUE
_optim_peek_user_binds = TRUE
_minimal_stats_aggregation= TRUE
_gs_anti_semi_join_allowed= TRUE
_optim_new_default_join_sel= TRUE
optimizer_dynamic_sampling= 2
_pre_rewrite_push_pred = TRUE
_optimizer_new_join_card_computation= TRUE
_union_rewrite_for_gs = "YES_GSET_MVS"
_generalized_pruning_enabled= TRUE
_optim_adjust_for_part_skews= TRUE
_optimizer_compute_index_stats= TRUE
_optimizer_push_pred_cost_based= TRUE
_optimizer_extend_jppd_view_types= TRUE
_optimizer_filter_pred_pullup= TRUE
_optimizer_connect_by_cost_based= TRUE
_optimizer_connect_by_combine_sw= TRUE
_optimizer_connect_by_elim_dups= TRUE
_connect_by_use_union_all= "TRUE"
_right_outer_hash_enable = TRUE
_replace_virtual_columns = TRUE
skip_unusable_indexes = TRUE
_optimizer_correct_sq_selectivity= TRUE
_optimizer_dim_subq_join_sel= TRUE
_query_rewrite_setopgrw_enable= TRUE
_optimizer_join_order_control= 3
_bloom_filter_enabled = TRUE
_bloom_folding_enabled = TRUE
_bloom_serial_filter = "ON"
_optimizer_join_elimination_enabled= TRUE
_gby_hash_aggregation_enabled= TRUE
_globalindex_pnum_filter_enabled= TRUE
_sql_model_unfold_forloops= "RUN_TIME"
_optimizer_cost_hjsmj_multimatch= TRUE
_optimizer_transitivity_retain= TRUE
_px_pwg_enabled = TRUE
_optimizer_cbqt_no_size_restriction= TRUE
_optimizer_enhanced_filter_push= TRUE
_optimizer_rownum_pred_based_fkr= TRUE
_optimizer_better_inlist_costing= "ALL"
_optimizer_or_expansion = "DEPTH"
_optimizer_outer_to_anti_enabled= TRUE
_optimizer_order_by_elimination_enabled= TRUE
_optimizer_star_tran_in_with_clause= TRUE
_optimizer_sortmerge_join_inequality= TRUE
_selfjoin_mv_duplicates = TRUE
_dimension_skip_null = TRUE
_optimizer_complex_pred_selectivity= TRUE
_bloom_pruning_enabled = TRUE
_first_k_rows_dynamic_proration= TRUE
_optimizer_distinct_elimination= TRUE
_optimizer_multi_level_push_pred= TRUE
_optimizer_group_by_placement= TRUE
_optimizer_distinct_placement= TRUE
_optimizer_coalesce_subqueries= TRUE
_optimizer_enable_density_improvements= TRUE
_optimizer_rownum_bind_default= 10
_optimizer_improve_selectivity= TRUE
_optimizer_native_full_outer_join= "FORCE"
_optimizer_ansi_join_lateral_enhance= TRUE
_optimizer_multi_table_outerjoin= TRUE
_optimizer_null_accepting_semijoin= TRUE
_optimizer_ansi_rearchitecture= TRUE
_optimizer_aggr_groupby_elim= TRUE
_optimizer_enable_extended_stats= TRUE
_pivot_implementation_method= "CHOOSE"
_optimizer_interleave_jppd= TRUE
_optimizer_fkr_index_cost_bias= 10
_optimizer_extended_stats_usage_control= 192
_optimizer_fast_pred_transitivity= TRUE
_optimizer_fast_access_pred_analysis= TRUE
_optimizer_unnest_disjunctive_subq= TRUE
_optimizer_unnest_corr_set_subq= TRUE
_optimizer_distinct_agg_transform= TRUE
_aggregation_optimization_settings= 0
_optimizer_eliminate_filtering_join= TRUE
_optimizer_join_factorization= TRUE
_optimizer_use_cbqt_star_transformation= TRUE
_optimizer_table_expansion= TRUE
_and_pruning_enabled = TRUE
_optimizer_use_feedback = TRUE
_optimizer_gather_feedback= TRUE
_optimizer_try_st_before_jppd= TRUE
_px_partition_scan_enabled= TRUE
_optimizer_false_filter_pred_pullup= TRUE
_optimizer_enable_table_lookup_by_nl= TRUE
_optimizer_cube_join_enabled= TRUE
_optimizer_outer_join_to_inner= TRUE
_optimizer_hybrid_fpwj_enabled= TRUE
_px_object_sampling_enabled= TRUE
_px_concurrent = TRUE
_px_replication_enabled = TRUE
_optimizer_full_outer_join_to_outer= TRUE
_px_filter_parallelized = TRUE
_px_filter_skew_handling = TRUE
_px_groupby_pushdown = "FORCE"
_px_parallelize_expression= TRUE
_optimizer_gather_stats_on_load= TRUE
_optimizer_batch_table_access_by_rowid= TRUE
_px_wif_dfo_declumping = "CHOOSE"
_px_wif_extend_distribution_keys= TRUE
_px_join_skew_handling = TRUE
_px_adaptive_dist_method = "CHOOSE"
_px_partial_rollup_pushdown= "ADAPTIVE"
_optimizer_dsdir_usage_control= 126
_px_cpu_autodop_enabled = TRUE
_px_single_server_enabled= TRUE
_optimizer_use_gtt_session_stats= TRUE
_optimizer_adaptive_plans= TRUE
_optimizer_strans_adaptive_pruning= TRUE
_optimizer_proc_rate_level= "BASIC"
_optimizer_use_histograms= TRUE
_adaptive_window_consolidator_enabled= TRUE
_optimizer_cluster_by_rowid= TRUE
_optimizer_cluster_by_rowid_control= 129
_distinct_agg_optimization_gsets= "CHOOSE"
_px_scalable_invdist = TRUE
_optimizer_reduce_groupby_key= TRUE
_optimizer_cluster_by_rowid_batched= TRUE
_optimizer_inmemory_table_expansion= TRUE
_optimizer_inmemory_gen_pushable_preds= TRUE
_optimizer_inmemory_autodop= TRUE
_optimizer_inmemory_access_path= TRUE
_px_external_table_default_stats= TRUE
_optimizer_nlj_hj_adaptive_join= TRUE
_optimizer_inmemory_bloom_filter= TRUE
_optimizer_inmemory_cluster_aware_dop= TRUE
_optimizer_inmemory_minmax_pruning= TRUE
dg_broker_start = TRUE
_diag_adr_trace_dest = "/u01/app/oracle/diag/rdbms/stby/STBY/trace"
enable_pluggable_database= TRUE
NOTE: remote asm mode is local (mode 0x1; from cluster type)
Starting background process PMON
Mon Apr 10 23:41:46 2017
PMON started with pid=2, OS id=1315
Starting background process PSP0
Mon Apr 10 23:41:46 2017
PSP0 started with pid=3, OS id=1317
Starting background process VKTM
Starting background process GEN0
Mon Apr 10 23:41:47 2017
VKTM started with pid=4, OS id=1321 at elevated (RT) priority
Mon Apr 10 23:41:47 2017
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Mon Apr 10 23:41:47 2017
GEN0 started with pid=5, OS id=1327
Starting background process MMAN
Mon Apr 10 23:41:47 2017
MMAN started with pid=6, OS id=1329
Starting background process DIAG
Starting background process DBRM
Mon Apr 10 23:41:47 2017
DIAG started with pid=8, OS id=1333
Starting background process VKRM
Mon Apr 10 23:41:47 2017
DBRM started with pid=9, OS id=1335
Starting background process DIA0
Mon Apr 10 23:41:47 2017
VKRM started with pid=10, OS id=1337
Starting background process DBW0
Mon Apr 10 23:41:47 2017
DIA0 started with pid=11, OS id=1339
Starting background process LGWR
Mon Apr 10 23:41:47 2017
DBW0 started with pid=12, OS id=1342
Starting background process CKPT
Mon Apr 10 23:41:47 2017
LGWR started with pid=13, OS id=1344
Mon Apr 10 23:41:47 2017
CKPT started with pid=14, OS id=1347
Starting background process SMON
Mon Apr 10 23:41:47 2017
SMON started with pid=16, OS id=1355
Starting background process RECO
Starting background process LREG
Mon Apr 10 23:41:47 2017
RECO started with pid=18, OS id=1359
Starting background process PXMN
Mon Apr 10 23:41:47 2017
LREG started with pid=19, OS id=1361
Starting background process MMON
Mon Apr 10 23:41:47 2017
PXMN started with pid=20, OS id=1363
Starting background process MMNL
Mon Apr 10 23:41:47 2017
MMON started with pid=21, OS id=1365
Starting background process DMON
Mon Apr 10 23:41:47 2017
MMNL started with pid=22, OS id=1367
ORACLE_BASE from environment = /u01/app/oracle
Mon Apr 10 23:41:47 2017
DMON started with pid=23, OS id=1369
Mon Apr 10 23:41:47 2017
ALTER DATABASE MOUNT
Mon Apr 10 23:41:50 2017
Using default pga_aggregate_limit of 2048 MB
Mon Apr 10 23:41:52 2017
Successful mount of redo thread 1, with mount id 2528976638
Mon Apr 10 23:41:52 2017
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE MOUNT
Mon Apr 10 23:41:55 2017
ALTER DATABASE OPEN
Mon Apr 10 23:41:55 2017
Data Guard Broker initializing...
Mon Apr 10 23:41:55 2017
Starting Data Guard Broker (DMON)
Starting background process INSV
Mon Apr 10 23:41:55 2017
INSV started with pid=24, OS id=1402
Mon Apr 10 23:41:56 2017
Data Guard Broker initialization complete
Data Guard: verifying database primary role...
Starting background process NSV0
Mon Apr 10 23:41:58 2017
NSV0 started with pid=25, OS id=1410


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:01
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=rac_DGB.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:11
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Starting background process RSM0
Mon Apr 10 23:42:11 2017
RSM0 started with pid=26, OS id=1452
Mon Apr 10 23:42:12 2017
Data Guard: version check completed
Data Guard: primary role verified
Ping without log force is disabled
.
Starting background process TMON
Mon Apr 10 23:42:12 2017
LGWR: STARTING ARCH PROCESSES
Starting background process ARC0
Mon Apr 10 23:42:12 2017
TMON started with pid=27, OS id=1456
Mon Apr 10 23:42:12 2017
ARC0 started with pid=28, OS id=1460
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:42:12 2017
ARC0: STARTING ARCH PROCESSES
Starting background process ARC1
Mon Apr 10 23:42:12 2017
ARC1 started with pid=29, OS id=1462
Starting background process ARC2
Starting background process ARC3
Mon Apr 10 23:42:13 2017
ARC2 started with pid=30, OS id=1464
Mon Apr 10 23:42:13 2017
ARC3 started with pid=31, OS id=1466
ARC1: Archival started
ARC2: Archival started
Mon Apr 10 23:42:13 2017
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
Mon Apr 10 23:42:13 2017
ARC2: Becoming the heartbeat ARCH
Mon Apr 10 23:42:13 2017
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Apr 10 23:42:13 2017
Thread 1 advanced to log sequence 74 (thread open)
Thread 1 opened at log sequence 74
Current log# 2 seq# 74 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283
Current log# 2 seq# 74 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287
Successful open of redo thread 1
Mon Apr 10 23:42:13 2017
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 10 23:42:13 2017
SMON: enabling cache recovery
Mon Apr 10 23:42:13 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Error 12528 for archive log file 2 to 'rac1'
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:13 2017
Archived Log entry 52 added for thread 1 sequence 73 ID 0x96bd0722 dest 1:


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:13
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc0_1460.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
FAL[server, ARC0]: Error 12528 creating remote archivelog file 'rac1'
ARC0: FAL archive failed with error 12528. See trace for details
Mon Apr 10 23:42:13 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc0_1460.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Mon Apr 10 23:42:13 2017
ORACLE Instance STBY - Archival Error. Archiver continuing.
Mon Apr 10 23:42:14 2017
[1400] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:4809484 end:4810074 diff:590 ms (0.6 seconds)
Verifying minimum file header compatibility (11g) for tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Mon Apr 10 23:42:14 2017
SMON: enabling tx recovery
Starting background process SMCO
Mon Apr 10 23:42:14 2017
SMCO started with pid=34, OS id=1474
Mon Apr 10 23:42:14 2017
Database Characterset is AL32UTF8
No Resource Manager plan active
Mon Apr 10 23:42:16 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Mon Apr 10 23:42:16 2017
******************************************************************
TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Error 12528 received logging on to the standby


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Mon Apr 10 23:42:16 2017
Error 12528 received logging on to the standby
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc2_1464.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
PING[ARC2]: Heartbeat failed to connect to standby 'rac1'. Error is 12528.
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:16 2017
Error 12528 for archive log file 2 to 'rac1'
Mon Apr 10 23:42:16 2017
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


***********************************************************************

Fatal NI connect error 12528, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.78.61)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=RAC.racattack)(CID=(PROGRAM=oracle)(HOST=oem132.racattack)(USER=oracle))))

VERSION INFORMATION:
TNS for Linux: Version 12.1.0.2.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.1.0.2.0 - Production
Time: 10-APR-2017 23:42:16
Tracing not turned on.
Tns error struct:
ns main err code: 12564

TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Mon Apr 10 23:42:16 2017
Error 12528 received logging on to the standby
Mon Apr 10 23:42:16 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_arc2_1464.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
PING[ARC2]: Heartbeat failed to connect to standby 'rac1'. Error is 12528.
Mon Apr 10 23:42:17 2017
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='STBY';
Mon Apr 10 23:42:17 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:17 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_tt00_1468.trc:
ORA-12528: TNS:listener: all appropriate instances are blocking new connections
Mon Apr 10 23:42:17 2017
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='STBY';
Mon Apr 10 23:42:17 2017
replication_dependency_tracking turned off (no async multimaster replication found)
Mon Apr 10 23:42:18 2017
Thread 1 advanced to log sequence 75 (LGWR switch)
Current log# 1 seq# 75 mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279
Current log# 1 seq# 75 mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281
Mon Apr 10 23:42:18 2017
Archived Log entry 53 added for thread 1 sequence 74 ID 0x96bd0722 dest 1:
Starting background process AQPC
Mon Apr 10 23:42:18 2017
AQPC started with pid=37, OS id=1489
Mon Apr 10 23:42:21 2017
Database Characterset for PDB$SEED is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Opening pdb PDB$SEED (2) with no Resource Manager plan active
Mon Apr 10 23:42:35 2017
ALTER SYSTEM SET log_archive_dest_state_2='RESET' SCOPE=BOTH;
Starting background process CJQ0
Completed: ALTER DATABASE OPEN
Mon Apr 10 23:42:39 2017
CJQ0 started with pid=50, OS id=1904
Mon Apr 10 23:42:58 2017
Shared IO Pool defaulting to 36MB. Trying to get it from Buffer Cache for process 1365.
===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================
Mon Apr 10 23:42:59 2017
db_recovery_file_dest_size of 16384 MB is 0.84% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
alter pluggable database pdb open
Mon Apr 10 23:43:00 2017
Database Characterset for PDB is AL32UTF8
Due to limited space in shared pool (need 6094848 bytes, have 3981120 bytes), limiting Resource Manager entities from 2048 to 32
Mon Apr 10 23:43:15 2017
Opening pdb PDB (3) with no Resource Manager plan active
Pluggable database PDB opened read write
Completed: alter pluggable database pdb open
Mon Apr 10 23:45:07 2017
alter pluggable database pdb close
ALTER SYSTEM: Flushing buffer cache inst=0 container=3 local
Mon Apr 10 23:45:12 2017
Pluggable database PDB closed
Completed: alter pluggable database pdb close
Shutting down instance (immediate)
Stopping background process SMCO
Shutting down instance: further logons disabled
Stopping background process CJQ0
Killed process oracle@oem132.racattack (Q002) with pid is 48, OS pid 1537
Stopping background process MMNL
Stopping background process MMON
License high water mark = 9
ALTER DATABASE CLOSE NORMAL
Mon Apr 10 23:45:20 2017
SMON: disabling tx recovery
Mon Apr 10 23:45:20 2017
Stopping Emon pool
Stopping Emon pool
Mon Apr 10 23:45:20 2017
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:45:24 2017
ARCH shutting down
Mon Apr 10 23:45:24 2017
ARCH shutting down
ARC2: Archival stopped
Mon Apr 10 23:45:24 2017
ARC3: Archival stopped
Mon Apr 10 23:45:24 2017
ARCH shutting down
Mon Apr 10 23:45:24 2017
Mon Apr 10 23:45:24 2017
ARC1: Archival stoppedARCH shutting down

Mon Apr 10 23:45:24 2017
ARC0: Archival stopped
Mon Apr 10 23:45:25 2017
Thread 1 closed at log sequence 75
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
Mon Apr 10 23:45:27 2017
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Mon Apr 10 23:45:31 2017
Completed: Data Guard Broker shutdown
Mon Apr 10 23:45:33 2017
Mon Apr 10 23:45:33 2017
ARCH: Archival disabled due to shutdown: 1089
Stopping background process VKTM
Shutting down archive processes
Archiving is disabled
Mon Apr 10 23:45:41 2017
Instance shutdown complete


Ignore the Fatal NI and TNS errrors.  I plan to address them later.

.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8g : Switchover from RAC Primary to SingleInstance Standby

Mon, 2017-04-10 10:40
Continuing this series of posts where I have a SingleInstance/FileSystem Standby database for a RAC/ASM database ...


Checking the status of the configuration :

DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 28 seconds ago)

DGMGRL>


SQL> select thread#, max(sequence#)
2 from v$archived_log
3 group by thread#
4 order by 1
5 /

THREAD# MAX(SEQUENCE#)
---------- --------------
1 66
2 36

SQL> alter system archive log current;

System altered.


RFS[2]: Selected log 8 for thread 2 sequence 38 dbid 2519807290 branch 931825279
Mon Apr 10 23:03:17 2017
Archived Log entry 39 added for thread 2 sequence 37 ID 0x96312536 dest 1:
Mon Apr 10 23:03:18 2017
Media Recovery Waiting for thread 2 sequence 38 (in transit)
Mon Apr 10 23:03:18 2017
Recovery of Online Redo Log: Thread 2 Group 8 Seq 38 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_8.300.937936389
Mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_8.306.937936389
RFS[1]: Selected log 6 for thread 1 sequence 68 dbid 2519807290 branch 931825279
Mon Apr 10 23:03:23 2017
Archived Log entry 40 added for thread 1 sequence 67 ID 0x96312536 dest 1:
Mon Apr 10 23:03:23 2017
Media Recovery Waiting for thread 1 sequence 68 (in transit)
Mon Apr 10 23:03:23 2017
Recovery of Online Redo Log: Thread 1 Group 6 Seq 68 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_6.298.937936361
Mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_6.304.937936363


The dgmgrl status above is from node1 (collabn1) of the RAC (Primary)database.
The query on v$archived_log is from node2 (collabn2), instance RAC2 of the RAC (Primary) database.
The listing of messages in the alert log are from the Standby database.

Let me add some rows to the data I have in the PDB in the RAC database.

[oracle@collabn1 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 10 23:06:31 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 03 2017 22:49:41 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select count(*) from my_data;

COUNT(*)
----------
100

SQL> insert into my_data select rownum+100, to_char(rownum+100)
2 from dual
3 connect by level < 201;

200 rows created.

SQL> select count(*) from my_data;

COUNT(*)
----------
300

SQL> commit;

Commit complete.

SQL>


Let me try a SWITCHOVER now.

DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 102 seconds ago)

DGMGRL> switchover to stby;
Performing switchover NOW, please wait...
Operation requires a connection to instance "STBY" on database "stby"
Connecting to instance "STBY"...
Connected as SYSDBA.
New primary database "stby" is opening...
Oracle Clusterware is restarting database "rac" ...
Switchover succeeded, new primary is "stby"
DGMGRL>


Querying on the new "standby" that is RAC :

[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 10 23:36:12 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
MOUNTED PHYSICAL STANDBY

SQL> select instance_name, host_name from gv$instance;

INSTANCE_NAME
----------------
HOST_NAME
----------------------------------------------------------------
RAC1
collabn1.racattack

RAC2
collabn2.racattack


SQL>


While, the old Standby :

[oracle@oem132 trace]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 10 23:37:34 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysdba

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select open_mode, databas_role from v$database;
select open_mode, databas_role from v$database
*
ERROR at line 1:
ORA-00904: "DATABAS_ROLE": invalid identifier


SQL> select open_mode, database_role from v$database;

OPEN_MODE DATABASE_ROLE
-------------------- ----------------
READ WRITE PRIMARY

SQL> select instance_name, host_name from gv$instance;

INSTANCE_NAME
----------------
HOST_NAME
----------------------------------------------------------------
STBY
oem132.racattack


SQL>


So, host "oem132.racattack" now has the Primary database in Read Write mode. The instance name is STBY because I chose that to be the instance name through the testing.  (I could have chosen BOSTON and CHICAGO as is used in some Oracle examples / documentation).

Let me verify my data on STBY :

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
Database mounted.
Database opened.
SQL> alter pluggable database pdb open;

Pluggable database altered.

SQL> connect hemant/hemant@PDB
Connected.
SQL> select count(*) from my_data;

COUNT(*)
----------
300

SQL>


The rows inserted at the RAC Primary are now in the Pluggable Database PDB at STBY.

.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 8f : Accessing data in the PDB in the Standby

Sun, 2017-04-09 03:03
Apparently, the error :
< br />
alter pluggable database pdb open
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.

in my previous post was a spurious error.

I am able to access the PDB in the Standby.  Thus :

[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [STBY] ? STBY
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 9 15:57:27 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL> rem here I notice the alert log showing :
SQL> rem ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
SQL> rem Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT NODELAY
SQL> recover managed standby database cancel;
Media recovery complete.
SQL> rem I cancel the automatic recovery as I do not want to run Active Data Guard
SQL> alter database open read only;

Database altered.

SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.


SQL>
SQL> select con_id, name, open_mode, open_Time from v$pdbs;

CON_ID NAME OPEN_MODE
---------- ------------------------------ ----------
OPEN_TIME
---------------------------------------------------------------------------
2 PDB$SEED READ ONLY
09-APR-17 04.00.48.232 PM +08:00

3 PDB READ ONLY
09-APR-17 04.00.59.105 PM +08:00


SQL>
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Sun Apr 9 16:02:02 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 03 2017 22:49:41 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
MY_DATA

SQL> select count(*) from my_data;

COUNT(*)
----------
100

SQL> show con_id;

CON_ID
------------------------------
3
SQL>


Thus, the PDB on the Standby *does* OPEN READ ONLY and I can see the data that had been populated from the Primary.
.
.
.
Categories: DBA Blogs

12cR1 RAC Posts -- 8e : Redo Shipping and Apply (RAC to nonRAC)

Mon, 2017-04-03 10:22
Continuing the series of posts on the RAC/ASM to SingleInstance/FileSystem  DataGuard configuration ...

With both instances RAC1 and RAC2 running, I have redo shipping from both instances to the single instance STBY.
(In my test server with 3 VMs, it takes a number of minutes to have all 3 instances running and communicating with each other ... so there are spurious ORA and TNS errors until the 3 instances have stabilized. {in fact, it would be best to start the Standby before starting the RAC Primary instances}. I am posting messages after the stabilization).

Thus, on RAC 1 :
Mon Apr 03 22:43:51 2017
Archived Log entry 97 added for thread 1 sequence 59 ID 0x96312536 dest 1:
Mon Apr 03 22:43:52 2017
TT00: Standby redo logfile selected for thread 1 sequence 60 for destination LOG_ARCHIVE_DEST_2

And on RAC 2:
Mon Apr 03 22:44:12 2017
Thread 2 cannot allocate new log, sequence 32
Checkpoint not complete
Current log# 3 seq# 31 mem# 0: +DATA/RAC/ONLINELOG/group_3.290.931826413
Current log# 3 seq# 31 mem# 1: +FRA/RAC/ONLINELOG/group_3.259.931826417
Mon Apr 03 22:44:19 2017
Thread 2 advanced to log sequence 32 (LGWR switch)
Current log# 4 seq# 32 mem# 0: +DATA/RAC/ONLINELOG/group_4.291.931826417
Current log# 4 seq# 32 mem# 1: +FRA/RAC/ONLINELOG/group_4.260.931826421
Mon Apr 03 22:44:20 2017
Archived Log entry 99 added for thread 2 sequence 31 ID 0x96312536 dest 1:
Mon Apr 03 22:44:21 2017
TT00: Standby redo logfile selected for thread 2 sequence 32 for destination LOG_ARCHIVE_DEST_2

And on STBY :
Mon Apr 03 22:43:13 2017
Media Recovery Waiting for thread 2 sequence 31 (in transit)
RFS[2]: Selected log 5 for thread 1 sequence 60 dbid 2519807290 branch 931825279
Mon Apr 03 22:43:52 2017
Archived Log entry 25 added for thread 1 sequence 59 ID 0x96312536 dest 1:
Mon Apr 03 22:44:20 2017
Archived Log entry 26 added for thread 2 sequence 31 rlc 931825279 ID 0x96312536 dest 2:
Mon Apr 03 22:44:20 2017
Media Recovery Log /u01/app/oracle/fast_recovery_area/STBY/archivelog/2017_04_03/o1_mf_2_31_dg4qr2qw_.arc
RFS[1]: Selected log 7 for thread 2 sequence 32 dbid 2519807290 branch 931825279
Mon Apr 03 22:44:21 2017
Media Recovery Log /u01/app/oracle/fast_recovery_area/STBY/archivelog/2017_04_03/o1_mf_1_59_dg4qx70d_.arc
Resize operation completed for file# 3, old size 768000K, new size 778240K
Media Recovery Waiting for thread 1 sequence 60 (in transit)
Mon Apr 03 22:44:24 2017
Recovery of Online Redo Log: Thread 1 Group 5 Seq 60 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/STBY/onlinelog/group_5.292.937936339
Mem# 1: /u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_5.303.937936343

Thus, Thread1 (instance RAC1) is currently at Sequence#60, Thread2 (instanc RAC2) is currently at Sequence=32.  The STBY alert log shows that it is receiving Redo for both Threads.

Let me login to the PDB in the RAC database and create some data.

[oracle@collabn1 ~]$ tnsping PDB

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 03-APR-2017 22:49:34

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn-cluster-scan.racattack)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = PDB.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ sqlplus hemant/hemant@PDB

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 22:49:39 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Last Successful login time: Mon Apr 03 2017 22:48:56 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> show con_id

CON_ID
------------------------------
3
SQL> create table my_data (id_col number, data_col varchar2(15));

Table created.

SQL> insert into my_data select rownum, 'Row:' || to_char(rownum)
2 from dual connect by level < 101;

100 rows created.

SQL> commit;

Commit complete.

SQL>


So, user HEMANT in the Pluggable Database PDB has a table with 100 rows.

I open the Standby Database Read Only.

[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [oracle] ? STBY
The Oracle base has been set to /u01/app/oracle
[oracle@oem132 ~]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 22:53:05 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysdba

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> recover managed standby database cancel;
Media recovery complete.
SQL> alter database open;

Database altered.

SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.


SQL> alter pluggable database pdb open;
alter pluggable database pdb open
*
ERROR at line 1:
ORA-65019: pluggable database PDB already open

SQL> alter pluggable database pdb close;

Pluggable database altered.


So, there is an error opening the PDB in the Standby database.  I can't find a note about it on MoS.  This will need more research. Let my try with a Common User in CDB$ROOT.

Restart the Standby.
SQL> shutdown
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ sqlplus

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 23:14:54 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Enter user-name: / as sysdba
Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL> recover managed standby database using current logfile disconnect from session;
Media recovery complete.
SQL>


Create a Common User with data in the RAC Primary.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 3 23:17:00 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> create user C##HKCCOMMON identified by hemant;

User created.

SQL> grant connect, resource, dba to c##HKCCOMMON;

Grant succeeded.

SQL> connect C##HKCCOMMON/hemant
Connected.
SQL> create table root_my_table (id_col number, data_col varchar2(15));

Table created.

SQL> insert into root_my_table select rownum, 'AA' || to_char(rownum)
2 from dual connect by level < 101;

100 rows created.

SQL> commit;

Commit complete.

SQL>


Now, open the Standby Read Only and verify the ROOT_MY_TABLE.

SQL> recover managed standby database cancel;
Media recovery complete.

SQL> alter database open read only;

Database altered.

SQL> connect c##HKCCOMMON/hemant
Connected.
SQL> select count(*) from root_my_table;

COUNT(*)
----------
100

SQL>


Yes, the CDB ROOT user and data have gone over to the Standby !

.
.


Categories: DBA Blogs

12cR1 RAC Posts -- 8d : Registering the two databases in DataGuard Broker

Mon, 2017-03-27 10:19
Continuing my series on setting up a non-RAC, FileSystem Standby for a RAC, ASM Database....

On the Standby:

[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [oracle] ? STBY
The Oracle base has been set to /u01/app/oracle
[oracle@oem132 ~]$ lsnrctl start

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:24:54

Copyright (c) 1991, 2014, Oracle. All rights reserved.

Starting /u01/app/oracle/product/12.1.0/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 12.1.0.2.0 - Production
System parameter file is /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Log messages written to /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oem132.racattack)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date 27-MAR-2017 22:24:58
Uptime 0 days 0 hr. 0 min. 2 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
Services Summary...
Service "STBY" has 1 instance(s).
Instance "STBY", status UNKNOWN, has 2 handler(s) for this service...
The command completed successfully
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Mon Mar 27 22:25:07 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> alter database mount standby database;

Database altered.

SQL>
SQL> show parameter dg_broker

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
dg_broker_config_file1 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr1STBY.dat
dg_broker_config_file2 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr2STBY.dat
dg_broker_start boolean FALSE
SQL>
SQL> alter system set dg_broker_Start=TRUE;

System altered.

SQL>


On the first instance of the Primary :

SQL> show parameter dg_broker

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
dg_broker_config_file1 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr1RAC.dat
dg_broker_config_file2 string /u01/app/oracle/product/12.1.0
/dbhome_1/dbs/dr2RAC.dat
dg_broker_start boolean FALSE
SQL> alter system set dg_broker_config_file1='+DATA/RAC/dgbroker1.dat' sid='*' scope=BOTH;

System altered.

SQL> alter system set dg_broker_config_file2='+FRA/RAC/dgbroker2.dat' sid='*' scope=BOTH;

System altered.

SQL> alter system set dg_broker_start=TRUE;

System altered.

SQL>

[oracle@collabn1 ~]$ tnsping RAC1

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:37:41

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = RAC.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ sqlplus sys/racattack@STBY

SQL*Plus: Release 12.1.0.2.0 Production on Mon Mar 27 22:37:51 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

ERROR:
ORA-01033: ORACLE initialization or shutdown in progress
Process ID: 0
Session ID: 0 Serial number: 0


Enter user-name: sys/racattack@STBY as SYSDBA

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@collabn1 ~]$ tnsping RAC1

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:38:07

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = collabn1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = RAC.racattack)))
OK (0 msec)
[oracle@collabn1 ~]$ tnsping STBY

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 27-MAR-2017 22:38:10

Copyright (c) 1997, 2014, Oracle. All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = oem132)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = STBY)))
OK (0 msec)
[oracle@collabn1 ~]$ dgmgrl
DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production

Copyright (c) 2000, 2013, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
DGMGRL> connect sys/racattack@RAC1
Connected as SYSDBA.
DGMGRL> create configuration RAC as primary database is RAC connect identifier is RAC1;
Configuration "rac" created with primary database "rac"
DGMGRL> add database stby as connect identifier is stby maintained as physical;
Database "stby" added
DGMGRL> enable configuration;
Enabled.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database
Error: ORA-16664: unable to receive the result from a database

Fast-Start Failover: DISABLED

Configuration Status:
ERROR (status updated 7 seconds ago)

DGMGRL>


Errors in the STBY DataGuard Broker Logfile drcSTBY.log :

03/27/2017 22:45:12
Warning: Property 'StandbyFileManagement' has inconsistent values:METADATA='MANUAL', SPFILE='', DATABASE='MANUAL'
Warning: Property 'ArchiveLagTarget' has inconsistent values:METADATA='0', SPFILE='', DATABASE='0'
Warning: Property 'LogArchiveMaxProcesses' has inconsistent values:METADATA='4', SPFILE='', DATABASE='4'
Warning: Property 'LogArchiveMinSucceedDest' has inconsistent values:METADATA='1', SPFILE='', DATABASE='1'
SPFILE is missing value for property 'LogArchiveTrace' with sid='STBY'
Warning: Property 'LogArchiveTrace' has inconsistent values:METADATA='0', SPFILE='(missing)', DATABASE='0'
SPFILE is missing value for property 'LogArchiveFormat' with sid='STBY'
Warning: Property 'LogArchiveFormat' has inconsistent values:METADATA='%t_%s_%r.dbf', SPFILE='(missing)', DATABASE='%t_%s_%r.dbf'
Failed to send message to site rac. Error code is ORA-16501.

Errors in the RAC1 DataGuard Broker Logfile drcRAC1.log :

Site stby returned ORA-16664.
Data Guard Broker Status Summary:
Type Name Severity Status
Configuration rac Warning ORA-16607
Primary Database rac Success ORA-00000
Physical Standby Database stby Error ORA-16664


Actions taken on the Standby :

SQL> show parameter standby

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
standby_archive_dest string ?/dbs/arch
standby_file_management string MANUAL
SQL> alter system set standby_file_management='AUTO' scope=BOTH;

System altered.

SQL> alter system set archive_lag_target=0 scope=BOTH;

System altered.

SQL> alte system set log_archive_max_processes=4 scope=BOTH;
SP2-0734: unknown command beginning "alte syste..." - rest of line ignored.
SQL> alter system set log_archive_max_processes=4 scope=BOTH;

System altered.

SQL> alter system set log_archive_min_succeed_dest=1 scope=BOTH;

System altered.

SQL> alter system set log_archive_trace=0 scope=BOTH;

System altered.

SQL> alter system set log_archive_format='%t_%s_%r.dbf' scope=BOTH;
alter system set log_archive_format='%t_%s_%r.dbf' scope=BOTH
*
ERROR at line 1:
ORA-02095: specified initialization parameter cannot be modified


SQL>


I also had to edit the TNSNAMES.ORA on the STBY server for the correct *VIP* entry (not the physical hostname) for the RAC1 instance.

RAC1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.78.61)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = RAC.racattack)
)
)

RAC1_DGB =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.78.61)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = RAC_DGMGRL)
)
)


Thereafter, it worked.  Note : You sometimes have to wait a while to enter the SHOW CONFIGURATION, the first result may show an error.  So I entered it twice.

DGMGRL> disable configuration;
Disabled.
DGMGRL> enable configuration;
Enabled.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database
Error: ORA-16664: unable to receive the result from a database

Fast-Start Failover: DISABLED

Configuration Status:
ERROR (status updated 125 seconds ago)

DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 14 seconds ago)

DGMGRL>


These are the drcSTBY.log entries when the TNSNAMES.ORA for RAC1 was incorrect on the Standby Server :

Failed to connect to remote database rac. Error is ORA-12541
Failed to send message to site rac. Error code is ORA-12541.


This is the drcSTBY.log entry on success :

03/27/2017 23:00:51
Apply Instance for Database stby set to STBY


And the entry in drcRAC1.log on success :

03/27/2017 23:00:10
DISABLE CONFIGURATION
03/27/2017 23:00:46
ENABLE CONFIGURATION
03/27/2017 23:00:57
EDIT DATABASE stby SET PROPERTY ActualApplyInstance = STBY
03/27/2017 23:01:00
Apply Instance for database stby is STBY


Setting Maximum Performance mode (command issued on the *Primary*) :

SQL> alter database set standby to maximize performance;

Database altered.

SQL>

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxPerformance;
Succeeded.
DGMGRL> show configuration;

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 34 seconds ago)

DGMGRL>
DGMGRL> show configuration verbose

Configuration - rac

Protection Mode: MaxPerformance
Members:
rac - Primary database
stby - Physical standby database

Properties:
FastStartFailoverThreshold = '30'
OperationTimeout = '30'
TraceLevel = 'USER'
FastStartFailoverLagLimit = '30'
CommunicationTimeout = '180'
ObserverReconnect = '0'
FastStartFailoverAutoReinstate = 'TRUE'
FastStartFailoverPmyShutdown = 'TRUE'
BystandersFollowRoleChange = 'ALL'
ObserverOverride = 'FALSE'
ExternalDestination1 = ''
ExternalDestination2 = ''
PrimaryLostWriteAction = 'CONTINUE'

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS

DGMGRL>


Will be testing Redo Transmission and Apply later.
.
.
.

Categories: DBA Blogs

1.5million PageViews

Mon, 2017-03-13 10:01
My Oracle Blog now has had 1.5million PageViews.


The 1million PageViews mark was hit in March 2015.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8c : Ignorable "Errors" during the DUPLICATE

Mon, 2017-03-13 09:56
In yesterday's post, I had shown a DUPLICATE DATABASE from RAC-ASM to SingleInstance-FileSystem.

During the course of the DUPLICATE DATABASE run, the Standby alert log seemingly reported errors.  I chose to ignore the "errors" as I know the DUPLICATE was running successfully.

For your reference, here are some of the good messages :

Sun Mar 12 23:17:55 2017
ALTER SYSTEM SET control_files='/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl','/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl' COMMENT='Set by RMAN' SCOPE=SPFILE;
Sun Mar 12 23:17:59 2017
ALTER SYSTEM SET control_files='/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl','/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl' COMMENT='Set by RMAN' SCOPE=SPFILE;

Sun Mar 12 23:21:41 2017
Switch of datafile 1 complete to datafile copy
checkpoint is 3315193
Switch of datafile 3 complete to datafile copy
checkpoint is 3315211
Switch of datafile 4 complete to datafile copy
checkpoint is 3315270
Switch of datafile 5 complete to datafile copy
checkpoint is 1754712
Switch of datafile 6 complete to datafile copy
checkpoint is 3315280
Switch of datafile 7 complete to datafile copy
checkpoint is 1754712
Switch of datafile 8 complete to datafile copy
checkpoint is 3315276
Switch of datafile 9 complete to datafile copy
checkpoint is 2978862
Switch of datafile 10 complete to datafile copy
checkpoint is 2978862
Switch of datafile 11 complete to datafile copy
checkpoint is 2978862


And here are the errors (about the online redo logs) continuously being reported.  These are messages which I ignored because the Standby has no online redo logs yet.

Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_1.283.931825279'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_2.258.931825287'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/STBY/onlinelog/group_2.284.931825283'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 3 of thread 2
ORA-00312: online log 3 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 3 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 3 of thread 2
ORA-00312: online log 3 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_3.259.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 3 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_3.290.931826413'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 4 of thread 2
ORA-00312: online log 4 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 4 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 4 of thread 2
ORA-00312: online log 4 thread 2: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_4.260.931826421'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 4 thread 2: '/u01/app/oracle/oradata/STBY/onlinelog/group_4.291.931826417'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 5 of thread 0
ORA-00312: online log 5 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_5.303.937936343'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 5 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_5.292.937936339'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 5 of thread 0
ORA-00312: online log 5 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_5.303.937936343'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 5 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_5.292.937936339'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 6 of thread 0
ORA-00312: online log 6 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_6.304.937936363'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 6 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_6.298.937936361'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 6 of thread 0
ORA-00312: online log 6 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_6.304.937936363'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 6 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_6.298.937936361'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 7 of thread 0
ORA-00312: online log 7 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_7.305.937936377'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 7 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_7.299.937936375'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 7 of thread 0
ORA-00312: online log 7 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_7.305.937936377'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 7 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_7.299.937936375'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 8 of thread 0
ORA-00312: online log 8 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_8.306.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 8 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_8.300.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 8 of thread 0
ORA-00312: online log 8 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_8.306.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 8 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_8.300.937936389'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 9 of thread 0
ORA-00312: online log 9 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_9.307.937936405'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 9 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_9.301.937936403'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sun Mar 12 23:18:22 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_lgwr_16416.trc:
ORA-00313: open failed for members of log group 9 of thread 0
ORA-00312: online log 9 thread 0: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_9.307.937936405'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 9 thread 0: '/u01/app/oracle/oradata/STBY/onlinelog/group_9.301.937936403'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3

alter database clear logfile group 1
Clearing online log 1 of thread 1 sequence number 41
Sun Mar 12 23:21:42 2017
Errors in file /u01/app/oracle/diag/rdbms/stby/STBY/trace/STBY_ora_16474.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/u01/app/oracle/fast_recovery_area/STBY/onlinelog/group_1.257.931825281'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3



It is also interesting to see how CONTROL_FILES, DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT are reported in the alert log :

  
control_files = "/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl"
control_files = "/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl"
db_file_name_convert = "+DATA/RAC"
db_file_name_convert = "/u01/app/oracle/oradata/STBY"
db_file_name_convert = "+FRA/RAC"
db_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"
log_file_name_convert = "+DATA/RAC"
log_file_name_convert = "/u01/app/oracle/oradata/STBY"
log_file_name_convert = "+FRA/RAC"
log_file_name_convert = "/u01/app/oracle/fast_recovery_area/STBY"

Each component of the entry in the init parameter file is reported on a separate line in the alert log.
.
.
.

Categories: DBA Blogs

12cR1 RAC Posts -- 8b : DUPLICATE DATABASE FOR STANDBY

Sun, 2017-03-12 10:27
Continuing the previous post, where I have done the pre-setup...
(A reminder : This is a SingleInstance Standby on FileSystem for a RAC Database with a PDB on ASM)


On the SingleInstance Node (for the Standby database)

Precreate the required folders for the database (and redo and control) files (I later realised that the "onlinelog" folders for the SEED and PDB are not required)

[oracle@oem132 ~]$ cd /u01/app/oracle/oradata
[oracle@oem132 oradata]$ mkdir STBY
[oracle@oem132 oradata]$ cd STBY
[oracle@oem132 STBY]$ mkdir datafile
[oracle@oem132 STBY]$ mkdir onlinelog
[oracle@oem132 STBY]$ mkdir 44bbc69ce8f552aee053334ea8c07365
[oracle@oem132 STBY]$ mkdir fd9ac20f64d244d7e043b6a9e80a2f2f
[oracle@oem132 STBY]$ cd 44*
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ mkdir datafile tempfile
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ cd ../fd*
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$ mkdir datafile tempfile
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$
[oracle@oem132 STBY]$
[oracle@oem132 STBY]$ cd /u01/app/oracle/fast_recovery_area
[oracle@oem132 fast_recovery_area]$ mkdir STBY
[oracle@oem132 fast_recovery_area]$ cd STBY
[oracle@oem132 STBY]$ mkdir onlinelog
[oracle@oem132 STBY]$ mkdir 44bbc69ce8f552aee053334ea8c07365
[oracle@oem132 STBY]$ mkdir fd9ac20f64d244d7e043b6a9e80a2f2f
[oracle@oem132 STBY]$ cd 44*
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ mkdir onlinelog
[oracle@oem132 44bbc69ce8f552aee053334ea8c07365]$ cd ../fd*
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$ mkdir onlinelog
[oracle@oem132 fd9ac20f64d244d7e043b6a9e80a2f2f]$
[oracle@oem132 STBY]$

Note that the names of the two folders for the SEED and the PDB are in *lower case*.

Next start the listener for STBY.

[oracle@oem132 STBY]$ cd
[oracle@oem132 ~]$ tail -1 /etc/oratab
STBY:/u01/app/oracle/product/12.1.0/dbhome_1:N
[oracle@oem132 ~]$ . oraenv
ORACLE_SID = [oracle] ? STBY
The Oracle base has been set to /u01/app/oracle
[oracle@oem132 ~]$ ls -l $ORACLE_HOME/dbs/*STBY*
-rw-r--r-- 1 oracle oinstall 508 Mar 6 22:11 /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initSTBY.ora
-rw-r----- 1 oracle oinstall 7680 Mar 6 22:13 /u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwSTBY
[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Mar 12 23:03:43 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount
ORACLE instance started.

Total System Global Area 838860800 bytes
Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ lsnrctl start

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 12-MAR-2017 23:04:28

Copyright (c) 1991, 2014, Oracle. All rights reserved.

Starting /u01/app/oracle/product/12.1.0/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 12.1.0.2.0 - Production
System parameter file is /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Log messages written to /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oem132.racattack)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date 12-MAR-2017 23:04:28
Uptime 0 days 0 hr. 0 min. 12 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/oem132/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oem132.racattack)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
Services Summary...
Service "STBY" has 1 instance(s).
Instance "STBY", status UNKNOWN, has 2 handler(s) for this service...
The command completed successfully
[oracle@oem132 ~]$

I am now ready to issue the DUPLICATE DATABASE command from the RAC Primary.

[oracle@collabn1 ~]$ . oraenv
ORACLE_SID = [RAC1] ? RAC1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@collabn1 ~]$ rman

Recovery Manager: Release 12.1.0.2.0 - Production on Sun Mar 12 23:17:27 2017

Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.

RMAN> connect target sys/racattack

connected to target database: RAC (DBID=2519807290)

RMAN> connect auxiliary sys/racattack@STBY

connected to auxiliary database: RAC (not mounted)

RMAN> DUPLICATE TARGET DATABASE FOR STANDBY FROM ACTIVE DATABASE ;

Starting Duplicate Db at 12-MAR-17
using target database control file instead of recovery catalog
allocated channel: ORA_AUX_DISK_1
channel ORA_AUX_DISK_1: SID=12 device type=DISK

contents of Memory Script:
{
backup as copy reuse
targetfile '+DATA/RAC/PASSWORD/pwdrac.277.931824933' auxiliary format
'/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwSTBY' ;
}
executing Memory Script

Starting backup at 12-MAR-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=273 instance=RAC1 device type=DISK
Finished backup at 12-MAR-17

contents of Memory Script:
{
sql clone "alter system set control_files =
''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment=
''Set by RMAN'' scope=spfile";
backup as copy current controlfile for standby auxiliary format '/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl';
restore clone primary controlfile to '/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl' from
'/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl';
sql clone "alter system set control_files =
''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment=
''Set by RMAN'' scope=spfile";
shutdown clone immediate;
startup clone nomount;
}
executing Memory Script

sql statement: alter system set control_files = ''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment= ''Set by RMAN'' scope=spfile

Starting backup at 12-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
copying standby control file
output file name=/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl tag=TAG20170312T231755
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 12-MAR-17

Starting restore at 12-MAR-17
using channel ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: copied control file copy
Finished restore at 12-MAR-17

sql statement: alter system set control_files = ''/u01/app/oracle/oradata/STBY/controlfile/o1_mf_ddbs0p3w_.ctl'', ''/u01/app/oracle/fast_recovery_area/STBY/controlfile/o1_mf_ddbs0p40_.ctl'' comment= ''Set by RMAN'' scope=spfile

Oracle instance shut down

connected to auxiliary database (not started)
Oracle instance started

Total System Global Area 838860800 bytes

Fixed Size 2929936 bytes
Variable Size 230689520 bytes
Database Buffers 599785472 bytes
Redo Buffers 5455872 bytes

contents of Memory Script:
{
sql clone 'alter database mount standby database';
}
executing Memory Script

sql statement: alter database mount standby database

contents of Memory Script:
{
set newname for tempfile 1 to
"/u01/app/oracle/oradata/STBY/tempfile/temp.285.931825311";
set newname for tempfile 2 to
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/pdbseed_temp012016-12-29_12-23-03-am.dbf";
set newname for tempfile 3 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/tempfile/temp.295.931827153";
switch clone tempfile all;
set newname for datafile 1 to
"/u01/app/oracle/oradata/STBY/datafile/system.279.931825083";
set newname for datafile 3 to
"/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019";
set newname for datafile 4 to
"/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149";
set newname for datafile 5 to
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323";
set newname for datafile 6 to
"/u01/app/oracle/oradata/STBY/datafile/users.280.931825149";
set newname for datafile 7 to
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323";
set newname for datafile 8 to
"/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143";
set newname for datafile 9 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089";
set newname for datafile 10 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089";
set newname for datafile 11 to
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189";
backup as copy reuse
datafile 1 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/system.279.931825083" datafile
3 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019" datafile
4 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149" datafile
5 auxiliary format
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323" datafile
6 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/users.280.931825149" datafile
7 auxiliary format
"/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323" datafile
8 auxiliary format
"/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143" datafile
9 auxiliary format
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089" datafile
10 auxiliary format
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089" datafile
11 auxiliary format
"/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189" ;
sql 'alter system archive log current';
}
executing Memory Script

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

renamed tempfile 1 to /u01/app/oracle/oradata/STBY/tempfile/temp.285.931825311 in control file
renamed tempfile 2 to /u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/pdbseed_temp012016-12-29_12-23-03-am.dbf in control file
renamed tempfile 3 to /u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/tempfile/temp.295.931827153 in control file

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

Starting backup at 12-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile file number=00001 name=+DATA/RAC/DATAFILE/system.279.931825083
output file name=/u01/app/oracle/oradata/STBY/datafile/system.279.931825083 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:45
channel ORA_DISK_1: starting datafile copy
input datafile file number=00003 name=+DATA/RAC/DATAFILE/sysaux.278.931825019
output file name=/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00007 name=+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/sysaux.286.931825323
output file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00010 name=+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/sysaux.294.931827089
output file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:25
channel ORA_DISK_1: starting datafile copy
input datafile file number=00009 name=+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/system.293.931827089
output file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00005 name=+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/system.287.931825323
output file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00004 name=+DATA/RAC/DATAFILE/undotbs1.281.931825149
output file name=/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00008 name=+DATA/RAC/DATAFILE/undotbs2.289.931826143
output file name=/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting datafile copy
input datafile file number=00006 name=+DATA/RAC/DATAFILE/users.280.931825149
output file name=/u01/app/oracle/oradata/STBY/datafile/users.280.931825149 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
channel ORA_DISK_1: starting datafile copy
input datafile file number=00011 name=+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/users.296.931827189
output file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189 tag=TAG20170312T231824
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 12-MAR-17

sql statement: alter system archive log current

contents of Memory Script:
{
switch clone datafile all;
}
executing Memory Script

datafile 1 switched to datafile copy
input datafile copy RECID=3 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/system.279.931825083
datafile 3 switched to datafile copy
input datafile copy RECID=4 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019
datafile 4 switched to datafile copy
input datafile copy RECID=5 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149
datafile 5 switched to datafile copy
input datafile copy RECID=6 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.287.931825323
datafile 6 switched to datafile copy
input datafile copy RECID=7 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/users.280.931825149
datafile 7 switched to datafile copy
input datafile copy RECID=8 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.286.931825323
datafile 8 switched to datafile copy
input datafile copy RECID=9 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143
datafile 9 switched to datafile copy
input datafile copy RECID=10 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.293.931827089
datafile 10 switched to datafile copy
input datafile copy RECID=11 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.294.931827089
datafile 11 switched to datafile copy
input datafile copy RECID=12 STAMP=938474501 file name=/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296.931827189
Finished Duplicate Db at 12-MAR-17

RMAN>

The DUPLICATE ... FOR STANDBY has completed.  Let me just run two verification checks.

On the Standby :

[oracle@oem132 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Mar 12 23:24:05 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY

SQL> select open_mode from v$database;

OPEN_MODE
--------------------
MOUNTED

SQL>
SQL> select file#, name from v$datafile order by 1;

FILE#
----------
NAME
--------------------------------------------------------------------------------
1
/u01/app/oracle/oradata/STBY/datafile/system.279.931825083

3
/u01/app/oracle/oradata/STBY/datafile/sysaux.278.931825019

4
/u01/app/oracle/oradata/STBY/datafile/undotbs1.281.931825149

5
/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/system.28
7.931825323

6
/u01/app/oracle/oradata/STBY/datafile/users.280.931825149

7
/u01/app/oracle/oradata/STBY/fd9ac20f64d244d7e043b6a9e80a2f2f/datafile/sysaux.28
6.931825323

8
/u01/app/oracle/oradata/STBY/datafile/undotbs2.289.931826143

9
/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/system.29
3.931827089

10
/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/sysaux.29
4.931827089

11
/u01/app/oracle/oradata/STBY/44bbc69ce8f552aee053334ea8c07365/datafile/users.296
.931827189


10 rows selected.

SQL>
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
[oracle@oem132 ~]$ cd $ORACLE_HOME/dbs
[oracle@oem132 dbs]$ ls -ltr *STBY*
-rw-r--r-- 1 oracle oinstall 508 Mar 6 22:11 initSTBY.ora
-rw-r----- 1 oracle oinstall 24 Mar 12 23:07 lkSTBY
-rw-r----- 1 oracle oinstall 7680 Mar 12 23:17 orapwSTBY
-rw-rw---- 1 oracle oinstall 1544 Mar 12 23:18 hc_STBY.dat
-rw-r----- 1 oracle oinstall 17920 Mar 12 23:18 spfileSTBY.ora
[oracle@oem132 dbs]$

On the Primary :

RMAN> exit


Recovery Manager complete.
[oracle@collabn1 ~]$ sqlplus '/ as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Mar 12 23:26:17 2017

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PRIMARY

SQL> select open_mode from v$database;

OPEN_MODE
--------------------
READ WRITE

SQL> set pages600
SQL> select file#, name from v$datafile order by 1;

FILE#
----------
NAME
--------------------------------------------------------------------------------
1
+DATA/RAC/DATAFILE/system.279.931825083

3
+DATA/RAC/DATAFILE/sysaux.278.931825019

4
+DATA/RAC/DATAFILE/undotbs1.281.931825149

5
+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/system.287.931825323

6
+DATA/RAC/DATAFILE/users.280.931825149

7
+DATA/RAC/FD9AC20F64D244D7E043B6A9E80A2F2F/DATAFILE/sysaux.286.931825323

8
+DATA/RAC/DATAFILE/undotbs2.289.931826143

9
+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/system.293.931827089

10
+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/sysaux.294.931827089

11
+DATA/RAC/44BBC69CE8F552AEE053334EA8C07365/DATAFILE/users.296.931827189


10 rows selected.

SQL>

All datafiles (CDB, SEED and 1 PDB) are replicated.  Note how the folder names for the SEED and PDB are in lower-case on the Standby.  Since the source database is RAC, it has two Undo Tablespaces.  (This is 12.1 so I do not have the 12.2 feature of Local Undo in PDBs)

The next few posts will be on the DataGuard Broker and Redo Shipping and Applly.
(Should I also do a SWITCHOVER ?)
.
.
.


Categories: DBA Blogs

12cR1 RAC Posts -- 8a : Setting up SingleInstance DG Standby for RAC

Wed, 2017-03-08 09:08
This is the first of a small series of subposts on setting up SingleInstance DataGuard Standby for my 12cR1 RAC environment.

Primary Database : 12cR1 MultiTenant RAC : 2 nodes, database on ASM
Standby Database : 12cR1 MultiTenant SingleInstance : 1 node, database on FileSystem

The "difficulties" are :
a.  The Primary Database is MultiTenant with 1 PDB besides the SEED
b.  The Primary Database is on ASM but the Standby is to be built on FileSystem
c.  The Primary is a 2-node Cluster while the Standby is a SingleInstance installatin

This post is on the initial setup.

RAC Nodes
collabn1.racattack   192.168.78.51
collabn2.racattack   192.168.78.52

SingleInstance Node 
oem132.racattack     192.168.78.100


Database Changes :
SQL> alter database force logging;

Database altered.

SQL> alter database flashback on;

Database altered.

SQL>
SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL> alter database add standby logfile size 50M;

Database altered.

SQL>
SQL>select  group# , status, type, member from v$logfile order by 1;


Listener on Standby Server :
LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = oem132.racattack)(PORT = 1521))
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
    )
  )

SID_LIST_LISTENER=
(SID_LIST =
  (SID_DESC =
    (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1)
    (SID_NAME = STBY)
   )
  (SID_DESC =
    (GLOBAL_NAME = STBY_DGMGRL)
    (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1)
    (SID_NAME = STBY)
   )
)


tnsnames.ora on first RAC Node :
STBY =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = oem132)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = STBY)
    )
  )

STBY_DGB =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = oem132)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = STBY_DGMGRL)
    )
  )


STBY Instance Parameter File :
compatible=12.1.0.2.0
db_name=RAC
db_unique_name=STBY
enable_pluggable_database=TRUE
sga_target=800M
db_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY'
log_file_name_convert='+DATA/RAC','/u01/app/oracle/oradata/STBY','+FRA/RAC','/u01/app/oracle/fast_recovery_area/STBY'
db_create_file_dest='/u01/app/oracle/oradata'
db_recovery_file_dest='/u01/app/oracle/fast_recovery_area'
db_recovery_file_dest_size=16G
remote_login_passwordfile=EXCLUSIVE


Copying the Password File :
ASMCMD> pwcopy +DATA/RAC/PASSWORD/pwdrac.277.931824933 /var/tmp/RACPasswordFile
copying +DATA/RAC/PASSWORD/pwdrac.277.931824933 -> /var/tmp/RACPasswordFile
ASMCMD> exit
scp /var/tmp/RACPasswordFile oracle@oem132:/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwSTBY


to be continued in post 8b on the filesystem directories for the STBY instance and the execution of the DUPLICATE DATABASE command.

Categories: DBA Blogs

Pages