Distribute RMAN channels across RAC nodes

From: <avramil_at_avramil.cnc.net>
Date: Wed, 05 Nov 2014 15:13:07 -0500
Message-ID: <20141105151307.flj5oq6rcw4soss4_at_hostingemail.xo.com>

    Hello,

We have a 2-node 11.2.0.2 RAC on Solaris 10.  This application started out relatively small, but recently has grown into a 2-TB database. Our RMAN backup times have increased with the growth of the database.  Full Backups over a dedicated backup network (BAN) to a deduplication server now take over 7 hours.

We've discovered that our backups are being bottlenecked by a local network switch that can onlyhandle 1-Gbit of traffic. We don't have the option of upgrading the network switch.  There is a plan in place to upgrade the application and migrate all of it to a more robust environment, but that will take several months before it can be implemented.

In the meantime, we thought that we might be able to optimize the backup process through the BAN by distributing the RMAn channels among the nodes of the RAC.
We have been able to do this on Linux servers and it works.  But on Solaris 10 RACs, running both 11.2.0.2 and 11.2.0.4, this fails.

We are using the EMC software DD Boost to write our RMAN backups to our backup deduplication environment.  We have 200 database backups running via DD Boost, so we don't think it is that.

For example, trying to connect to the database and load the DD Boost library locally on Solaris 10 works fine:

$ cat test.rman
connect  target /
run {
ALLOCATE CHANNEL c1 TYPE SBT_TAPE PARMS
'SBT_LIBRARY=/u01/app/oracle/product/11.2.0/db/lib/libddobk.so,
ENV=(STORAGE_UNIT=USERID,BACKUP_HOST=DEDUPSERVER,ORACLE_HOME=/u01/app/oracle/product/11.2.0/db)'; }

$ rman _at_test.rman

Recovery Manager: Release 11.2.0.4.0 - Production on Tue Oct 28 13:04:13 2014 Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

RMAN> connect  target *
2> run {
3> ALLOCATE CHANNEL c1 TYPE SBT_TAPE PARMS
'SBT_LIBRARY=/u01/app/oracle/product/11.2.0/db/lib/libddobk.so,
4>
ENV=(STORAGE_UNIT=username,BACKUP_HOST=dedupehost,ORACLE_HOME=/u01/app/oracle/product/11.2.0/db)'; 5> }
6>

connected to target database: ORDB (DBID=123456789) using target database control file instead of recovery catalog allocated channel: c1
channel c1: SID=388 instance=ORDB device type=SBT_TAPE channel c1: Data Domain Boost API
released channel: c1  

Recovery Manager complete.

But when trying to make a connection to another node, it fails:

$ cat test2.rman

connect  target  sys/pass_at_node2
run {
ALLOCATE CHANNEL c1 TYPE SBT_TAPE PARMS
'SBT_LIBRARY=/u01/app/oracle/product/11.2.0/db/lib/libddobk.so,
ENV=(STORAGE_UNIT=username,BACKUP_HOST=deduphost,ORACLE_HOME=/u01/app/oracle/product/11.2.0/db)'; }

$ rman _at_test2.rman

Recovery Manager: Release 11.2.0.4.0 - Production on Tue Oct 28 13:44:55 2014

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

RMAN> connect  target *
2> run {
3> ALLOCATE CHANNEL c1 TYPE SBT_TAPE PARMS
'SBT_LIBRARY=/u01/app/oracle/product/11.2.0/db/lib/libddobk.so,
4>
ENV=(STORAGE_UNIT=oracle,BACKUP_HOST=dddsnq08,ORACLE_HOME=/u01/app/oracle/product/11.2.0/db)'; 5> }
6>
connected to target database: ORDB (DBID=123456789)

using target database control file instead of recovery catalog

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================

RMAN-03009: failure of allocate command on c1 channel at 10/28/2014 13:44:57 ORA-19554: error allocating device, device type: SBT_TAPE, device name: ORA-27211: Failed to load Media Management Library Additional information: 3986

Recovery Manager complete.

 It works on Linux, but not on Solaris 10. If anyone has any insights, they would be greatly appreciated.

Thanks,
Lou Avrami   

--
http://www.freelists.org/webpage/oracle-l

Received on Wed Nov 05 2014 - 21:13:07 CET