Fuad Arshad

Subscribe to Fuad Arshad feed Fuad Arshad
This is Just stuff i find on Oracle From various different sources or from my own personal experience I can also be found at http://www.twitter.com/fuadar
Updated: 17 hours 25 min ago

Collaborate 2013 - Lots Of learning Lots of Fun

Tue, 2013-03-26 15:56
Collaborate 2013 is right around the corner and that is the place to be this April in Denver.
 There are a lot of Learning and a lot of social Activities. I'm goign to be down there presenting an Paper on Oracle Database Appliance http://bit.ly/YFmPpu .
I will be talking about how to deploy the Database Appliance and how it has changed how i do my day to day work. If you are interested in listening to me talk about Oracle Database Appliance , How it works , how easy it is to deploy etc . Please  join me at http://bit.ly/YFmPpu
There will be a lot of Tracks and Sessions and even time with a lot of influential People.
Below is a sneak peak of what i will be talking about. So if you feel that the Topic interests you. Please Come and join me or just come by to say Hi.

Moving Datafile from Physical Standby to Primary via Rman

Tue, 2013-03-26 10:08
Rman is an interesting tool and it seems everyday you learn something new. As part of a production issue we lost a diskgroup . This Diskgroup was only a small subset of the actual data but due to an issue in the underlying disk (External Redundancy). The database crashed. As we were debugging the issue with our vendor . We decided to use Rman and our Physical Standby to bring back the datafiles to help mitigate the outage. Now we could have failed over to our standby but since  our Standby hardware was not sized to handle production workload that was a risk the business was not willing to take.
As Oracle 11.1 you can use file from a standby(physical) database and move them to the primary.
We went thru to the Standby and determined which datafiles were in the particular group and then connected to primary from the standby as an auxiliary and copied the files over the network to the different diskgroup.

connect sys@standby AUXILIARY sys@primary;
Once the Files are copied you can switch the datafiles .

This will allow for a move of the datafile as well as rename on the primary.
you will then  have do a crash recovery on the datafile(s) in question.

recover database 2;
alter database datafile 2 online;
you have just copied a file over the network and plugged it into your primary from standby. This is a pretty convenient if accidents happens.

Changing Enterprise Manager 12c Default inactive Timeout

Tue, 2013-02-26 14:27
Oracle Enterprise defaults to a timeout of 45 minutes . Depending on the organization or security policies you might want to change that to a lesser amount of time or in my case a longer period .
To check what the timeout is set
em@emap1>./emctl get property -name oracle.sysman.eml.maxInactiveTime
Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
SYSMAN password:
Value for property oracle.sysman.eml.maxInactiveTime for oms All Management Servers is null null = default of 45 minutes
After you have checked the value . you can change it
em@emap1>./emctl set property -name oracle.sysman.eml.maxInactiveTime -value 90
Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
SYSMAN password:
Property oracle.sysman.eml.maxInactiveTime has been set to value 90 for all Management Servers
OMS restart is required to reflect the new property value
This will set the inactive time to 90 minutes . you will need an OMS bounce for this to take effect. Please ensure that whatever value you set meets your requirements and the security requirements of your company

Deploying a Virtualized Oracle Database Appliance ODA 2.5

Mon, 2013-02-11 17:12

So I finally got the opportunity  to deploy ODA 2.5 on a development Oracle Database Appliance. The Documentation  is very lacking and needs more refinement for the masses.
Here are the steps to deploy ODA 2.4 by reimaging the ODA . Concepts and procedures for bare maetaling the box remain the same.
    1. Use the ILOM to connect to the box via remote control and mount the VM Image  ( Need to do this on both hosts individually)
    2. After the reboot . The imaging Process will start and should takes between 1-2 hrs (Took about 2 hrs 30 minutes for me)
    3. Once you get the Oracle VM Server 3.1.1 screen. Your box has been imaged with a dom0 image.
    4. If you are using the ILOM you can ALT-F2 to get a login prompt
    5. Login as root
    6. Ensure that both boxes have been reimaged before starting the next step
    7. Run /opt/oracle/oak/bin/oakcli configure firstnet  ( on the first node)
    8. You have 2 options  (Local or Global)
    9. Global should be selectedi f both nodes are ready ot be ip'ed
    10. Select the network net1 , net2, net3, net4)
    11. Please note it is a little confusing but Here is a break down
        a.  priv1=bond0 (Interconnect)
        b.  Net1=bond1
        c. Net2=bond2
        d. Net3=bond3
        d. Net4=xbond0
    12. Oracle Failed to mention this but on startup does provide the MAC Addresses as well as ethernet names and bond info so be careful and ensure that  you understand your network topology prior to installing.
    13. You do want to make sure you have a DNS entry and a new IP Address for the DOM0 for each Server Node (2 x Dom0)
    14. Needless to say the network should be same on both nodes e.g public should be cabled on net1 on both nodes for consistency
    15. The network config will configure the public on both nodes  for Dom0 only
    16. After the config scp patch 16186172 into /OVS on the Dom0 box 0
    17. Unzip the patch 1 & 2 files and cat them together  cat a b >templates.
    18. Deploy the oda_base
 [root@odadb1-dom0 bin]# ./oakcli deploy oda_base
Enter the template location: /OVS/templateBuild-2013-01-15-08-53.tar.gz
Core Licensing Options:
1. 2 CPU Cores
2. 4 CPU Cores
3. 6 CPU Cores
4. 8 CPU Cores
5. 10 CPU Cores
6. 12 CPU Cores
Selection[1 : 6] : 5
ODA base domain memory in GB(min 8, max 88)[default 80] :
INFO: Using default memory size i.e. 80 GB
INFO: Node 0
INFO: Deployment in non local mode
INFO: Running the command to copy the template /OVS/templateBuild-2013-01-15-08- 53.tar.gz to remote node 1
templateBuild-2013-01-15-08-53.tar.gz 100% 4620MB 47.6MB/s 01:37
INFO: Node 0
INFO: Spawned the process 26679 in the deployment node 0
INFO: Trying to setup on deployment node 0
INFO: Spawned the process 26680 in the node 1
INFO: Trying to setup on node 1
Using config file "/OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/vm.cfg" .
Started domain oakDom1 (id=1)
INFO: Deployment in local mode
INFO: Node 1
INFO: Extracted the image files on node 1
INFO: Node 1
INFO: The VM Configuration data is written to /OVS/Repositories/odabaseRepo/Virt ualMachines/oakDom1/vm.cfg file
INFO: Running /sbin/losetup /dev/loop0 /OVS/Repositories/odabaseRepo/VirtualMach ines/oakDom1/System.img command to mount the image file
INFO: Mount is successfully completed on /dev/loop0
INFO: Making change to the /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1 /tmpmnt/boot/grub/grub.conf file
INFO: Node 1
INFO: Node 1
INFO: Assigning IP to the second node...
INFO: Node 1
INFO: Created oda base pool
INFO: Starting ODA Base...
Using config file "/OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/vm.cfg" .
Started domain oakDom1 (id=1)
INFO: Deployment in local mode
INFO: Node 0
INFO: Extracted the image files on node 0
INFO: Node 0
INFO: The VM Configuration data is written to /OVS/Repositories/odabaseRepo/Virt ualMachines/oakDom1/vm.cfg file
INFO: Running /sbin/losetup /dev/loop0 /OVS/Repositories/odabaseRepo/VirtualMach ines/oakDom1/System.img command to mount the image file
INFO: Mount is successfully completed on /dev/loop0
INFO: Making change to the /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1 /tmpmnt/boot/grub/grub.conf file
INFO: Node 0
INFO: Node 0
INFO: Assigning IP to the first node...
INFO: Node 0
INFO: Created oda base poo1
INFO: Starting ODA Base…
19. Once oda_base is deployed
 [root@podadb31-dom0 bin]# ./oakcli show oda_base
ODA base domain
ODA base CPU cores :10
ODA base domain memory :80
ODA base template :/OVS/templateBuild-2013-01-15-08-53.tar.g
20.Once the oda_base is installed you will have to vnc in using the dom0 port 5900 to get access to the database server ( Due to a bug you will need to vnc in on both servers first and press the press any key to continue).
21. Once logged in you will need to IP the oda_base
22.You can either use the /opt/oracle/oak/bin/oakcli configure firstnet ( Please note it detects VM Environment and gives eth1,eth2,eth3 and eth4 as options )
23. Better to use  ./oakcli deploy
24. Oakcli now has option to change the proxy port for ASR as wella s configure external ASR server
25. External ASR server needs a server name as well as port (no definition of what the port is supposed to be )
26.  Also due to a bug if vm manager is bounced   you will have to vnc in and hit "press any key to continue". you can see that below
 The deployment process has not changed and will follow the same deployment steps.
This is the first release of ODA on a virtualized platform and glitches are to be expected. but it does seems to have been rushed out .
Please feel free to comment or ask questions. I have only deployed a DOm0 and the ODA_base here but i will deploy an app shortly and post my experience

Update: Edited steps and changed the network names

Oracle Database Appliance 2.5 And Virtualization (Updated)

Fri, 2013-01-25 12:21
So the cats out of the bag and Oracle has finally announced that Jan 31st ODA 2.5 will be available. The ODA 2.5 is a pretty significant update to the ODA Platform. it will have 1. Virtualization as an option with OVM 3.1.1 2. Official External ASR management. So The virtualization is huge since it allows for using the capacity of the ODA for things other than the Database. The concept is a DOM0 and an ODA_BASE which is a privileged domain that ensures that The Database Appliance performs optimally. Intially the DOMU or user Domain will not have access to ASM Disks or ACFS mounts but that is something that will show up in a future release . The User Domain will not be patched as part of the Single Patching technique which mean if you go with virtualization the user domain needs to be patched independently. Also initially only 250GB extra slice is the only space available for the DOM0 and DOMU areas. This can bring in a little more complexity but from a DR perspective it helps in building idle capacity with a smaller footprint . The question that is important is should ODA deployments start with a Virtual deployment even though there might be no requirement for it initially. Oracle will ship the physical deployment and customers can bare metal it to be virtualized. Oracle Claims to be able to provide virtualization with ease of use but only time will tell. I am pretty excited about this since i can build virtualized boxes and do DR at a cheaper cost but the same horse power that i need. I do intend to test this and see how the ease of use to determine if that is my goign in stance for all new ODA deployments . Update: The Database Appliance has 500GB of Local Disk space which means DOM0 and DOMU will have access to 250GB and ODA_BASE will have 250GB available to it.

Reinstalling an Oracle Database Appliance Part 3

Fri, 2012-11-23 10:47
An Oracle Database appliance is a set of standard hardware and software molded together to created a 2 node RAC cluster that is easy to deploy. In parts 1 & 2 i talked about what methods you can use to reformat or erase the Oracle Database Appliance and rebuild it. In this post  i will talk about 2 methods to complete a deployment on the Oracle Database Appliance (ODA).
You can use the GUI tool oakcli deploy that will walk you thru all the steps and let you add all the IP's and then do the deploy.
You can use the Oracle Appliance Kit offline configurator to generate the conf file and then either run  oakcli deploy to quickly validate all the information and runa graphical deploy.
The Oracle Appliance Kit Offline Configurator is available at the Oracle Technology Network ODA Page .  The Offline configurator is an exact replica of the oakcli deploy tool except it can be done on your laptop so you dont have to type all the info on the server. The Offline Configurator allows to save a text file in the below format.
#onecommand.params generated by GUI tool
# Confuration information


DbNodeArr=(ALL )

This text file contains a complete snapshot of the ODA being Built. Below is how the ODA configurator looks like.

Once the file is loaded you can go thru a standard GUI deployment.
The Second method which is very useful to deploy an ODA is the silent method. you can use the text file generated from the ODA offline configurator and copy the config file

./oakcli copy -conf /software/test11_12.txt
Successfully copied the config file!!!

Once the config file has been copied you list what the GridInst.pl steps are

[root@oak1 onecmd]# cd /opt/oracle/oak/onecmd
[root@oak1 onecmd]# ./GridInst.pl -l
INFO : Logging all actions in /dev/null and traces in /dev/null
INFO : Loading configuration file /opt/oracle/oak/onecmd/onecommand.params...
The steps in order are...
Step 0 = ValidateParamFile
Step 1 = SetupNetwork
Step 2 = WriteNodelists
Step 3 = SetupSSHroot
Step 4 = SetupDNS
Step 5 = UpdateEtcHosts
Step 6 = SetTimezone
Step 7 = SetupNTP
Step 8 = SetupILOM
Step 9 = ValidateEnv
Step 10 = CreateUsers
Step 11 = SetupStorage
Step 12 = SetupSSHusers
Step 13 = InstallGIClone
Step 14 = RunGIClonePl
Step 15 = RunRootScripts
Step 16 = GIConfigAssists
Step 17 = CreateASMDiskgroups
Step 18 = InstallDBClone
Step 19 = RunDBClonePl
Step 20 = DbcaDB
Step 21 = SetupEMDbControl
Step 22 = SetupACFS
Step 23 = SetupTFA
Step 24 = SetupASR
Step 25 = ResecureMachine
[root@oak1 onecmd]#
It is very important to validate how many steps are there since Oracle can change the number between the versions. The above is a snapshot of ODA 2.4 Steps. you can decide to break down the number of steps run this in debug mode in which case you will have interaction on some steps. or just execute GridInst.pl via a shell script
[root@oak1 onecmd]# ./GridInst.pl -r 1-25

This will execute the ODA deploy in non GUI mode with all logging happening as normal.
This concludes this 3 part series on how to reinstall an Oracle Database Appliance. Please feel free to ask any questions regarding the ODA and i will try to the best of my knowledge to answer them

Reinstalling an Oracle Database Appliance part 1Reinstalling an Oracle Database Appliance Part 2

Restoring a Database with Rman Controlfile Parameters

Tue, 2012-11-20 19:08
There are occasions when you recieve a complete backup of a database from a  place different then yours and have to do a restore. normally a restore is simple . put the files in a directory run rman catalog on them and you are done. but what if you have rman configuration parameters stored in the controlfile and you are trying to duplicate the database. During a duplicate rman will restore the controlfile and then use the parameters associated in the controlfile for the next steps.
I hit this issue when the db i was trying to duplicate  has sbt_tape parameter and the host i was trying to duplicate on  has no MML configured to it.
After trying a couple of times with no luck. I opened an SR and got a response from a very helpful Oracle Support guy (even on Sev 3).
Below are the procedures i followed to solve my rather simple problem.
1. Startup the auxiliary database in nomount
2. Restore the controlfile from the autobackup piece
RESTORE CONTROLFILE FROM '.... location and name of the autobackup piece ...';
3. Mount the database:
4. Clear the SBT related configurations, CHANNEL and DEVICE - Do a show all and make sure there are no parameters that you might have an issue with
See the RMAN reference guide under CONFIGURE for details on how to do this.
5. In SQLPLUS backup the controlfile:
ALTER DATABASE BACKUP CONTROLFILE TO '/location/of/backups/control.dbf';
6. Startup the auxiliary database in nomount
7. Move the controlfile autoback piece out of the backup location
8. Run the Duplicate, which now will use the controlfile copy we created with correct RMAN configuration.

I was unaware of doing something this way and i guess its good to learn something new everyday.


Thu, 2012-11-15 09:03
To install an new EM12c environment it is highly recommended you run the emprereqkit  utility . This utility is accessible thru the installation binaries  in the $SHARED_SOFT_PATH/ directory. if you are running a prereq check via a shared mount point . you might hit an error like the one below. This error happens if you ran an emprereqkit   on another environment and never completed all the tasks.

Following exception occured in EMPrereqKit
Calling sequence (top to bottom)
--level 1--------------------------------------
Class/Method: EMPrereqKitManager/getInitialState
Message     :  Please end the session in last db (runPrereqs, runCorrectiveActions, runPostCorrectiveActions). Then only you can start a new session at a new db

         at oracle.sysman.emdrep.emprereqkit.emprerequisitemanager.EMPrereqKitManager.getInitialState(EMPrereqKitManager.java:353)
         at oracle.sysman.emdrep.emprereqkit.emprerequisitemanager.EMPrereqKitManager.processActionsList(EMPrereqKitManager.java:330)
         at oracle.sysman.emdrep.emprereqkit.emprerequisitemanager.EMPrereqKitManager.executeMainOperation(EMPrereqKitManager.java:171)
         at oracle.sysman.emSDK.emdrep.emprereqkit.EMPrerequisiteKit.executeMainOperation(EMPrerequisiteKit.java:83)
         at oracle.sysman.emdrep.emprereqkit.core.EMPrereqKit.main(EMPrereqKit.java:72)

 Caused by: null

empreqkit creates a state file that keep tab on the state of the job.
the Statefile is created in $SHARED_SOFT_PATH/
a cat of the statefile will show something like this

If this state file is there it will not  allow you to run a seperate session until either the old session on the old database server  is completed or the statefile is removed or renamed.

Reinstalling an Oracle Database Appliance Part 2

Tue, 2012-11-06 08:33
This is the second post in the series for Reinstalling the Oracle Database Appliance. In the last post i discussed using Bare matal install to reimage the ODA and perform a new deployment. In this post i will talk about  cleanupDeploy.pl

Cleanupdeploy.pl is a utility provided by Oracle as Part of the Oracle Appliance Kit (OAK). This utility performs the following

The Cleanup Steps are the following:

    1. uninstall ASR
    2. de-install All Oracle Software except the Oracle applicance Kit and the OS
    3. Cleanup Disk
    4. drop all users
    5. rollback various system configs
    6. remove public and private IP info
The CleanupDeploy.pl script is available in /opt/oracle/oak/onecmd directory and will  ask you to confirm 
[root@oak1 ]# ./cleanupDeploy.pl

Please enter the root password for performing cleanup:
Re-enter root password:
About to clear up OAK deployment,public network connectivity will be lost,root password will be set to default and both nodes will be rebooted
Do you want to continue(yes/no):
This will cleanup all the software on both the nodes. Please note while the OAK software will remain. you will lose access to the box since the public IP will be wiped out. you will need to access the ILOM and remote console in to reconfig the IP's on both nodes 

./oakcli configure firstnet
This will allow you to either further patch the system or run a deployment .
The difference between the CleanupDeploy.pl vs Baremetal can be attributed to 2 main things and while both must be used with caution and provide loss of data. Cleanupdeploy will not format the root filesystems so anything not on shared disk is preserved but will keep the old version of the OAK software in tact. Bare Metal restore on the other hand will wipe out everything but will also give you the software version you need. For new deployments if your intention is to go to 2.4 and your ODA is running version 2.1 my recommendation would be to do a Bare metal prior to deployment. The CleanupDeploy.pl will help in redploy an ODA for cases like moving from HIGH redundancy to NORMAL redundancy if you have not yet deployed an ODA or of you want the additional space and have something like Dataguard to failover you database . rebuild the ODA and fail it back.

In the 3rd Part of this series i will walk thru the deployment process and how to do deployment manually or via the OAK Configuration Assistant.

Reinstalling an Oracle Database Appliance part 1Reinstalling an Oracle Database Appliance Part 3

Reinstalling an Oracle Database Appliance Part 1

Sun, 2012-11-04 15:01
Oracle Database Appliance comes with factory installed software. If you need to reinstall or reimage the ODA you can do that using a couple of methods. I'm going to have  series of posts talking about reinstalling and redeploying an ODA using various methods
A reinstall needs to start with cleaning up
There are 2 methods for cleanup..
  1. Using a Bare Metal Install
  2. Using cleandeploy.pl
A Bare metal Install is done by downloading an ISO from the MOS. The latest 2.4 ISO can be downloaded Here.
While there is a complete document available via Note. 1373599.1. 
I will give a brief synopsis of what is needed. A Bare install needs to be done per node and the instructions are the same. Please note a bare metal install will format all disks. All data will be lost. All instructions here will cause loss of data and should be done with extreme caution.
  1. . Download the 2.4(or version you need) ISO.
  2.  Login to the ILOM 
  3.  Go to the Remote Control tab and launch remote control.
  4.  In the java window that opens Select Devices --> Cdrom Image
  5. Select where the image resides on your laptop and click ok. This will map the cdrom image to your device
  6. Once the mapping is confirmed . go to the ILOM Remote control tab - >Host Control  and Select Cdrom as the next boot device
  7. In the Remote Control -->Remote Host Control  Select Power Cycle and OK
  8. This will reboot the node and start the install process. 
  9. If you are installing over a network. Please make sure not to disconnect until the install is completed.
  10. The reimage process can take between 1 to 2 hrs  to complete.
  11. After the reimage is complete.  the node is now akin to a factory installed node for the version you just imaged on.
  12. Follow the same steps to image the second node . Both nodes can be imaged in parallel.
  13. The ILOM and Bios are not impacted and a deploy/patch will need to be run to get the ILOM/BIOS to update.
  14. Run a ./oakcli configure firstnet to IP and bring the node on the public network
  15. Follow the steps in the read-me of the End user bundle to deploy the ODA.
 In the next post I'll talk about cleandploy.pl and how you can run a cleanupdeploy to reinstall the ODA.

    Reinstalling an Oracle Database Appliance part 2Reinstalling an Oracle Database Appliance Part 3

    Oracle Database Appliance (ODA) Patch 2.4 is out

    Fri, 2012-11-02 21:22
    This is just a post ot point out that ODA Patch 2.4 is out. ODA Patch 2.4 requires 2.2 or above to install.
    A couple of Highlights of this release seem to be
    1. Enhanced odachk 2.16
    2. A trace file analyzer 
    3. The ability to now use normal redundancy for DATA and RECO diskgroups
    The ability to double mirror instead of triple mirror move the ODA's capacity from 4Tb to 6TB and is a choice in the offline configurator
    if you have an old deploy file you can load it in the 2.4 Offline configurator by changing
    I will play around with the trace file analyzer to see what it brings to the table.