ASM "disk" replacement

From: Don Granaman <>
Date: Thu, 7 May 2009 14:57:43 -0500
Message-ID: <>

We are using ASM for RAC on Linux x86-64 and will need to replace a number of disks in the storage array.  

I am wondering if anyone here has had any "interesting" experience with this sort of thing - good or bad.  

This can also be a sanity check on my embryonic "plan". I just heard about the need for this today and need to come up with a plan. We don't have to do this soon, but it absolutely must be done well before the end of the year.  

This may be complicated (or simplified) a bit by the fact that we have a sort of "minimal ASM" setup. ("New-fangled gizmos anywho - don't entirely trust 'em." - the OraSaurus in me says.)  

The basics:  

Only datafiles and tempfiles are on ASM - in a single diskgroup.

Redo is on "real" raw devices, not ASM. Archive destinations on OCFS2 filesystem. RMAN backups to NAS.

2.5+ TB database - hot, mostly with massive 24xforever inserts. (This beast generates 200+ GB of archive on a slow day.)

The six existing ASM "disks" are actually RAID-10 sets striped and mirrored in hardware - about 500 GB each.

Only "external redundancy" in ASM.

Physical (exclusive) standby also on ASM - different hardware and location, but same layout, but a different diskgroup name.

[Network link between primary and standby is rather busy - bandwidth
about 50% utilized just by log shipping.]

Notable downtime is not an option.  

In theory, this should be easy (famous last words) - alter diskgroup ... add <some new "disks"> drop <some old "disks">;  

Adding the new replacement "disks" (with similarly sized/performing hardware RAID-10 sets) and dropping the old ones in a single "alter diskgroup" command *should* minimize the ASM rebalancing.  

We would likely do this first on the standby, then on the primary once the standby "reorg" is complete.

[Hopefully, we will have a complete production mirror - RAC, standby and
all, with somewhat weaker hardware though - to test all this on first.]  

So, does anyone have any relevant experiences, horror stories, caveats, hearsay, et cetera to share?  

Thanks for any info!

Don Granaman

Received on Thu May 07 2009 - 14:57:43 CDT

Original text of this message