Storage (EMC) layout advice..

Hello,

We have recently received an EMC CX500, which is going to be dedicated exclusively for database use. The storage array is going to be attached to 3 v40Z Opteron servers, which will be running Red Hat AS 4.0
Oracle 9i
Veritas Cluster/DMP 4.1 (to be released Aug 1st) no RAC, databases on different hosts support different (but often similar) applications.

Our EMC setup involves 4x15drive shelves with 73G 15K drives (for a total of 60 drives)

If you know EMC, you might know that the first 4-6 disks in EMC array are semi-reserved for array needs (~12G of each disk is unusable). Therefore, the first 6 disks we'll have either 3x3 RAID 10, or 3x RAID1s (please advice, in the past, when using JBODs under VxVM, I've noticed that striping does slow down log writes slightly, however, 3 disks would even out IO better). We'll carve a few small LUNs out of these to present to host for redo log placement

Then we plan to use the next 8 disks in the first shelve also in RAID 10 configuration for archived log files. EMC recommends that one disables cacheing for archived logs, so that cache is not 'saturated' with unneeded data.

This leaves 1 disk left on shelve one (which we'll leave as hot spare), and then pretty much 3 empty shelves for data/indexes/system/rbs. Since EMC does not support more than 16 drives per LUN, we are thinking to use MetaLUNS in order to utilize most disks and spready the IO evenly across the rest of the drives. with about 45 disks, we were thinking of creating nine or ten 2x2 RAID 10 groups, and striping across them.. (using another 36 or 40 drives respectfully). However, this creates a stripe size which I am not that comfortable with. With default 'stripe unit size' or (stripe element size, as EMC calls it) of 64K, a stripe size of a 2x2 RAID10 LUN becomes 128K (2x64K), and an 8 x 2x2 RAID10 MetaLUN would have a stripe size of 1M, while adding 9th 2x2 LUN to MetaLUN will increment that by 128K.

So, question is, should I worry about using 1M + 128K stripe (metalun stripe) size, or even 1M + 256K metalun stripe size? Theoretically, it shouldn't matter much, provided max io size could be higher than 1M on OS, database, file system and HBA.

I am pretty sure Oracle can do more than 1M, so can Veritas File System. I couldn't verify if Linux can do more than 1M (but I think it can), and couldn't find any info on Emulex LP9802-E HBA.

Your thoughts?

BTW, the reason we thought 4 disk RAID10's striped across are good, is b/c it would be easy to either add or substruct from a MetaLUN, but we'd be happy to hear your opinions as well..

I have also yet to completely understand how "element size multiplier" affects all of this. It's not clear from the docs I've reviewed so far.

Any advice, pointers is greately appreciated.

Thanks.

.......
We use Oracle 8.1.7.4 and 9.2.0.5 on Solaris 2.7 boxes remove NSPAM to email Received on Tue Jul 26 2005 - 16:42:12 CDT