Re: need advice on filesystem layout

brunkow_at_btree.brooktree.com (Susan Brunkow) wrote:

>
>I am setting up a new HP K200 with a RAID array
>of 20 1 MB disks. After reading all the comments
>about lousy performance with RAID level 5,
>we've decided to use RAID level 1.
>
>We will have at least 3 production databases on the system.
>
>My boss has suggested striping the whole 20 MB into 20
>file systems (meaning that each file system would be spread over
>all of the disks.)
>I am afraid that if I do this, we will have lots of contention
>for the disks, since each file system will be on all the disks.
>
>I've heard that it's a good idea to separate the data tablespaces,
>system tablespaces, indexes, rollback segments and redo logs, since
>they are all used at once in a transaction.
>
>I am leaning toward setting up 5 files systems on small groups of disks
>maybe group 3 disks into 1 file system for the index tablespaces,
>another 2 disks into a 2MB file system for all the system tablespaces,
>another 3 disks into a file system for the redo logs,
>1 disk for the rollback segments,
>and the rest of the disks into 2 or 3 large files system for all the data
>tablespaces.
>
>Has anyone tried either of these approaches? Would one of them
>work better than the other?
>
>
>Sue Brunkow
>Brooktree Corp
>San Diego, CA
>
>brunkow_at_brooktree.com
>

Raid 5 performs fine if you set it up properly. Modern Raid systems do not overload the processor, and the cache speeds up writes. Striping IS still faster, but Raid 5 is fine for most uses. We have an 8Gig Oracle DB with 20 concurrent active users doing OLTP and get 60ms writes on Raid 5. True when we set it up on a striped system we got 8ms writes, but there was *NO* observable difference on the stopwatch at the users terminal. No difference at all.
Anyway, the array I set us is like this: There are 18 X 2Gig disks in the array. One disk is set aside for a hot spare and has nothing on it.

There are 8 swap volumes. Disk 1 has the logs (8 of 'em) for the swap mirrors. Disks 2 - 9 have a swap volume, and disks 10-17 have mirrors of the swap volumes on 2 -9. Swap is not striped, just 8 small volumes mirrored.

Disks 1-8 have the Oracle and Mincom software on them (Raid 5), a "demo" version of the database (striped only), A file system for all log files including Oracle archive log files, alert files etc (Raid 5). Then there is a striped fs for reports, export files etc (things we could do without if we lost a disk). Then there is a striped mirror for production redo logs. Thus Each disk in the 1-8 range has
Swap, Software, Oracle demo, Logging, reports, prod redo.

Disks 9 - 17 have more swap, 8 Gig production DB Raid 5, 8 Gig test DB Raid 5 and a "marketing" version on disk 17 in a broken concat volume (1.5 Gig).

Main points: the Production DB is on different disks that the redo logs, even though striping + Raid are used. The production redo logs are on striped mirror because the suppliers said so. I would have run this Raid 5 too.

The only thing I will be changing is our produciton rollback segements are in the logging directory (raid 5 on the same disks as the redo logs). I will probably shift this.

Basically any file system that is required (i.e. your app stops if this filesystem gets a problem) should be Raid 5 or some type of mirror. That's why our software and archive logs etc are on Raid 5.

The test/demo/marketing databases and the export/reports file system have no redundancy, but the system will work without these file systems.

Remember that your mirrors and Raid 5 volumes (if you try them) also require logs on different drives than your volumes. So when we got 18 x 2gig disks, we could not create 18 Gig of mirrored disk. One disk had to be set aside for a hot spare because of the physical design on the Sun hardware. One other has to be taken up with the log for the mirror. So you have to juggle smaller file systems mixed with logs around to get the most out of your dollars.

I would go with the idea of groups of disks for various bits of the system, but to fit the log files in, I would have parts of more than one file system on each disk. The more disks you can stipe over the better (provided you have some form of redundancy), but this should be balanced with the ususal ideas of separating redo logs from DB etc.

Regards, George Dau
gedau_at_mim.com.au Received on Sun Nov 05 1995 - 00:00:00 CET