Changes in Oracle disk configuration and resulting changes in performance

From: Alan Johnson <alan.johnson_at_gel.com>
Date: 1996/11/25
Message-ID: <3299CE14.64C9@gel.com>#1/1

Folks, (NOTE: some of this is VERY WIDE!)

I've recently made (had to) some changes in the layout of my Oracle "disk farm". The specifics:

6x2.1 GB Fast SCSI disk drives, each with its own controller, used as:

4x2.1GB as raw partition data space (4 GB total, with mirroring) 2x2.1GB as raw partition index space (2GB total, with mirroring)

4x.911GB 6 mb/sec IPI-2 drives on a single IPI controller, used as:

2x.911GB as /,/usr,/oracle (containing apps, rollback, system ts,

again mirrored)
2x.911GB as /usr02 (containing archived log files)

This is a Sun 690 with 4x125Mhz HyperSparc chips, 256 MB RAM. I've changed it as follows:

8x2.1 GB Fast SCSI disk drives, 2 doubled up on a single controllers, and 1 on the "slow" 5 MB/sec controller on the MB. Configured as:

6x2.1 GB Fast SCSI disk drives, each with its own controller, used as:

4x2.1GB as raw partition data space (4 GB total, with mirroring) 2x2.1GB as raw partition index space (2GB total, with mirroring)

2x2.1GB SCSI drives, used as:

2x2.1GB containing /,/usr,/oracle,/usr02 as before, again mirrored.

All else is the same.

My reasoning regarding the 2 drive on 1 bus was due to the fact that neither drive can consume the full 10MB bus bandwidth, due to internal limitations. Similar arguement for the FAST SCSI on the 5 mb/sec system bus.
I had to do that (put it on the system bus) to get it to boot (at the time).

Tho the system is running, I'm not please with the performance. I'm getting iostats that look like:

WIDE STUFF (132 columns)

iostat -D -l 10 5

	      dk0	        dk1	          dk2	        dk3	          dk4	       
dk5	          dk6	        dk7

rps wps util rps wps util rps wps util rps wps util rps wps util rps wps util rps wps util rps wps util

   1   7 13.2	 5   0  5.7	   5   1  6.6	 4   1  5.9	   5   0  7.1	 4  
1  6.1	   1   7 10.6	 5   1  8.7
  14   3 18.6   15   0 17.4   15   1 11.7	 6   1  8.7	   9   0 13.5	 6  
1  7.3   14   3 21.4	 9   1 17.4

31 3 41.3 18 0 31.2 18 1 18.3 11 1 16.1 12 0 22.2 11 1 15.9 31 3 44.1 12 0 21.8
25 2 32.3 21 0 30.7 21 1 22.4 6 1 7.3 11 0 18.4 7 1 8.3 25 2 35.1 11 1 23.0

   5   7 25.6   19   0 23.9   19   2 20.9	 4   0  5.2   11   2 26.0	 4  
2  8.4	   5   7 20.3   11   2 23.9
   6   5 24.8   19   0 26.0   19   2 20.8	 7   2 10.4   12   0 19.9	 7  
2 10.8	   6   5 20.1   12   2 23.8
  24  10 59.6   14   0 17.9   14   1 12.1	 3   1  3.8   10   0 17.5	 4  
0  4.0   24  10 52.4   10   2 22.4
  32   2 41.4   16   0 18.5   15   1 15.3	 5   1  9.0	   9   0 16.9	 5  
1  8.0   32   2 43.8	 9   1 18.3
  33   2 45.3   17   3 28.0   17   3 20.1	 7   1  8.0   12   0 22.9	 7  
1 10.7   33   2 46.5   12   0 23.9
  25   3 30.2   18   0 20.3   18   1 16.3	 5   3 10.3   11   0 18.3	 6  
4 12.5   26   3 41.2   10   1 20.5
  29   3 42.9   18   0 20.8   18   0 17.3	 6   0  7.7   14   1 25.2	 6  
0  6.8   29   3 43.9   14   1 27.8
  28   3 44.3   19   0 21.2   19   1 18.9   10   2 13.3	   8   0 14.1  
10   2 18.9   28   3 43.5	 8   1 17.5
  25   8 61.1   13   0 15.3   13   2 12.5	 6   1  7.5	   6   0 11.3	 6  
1 10.5   22   8 48.8	 6   2 14.5
  30   4 40.6   14   0 17.2   14   1 16.2	 6   1  9.7   10   0 19.8	 7

1 9.3 29 4 48.9 10 1 22.2

The slow drive is dk0. As you can see, it looks overutilized. The mirror of the slow drive is dk6. It seems to be having the same troubles. My questions:

Are my diagnoses correct?
What a good rule of thumb WRT ios for a Fast SCSI drive
Why were the IPIs so much better (they crash lots and are getting hard to find, hence the change)
Any simple ideas on how to improve this?

TIA Alan Johnson Received on Mon Nov 25 1996 - 00:00:00 CST