RE: Using SAN or local drives?

From: Kevin Closson <kevinc_at_polyserve.com>
Date: Tue, 3 Oct 2006 11:15:24 -0700
Message-ID: <5D2570CAFC98974F9B6A759D1C74BAD0E5B2B1@ex2.ms.polyserve.com>

>>>Now we do simple math:
>>>792 sectors x 512 bytes = 396Kbytes/track A single track
>>>read plus move heads to the track nextdoor is 4ms (single

.
.
.

>>>Amount of data read per second = 396Kbytes x 227 reads = 87.8Mbytes/s

 I don't argue your simple math. I argue what it has to do with anything. Measurements are important too. After all, there is a track buffer. Why stop short with 87.8MB/s. If I let all the planets align, I
can get nearly 180MB/s from a single 15K RPM spindle. The following spam shows a dd_direct binary (linux coreutils 5.2.1 with open(,O_DIRECT,)) executed within a script with each invocation reading only 2MB (fits in the track buffer). All told the dd processes read an aggregate of 15.62GB in 89.8 seconds, or 178MB/s.

If I change that to random, async 4KB I/O in a 4GB file, the same drive tops
out at what one would expect--about ~1,700 IOps or ~6.6MB/s . Worse yet, throw an
occasional write in there, the number drops to more what one would expect for
real life where Oracle is concerned--~250 IOps.

[root_at_tmr6s14 ~]# iostat -x 5 | egrep '^Dev|c0d0' Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 79.04 4.99 2881.44 1.40 359166.47 51.10 179583.23 25.55 124.61 123.79 42.88 0.35 99.64 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 79.52 3.41 2848.19 1.20 358811.24 36.95 179405.62 18.47 125.94 124.59 43.68 0.35 100.26 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 98.80 2.79 2911.18 1.40 354414.37 33.53 177207.19 16.77 121.70 122.93 42.21 0.34 99.62 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 79.20 3.00 2878.80 1.40 359884.80 35.20 179942.40 17.60 124.96 124.33 43.19 0.35 99.88

[root_at_tmr6s14 ~]# time sh ./z

real 1m29.743s
user 0m2.930s
sys 0m20.031s
[root_at_tmr6s14 ~]# cat z

xcnt=1
until [ $xcnt -eq 80 ]
do

        ycnt=1
        while [ $ycnt -lt 100 ]
        do
                dd_direct if=foo of=/dev/null bs=128k count=16 2>
/dev/null &
                (( ycnt = $ycnt + 1 ))
        done

        wait

(( xcnt = $xcnt + 1 ))
done

--
http://www.freelists.org/webpage/oracle-l

Received on Tue Oct 03 2006 - 13:15:24 CDT