Re: IO Latency

From: Alex Gorbachev <ag_at_oracloid.com>
Date: Tue, 22 Apr 2008 21:58:12 -0400
Message-Id: <76C453D3-80A2-4D83-8AB6-9A99A1532B88@oracloid.com>


Stefan,

As a member of the BAARF party I would guess that it's RAID5 there. However, the BAAG membership doesn't let me do it this way.

As a compromise, let me ask you what RAID level this baby are configured with? I know that they support RAID 5 & 10.

Alex

On 17-Apr-08, at 5:54 AM, Stefan Knecht wrote:

> Thanks for the feedback everyone!
>
> However, it's not quite *as* simple :-)
>
> The IO waits that I posted were just the peaks occurring in a larger
> batch.
>
> Some more facts:
>
> - IO response seems very random in nature (You repeat the same
> process 5 times, 3 times it runs dead slow (60-100 rows per second),
> twice it runs lighting fast (3000 rows per second), overall system
> load was more or less constant during the tests)
> - Out of a typical nightly batch, where 500'000 sequential IOs are
> performed, only about 5% of the WAITs are in the range of +100ms
> (and up to 4seconds). They however, account to more than 50% of the
> total WAIT caused by IO on the batch
> - CPUs are mostly idle, the system is not at all suffering from CPU
> contention (got 12 CPUs in this LPAR)
> - IBM has performed measurements on the storage subsystem directly.
> But they've only looked at averages (over 15min periods). Taking the
> batch from above, where the majority of IOs is very fast, and a few
> are slow, the average ends up looking rather fast too, due to the
> high number of fast IOs. Yet total response time is very slow.
>
> What really gets me confused is how an 8k IO can even take up to 4
> seconds. Even if the disks are busy and suffering contention, there
> has still got to be something else causing such enormous latency.
>
> We have also run ORION to measure the IO performance, and it's not
> excellent, but it's also nowhere near as disastrous (120MB/s
> throughput with 20 concurrent large IOs, 40ms latency with 30
> concurrent small random IOs). That looks "OK" to me.
>
> Cheers
>
> Stefan
>
> --
> =========================
>
> Stefan P Knecht
> Senior Consultant
> Infrastructure Managed Services
>
> Trivadis AG
> Europa-Strasse 5
> CH-8152 Glattbrugg
>
> Phone +41-44-808 70 20
> Fax +41-808 70 12
> Mobile +41-79-571 36 27
> stefan.knecht_at_trivadis.com
> http://www.trivadis.com
>
> OCP 9i/10g SCSA SCNA
> =========================

--
http://www.freelists.org/webpage/oracle-l
Received on Tue Apr 22 2008 - 20:58:12 CDT

Original text of this message