Queue depth on QLogic/3par

From: Jeremy Schneider <jeremy.schneider_at_ardentperf.com>
Date: Wed, 30 Jan 2008 15:53:17 -0500
Message-ID: <611ad3510801301253s34f3127fq2110a190801c700@mail.gmail.com>

Just wondering if anyone has done much with tweaking the queue depth on the HBA. I'm running some benchmarks with Oracle's Orion tool (Redhat5/Opteron/QLogic/3par) and noticed that with the small random I/O tests the bottleneck really seems to be the queue depth - even with hundreds of threads iostat reports sub-millisecond service time yet shows long queues and long wait times. Since this database being migrated to this new system is like 95% "sequential read" from the wait events I figure that there will be a lot of these small I/O requests and it seems worth tuning.

There seems to be a option "ql2xmaxqdepth" on the qlogic driver that controls the queue depth. It defaults to 32 but I saw some chatter on the VMware forums about increasing it to 64. But I also saw a note for Solaris/Symmetrix saying something about using a max queue depth less than or equal to the max queue depth of the LUN (perhaps configured on the SAN?) to avoid commands getting rejected by a full queue condition.

I'm not sure exactly how all of this works internally - can different HBA's or SAN's only handle a certain max queue depth? Has anyone else experimented with different queue depths? Does this change something on the HBA or just in kernel-land? I think I'm going to try increasing it and re-run the Orion benchmark to see if I get any errors or performance difference - but I'm curious what others' experience has been.


Jeremy Schneider
Chicago, IL

Received on Wed Jan 30 2008 - 14:53:17 CST

Original text of this message