caution on Linux asmlib in DW setups

Just a note so anyone else thinking about this can hopefully avoid the same problems I ran into. I haven't found any other mentions of this.

I've been working on a new 10g install on Linux. We have been using ASM, with Oracle's asmlib kernel module to simply disk configuration.

We have a system setup with 5 10k RPM SCSI disks and were only getting about 60 megabytes/sec out during full table scans. Each disk by itself can do that, so something didn't add up. Since this system is a data warehouse system, this sort of throughput is somewhat problematic.

When I dug into it, I found that the asmlib kernel module limited each IO to 64k! From the code:

/*

• Max I/O is 64K. Why 64? Because the MegaRAID card supports only
• 26 scatter/gather segments. 26 * 4k page == 104k. The power of 2
• below that is 64K. The intention is for per-device discovery of this
• value in the future. FIXME. */ #define ASM_MAX_IOSIZE (1024 * 64)

Needless to say, we aren't using a megaraid, and 64k IOs are pretty limiting.

When I changed this to 256k, throughput during full table scans went up from 60 megabytes/sec to 180 megabytes/sec.

I have opened a TAR on the issue, but since we are strapped for time have just decided to skip asmlib since I'm not too keen on the idea of all my IO going through a not-yet-ready-for-primetime kernel module. You don't need to use asmlib to use ASM, it just makes administering it a bit easier.

Anyway, just FYI in case you are thinking of using asmlib on Linux in an environment where full table scans are frequent. I woudln't expect it would make too much difference in the canonical OLTP environment... but since full table scans are increasingly becoming relevant in a lot of environments, you never know.

--
http://www.freelists.org/webpage/oracle-l