Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Mailing Lists -> Oracle-L -> Re: caution on Linux asmlib in DW setups

Re: caution on Linux asmlib in DW setups

From: Marc Slemko <identd_at_gmail.com>
Date: Mon, 7 Feb 2005 14:20:31 -0800
Message-ID: <9662496505020714201639e846@mail.gmail.com>


On Mon, 7 Feb 2005 02:28:04 -0800, Marc Slemko <identd_at_gmail.com> wrote:
> Just a note so anyone else thinking about this can hopefully avoid the
> same problems I ran into. I haven't found any other mentions of this.
>
> I've been working on a new 10g install on Linux. We have been using
> ASM, with Oracle's asmlib kernel module to simply disk configuration.
>
> We have a system setup with 5 10k RPM SCSI disks and were only getting
> about 60 megabytes/sec out during full table scans. Each disk by
> itself can do that, so something didn't add up. Since this system is
> a data warehouse system, this sort of throughput is somewhat
> problematic.
>
> When I dug into it, I found that the asmlib kernel module limited each
> IO to 64k! From the code:
>
> /*
> * Max I/O is 64K. Why 64? Because the MegaRAID card supports only
> * 26 scatter/gather segments. 26 * 4k page == 104k. The power of 2
> * below that is 64K. The intention is for per-device discovery of this
> * value in the future. FIXME.
> */
> #define ASM_MAX_IOSIZE (1024 * 64)
>
> Needless to say, we aren't using a megaraid, and 64k IOs are pretty limiting.
>
> When I changed this to 256k, throughput during full table scans went
> up from 60 megabytes/sec to 180 megabytes/sec.
>
> I have opened a TAR on the issue, but since we are strapped for time
> have just decided to skip asmlib since I'm not too keen on the idea of
> all my IO going through a not-yet-ready-for-primetime kernel module.
> You don't need to use asmlib to use ASM, it just makes administering
> it a bit easier.
>
> Anyway, just FYI in case you are thinking of using asmlib on Linux in
> an environment where full table scans are frequent. I woudln't expect
> it would make too much difference in the canonical OLTP environment...
> but since full table scans are increasingly becoming relevant in a lot
> of environments, you never know.

Just to keep anyone who is interested updated... I got the following response to my TAR:

  This was reported in internal bug 4223517 about 2 weeks ago. I will   update the bug to let
  development know that you are experiencing the same issue.

  Hdr: 4113517 10.1 8 1.0.3 PRODID-1309 PORTID-46   Abstract: ASMLIB FOR LINUX REPORTS A MAXIO SIZE OF 64K

For now we've just switched to not use it, it just isn't worth it and doesn't seem fully baked. Since asmlib isn't just used for discovery of devices, but every single IO operation goes through another layer, it isn't something to toy with.

Also note the patch notice Oracle has stuck up at http://www.oracle.com/technology/tech/linux/asmlib/index.html :

"It is recommended that customers using the ASMLIB for Linux download and apply the following patch from metalink to avoid some possible data corruption issues which might arise in extremely heavy workload environments. The patch number is 3800160 and it fixes a bug found on the Oracle database kernel side of the interface. The patch is currently available for x86 Linux only. It will be available on IA64 Linux and AMD64 Linux shortly and it will be included in the 10.1.0.4 patchset. "

The bug referenced by the patch doesn't seem to be public though so not sure of any details beyond that.

--
http://www.freelists.org/webpage/oracle-l
Received on Mon Feb 07 2005 - 17:23:13 CST

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US