Re: Excessive IOPS to shared ORACLE_HOME

Chen
  I have seen similar issue in Solaris platform. Can you see any difference as to how Oracle_home file system is mounted between these servers ? It's a common fallacy that Oracle home binaries are never accessed once the database is up and running. In reality, Oracle binaries are accessed constantly and if the file system holding oracle_home is mounted with mount options bypassing UNIX buffer cache, then IOPS to underlying file system increases sharply. So, you might want to check all your DB nodes and make sure Oracle home file system is using unix buffer cache.
  There are other reasons too. Enormous amount of audit files, logging in listener log, CRS spitting out log entries etc. Check that out too.

Cheers
Riyaj Shamsudeen
The Pythian Group - http://www.pythian.com Personal blog: http://orainternals.wordpress.com  

Chen Shapira wrote:
> Hi Oracle Fans,
>
> I need some help and advice on tracking down source of excessive IO.
>
> Our 10.2.0.[2,3] RAC enviroments are configured with shared
> ORACLE_HOME, residing on Netapp storage. The same volume also contains
> the voting disk and OCR for the RAC.
> This volume, that we assumed will have almost no load, actually has
> about 6000 IOPS (IO operations per second), sometimes up to 13000. For
> comparison, our data volume has about 1500 IOPS.
>
> Interesting findings:
> 1) From Netapp reports and from some network sniffing, it appears that
> over 90% of the IO is "access" and "get attribute" operations -
> neither reads, nor writes.
> 2) This does not happen to the voting disks on RAC systems that have
> no shared home. (although there are other configuration differences as
> well).
> 3) This also does not happen in stand alone systems that have
> ORACLE_HOME on the Netapp.
>
> I can't use IOSTAT to debug because it does not report NFS. Network
> monitoring aggregates over all NFS traffic, so I can't seperate normal
> data traffic from strange ORACLE_HOME traffic (it is all on same
> storage interface). Linux does not report network activity per process
> (not without special patches). Wait event tables will not show traffic
> to Oracle_home or to voting disk.
>
> Any suggestions where to look?
> Anyone can guess what we could have configured wrong to cause such an
> issue? ( I know that we are all against random guessing, but it feels
> like all investigation channels are insufficient).
>
> Thanks,
> Chen Shapira
> --
> http://www.freelists.org/webpage/oracle-l
>
>
>
>

--
http://www.freelists.org/webpage/oracle-l