Re: 11g: 30K of open file descriptors per oracle process to oraus.msb

> Hi,
>
> your feedback/experience sharing is much appreciated. Platform is oracle
> database 11.2 on linux x86_64.
> SR to oracle support has been made but no feedback yet that helps me any
> further.
>
> Last weekend when doing tests where packaged plsql stored procedures were
> being called many times by java code running on jboss side we saw following
> errormessage in the server.log file:
> ...java.sql.SQLException: ORA-00600: internal error code, arguments:
> [pesldl01_Get_Object: shm_open: errno 24 errmsg Too many open fil], [], [],
> [], [], [], [], [], [], [], [], []
>
> then when we checked on the oracle (rac) serverside we noticed that lsof
> was reporting more than 120000 lines pointing to oraus.msb like following:
>
> COMMAND PID USER FD TYPE DEVICE SIZE
> NODE NAME
> oracle *2780* oracle 27r REG 253,24 1074176
> 9404528 /opt/11.2.0/db_1/rdbms/mesg/oraus.msb
> ..repeated 30000 times
> oracle *4976* oracle 10r REG 253,24 1074176
> 9404528 /opt/11.2.0/db_1/rdbms/mesg/oraus.msb
> ..repeated 30000 times
> so each oracle shadow process was holding 30K of open filedescriptors to
> the same file if I understand correctly, which looks insane to me.
>
> I am trying to find the trigger that causes this behaviour leading to
> outages, which is not in line with oracle's high availability motto.
>
> Regards,
> Andre
>
>
>
>

Smells like a file descriptor not being closed after used by some Oracle code ( a very used one ) :( i think that you will need a patch from support. Put pressure on them, hope that you've marked your SR as severity 1 ( your system will hang randomly depending on what process will need a new file descriptor and the kernel will deny it - dbwriter, clusterware processes, server processes...-). Send a comment each 40 min, and request escalation after that.

My best wishes ( i suffered something similar years ago )

D.

--
http://www.freelists.org/webpage/oracle-l