Oracle FAQ Your Portal to the Oracle Knowledge Grid

Home -> Community -> Mailing Lists -> Oracle-L -> RE: RMAN "Memory fault"

RE: RMAN "Memory fault"

From: Herring Dave - dherri <>
Date: Mon, 29 Oct 2007 16:10:25 -0500
Message-ID: <>

Sorry, should have given more details. 2 databases on the server, gmdwt and gmanat. gmdwt is the one that causes the failure. Both have the same $ORACLE_HOME and I'm using oraenv to switch environments. A check of $LD_LIBRARY_PATH (that's what Tru64 uses) shows the same value for each. So does "ulimit -a", which is:

time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         4194304
stack(kbytes)        524288
memory(kbytes)       16285248
coredump(blocks)     0
nofiles(descriptors) 4096
vmemory(kbytes)      12582912

I did run "file ..." on each file returned from "ldd", which all came back as " COFF format alpha shared library, demand paged executable or object module not stripped - version 3.13-14".

I also ran the shell for-loop and didn't come up with any issues, other than $ORACLE_HOME/lib having a few newer files, which match in date to the most recent Oracle security patch. And for those wondering, this problem happened between security patches, as in months after the last one and before the most recent one, although each one does tend to bring its own problems.

for _Dir in `echo $LD_LIBRARY_PATH | sed 's/:/ /g'` do

    echo "\nSearching \"${_Dir}\" for files newer than \"\$ORACLE_HOME/bin/rman\"..."     find ${_Dir} -name "*.so" -newer $ORACLE_HOME/bin/rman 2>/dev/null | xargs -n1 ls -l done

Searching "/usr/lib" for files newer than "$ORACLE_HOME/bin/rman"...

Searching "/usr/local/lib" for files newer than "$ORACLE_HOME/bin/rman"...

Searching "/batchcdi/util/lib" for files newer than "$ORACLE_HOME/bin/rman"...

Searching "/u01/app/oracle/product/9.2.0/lib" for files newer than "$ORACLE_HOME/bin/rman"...

lrwxrwxrwx   1 oracle   dba           14 Oct 24 17:39 /u01/app/oracle/product/9.2.0/lib/ ->
lrwxrwxrwx   1 oracle   dba           16 Oct 24 18:02 /u01/app/oracle/product/9.2.0/lib/ ->
lrwxrwxrwx   1 oracle   dba           20 Oct 24 18:02 /u01/app/oracle/product/9.2.0/lib/ ->
-rwxr-x---   1 oracle   dba        18896 Jun 27 17:20 /u01/app/oracle/product/9.2.0/lib/

One hit on google about output from truss suggested the problem is an overflow on a floating type. I'm going the route of SR with Oracle and including their SQL (from 10046 on both ends).


David C. Herring, DBA  |   A c x i o m  Delivery Center Organization 630-944-4762 office | 630-430-5988 wireless | 630-944-4989 fax

From: Tim Gorman [] Sent: Monday, October 29, 2007 3:02 PM
To: Herring Dave - dherri
Cc: oracle-l
Subject: Re: RMAN "Memory fault"

LD_LIBRARY_PATH and/or SHLIB_PATH are UNIX environment variables, and easy to get wrong when switching environments, unless you've got your environment-setting scripts solid.

Also, could be fun to compare "ulimit -a" output in the environment where it works and where it doesn't...

Database parms are almost certainly blameless for this type of error.  It is probably environment (i.e. env variables, process limits, etc) or it could be an executable shared-library file that's been overwritten or had it's permissions hosed, or something like that.  That's why I suggest running "file" against each of the shared libraries returned in the "ldd" command -- one of those shared libraries might just come back as "data" instead of "executable", indicating that someone or something reset permissions so the file is no longer executable, for example...

Might also try something like the following: for _Dir in `echo $LD_LIBRARY_PATH | sed 's/:/ /'` do
    echo "\nSearching \"${_Dir}\" for files newer than \"\$ORACLE_HOME/bin/rman\"..."
    find ${_Dir}-name "*.so" -newer $ORACLE_HOME/bin/rman -exec ls -l {} \;

On the off-chance that a shared-library used by RMAN is newer than the RMAN executable file...     

The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged.

If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.

If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system.

Thank You.

Received on Mon Oct 29 2007 - 16:10:25 CDT

Original text of this message