Re: DMON killing RSM0?

From: Mladen Gogala <gogala.mladen_at_gmail.com>
Date: Sat, 13 Jun 2020 10:50:51 -0400
Message-ID: <05b05a10-9572-c6fc-8e3d-bcd955773229_at_gmail.com>



Hi Dave,

These errors are network timeout errors. RSM processes monitor the standby status. Oracle connects to the primary port, usually 1521, and then the the connection is handed to the dynamic ports. Firewall settings sometimes cut these ports off, at least some of them. The default setting with Oracle installation is something like:

net.ipv4.ip_local_port_range = 9000 65500

Your firewall may be configured to have dynamic ports between 32000 and 55000. The result is the situation in which Linux attempts to hand off the primary connection to the dynamic port which is blocked by firewall. Each killed remote status monitor (RSM) will produce its own trace. Please, check the trace and if you see something like "timeout on the port 55831" then you know that there is some configuration you need to do. Here is a decent article about the dynamic (local) ports:

https://blog.fpmurphy.com/2015/02/ip-dynamic-port-range.html

Fortunately, you don't have to deal with the logical standby. Now, that would be fun for the whole family. In addition to the archive delivery and the status monitoring, there is also a redo apply process.

Regards

On 6/12/20 5:39 PM, Herring, Dave (Redacted sender HerringD for DMARC) wrote:
>
> I have a situation where it looks like the DMON process is killing off
> RSM0 processes every night around the same time and I don't have a
> good explanation as to why. This is on a 4-node Exadata env running
> 18c with 6 dbs, all using DG (the standby is also a 4-node Exadata env).
>
> Every night between 20:12 and 21:35 we get a series of ORA-16665
> errors from all databases, errors found in the broker's logfile. 
> Checking each db's alert log I see messages like the following:
>
> Process RSM0, PID = 51310, will be killed
>
> Process termination requested for pid 51310 [source = rdbms], [info =
> 2] [request issued by pid: 76161, uid: 110]
>
> SPID 76161 is DMON, which means every night DMON kills off RSM0
> processes around the same time.  This is done for all databases.
>
> Is there a DG broker setting that says to wipe out all DGB resource
> processes and restart them?
>
> Regards,
>
> Dave
>

-- 
Mladen Gogala
Database Consultant
Tel: (347) 321-1217


--
http://www.freelists.org/webpage/oracle-l
Received on Sat Jun 13 2020 - 16:50:51 CEST

Original text of this message