RE: EM12c + instance monitoring template + deadlock detection

From: Stephens, Chris <Chris.Stephens_at_adm.com>
Date: Thu, 27 Feb 2014 10:30:08 -0600
Message-ID: <D95BD5AFADBB0F4E9BB6C53F14D3A05006BFAFF090_at_JRCEXC1V1.research.na.admworld.com>



Yep.

...
Event reported time=Feb 27, 2014 8:56:51 AM CST Target Lifecycle Status=Test
Comment=Test and development Oracle cluster Operating System=Linux
Platform=x86_64
Associated Incident Id=<removed>
Associated Incident Status=New
Associated Incident Owner=
Associated Incident Acknowledged By Owner=No Associated Incident Priority=None
Associated Incident Escalation Level=0
Event Type=Metric Alert
Event name=UserBlockChain:db_time
Metric Group=User Block Chain
Metric=<removed>
Metric value=527
Key Value=6029312_14023_882_36609
Key Column 1=ID(lockID1_lockID2_sesID_sesSerial#) Rule Name=XXX Incident Management Rule set,Create incident for critical metric alerts Rule Owner=CSTEPHENS
Update Details:
Total db time 527 seconds is consumed by 1 sessions blocked by session ... Incident created by rule (Name = XXX Incident Management Rule set, Create incident for critical metric alerts; Owner = CSTEPHENS).

From: Maaz Anjum [mailto:maazanjum_at_gmail.com] Sent: Thursday, February 27, 2014 10:24 AM To: Stephens, Chris
Cc: Niall Litchfield; oracle-l_at_freelists.org Subject: Re: EM12c + instance monitoring template + deadlock detection

Let's level set here for a minute - are you receiving alerts for other metrics within the incidient rule (if any)?

On Thu, Feb 27, 2014 at 11:20 AM, Stephens, Chris <Chris.Stephens_at_adm.com<mailto:Chris.Stephens_at_adm.com>> wrote: I've noticed that the database doesn't create an incident for the deadlock I'm generating and the "deadlock" metric is under the "incident" category in the "database instance" template.

So maybe there's no way to alert on a user generated deadlock?

Thu Feb 27 08:48:08 2014
Global Enqueue Services Deadlock detected. More info in file /u01/app/oracle/diag/rdbms/oract1db/oract1db2/trace/oract1db2_lmd0_21286.trc.

adrci> show incident

ADR Home = /u01/app/oracle/diag/rdbms/oract1db/oract1db2:


INCIDENT_ID          PROBLEM_KEY                                                 CREATE_TIME

-------------------- ----------------------------------------------------------- -------------------------
14897 ORA 600 [kdsgrp1] 2014-01-29 12:05:30.85000 17777 ORA 600 [kdsgrp1] 2014-01-29 12:05:30.87200
2 rows fetched

adrci> exit

From: Niall Litchfield [mailto:niall.litchfield_at_gmail.com<mailto:niall.litchfield_at_gmail.com>] Sent: Thursday, February 27, 2014 3:30 AM To: Stephens, Chris
Cc: oracle-l_at_freelists.org<mailto:oracle-l_at_freelists.org> Subject: Re: EM12c + instance monitoring template + deadlock detection

Chris

I don't immediately have access to a system to look, but is a GES Deadlock applicable to a "database instance" target type. I'd sort of expect that to be a rac_database issue.

On Wed, Feb 26, 2014 at 3:14 PM, Stephens, Chris <Chris.Stephens_at_adm.com<mailto:Chris.Stephens_at_adm.com>> wrote: 12.1.0.3 EM12c monitoring 11.2.0.4 RAC on Oracle Linux 6.

I've got a monitoring template for Target Type = Database Instance. In it I have the "Incident" metric class upload interval set to Every 5 minutes. Under incident I have "Deadlock" Comparison operator set to "Matches" ,"Warning Threshold" empty, "Critical Threshold" set to ".*".

I've applied the template to all of our instances but when I create a deadlock situation, an alert isn't raised. Messages like the following shows up in the alert log:

Global Enqueue Services Deadlock detected. More info in file /u01/app/oracle/diag/rdbms/oract1db/oract1db2/trace/oract1db2_lmd0_21286.trc.

I see alerts for other events that I have set up to monitor through this template like "Blocking Session DB Time" etc.

Anybody have any suggestions on what could be the problem?

I've very new to EM12c btw so it could very easily be something very obvious.

Thanks for any help!
Chris

CONFIDENTIALITY NOTICE:
This message is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by email reply.

--

Niall Litchfield
Oracle DBA
http://www.orawin.info

CONFIDENTIALITY NOTICE:
This message is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by email reply.

--

A life yet to be lived...

CONFIDENTIALITY NOTICE:
This message is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by email reply.

--

http://www.freelists.org/webpage/oracle-l Received on Thu Feb 27 2014 - 17:30:08 CET

Original text of this message