Re: Shell Script to Supress OEM alerts during shutdown immediate

From: Arun Chugh <arun.chugh1610_at_gmail.com>
Date: Sun, 27 Mar 2016 12:38:56 +0530
Message-Id: <3F5E88C7-04A8-4E1D-A226-4711A033ECB7_at_gmail.com>

We are not getting non-prod alerts over the oncall phone. Just wanted to reduce the no. of alerts flooding to our team mail id.

Already getting the one alert for db down ie minimum notification. Now even We do not want notifications in case of event like shutdown immediate but not in case of shutdown abort

Sent from my iPhone

> On 27-Mar-2016, at 3:49 AM, RajeevGM <rprabha01_at_gmail.com> wrote:
>
> Arun,
>
> The suppressing for non-prod DBs that you
> mentioned - is this only for the Oncall phone
> or to the DBA team's group mail or both ?
>
> What we have done at our place is - decouple
> Prod DB groups from Non-prod DB groups,
> so, the non availability of non-prod DBs
> does not reach the Oncall phone/pager.
>
> If non-availability (let's say via shutdown
> immediate of non-prod/non critical DBs brings
> lots of emails to the generic/group mail box,
> can't you try keeping the repeat notifications
> to the minimum.
>
> This will avoid the zero notification scenario
> and will also not flood the team's inbox.
>
> This may or may not serve your purpose or for
> some of the clients, but, wanted to share how
> we reduce the annoyance factor on the Oncall
> phone.
>
> Rajeev

>> On Mar 26, 2016, at 3:35 PM, Arun Chugh <arun.chugh1610_at_gmail.com> wrote:
>> 
>> We want to supress the alert for lower env like SIT/DEV etc.. Not for prod. If anyone has some idea for the same or handy scripts / procedure, please share.
>> 
>> Regards,
>> Arun Chugh
>> 
>> Sent from my iPhone
>> 
>>> On 26-Mar-2016, at 10:35 PM, Kellyn Pot'Vin-Gorman <dbakevlar_at_gmail.com> wrote:
>>> 
>>> Thank you for letting me know.  Also be aware, by suppressing alerts that are generated for outages, if you're not careful, you could set yourself up for a real situation being disregarded as a planned outage by the EM.  I would recommend investigating corrective actions that could be applied to a carefully planned metric extension to monitor for this type of situation.
>>> 
>>> Hope this helps,
>>> Kellyn
>>> 
>>>  
>>> 
>>>  	
>>> Kellyn Pot'Vin-Gorman
>>> about.me/dbakevlar
>>>  
>>> 
>>>> On Sat, Mar 26, 2016 at 9:25 AM, Arun Chugh <arun.chugh1610_at_gmail.com> wrote:
>>>> Acutally the motive behind this is to reduce the number of incidents we are getting due to database went down (intentionally) due to any DBA activity, so we want the automatic supress of alerts.
>>>> 
>>>> No there is no such plan for upgrade in near future.
>>>> 
>>>> Sent from my iPhone
>>>> 
>>>>> On 26-Mar-2016, at 8:27 PM, Kellyn Pot'Vin-Gorman <dbakevlar_at_gmail.com> wrote:
>>>>> 
>>>>> I'm going to make a few assumptions here, so please let me know if this is correct-
>>>>> 
>>>>> 1.  You begin maintenance and submit the blackout.
>>>>> 2.  A flurry of alert notifications are sent and alarm the DBA on call
>>>>> 3.  You are looking for a way to avoid these.
>>>>> 
>>>>> The reason behind this situation is that the blackout signal to the agent is controlled by the OMS in any verion 12.*.  With the introduction of EM13c, the AGENT now controls the blackout, the communication to the OMS and then the agent availability.  This change is valuable to correcting this notification issue that [I believe] you're describing.
>>>>> 
>>>>> Let me know, as I can tell you, EM13c will solve this if you're already planning on upgrading in the near future.
>>>>> 
>>>>> Kellyn Pot'Vin-Gorman
>>>>> Consulting Member of Technical Team, SCP
>>>>> Oracle Enterprise Manager and Oracle Management Cloud  
>>>>> 
>>>>>  
>>>>> 
>>>>>  	
>>>>> Kellyn Pot'Vin-Gorman
>>>>> about.me/dbakevlar
>>>>>  
>>>>> 
>>>>>> On Sat, Mar 26, 2016 at 7:35 AM, Arun Chugh <arun.chugh1610_at_gmail.com> wrote:
>>>>>> Thanks for the suggestion.
>>>>>> 
>>>>>> It would be great if anyone of you is having the procedure handy for this task.
>>>>>> 
>>>>>> But one thing I want to mentioned here is that the database is running thru oracle user and emagent is running thru grid user, so we need to setup passwordless connection(ssh) for every server in order to run emcli command via procedure and it would be very tedious task for entire setup or correct me if I am wrong.
>>>>>> 
>>>>>> Regards,
>>>>>> Arun Chugh
>>>>>> 
>>>>>> Regards,
>>>>>> Arun Chugh
>>>>>> 
>>>>>> Sent from my iPhone
>>>>>> 
>>>>>>> On 26-Mar-2016, at 5:59 PM, Niall Litchfield <niall.litchfield_at_gmail.com> wrote:
>>>>>>> 
>>>>>>> You should
>>>>>>> 
>>>>>>> a) have procedures and script(s) for shutting down databases that everyone follows
>>>>>>> b) include the necessary emcli commands in the scripts 
>>>>>>> 
>>>>>>> If someone doesn't follow procedure then I'd say that an alert and an on-call page is exactly what you want to happen.
>>>>>>> 
>>>>>>> 
>>>>>>>> On Sat, Mar 26, 2016 at 10:54 AM, Arun Chugh <arun.chugh1610_at_gmail.com> wrote:
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> Thanks for the response.
>>>>>>>> 
>>>>>>>> Yes, this could be one of the solution but what happens, we forget to put the blackout in OEM and start getting the alerts for the same, so we want the automatic solution of it, if any DBA put down the DB with "shutdown immediate" we should not receive any alerts.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Arun Chugh
>>>>>>>> 
>>>>>>>> Sent from my iPhone
>>>>>>>> 
>>>>>>>>> On 26-Mar-2016, at 4:05 PM, Ls Cheng <exriscer_at_gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi
>>>>>>>>> 
>>>>>>>>> You mean Blackout? You can run emcli, this is an example
>>>>>>>>> 
>>>>>>>>> emcli create_blackout \
>>>>>>>>> -name="MAINTENANCE" \
>>>>>>>>> -add_targets="XXXX:oracle_database;ZZZZ:oracle_listener" \
>>>>>>>>> -reason="MAINTENANCE WINDOW" \
>>>>>>>>> -schedule="frequency:once;duration:0:30"
>>>>>>>>> 
>>>>>>>>> It blackouts 2 targets, one database and one listener for 30 minutes
>>>>>>>>> 
>>>>>>>>> Regards
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Sat, Mar 26, 2016 at 11:25 AM, Arun Chugh <arun.chugh1610_at_gmail.com> wrote:
>>>>>>>>>> Hello Team,
>>>>>>>>>> 
>>>>>>>>>> Our requirement is to automatically supress the OEM alerts whenever the database undergoes "shutdown immediate" during the refresh or maintenance windows.
>>>>>>>>>> 
>>>>>>>>>> If someone have any script, please share. Thanks !!!
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Arun Chugh
>>>>>>>>>> 
>>>>>>>>>> Sent from my iPhone--
>>>>>>>>>> http://www.freelists.org/webpage/oracle-l
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> -- 
>>>>>>> Niall Litchfield
>>>>>>> Oracle DBA
>>>>>>> http://www.orawin.info
>>> 

--
http://www.freelists.org/webpage/oracle-l

Received on Sun Mar 27 2016 - 09:08:56 CEST