Re: Backups failing

From: Niall Litchfield <niall.litchfield_at_gmail.com>
Date: Tue, 30 Aug 2011 12:00:57 +0100
Message-ID: <CABe10sbAizf05CeR_Pjr_MRxTE8iRDe=PhgvZj45bu8a3fPQjA_at_mail.gmail.com>



Hi

Oliver, John's note is for Grid Control and not for Database Control, that doesn't mean it won't work - but it may not do so. If the problem you are trying to address is RMAN backups failing then using the rman command line directly maybe of more immediate use. At the very least I would do the following in a terminal/command window on the server ( & having set the oracle environment correctly for your database. )

rman target /
list backup of database;

On Tue, Aug 30, 2011 at 9:59 AM, Oliver <ofabelo_at_gmail.com> wrote:

> I did that yesterday ... I will try again.
> Thanks.
>
>
> 2011/8/30 Pavel <ocp.pauler_at_gmail.com>
>
>> Hi
>> Try this:
>> emctl stop agent
>> cd $AGENT_HOME/sysman/emd
>> rm -rf collection/*
>> rm -rf state/*
>> rm -rf upload/*
>> emctl start agent
>>
>> Regards,
>> Pavel.
>>
>>
>> 2011/8/30 Oliver <ofabelo_at_gmail.com>
>>
>>> I'm looking emagent.trc, just when it crashes ... The log output is:
>>>
>>> 2011-08-30 00:11:12 Thread-4039011216 ERROR scheduler: Collection
>>> oracle_listener:LISTENER_xxx.xxx.xx:Response has not finished in 721 minutes
>>> (allowed: 720).
>>> Exiting agent.
>>> 2011-08-30 00:11:12 Thread-4039011216 ERROR : Signalled to Exit with
>>> status 3
>>> 2011-08-30 00:11:12 Thread-4136433344 ERROR main: EMAgent abnormal
>>> terminating
>>> 2011-08-30 00:11:41 Thread-4136175296 WARN command: Job Subsystem
>>> Timeout set at 600 seconds
>>> 2011-08-30 00:11:41 Thread-4136175296 WARN upload: Upload manager has
>>> no Failure script: disabled
>>> 2011-08-30 00:11:41 Thread-4136175296 WARN upload: Recovering left over
>>> xml files in upload directory
>>> 2011-08-30 00:11:42 Thread-4136175296 WARN upload: Recovered 0 left
>>> over xml files in upload directory
>>> 2011-08-30 00:11:42 Thread-4136175296 WARN metadata: Metric Disk_Path
>>> does not have any data columns
>>> 2011-08-30 00:11:42 Thread-4136175296 WARN metadata: Metric
>>> osm_diskGroupPolicies does not have any data columns
>>> 2011-08-30 00:12:42 Thread-4136175296 ERROR TargetManager: TIMEOUT when
>>> compute dynamic properties for target xxx.xxx.xxx
>>> 2011-08-30 00:12:42 Thread-4136175296 ERROR TargetManager: target
>>> {xxx.xxx.xxx, oracle_database} is broken: cannot compute dynamic properties
>>> in time.
>>> 2011-08-30 00:12:43 Thread-4136175296 WARN collector: the column name
>>> DiskActivityavwait in this condition does not exist
>>> 2011-08-30 00:12:43 Thread-4119530384 ERROR upload: Error in
>>> uploadXMLFiles. Trying again in 60.00 seconds.
>>>
>>> I'm "googling" but without success for now ... I don't find the solution.
>>> Thanks beforehand.
>>>
>>>
>>>
>>> 2011/8/30 Oliver <ofabelo_at_gmail.com>
>>>
>>>> Same error today, initialization failure in status of the job and in the
>>>> error log, "Step aborted because agent went down".
>>>>
>>>>
>>>>
>>>>
>>>> 2011/8/29 Oliver <ofabelo_at_gmail.com>
>>>>
>>>>> This is the output:
>>>>>
>>>>> Oracle Enterprise Manager 10g Database Control Release 10.2.0.1.0
>>>>> Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
>>>>> ---------------------------------------------------------------
>>>>> Agent Version : 10.1.0.4.1
>>>>> OMS Version : 10.1.0.4.0
>>>>> Protocol Version : 10.1.0.2.0
>>>>> Agent Home : /u01/app/oracle/product/10.2.0/db_1/xxx_ora10g
>>>>> Agent binaries : /u01/app/oracle/product/10.2.0/db_1
>>>>> Agent Process ID : 19051
>>>>> Parent Process ID : 18114
>>>>> Agent URL : http://xxx:3938/emd/main
>>>>> Started at : 2011-08-29 12:09:00
>>>>> Started by user : oracle
>>>>> Last Reload : 2011-08-29 12:14:14
>>>>> Last successful upload : 2011-08-29 12:14:23
>>>>> Total Megabytes of XML files uploaded so far : 4.40
>>>>> Number of XML files pending upload : 0
>>>>> Size of XML files pending upload(MB) : 0.00
>>>>> Available disk space on upload filesystem : 87.14%
>>>>> ---------------------------------------------------------------
>>>>> Agent is Running and Ready
>>>>>
>>>>> I see it all ok, isn't it?
>>>>> Jobs are many hours for seeing it has failed ... I will look tomorrow,
>>>>> but I see that it doesn't writing in the backup dir ...
>>>>>
>>>>>
>>>>> 2011/8/29 Niall Litchfield <niall.litchfield_at_gmail.com>
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> what is the full output on the target machine of emctl status agent?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 29, 2011 at 11:49 AM, Oliver <ofabelo_at_gmail.com> wrote:
>>>>>>
>>>>>>> When I do "emctl upload agent" step, it shows:
>>>>>>>
>>>>>>> EMD upload error: uploadXMLFiles skipped :: OMS version not checked
>>>>>>> yet..
>>>>>>>
>>>>>>> Jus the error of that page ...
>>>>>>>
>>>>>>>
>>>>>>> 2011/8/29 Oliver <ofabelo_at_gmail.com>
>>>>>>>
>>>>>>>> Yes, I did crosscheck in rman ...
>>>>>>>> I'm going to do the steps of:
>>>>>>>>
>>>>>>>>
>>>>>>>> http://jhdba.wordpress.com/2008/12/10/clearing-an-enterprise-manager-agent-that-fails-to-upload/
>>>>>>>>
>>>>>>>> Is it ok?
>>>>>>>>
>>>>>>>> Thanks beforehand.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2011/8/29 Pavel <ocp.pauler_at_gmail.com>
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>> Did you crosscheck after delete some backups? Clear state don't
>>>>>>>>> remove created jobs.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Pavel.
>>>>>>>>>
>>>>>>>>> 2011/8/29 Oliver <ofabelo_at_gmail.com>
>>>>>>>>>
>>>>>>>>>> Thanks, but I'm sorry, I'm a newbie, the dba is on vacation and I
>>>>>>>>>> have to look it because I'm the person with more knowledges of Oracle.
>>>>>>>>>> I've tried stop and start agent/dbconsole but it is failing again.
>>>>>>>>>> I understand your commands, except remove all uploaded files :-?
>>>>>>>>>> Clear state or remove uploads files remove created jobs?
>>>>>>>>>> Ah, one thing, I think that it started to fail when it surpassed
>>>>>>>>>> the flash recovery quota. I had 300GB, and it had 330GB written, in
>>>>>>>>>> destination of backup dir. I deleted some backups and I will increase to
>>>>>>>>>> 500GB the flash recovery quota (I've not restarted database, is it
>>>>>>>>>> necessary?)
>>>>>>>>>> Thanks beforehand.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2011/8/29 Pavel <ocp.pauler_at_gmail.com>
>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>> Could you try these one:
>>>>>>>>>>>
>>>>>>>>>>> Issue emctl stop agent, emctl clear state agent, emctl start
>>>>>>>>>>> agent.
>>>>>>>>>>>
>>>>>>>>>>> And I think remove all uploads files.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Pavel.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2011/8/29 Oliver <ofabelo_at_gmail.com>
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> backups of EM are failing suddenly. It shows "Initialization
>>>>>>>>>>>> failure" in status and error log shows "Step aborted because agent went
>>>>>>>>>>>> down". The error comes after many hours, I see that it doesn't write nothing
>>>>>>>>>>>> in the oputut dir of the backup.
>>>>>>>>>>>> If I do "$ tail -f emagent.trc", it shows some warns and the
>>>>>>>>>>>> errors:
>>>>>>>>>>>>
>>>>>>>>>>>> 2011-08-29 09:43:25 Thread-4136277696 ERROR TargetManager:
>>>>>>>>>>>> TIMEOUT when compute dynamic properties for target
>>>>>>>>>>>> ora10g.localhost.localdomain
>>>>>>>>>>>> 2011-08-29 09:43:25 Thread-4136277696 ERROR TargetManager:
>>>>>>>>>>>> target {ora10g.localhost.localdomain, oracle_database} is broken: cannot
>>>>>>>>>>>> compute dynamic properties in time.
>>>>>>>>>>>> 2011-08-29 09:43:25 Thread-4136277696 WARN collector: the
>>>>>>>>>>>> column name DiskActivityavwait in this condition does not exist
>>>>>>>>>>>> 2011-08-29 09:43:25 Thread-4119632784 ERROR upload: Error in
>>>>>>>>>>>> uploadXMLFiles. Trying again in 60.00 seconds.
>>>>>>>>>>>>
>>>>>>>>>>>> It is 10.2.0.1.0 on Linux. Someone can help me?
>>>>>>>>>>>> Thanks beforehand.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers...
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Niall Litchfield
>>>>>> Oracle DBA
>>>>>> http://www.orawin.info
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

-- 
Niall Litchfield
Oracle DBA
http://www.orawin.info

--
http://www.freelists.org/webpage/oracle-l
Received on Tue Aug 30 2011 - 06:00:57 CDT

Original text of this message