Re: OEM DB console on 10gR2 broken.

On Nov 29, 7:29 am, gba..._at_ravenpack.com wrote:
> Hello,
>
> I have an issue with an OEM database control installed with Oracle
> Database
> 10.2.0.1.0 on Linux(Flavour of linux) X86_64 platform. This is a
> production environment.
>
> Here is a summary of my problem:
> About a week ago, I noticed that an RMAN backup session scheduled from
> the Database Control page had hung for a long period of time.
> I killed the Perl OS Process for the backup session and tried to
> restart it.
> After restarting, the new Perl process also hung, this time consuming
> 100% CPU.
> I also killed this.
>
> At this point, I decided to drop and recreate the OEM repository using
> emca script:
>
> > emca -dbconfig dbcontrol db -repos recreate
>
> The script hung while dropping the repository. Once again, I saw a
> perl process at 100% CPU.
> It exactly hang at:
> $ORACLE_HOME/sysman/admin/emdrep/bin/emrepmgr.pl:
> getTempTablespace() method
> <code>
> my $lda = DBI->connect('dbi:Oracle:', "SYS@".$EM_REPOS_CONNSTRING,
> "$EM_REPOS_DBSYSPWD",
> {ora_session_mode => 2, PrintError => 0, RaiseError => 0,
> AutoCommit =>
> 0})
> or (info "Could not connect to SYS/$EM_REPOS_CONNSTRING:
> $DBI::errstr\n"
> && exit 50);
> </code>
>
> I made a simple Perl file to test connections to the database using
> DBI
> Perl interface with different users and all attempts to connect hung.
> No error is
> returned. I am starting to believe that perhaps the Perl environment
> could
> be corrupted.
>
> I manage to recreate the OEM environment by following metalink Note:
> 278100.1 instructions:
> 1) Deleting DB Control Configuration Files using emca script:> emca -deconfig dbcontrol db
>
> 2) Manually dropping OEM related objects:
> Logon SQLPLUS as user SYS or SYSTEM, and drop the sysman account and
> management objects:
> SQL> drop user sysman cascade;
> SQL> drop role MGMT_USER;
> SQL> drop user MGMT_VIEW cascade;
> SQL> drop public synonym MGMT_TARGET_BLACKOUTS;
> SQL> drop public synonym SETEMVIEWUSERCONTEXT;
> 3) Create the DB Control Repository Objects and Configuration Files.
>
> > emca -config dbcontrol db -repos create
>
> However the database control page shows the general status as "?Status
> Pending?"
> and no information available. The home page shows the error:
> java.lang.Exception: Exception in sending Request :: null
>
> I forced the agent to do an ?upload? by issuing:> emctl upload
>
> And finally got access to the OEM database control page.
>
> My current situation is:
> 1) The OS process emagent is consuming 100% of 1 CPU while the db
> console
> is started.
> 2) The database control page shows incomplete information. For
> instance, I
> have access to performance and database objects information, but I
> cannot
> get system statistics, ADDM recommendations, most of the metrics, and
> lots of
> other valuable information.
>
> The emca.trc file shows, among others, the following errors:
> 2007-11-29 11:44:56 Thread-4136450272 ERROR TargetManager: target
> {[SID], oracle_database} is broken: cannot compute dynamic properties
> in time.
> 2007-11-29 11:44:56 Thread-4136450272 ERROR util.files: ERROR:
> nmeufis_new: failed in lfiopn on file: $ORACLE_HOME/[host]_[sid]/
> sysman/emd/agntstmp.txt. error = 2 (No such file or directory)
> 2007-11-29 11:44:56 Thread-4136450272 ERROR collector:
> nmecs_validateStateFile(): state file invalid : $ORACLE_HOME/
> [host]_[sid]/sysman/emd/state/snapshot
>
> The DB is completely operational and I can perform Backups using the
> RMAN client.
>
> My questions are:
> 1. Does anyone have an idea of how to resolve this problems?
> 2. Is there a sure way to reinstall, purge or completely clean the OEM
> environment and start from scratch?
>
> Best Regards,
>
> G. Bahut

Is there a specific reason why you are sitting at 10.2.0.1 instead of moving into 10.2.0.3?

There's a bunch of bugs related to the agent stuff that get fixed along the way ... while you can and perhaps should contact oracle support a shot in the dark is that perhaps going up to 10.2.0.3 may fix many things.

You may also want to take a look at note 423099.1 in metalink it describes a workaround related to agent status.

Make sure that you have the correct ORACLE_SID value set before stopping and starting agent. Received on Thu Nov 29 2007 - 07:11:54 CST