Home » RDBMS Server » Enterprise Manager » Agent unreachable
Agent unreachable [message #284390] Thu, 29 November 2007 09:01 Go to next message
monto
Messages: 60
Registered: November 2007
Location: DALLAS
Member
Hi,
I have installed agent 10g on unix machine(HPUX-11.1) agent runs for 6-8 hrs
after that it dies out and when i see the instance on that server through grid control it says agent unreachable , iam not sure whats happening here.


[oracle:leopard:NOSID] /oracle/product/agent10g/sysman/log > ps -ef|grep agen>
root 2661 1 0 May 20 ? 0:00 /etc/opt/resmon/lbin/emsagent
root 2617 1 0 May 20 ? 2:38 /usr/sbin/swagentd -r
oracle 23822 23690 0 Nov 20 ? 13:24 /oracle/product/agent10g/bin/emagent
oracle 23690 1 0 Nov 20 ? 2:54 /oracle/product/agent10g/perl/bin/perl /oracle/product/agent10g
aduaslq 16850 22944 0 Nov 23 ? 4:58 /amuaslq01s/app/amuaslqdb/10.2.0/bin/emagent---->agent is for OAM


[oracle:leopard:NOSID] /oracle/product/agent10g/bin > emctl status agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent is Not Running


[oracle:leopard:NOSID] /oracle/product/agent10g/bin > emctl start agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
Starting agent ....... failed.
Failed to start HTTP listener.
Consult the log files in: /oracle/product/agent10g/sysman/log

[oracle:leopard:NOSID] /oracle/product/agent10g/bin > ps -ef|grep agent
root 2661 1 0 May 20 ? 0:00 /etc/opt/resmon/lbin/emsagent
root 2617 1 0 May 20 ? 2:38 /usr/sbin/swagentd -r
oracle 23822 23690 0 Nov 20 ? 13:23 /oracle/product/agent10g/bin/emagent
oracle 3913 15344 0 21:42:27 pts/7 0:00 grep agent
oracle 23690 1 0 Nov 20 ? 2:53 /oracle/product/agent10g/perl/bin/perl /oracle/product/agent10g
aduaslq 16850 22944 0 Nov 23 ? 4:57 /amuaslq01s/app/amuaslqdb/10.2.0/bin/emagent


/oracle/product/agent10g/sysman/log :vi emagent.trc

2007-11-27 17:48:44 Thread-189993 ERROR util.files: ERROR: nmeufos_new: failed i
n lfiopn on file: /oracle/product/agent10g/sysman/emd/agntstmp.txt.error = 24 (T
oo many open files)
2007-11-27 17:48:44 Thread-189993 ERROR pingManager: Error in updating the agent
time stamp file
2007-11-27 17:48:48 Thread-189994 ERROR util.fileops: ERROR: snmeuf_dirlist can'
t list directory: /oracle/product/agent10g/sysman/emd/upload: Too many open file
s (errno=24)
2007-11-27 17:48:51 Thread-189995 ERROR engine: Failed when generating a new ECI
D.
2007-11-27 17:48:51 Thread-189995 ERROR fetchlets.healthCheck: GIM-00104: file n
ot found
LEM-00031: file not found; arguments: [lempgmh] [lmserr]
LEM-00033: file not found; arguments: [lempgfm] [Couldn't open message file]
LEM-00031: file not found; arguments: [lempgmh] [lmserr]
2007-11-27 17:48:51 Thread-189995 ERROR engine: [oracle_database,leopard-amuaslq
.am,health_check] : nmeegd_GetMetricData failed : Instance Health Check initiali
zation failed due to one of the following causes: the owner of the EM agent proc
ess is not same as the owner of the Oracle instance processes; the owner of the
EM agent process is not part of the dba group; or the database version is not 10
g (10.1.0.2) and above.


The main error is this
2007-11-27 17:48:44 Thread-189993 ERROR util.files: ERROR: nmeufos_new: failed i
n lfiopn on file: /oracle/product/agent10g/sysman/emd/agntstmp.txt.error = 24 (T
oo many open files)
2007-11-27 17:48:44 Thread-189993 ERROR pingManager: Error in updating the agent
time stamp file

and after some gooling i found that It is giving too many files open error. It seems like kernel parameter settings for open files is low.I need to increase the kernel parameter maxfiles and current value is below but i have agent on another server which has exact same number of maxfiles(2048)runs
with no issues why is that this (server)has an issue with it,is it got to do with the number of instances running on the box or memory or different versions of oracle?

/etc/sysdef | grep maxfiles
maxfiles 2048 - 30-2048 -
maxfiles_lim 2048 - 30-2048 -


Re: Agent unreachable [message #284440 is a reply to message #284390] Thu, 29 November 2007 12:23 Go to previous messageGo to next message
DreamzZ
Messages: 1666
Registered: May 2007
Location: Dreamzland
Senior Member
Check the permissions and ownership of sysman/log files

my box:/opt/oracle/10.2.0AGENT/agent10g/sysman/log$ ls -lt
total 42498
-rw-r-----   1 oracle   dba      3145189 Nov 29 18:22 emagent.trc
-rw-r-----   1 oracle   dba      5095273 Nov 29 18:15 emctl.log
-rw-r-----   1 oracle   dba      4653920 Nov 28 23:39 emagent_perl.trc
-rw-r-----   1 oracle   dba      4194524 Nov 28 09:45 emagent.trc.1
-rw-r-----   1 oracle   dba      4194337 Nov 17 02:37 emagent.trc.2
-rw-r-----   1 oracle   dba          937 Oct 26 20:29 nmcfsga.log
-rw-r-----   1 oracle   dba          856 Oct 26 20:29 nmcfsga.err
-rw-r-----   1 oracle   dba         8276 Oct 25 23:28 emagent.nohup
-rw-r-----   1 oracle   dba         3299 Oct 24 23:28 emagent.log
-rw-r-----   1 oracle   dba         2087 Oct 24 23:28 emdctl.trc
-rw-r-----   1 oracle   dba          126 Oct 24 23:27 agabend.log
-rw-r-----   1 oracle   dba            0 Sep 25 20:53 nfsPatchPlug.log
-rw-r-----   1 oracle   dba            0 Sep 25 20:52 emagentfetchlet.trc
-rw-r-----   1 oracle   dba            0 Sep 25 20:52 emagentfetchlet.log
-rw-r-----   1 oracle   dba         1176 Sep 25 20:52 secure.log
-rw-r-----   1 oracle   dba            0 Sep 25 20:51 emdctl.log


[Updated on: Thu, 29 November 2007 12:23]

Report message to a moderator

Re: Agent unreachable [message #284460 is a reply to message #284390] Thu, 29 November 2007 14:38 Go to previous message
monto
Messages: 60
Registered: November 2007
Location: DALLAS
Member
Here is the ouput
-rw-r----- 1 oracle dba 1280 Nov 28 18:50 secure.log
-rw-rw---- 1 oracle dba 9244 Nov 28 18:51 emtgtctl.log
-rw-rw---- 1 oracle dba 79500 Nov 28 18:51 emtgtctl.trc
-rw-r----- 1 oracle dba 9244 Nov 28 18:51 emdctl.log
-rw-r----- 1 oracle dba 42 Nov 28 18:51 agabend.log
-rw-r----- 1 oracle dba 0 Nov 28 18:52 nfsPatchPlug.log
-rw-r----- 1 oracle dba 2719 Nov 28 18:54 emagent.log
-rw-r----- 1 oracle dba 19425 Nov 28 22:39 emagent_perl.trc
-rw-r----- 1 oracle dba 4194336 Nov 29 04:19 emagent.trc.3
-rw-r----- 1 oracle dba 4194452 Nov 29 11:04 emagent.trc.2
-rw-r----- 1 oracle dba 8691 Nov 29 16:56 emctl.log
-rw-r----- 1 oracle dba 4194426 Nov 29 17:48 emagent.trc.1
-rw-r----- 1 oracle dba 267666 Nov 29 20:35 emagentfetchlet.log
-rw-r----- 1 oracle dba 267666 Nov 29 20:35 emagentfetchlet.trc
-rw-r----- 1 oracle dba 148577 Nov 29 20:35 emagent.nohup
-rw-r----- 1 oracle dba 767467 Nov 29 20:35 emdctl.trc
-rw-r----- 1 oracle dba 1779313 Nov 29 20:35 emagent.trc
Previous Topic: OEM problem[merged]
Next Topic: is it everything in sql?
Goto Forum:
  


Current Time: Sun Dec 11 00:20:36 CST 2016

Total time taken to generate the page: 0.09900 seconds