OPatchAuto failed

From: Sameer Choudhari <sameer.choudhari_at_ezidebit.com.au>
Date: Fri, 4 Oct 2019 13:40:58 +1000
Message-ID: <CAMzOyFE7vMbta-R3dWajm-BN3J_9pH=+r8LwPFf2c_vnx_9gaQ_at_mail.gmail.com>

Team

I am applying JULY 2019 Proactive Bunble Patch on our UAT env.

Env :

DB : 12.1.0.2.0 (2 node rac)
OS : Oracle Linux 7

I am able to patch node-1 i.e. *uat-db-001 *without any issue but got below error while patching node -2 i.e. uat-db-002 .

[root_at_uat-db-002 .ssh]# export PATH=$PATH:$ORACLE_HOME/OPatch
[root_at_uat-db-002 .ssh]# export PATH=$PATH:/u01/g01/grid/12.1/OPatch
[root_at_uat-db-002 .ssh]# cd  /u01/stage/COMBO_JVM_DBBP_JUL2019/29699255
[root_at_uat-db-002 29699255]# $ORACLE_HOME/OPatch/opatchauto apply

/u01/stage/COMBO_JVM_DBBP_JUL2019/29699255/29698629

OPatchauto session is initiated at Fri Oct 4 13:13:54 2019

System initialization log file is /u01/g01/grid/12.1/ cfgtoollogs/opatchautodb/systemconfig2019-10-04_01-14-00PM.log.

Session log file is
/u01/g01/grid/12.1/cfgtoollogs/opatchauto/opatchauto2019-10-04_01-14-44PM.log
The id for this session is LBTP
Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1

/u01/g01/grid/12.1/perl/lib/5.14.1/x86_64-linux-thread-multi

/u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverException: Execution failed for host uat-db-001 due to : java.lang.Exception: Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1

/u01/g01/grid/12.1/perl/lib/5.14.1/x86_64-linux-thread-multi

OPatchauto session completed at Fri Oct 4 13:15:08 2019 Time taken to complete the session 1 minute, 14 seconds

opatchauto failed with error code 42
[root_at_uat-db-002 29699255]#

Checked MOS and Google but no help. I have shared partial session log below i.e.
/u01/g01/grid/12.1/cfgtoollogs/opatchauto/opatchauto2019-10-04_01-14-44PM.log .
Could you please let me know what could be the issue.

Oct 04, 2019 1:15:06 PM
com.oracle.cie.remote.linux.LinuxCommandExecutionHelper getSSHCmd INFO: SSH path::/bin/ssh
Oct 04, 2019 1:15:06 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: Command execution command: /bin/ssh -o FallBackToRsh=no -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o NumberOfPasswordPrompts=0 uat-db-001 ls /u01/g01/grid/12.1 > /dev/null Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: Command execution return code: 0
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: command execution message:
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: command executon failure message:
*** Authorized uses only. All activity may be monitored and reported. ***

Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceCommandExecutor executeAndNotify INFO: Executing command on host uat-db-001 :
/u01/g01/grid/12.1//perl/bin/perl
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl
-GRID_HOME=/u01/g01/grid/12.1 -OBJECTLOC=/u01/g01/grid/ 12.1//cfgtoollogs/opatchautodb/hostdata.obj -CRS_ACTION=check -CLUSTERNODES=uat-db-001,uat-db-002
-JVM_HANDLER=oracle/dbsysmodel/driver/sdk/productdriver/remote/RemoteOperationHelper Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxCommandExecutionHelper getSSHCmd INFO: SSH path::/bin/ssh
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: Command execution command: /bin/ssh -o FallBackToRsh=no -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o NumberOfPasswordPrompts=0 uat-db-001 /u01/g01/grid/12.1//perl/bin/perl
/u01/g0

1/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl
-GRID_HOME=/u01/g01/grid/12.1
-OBJECTLOC=/u01/g01/grid/12.1//cfgtoollogs/opatchautodb/hostdata.obj

-CRS_ACTION=check -CLUSTERNODES=uat-db-001 ,uat-db-002
-JVM_HANDLER=oracle/dbsysmodel/driver/sdk/productdriver/remote/RemoteOperationHelper Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: Command execution return code: 2
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: command execution message:
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl executeCommand
INFO: command executon failure message:
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceCommandExecutor executeAndNotify WARNING: Error while executing command "/u01/g01/grid/12.1//perl/bin/perl
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl
-GRID_HOME=/u01/g01/grid/12.1 -OBJECTLOC=/u01/g01/grid/12.1/
/cfgtoollogs/opatchautodb/hostdata.obj -CRS_ACTION=check
-CLUSTERNODES=uat-db-001,uat-db-002
-JVM_HANDLER=oracle/dbsysmodel/driver/sdk/productdriver/remote/RemoteOperationHelper " on host uat-db-001 :
Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceThreadHandler$SSHEquiCommandListener notifyResult
INFO: Received command execution result for host uat-db-001 Oct 04, 2019 1:15:07 PM
oracle.dbsysmodel.driver.sdk.productdriver.remote.HostDataCollector notifyResult
INFO: Consolidating result from host : uat-db-001 Oct 04, 2019 1:15:07 PM
oracle.dbsysmodel.driver.sdk.productdriver.remote.HostDataCollector notifyResult
INFO: Execution failed for host uat-db-001 due to : Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

2019-10-04 13:15:07,859 INFO [1]
oracle.dbsysmodel.driver.sdk.util.OsysUtility - Output message ::: 2019-10-04 13:15:07,870 INFO [1]
com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output : Message{id='null', message='Remote command execution failed due to *** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output: '}
2019-10-04 13:15:07,873 INFO [1]
com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport - Space available after session: 33580 MB 2019-10-04 13:15:07,996 SEVERE [1] com.oracle.glcm.patch.auto.OPatchAuto - OPatchAuto failed.
com.oracle.glcm.patch.auto.OPatchAutoException: oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverException: Execution failed for host uat-db-001 due to : java.lang.Exception: Remote command exe
cution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

at
com.oracle.glcm.patch.auto.OPatchAuto.prepareOrchestration(OPatchAuto.java:775)

at
com.oracle.glcm.patch.auto.OPatchAuto.orchestrate(OPatchAuto.java:397)

at
com.oracle.glcm.patch.auto.OPatchAuto.orchestrate(OPatchAuto.java:344)

at com.oracle.glcm.patch.auto.OPatchAuto.main(OPatchAuto.java:212) Caused by:
oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverException: Execution failed for host uat-db-001 due to : java.lang.Exception: Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.checkForException(OUIDriver.java:846)

at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.handleExceptions(OUIDriver.java:832)

at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.fetchRemoteHostData(OUIDriver.java:791)

at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.fetchRemoteNodeData(OUIDriver.java:690)

at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.DBPatchingHelper.fetchPreInitiatedNodes(DBPatchingHelper.java:899)

at
com.oracle.glcm.patch.auto.db.product.validation.validators.RemoteGIStatusValidation.isFirstNodeOfPatching(RemoteGIStatusValidation.java:274)

at
com.oracle.glcm.patch.auto.db.product.validation.validators.RemoteGIStatusValidation.validate(RemoteGIStatusValidation.java:185)

at
com.oracle.glcm.patch.auto.db.product.validation.DBValidationController.invoke(DBValidationController.java:91)

at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.CRSTopologyBuilder.validate(CRSTopologyBuilder.java:367)

at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.AbstractTopologyBuilder.buildTopology(AbstractTopologyBuilder.java:117)

at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.CRSTopologyBuilder.buildTopology(CRSTopologyBuilder.java:112)

at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport.loadTopology(DBBaseProductSupport.java:192)

at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBProductSupport.loadTopology(DBProductSupport.java:69)

at
com.oracle.glcm.patch.auto.OPatchAuto.loadTopology(OPatchAuto.java:1732)

at
com.oracle.glcm.patch.auto.OPatchAuto.prepareOrchestration(OPatchAuto.java:730)

... 3 more
2019-10-04 13:15:07,999 INFO [1]
com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output : Message{id='null',
message='oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverE xception: Execution failed for host uat-db-001 due to : java.lang.Exception: Remote command execution failed due to *** Authorized uses only. All activity may be monitored and reported. ***

Can't locate Cwd.pm in _at_INC (_at_INC contains:


/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi

So far , I tried

For "Can't locate Cwd.pm in _at_INC" , I checked it's there. Also, checked the permission for /u01/g01/grid/12.1/cfgtoollogs/opatchautodb and its owned by grid home user.

[root_at_uat-db-002 29699255]# find / -name Cwd.pm
/usr/lib64/perl5/vendor_perl/Cwd.pm
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_64-linux-thread-multi/Cwd.pm
/u01/g01/grid/12.1/perl/lib/5.14.1/Cwd.pm
/u01/d01/db/12.1.0/perl/lib/5.14.1/x86_64-linux-thread-multi/Cwd.pm
/u01/em_agent/agent_13.2.0.0.0/perl/lib/5.14.4/x86_64-linux-thread-multi/Cwd.pm
[root_at_uat-db-002 29699255]#

Cheers
Sameer

-- 
This is an email from Ezidebit Pty Ltd. The email and any attachments may 
be confidential, legally privileged and/or subject to copyright. If you are 
not the intended recipient, you must not disclose or use the information 
contained in this e-mail. If you have received this email in error, please 
notify us immediately and delete the email and all copies. We do not 
guarantee that this email and any attachments are free from virus or other 
errors. We will not be responsible for loss or damage resulting (either 
directly or indirectly) from any such virus or error. We will only be 
responsible for a change to a document if we made the change. The content 
of and opinions expressed in non-business emails are not that of Ezidebit 
Pty Ltd and are that of the sender.


--
http://www.freelists.org/webpage/oracle-l

Received on Fri Oct 04 2019 - 05:40:58 CEST

This message: [ Message body ]
Next message: dimensional.dba_at_comcast.net: "RE: OPatchAuto failed"
Previous message: Jonathan Lewis: "Re: Troubleshooting ORA-12704 errors"
Next in thread: dimensional.dba_at_comcast.net: "RE: OPatchAuto failed"
Reply: dimensional.dba_at_comcast.net: "RE: OPatchAuto failed"

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

Original text of this message