Darwin IT

Subscribe to Darwin IT feed
Darwin-IT professionals do ICT-projects based on a broad range of Oracle products and technologies. We write about our experiences and share our thoughts and tips.Martien van den Akkerhttps://plus.google.com/110503432901891966671noreply@blogger.comBlogger316125
Updated: 12 hours 23 min ago

Replacement of environment variables or properties in Bash

Sun, 2016-09-25 10:47
Earlier I wrote about the automatic installation of Fusion Middleware components using response files. A thing that lacked in my scripts was that although I had a FMW_HOME variable set in my enviroment shell script, the response files had the location hard coded in them. At the time I hadn't had the chance to figure out how to do property/variable replacement in shell. I do know how to do it with ANT. But I figured that installing ANT for only this was a bit too much, since with the installation of FMW you already get ANT as a module.

For an upgrade of my scripts to FMW 12.2.1.1, I did a Google-search on it and found: http://stackoverflow.com/questions/415677/how-to-replace-placeholders-in-a-text-file. The top 2 suggestions were:

  1. sed -e "s/\${i}/1/" -e "s/\${word}/dog/" template.txt
  2. i=32 word=foo envsubst < template.txt
Although the first option was favoured by many and considered the answer on the querstion, I personally favour the second. It turns out that sed does not accept references to the environment variables as a replacement. And that makes the replacements hardcoded again. The second does accept environment variable references. Actually, if the variable-reference in the template file  is already present in the environment, no actual replacement assignment have to be provided.

So let's say my response file template looks like:

[ENGINE]

#DO NOT CHANGE THIS.
Response File Version=1.0.0.0.0

[GENERIC]

#Set this to true if you wish to skip software updates
DECLINE_AUTO_UPDATES=true

#
MOS_USERNAME=

#
MOS_PASSWORD=<SECURE VALUE>

#If the Software updates are already downloaded and available on your local system, then specify the path to the directory where these patches are available and set SPECIFY_DOWNLOAD_LOCATION to true
AUTO_UPDATES_LOCATION=

#
SOFTWARE_UPDATES_PROXY_SERVER=

#
SOFTWARE_UPDATES_PROXY_PORT=

#
SOFTWARE_UPDATES_PROXY_USER=

#
SOFTWARE_UPDATES_PROXY_PASSWORD=<SECURE VALUE>

#The oracle home location. This can be an existing Oracle Home or a new Oracle Home
ORACLE_HOME=${FMW_HOME}

#Set this variable value to the Installation Type selected. e.g. Fusion Middleware Infrastructure, Fusion Middleware Infrastructure With Examples.
INSTALL_TYPE=Fusion Middleware Infrastructure

#Provide the My Oracle Support Username. If you wish to ignore Oracle Configuration Manager configuration provide empty string for user name.
MYORACLESUPPORT_USERNAME=

#Provide the My Oracle Support Password
MYORACLESUPPORT_PASSWORD=<SECURE VALUE>

#Set this to true if you wish to decline the security updates. Setting this to true and providing empty string for My Oracle Support username will ignore the Oracle Configuration Manager configuration
DECLINE_SECURITY_UPDATES=true

#Set this to true if My Oracle Support Password is specified
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false

#Provide the Proxy Host
PROXY_HOST=

#Provide the Proxy Port
PROXY_PORT=

#Provide the Proxy Username
PROXY_USER=

#Provide the Proxy Password
PROXY_PWD=<SECURE VALUE>

#Type String (URL format) Indicates the OCM Repeater URL which should be of the format [scheme[Http/Https]]://[repeater host]:[repeater port]
COLLECTOR_SUPPORTHUB_URL=



Saved as 'fmw_12.2.1.1.0_infrastructure.rsp.tpl'; note the reference ORACLE_HOME=${FMW_HOME}. And I have set FMW_HOME with an fmw12c_env.sh script, as described in former posts. Then I only have to do:
envsubst < fmw_12.2.1.1.0_infrastructure.rsp.tpl >>fmw_12.2.1.1.0_infrastructure.rsp
To have the file copied to fmw_12.2.1.1.0_infrastructure.rsp with a replaced FMW_HOME variable:

...
#
SOFTWARE_UPDATES_PROXY_PASSWORD=<SECURE VALUE>

#The oracle home location. This can be an existing Oracle Home or a new Oracle Home
ORACLE_HOME=/u01/app/oracle/FMW12211

#Set this variable value to the Installation Type selected. e.g. Fusion Middleware Infrastructure, Fusion Middleware Infrastructure With Examples.
INSTALL_TYPE=Fusion Middleware Infrastructure
...

Couldn't be more simple, I'd say. Nice thing is that this enables me to do more directives. So, learned something again, from a question dated 7,5 years ago...

How to solve JSchException Algorithm negotiation fail. And get logging out of JSch in SoapUI.

Wed, 2016-09-07 05:43
I was so glad with my SoapUI solution to SFTP files to a server. But so dissapointed I couldn't have it working at my customer.

After I changed the log entries to log with e.message, I got the line:
Wed Sep 07 11:17:43 CEST 2016:INFO:JSchException Algorithm negotiation fail

Now I needed more information than that. But the hint is at least that there is a mismatch in the available cyphers client side verses server side.

So first I wanted to get more logging out of Jsch. It turns out that it has it's own Logger framework, but the bright side of that is that you can easily wrap your own logging mechanism. In the case of SoapUI it is log4j. So create a java project with the libraries jsch-0.1.54.jar and from the SoapUI libs: log4j-1.2.14.jar. Then I created the following class file from an example from the answer in this stackoverflow question.

My version is:
package nl.darwin.jsch.log;


import com.jcraft.jsch.Logger;

/**
* Class to route log messages generated by JSch to Apache Log4J Logging.
*
* @author mrabbitt
* @see com.jcraft.jsch.Logger
*/
public class JSchLog4JLogger implements Logger {
private org.apache.log4j.Logger logger;

/**
* Constructor with custom category name
*
* @param logger the logger from Apache Log4J.
*/
public JSchLog4JLogger(org.apache.log4j.Logger logger) {
this.logger = logger;
}

/**
* Default constructor
*/
public JSchLog4JLogger() {
this(org.apache.log4j.Logger.getLogger(Logger.class.getName()));
}

/* (non-Javadoc)
* @see com.jcraft.jsch.Logger#isEnabled(int)
*/
public boolean isEnabled(int level) {
switch (level) {
case DEBUG:
return logger.isDebugEnabled();
case INFO:
return logger.isInfoEnabled();
case WARN:
return logger.isInfoEnabled();
case ERROR:
return logger.isInfoEnabled();
case FATAL:
return logger.isInfoEnabled();
}
return false;
}

/* (non-Javadoc)
* @see com.jcraft.jsch.Logger#log(int, java.lang.String)
*/
public void log(int level, String message) {
switch (level) {
case DEBUG:
logger.debug(message);
break;
case INFO:
logger.info(message);
break;
case WARN:
logger.warn(message);
break;
case ERROR:
logger.error(message);
break;
case FATAL:
logger.fatal(message);
break;
}
}
}

Then Jar it and add it to the bin/ext older of SoapUI (like in the previous blog post).
Now a simple extra line is needed and an import in your groovy script :
import nl.darwin.jsch.log.JSchLog4JLogger
...
JSch.setLogger(new JSchLog4JLogger(log))
JSch ssh = new JSch()

So simply set the logger on the JSch class, before the instantiation. Then the logging of JSch appears in the SoapUI logging, as easy as that.
It turned out that the server required the use of aes256-ctr, while the jre of SoapUI (which is Java 7 in SoapUI 5.2.1) has limited JCE policy. As is suggested here.

You can download the unlimited JCE policies here:
JDK
Unlimited JCE Download
JDK 1.6http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.htmlJDK 1.7http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.htmlJDK 1.8http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
For SoapUI, download the JDK 1.7 policy. Go to your SoapUI Home folder, and navigate to the security library folder within the JRE. For instance: c:\Program Files\SmartBear\SoapUI-5.2.1\jre\lib\security.

Unzip the JCE to a new folder UnlimitedJCEPolicy within the security folder. Create a another backup folder like LimitedJCEPolicy and copy the jars US_export_policy.jar and local_policy.jar to the LimitedJCEPolicy folder. And copy the corresponding files from UnlimitedJCEPolicy to the security folder, replacing the original ones.

Restart SoapUI and you're good to go.


Use SoapUI to test your SFTP based services

Wed, 2016-09-07 04:13
SoapUI is my favorite tool to do unit tests. I'd try to keep my self to test based development and build up tests along with the development service. For SOAP or REST based services this goes quite intuitively using SoapUI. For database driven it is a little harder, but SoapUI has a nice JDBC activity, that supports DML as well as callable statements as stored procedures.

But for files and especially SFTP its a little more complicated. For a while I'm working on a filebased integration with SAP as source system.

I configured and defined the SOASuite FTP adapter to use my SSH user (oracle) on my VirtualBox VM. Until now I tested it using the SFTP/SCP client from MobaXTerm (enthousiastically recommended: download here). But not so handy for unit tests.

I wanted to automate this using SoapUI. With some searching I found that JCraft Java Secure Channel library was the best and easiest option. I did take a look at Apache Commons Net. But couldn't get it to work so easily. Download the jsch-0.1.54.jar (or newer) file and copy it to the ./bin/ext folder in your SoapUI home:


And restart SoapUI.

Create a new empty SoapUI project, create a TestSuite called something like 'Utilities' and a TestCase called 'TC-FTP'. Add the following properties to the TestCase:

Property
Value
ftpHostdarlin-vce-dbftpPort22ftpUsernameoracleftpPasswordwelcome1localFilePathd:/Projects/2016MySapProject/ExampleFiles/SAP-MESSAGE.XMLremoteFileDir/home/oracle/SapHR/in
In the TestCase add a Groovy Script called FTP add the script below. I took the example from snip2code.com (also found elsewhere) and refactered it to:
//Send Files to SSH Location
//
//Download jsch-0.1.54.jar from http://www.jcraft.com/jsch/ and copy it to SoapUI-Home/bin/ext location
//Example from: https://www.snip2code.com/Snippet/413499/SoapUI-Groovy-Script-compatible-SFTP-fil

//import org.apache.commons.net.ftp.FTPSClient
import com.jcraft.jsch.*
//
// Properties
//
def testCase = testRunner.testCase;


def String ftpHost = testCase.getPropertyValue("ftpHost")
def int ftpPort = testCase.getPropertyValue("ftpPort").toInteger()
def String ftpUsername = testCase.getPropertyValue("ftpUsername")
def String ftpPassword = testCase.getPropertyValue("ftpPassword")
def String localFilePath = testCase.getPropertyValue("localFilePath")
def String remoteFileDir = testCase.getPropertyValue("remoteFileDir")
//
//
Session session = null
Channel channel = null
try {
log.info("Starting sftp upload process")
JSch ssh = new JSch()

session = ssh.getSession(ftpUsername, ftpHost, ftpPort)
session.setConfig("StrictHostKeyChecking", "no"); //auto accept secure host
session.setPassword(ftpPassword)
session.connect()
log.info("Connected to session")

channel = session.openChannel("sftp")
channel.connect()
log.info("Connected to channel")

ChannelSftp sftp = (ChannelSftp) channel;
sftp.put(localFilePath, remoteFileDir);
log.info("File Uploaded " + localFilePath + " TO " + remoteFileDir)

} catch (JSchException e) {
e.printStackTrace()
log.info("JSchException " + e.message)
} catch (SftpException e) {
e.printStackTrace()
log.info("SftpException " + e.message)
} finally {
if (channel != null) {
channel.disconnect()
log.info("Disconnected from channel")
}
if (session != null) {
session.disconnect()
log.info("Disconnected from session")
}
log.info("sftp upload process complete")
}

Changes I did was to define the input values based on the properties from the testcase. And move the session and channel variable declartions out of the try, to get it accessible from the finally branch. And to replace e.printStackTrace from the logging by e.message, to have a propery message (e.printStackTrace returns null) in the logging.

The reason that I suggest to have it in a separate test cases is to enable it to be called from actual testcases with parameters. To do so add to your test case a call-test case activity:

Set the following properties:

Choose 'Run primary TestCase (wait for running to finish, Thread-Safe)' option as Run Mode.

And provide the property values.

This script copies a file from a file location and uploads it. But I want to be able to insert some runtime specific options to refer to in asserts and later JDBC calls. To check on specific running instances. So I want to be able to adapt the content running in my test case. Actually I want to upload a string fetched from a property, maybe with expanded properties.

So I copied the testcase and groovy activity and adapted the script to:
//Send Files to SSH Location
//
//Download jsch-0.1.54.jar from http://www.jcraft.com/jsch/ and copy it to SoapUI-Home/bin/ext location
//Example from: https://www.snip2code.com/Snippet/413499/SoapUI-Groovy-Script-compatible-SFTP-fil

//import org.apache.commons.net.ftp.FTPSClient
import com.jcraft.jsch.*
import java.nio.charset.StandardCharsets
//
// Properties
//
def testCase = testRunner.testCase;
//
def String ftpHost = testCase.getPropertyValue("ftpHost")
def int ftpPort = testCase.getPropertyValue("ftpPort").toInteger()
def String ftpUsername = testCase.getPropertyValue("ftpUsername")
def String ftpPassword = testCase.getPropertyValue("ftpPassword")
def String fileContent = testCase.getPropertyValue("fileContent")
def String remoteFile = testCase.getPropertyValue("remoteFile")
//
Channel channel = null
Session session = null
try {
log.info("Starting sftp upload process")
JSch ssh = new JSch()

session = ssh.getSession(ftpUsername, ftpHost, ftpPort)
session.setConfig("StrictHostKeyChecking", "no"); //auto accept secure host
session.setPassword(ftpPassword)
session.connect()
log.info("Connected to session")

channel = session.openChannel("sftp")
channel.connect()
log.info("Connected to channel")

ChannelSftp sftp = (ChannelSftp) channel;

byte[] fileContentBytes = fileContent.getBytes(StandardCharsets.UTF_8)
InputStream fileInputStream = new ByteArrayInputStream(fileContentBytes);
log.info("Start uploaded to " + remoteFile)
sftp.put(fileInputStream, remoteFile);
log.info("File Content uploaded to " + remoteFile)

} catch (JSchException e) {
e.printStackTrace()
log.info("JSchException " + e.message)
} catch (SftpException e) {
e.printStackTrace()
log.info("SftpException " + e.message)
} catch (Exception e) {
e.printStackTrace()
log.info("Exception " + e.message)
} finally {
if (channel != null) {
channel.disconnect()
log.info("Disconnected from channel")
}
if (session != null) {
session.disconnect()
log.info("Disconnected from session")
}
log.info("sftp upload process complete")
}

here the lines and related properties:
def String localFilePath = testCase.getPropertyValue("localFilePath")
def String remoteFileDir = testCase.getPropertyValue("remoteFileDir")

are changed to:
def String fileContent = testCase.getPropertyValue("fileContent")
def String remoteFile = testCase.getPropertyValue("remoteFile")
Then the lines:
  byte[] fileContentBytes =   fileContent.getBytes(StandardCharsets.UTF_8)
InputStream fileInputStream = new ByteArrayInputStream(fileContentBytes);

convert the fileContent property value to an InputString. And that is given as an input to the statement sftp.put(fileInputStream, remoteFile);. Notice that since we upload file content, we need to provide a remoteFile path, including file name, insead of a remote directory. And that we need an extra import java.nio.charset.StandardCharsets.

It would be nice if the guys from SmartBear add both put and get as a seperate activity.  

.... this is only a psuedo object?

Fri, 2016-09-02 06:00
Yesterday I was working on a BPEL project that I created before the summer holidays. I wanted to implement it further. But on first redeployment I ran into:
[12:18:01 PM] ----  Deployment started.  ----
[12:18:01 PM] Target platform is (Weblogic 12.x).
[12:18:01 PM] Running dependency analysis...
[12:18:01 PM] Building...
[12:18:08 PM] Deploying profile...
[12:18:09 PM] Wrote Archive Module to D:\Projects\2016DWN\SOASuite\HRApplication\DWN_CdmHR\trunk\SOA\DWN_CdmHR\CDMHRDomainService\deploy\sca_CDMHRDomainService.jar
[12:18:18 PM] Deploying sca_CDMHRDomainService.jar to partition "default" on server SoaServer1 [http://darlin-vce-db.darwin-it.local:8001]
[12:18:18 PM] Processing sar=/D:/Projects/2016DWN/SOASuite/HRApplication/DWN_CdmHR/trunk/SOA/DWN_CdmHR/CDMHRDomainService/deploy/sca_CDMHRDomainService.jar
[12:18:18 PM] Adding sar file - D:\Projects\2016DWN\SOASuite\HRApplication\DWN_CdmHR\trunk\SOA\DWN_CdmHR\CDMHRDomainService\deploy\sca_CDMHRDomainService.jar
[12:18:18 PM] Preparing to send HTTP request for deployment
[12:18:18 PM] Creating HTTP connection to host:darlin-vce-db.darwin-it.local, port:8001
[12:18:18 PM] Sending internal deployment descriptor
[12:18:19 PM] Sending archive - sca_CDMHRDomainService.jar
[12:18:19 PM] Received HTTP response from the server, response code=500
[12:18:19 PM] Error deploying archive sca_CDMHRDomainService.jar to partition "default" on server SoaServer1 [http://darlin-vce-db.darwin-it.local:8001]
[12:18:19 PM] HTTP error code returned [500]
[12:18:19 PM] Error message from server:
There was an error deploying the composite on SoaServer1: Deployment Failed: Error occurred during deployment of component: HREmployeeProcess to service engine: implementation.bpel, for composite: CDMHRDomainService: ORABPEL-05215

Error while loading process.
The process domain is encountering the following errors while loading the process "HREmployeeProcess" (composite "default/CDMHRDomainService!1.0*soa_6e4206b5-3297-4f53-9944-734349aed8ab"): this is only a psuedo object.
This error contained an exception thrown by the underlying process loader module.
Check the exception trace in the log (with logging level set to debug mode). If there is a patch installed on the server, verify that the bpelcClasspath domain property includes the patch classes.
.

[12:18:19 PM] Check server log for more details.
[12:18:19 PM] Error deploying archive sca_CDMHRDomainService.jar to partition "default" on server SoaServer1 [http://darlin-vce-db.darwin-it.local:8001]
[12:18:19 PM] Deployment cancelled.
[12:18:19 PM] ---- Deployment incomplete ----.
[12:18:19 PM] Error deploying archive file:/D:/Projects/2016DWN/SOASuite/HRApplication/DWN_CdmHR/trunk/SOA/DWN_CdmHR/CDMHRDomainService/deploy/sca_CDMHRDomainService.jar
(oracle.tip.tools.ide.fabric.deploy.common.SOARemoteDeployer)

So the I was googling around, and found this blog entry. This one suggested a missmatch between the project and referenced wsdl's/xsd's in the MDS.

So I refreshed the MDS, restarted the whole SOA Server, but no luck.

At the doorstep of removing the lot of components and references, I decided to take a last closer look to the composite.xml.

The BPEL process component HREmployeeProcess had a reference to the service HREmployeeProcessSubscriber. The latter was based on a wsdl in the mds:
  <reference name="HREmployeeProcessSubscriber"
ui:wsdlLocation="oramds:/apps/CDM/services/domain/operations/hrm/v2/EmployeeDomainEntityEventService.wsdl">
<interface.wsdl interface="http://hhs.nl/services/domain/operations/hrm/v2/#wsdl.interface(EmployeeDomainEntityEventServicePortType)"/>
<binding.ws port="http://hhs.nl/services/domain/operations/hrm/v2/#wsdl.endpoint(hremployeeprocessa_client/EmployeeDomainEntityEventServicePort)"
location="http://darlin-vce-db:8001/soa-infra/services/default/HRSubscriberA/HREmployeeEventServiceA?WSDL"
soapVersion="1.1"/>
</reference>
But in the reference in the bpel component it refered to the BPEL process on the server:
<reference name="HREmployeeProcessSubscriber"
ui:wsdlLocation="http://darlin-vce-db:8001/soa-infra/services/default/HRSubscriberA/HREmployeeEventServiceA?WSDL">
<interface.wsdl interface="http://hhs.nl/services/domain/operations/hrm/v2/#wsdl.interface(EmployeeDomainEntityEventServicePortType)"/>
</reference>

Since the wsdl defined in the ui:wsdlLocation attribute needs to be available on compiling and loading of the component by the component-engine it's recommended to use a reference to an abstract wsdl in the mds. In this case I replaced the ui:wsdlLocation in the service reference by the mds. But apparently I forgot the BPEL Comnponent. To replace that, you should do this in the partnerlink definition in the BPEL Process. Because the composite.xml is automatically updated. Because the abastract wsdl lacks the partnerlink types, as you might know, JDeveloper suggests to create a wrapper wsdl for you.

Now, because of the synchronizations between bpel and the composite, you might need to hack the composite and the bpel process, to get thinks consistent again (at least I had to). But then, having it resolved, the composite was deployable again... And the BPEL process wasn't so pseudo anymore.

The third one on Creating weblogic user, now for SOA Suite

Thu, 2016-09-01 08:43
A few months ago I figured out how to create specific users with restricted access to Service Bus components. I blogged about it in part 2 of creating WebLogic users for ServiceBus 12c. But the series lacks an explanation on restricted user access on SOASuite.

Today in a question about Roles on Oracle Community Forums, the reference to this elaborate blog entry was given: Restricted View, by Antony Reynolds.

I think that blog explains it well. Unfortunately the link to 7.2 Partition Roles, Anthony mentioned, did not work. What I found (12.1.3) is 7.3 Securing Access to Partitions (12.1.3) and 7.3 Securing Access to SOA Folders (12.2.1). (Apparently from 12.2.1 onwards, partitions are called SOA folders...)




Mount NTFS disk in Linux 7

Fri, 2016-08-26 06:03
Today I wanted to pass an old disk in a usb-case to my son. It was from an old Windows Laptop and even though I'm administrator, I wasn't able to read the documents in an other user's folder.

So I thought, let's do it from an Oracle Linux 7 VM, as root. But it turns out that Oracle linux did not support NTFS by default.

But with the trick in this link I managed to do it.

To sum up, especially for my self:

Add the EPEL-7 repository from Fedora:
# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

# rpm -ivh epel-release-latest-7.noarch.rpm

This worked for me. Although I had to step over my issue that I add a Fedora repo to Oracle Linux.... An other option could have been (probably if under Redhat Linux or CentOS:
# yum install epel-release
...
# yum clean all
...
# yum update
...

Install the NTFS-3g package:
# yum install ntfs-3g -y

And enable NTFS support for FileManagers:
# yum install ntfsprogs -y

Then I got to the FileManager and I could browse the disk. It was mounted at '/run/media/oracle/803638DB3638D3BE'.

Generate Admin Channels to improve Weblogic Admin Console performance (and of FMW-Control)

Wed, 2016-08-24 11:02
At one of my customers we have quite an impressive domain configuration. It's a FMW domain with SOA, OSB, BAM, WSM, MFT in clusters of 4 nodes. The thing is that when having started all the servers, the console becomes slooooooowwwww. Not to speak of FMW Control (em).

One suggestion is to set the 'Invocation Timeout Seconds' under MyDomain->Configuration->General->Advanced to a value like 2. And 'Management Operation Timeout' under Preferences->Shared Preferences to a value like 5:

This surely makes the console responsive again. But it actually means that the console gives up right away when querying for the (health) state of the servers. So in stead of a health of 'OK', you get a 'server unreachable' message.

When having a lot of servers in the domain, they all share the same Admin Channel, and this seams to get over flooded. AdminServer does not get the responses in time. Sometimes a new request leads to a proper response, but in fact it takes a lot of time.

To reduce the load on the default channel, we created a admin server per managed server. Since it's a lot of servers, and we need to do it on several environments, so I created a wlst-script for it.
The script createAdminChannels.py:
#############################################################################
# Create AdminChannels for WebLogic Servers
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-08-24
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7001"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToadminServer(adminUrl, adminServerName):
try:
print(lineSeperator)
print('Try to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
except WLSTException:
message='Apparently AdminServer not Started!'
print (message)
raise Exception(message)
#
#
def createAdminChannel(serverName, adminListenPort):
print(lineSeperator)
channelName = serverName+'-AdminChannel'
try:
cd('/Servers/'+serverName+'/NetworkAccessPoints/'+channelName)
print('Channel '+channelName +' for '+serverName+' already exists.')
except WLSTException:
try:
print('Create Admin Channel for server: '+serverName+', with port: '+adminListenPort)
cd('/Servers/'+serverName)
cmo.createNetworkAccessPoint(channelName)
cd('/Servers/'+serverName+'/NetworkAccessPoints/'+channelName)
cmo.setProtocol('admin')
cmo.setListenPort(int(adminListenPort))
cmo.setEnabled(true)
cmo.setHttpEnabledForThisProtocol(true)
cmo.setTunnelingEnabled(false)
cmo.setOutboundEnabled(false)
cmo.setTwoWaySSLEnabled(false)
cmo.setClientCertificateEnforced(false)
print ('Succesfully created channel: '+channelName)
except WLSTException:
apply(traceback.print_exception, sys.exc_info())
message='Failed to create channel '+channelName+'!'
print (message)
raise Exception(message)
#
#
def main():
try:
print (lineSeperator)
print ('Start Osb Cluster')
print (lineSeperator)
print('\nConnect to AdminServer ')
connectToadminServer(adminUrl, adminServerName)
print(lineSeperator)
print('Start Edit Session')
edit()
startEdit()
#
#Create Admin Channels
# Administrators
print('\nCreate Admin Channels')
serverNameList=serverNames.split(',')
serverAdminPortList=serverAdminPorts.split(',')
#
idx=0
for serverName in serverNameList:
adminPort=serverAdminPortList[idx]
createAdminChannel(serverName, adminPort)
idx=idx+1
# Activate changes
print(lineSeperator)
print('Activate Changes')
save()
activate(block='true')
#
print('\nExiting...')
exit()
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#call main()
main()
exit()


The shell script to call it, createAdminChannels.sh:
#!/bin/bash
#############################################################################
# Create AdminChannels
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-08-24
#
#############################################################################
#
. fmw12c_env.sh
export PROPERTY_FILE=$1
echo
echo Create Admin Channels
wlst.sh ./createAdminChannels.py -loadProperties $PROPERTY_FILE

And the example property file, darlin-vce-db.properties:
#############################################################################
# Properties voor Creeëren SOADomain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.0, 2016-04-15
#
#############################################################################
#
# Properties for AdminServer
adminServerName=Adminserver
adminUrl=darlin-vce-db:7001
# AdminUser
adminUser=weblogic
adminPwd=welcome1
#
serverNames=AdminServer,OsbServer1,SoaServer1
serverAdminPorts=7101,8111,8101
#

Call the script as $> createAdminChannels.sh darlin-vce-db.properties

In the property file you'll need to name every server in the property serverNames. And for each server the particular Admin Listen Port in serverAdminPorts, in the exact same order. Start with the AdminServer.


At the end of the script the changes are activated and then the AdminServer listens over https on the changed port.

Important: the servers need to be down, except for the the AdminServer.

Unfortunately the infrastructure database was apparently down. So I haven't been able to start SOA, BAM, etc. to see if it is performant now. But I have high hopes...

Unable to login with a SQL Authenticator

Wed, 2016-08-24 06:25
For a project, we are migrating Forms to ADF.
There is also a number of reports which are not to be migrated yet.
Therefore, we need to keep the users in the database.
As we do not want to maintain two user stores, we thought it to be a good idea to create an authenticator in WebLogic to authenticate to the database.
There are loads of blog posts / support notes on how to configure a SQL Authenticator, so I won't repeat this procedure.
Take a look at Oracle Support Document 1342157.1 or a post from Edwin Biemond


However, we noticed a flaw in these (old) references.


In WebLogic 12c, when we create a SQL Authenticaotr, there is a filed named Identity Domain.
From the Oracle documentation, we learn:
All Authentication providers included in WebLogic Server support identity domains. If the identity domain attribute is set on an Authentication provider, that Authentication provider can authenticate only users who are defined in that identity domain.
...
An identity domain is a logical namespace for users and groups, typically representing a discrete set of users and groups in the physical datastore. Identity domains are used to identify the users associated with particular partitions.


As we do not use partitions in our domain, there is no use for an Identity domain.
But we did not know that when we setup the authenticator (and who reads the entire manual right??)


So following the previously mentioned resources, we created the SQL authenticator and entered the domain name in the Identity Domain field.
This resulted in a not working authenticator.
Symptoms:
  • When a user tried to login using database credentials, the autentication faild Always
  • No error message in the log
  • No activity in the datase (no query was executed to check the credentials)
To further analyse the issue, we added som Java options to the Admin server, using the setDomainEnv script:
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.kernel.debug=true -Dweblogic.log.StdoutSeverity=Debug -Dweblogic.log.LogSeverity=Debug -Dweblogic.StdoutDebugEnabled=true -Dweblogic.log.LoggerSeverity=Debug -Dweblogic.debug.DebugSecurityAtn=true" 
export JAVA_OPTIONS

This gave us more insight in the issue. The log file now revealled:
####<23-aug-2016 15:13:02 uur CEST> <Debug> <SecurityAtn> <BSCC6112> <DefaultServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <d489addb-c960-41f1-babb-f912aec329bc-00000036> <1471957982959> <[severity-value: 128] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000000> <weblogic.security.providers.authentication.shared.DBMSAtnLoginModuleImpl.login exception: 
java.security.PrivilegedActionException: javax.security.auth.login.LoginException: javax.security.auth.callback.UnsupportedCallbackException: Unrecognized Callback: class weblogic.security.auth.callback.IdentityDomainUserCallback
weblogic.security.auth.callback.IdentityDomainUserCallback@31789514
at java.security.AccessController.doPrivileged(Native Method)
at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
...


Unfortunately, we still had no clue.
As you have read from the beginning of the post, you might think that setting the Identity Domain field to DOMAIN might solve the issue (the log states partition-name: DOMAIN) but no, that's no solution either.

The trick is to leave the Identity Domain filed blank. After that the authenticator worked like a charm.

Set connection retry frequency on DataSource in WebLogic 12c.

Thu, 2016-07-14 11:45
I have encountered several times in WebLogic 12c that when the ConnectionPool of a DataSource in WebLogic could not be initialized because due to a connection error or invalid username or password, the server could not be started.

I can't remember having encounterd this problem in 11g, and this week I struggled with it with one of my customers. The perception of the DBA there was that in 11g the server did start up, but the DataSource would go in Suspended-state. Half a year ago, one of the admins just removed a datasource because it made the 12c WebLogic server unstartable. "Taking a short turn", we would say in Dutch.

Now, having a database that is down is a reality at my customers. At least in development and test environments. Also at some of my customers it is a reality to have databases refreshed with an earlier clone. Causing for instance database passwords becoming invalid. It's quite inconvenient not being able to start the servers. Especially when the AdminServer can't be started because of it. And with SOASuite and OSB this is a reality since some of the consoles and composers are targeted to the AdminServer.

This week I found there were 2 options to get the server started:
  • Temporarily untarget the datasource
  • Set the initial and minimum connections to 0. 
The first was acceptable for one of my customers because they have just one cluster to which the problematic datasources was targeted to. Untargeting would leave all the other settings. And retargetting doesn't raise questions because there's only one target option.

But in a complex domain with several clusters it might not be too obvious to which cluster(s) or server(s) the datasource should be retargeted.

Changing the initial/minimum connections is not ideal either. Because you need to remember what the preferred settings were. These are important when using them for web applications or services used by web applications where performance is key.

But today I stumbled upon a third option: 'Connection Creation Retry Frequency' which can be found under DataSource -> Configuration -> Connection Pool -> Advanced:

Set this to 300 to have a retry every 5 minutes. It is described as:
The number of seconds between attempts to establish connections to the database.

If you do not set this value, data source creation fails if the database is unavailable. If set and if the database is unavailable when the data source is created, WebLogic Server will attempt to create connections in the pool again after the number of seconds you specify, and will continue to attempt to create the connections until it succeeds.

When set to 0, connection retry is disabled.'

Learning all the time...



Start and stop a WebLogic (SOA/OSB) Domain

Thu, 2016-07-14 04:24
So let's start the day with a blog. In the past few months I created scripts to install FMW products and build a WebLogic domain for it. For most of my findings I did a blog already:
And then I did not list the blogs about the scripts to setup and patch the QuickStarts. Nice  to mention is that this week 12.2.1.1 is released. So I can adapt my scripts with the new software (I'm not going to post that, you should be able to figure that out yourself, maybe I'll provide the changed snippets when I come to it).

But the important missing part in this series are the start and stop of the domain. So let me provide and describe my scripts.

Oracle does provide scripts to start and stop your NodeManager and WebLogic servers. When you configure the NodeManager and Servers correctly you can use those to add the NodeManager to the init.d services on Linux or to your Windows Services. Then the NodeManager would start the AdminServer and other Servers automatically.

However, on my demo/training/development systems I'd like to be able to start/stop the parts myself.  So here we go.

SettingsI'm working with Linux and create shell scripts to kick-off the wlst scripts that I use to start the particular components. Therefor I use a settings script called fmw12c_env.sh to call the setWLSEnv.sh from the FMW home. I take it that you're able to translate them to Windows Command file if applicable. I provided it a couple of time already. But for completeness here it is:
#!/bin/bash
echo set Fusion MiddleWare 12cR2 environment
export FMW_HOME=/u01/app/oracle/FMW12210
export NODEMGR_HOME=/u01/app/work/domains/osb_domain/nodemanager

export SOA_HOME=$FMW_HOME/soa
export OSB_HOME=$FMW_HOME/osb
export MFT_HOME=$FMW_HOME/mft
#
echo call setWLSEnv.sh
. $FMW_HOME/wlserver/server/bin/setWLSEnv.sh
export PATH=$FMW_HOME/oracle_common/common/bin:$WL_HOME/common/bin/:$WL_HOME/server/bin:$PATH

Then the wlst scrips need a property file fmw.properties:
#############################################################################
# Set SOABPM Domain properties
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.0, 2016-06-27
#
#############################################################################
#
#Per domain nodemanager...
#############################################################################
# Set SOABPM Domain properties
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.0, 2016-06-27
#
#############################################################################
#
#Per domain nodemanager...
#nmHome=/u01/app/work/domains/osb_domain/nodemanager
nmHome=/u01/app/work/domains/soabpm_domain/nodemanager
#nmHome=/u01/app/work/domains/soabpm12c_dev/nodemanager
nmHost=darlin-vce-db
nmPort=5555
nmType=plain
#
#Domain
#wlsDomainName=osb_domain
#wlsDomainName=soabpm12c_dev
wlsDomainName=soabpm_domain
wlsDomainsHome=/u01/app/work/domains
#
#AdminServer
adminServerName=AdminServer
#adminServerName=OsbAdmin
adminUrl=darlin-vce-db:7001
adminUser=weblogic
adminPwd=welcome1
usrCfgFile=wlsCfgFile
usrKeyFile=wlsKeyFile
#Clusters
osbClr=OsbCluster
soaClr=SoaCluster

Create User Config and Key filesIn the property file above you'll see the adminUser and adminPwd (password). On a production environment you probably don't want to have those in plain sight. WebLogic provides a means to encrypt those in to a user config and a key file. When you generate those with the default names to your home folder you even can connect to the AdminServer without providing the username/password. In my scripts I use named files as defined in the usrCfgFile and usrKeyFile properties. But first we need to generate those and for that we need to connect to the AdminServer using the adminUser and adminPwd properties.

I created the script createUserConfigFiles.py to generate those files:
#############################################################################
# Stop AdminServer
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = 'stopAdmin.py'
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def main():
try:
print(lineSeperator)
print('Create user config files')
print(lineSeperator)
print('\nConnect to the AdminServer: '+adminServerName)
connect(adminUser, adminPwd, adminUrl)
#
print('\nStore Config files')
storeUserConfig(usrCfgFile,usrKeyFile)
#
print('\nExiting...')
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#
main();

After connecting to the AdminServer it is the storeUserConfig(usrCfgFile,usrKeyFile) that creates the files. If you don't supply the parameters it will create the default files in your home folder (for instance /home/oracle), then you can connect just by issueing connect('localhost:7001'). For more on this, read this blog.

To call the script above you can use the script createUserConfigFiles.sh or the contents of it:
#!/bin/bash
#############################################################################
# Create user config files using wlst.sh
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
#
. fmw12c_env.sh
echo
echo Create User Config files
wlst.sh ./createUserConfigFiles.py -loadProperties fmw.properties

After having greated the files you can (and should!) remove the adminUser and adminPwd properties from the fmw.properties file.
Start the NodeManager In de domain home/bin folder (for instance /u01/app/work/domains/soabpm_domain/bin) you'll find scripts to start and stop the NodeManager: startNodeManager.sh and stopNodeManager.sh. These are fine to use in the service configurations in Linux or Windows.

However, running them from the shell/command line will cause your session to be taken for the NodeManager. Stopping the session will cause the NodeManager to be stopped. Not so handy if you want to start the NodeManager in an overall script that starts the complete domain.

But you can start the NodeManager from wlst as well. And then it spawns the NodeManager into a seperate process in  the background.
The script startNodeManager.py is quite simple:
#############################################################################
# Start nodemanager
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def startNM(nmHost, nmPort, nmHome, nmType):
print(lineSeperator)
print ('Start NodeManager')
startNodeManager(verbose='true', NodeManagerHome=nmHome, ListenPort=nmPort, ListenAddress=nmHost);
print('Finished starting NodeManager')
#
def main():
try:
startNM(nmHost, nmPort, nmHome, nmType);
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#
main()

It's mainly about the startNodeManager(verbose='true', NodeManagerHome=nmHome, ListenPort=nmPort, ListenAddress=nmHost) statement in the  startNM() function, which is quite self explaining, I think.  No username password is required to start the NodeManager, but it does need to know what it's home and listenPort and Address should be.

As with all the scripts I  declare a scripName variable with the assignment of the 0-th argument: sys.argv[0]. This is used in the usage() function that is called in case of a NameError exception. This exception is raised when certain variable or function references are made but not found. Normally this would mean that a certain property is not set in the property file.

To run it from the shell you can use the startNodeManager.sh script:

#!/bin/bash
#############################################################################
# Start nodemanager using wlst
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
#
. fmw12c_env.sh
echo
echo Start NodeManager
wlst.sh ./startNodeManager.py -loadProperties fmw.properties
For every python script in the procession of this blog I have an accompanying shell script like this. So I won't provide those for the other scripts.

Stopping the NodeManager is a little more complicated:
#############################################################################
# Stop nodemanager
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType):
try:
print(lineSeperator)
print('Try to connect to the Node Manager')
try:
nmConnect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the NodeManager adminUser and adminPwd properties')
nmConnect(username=adminUser, password=adminPwd, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType)
print('Connected to the Node Mananger')
except WLSTException:
message='Apparently NodeManager not Started!'
print (message)
raise Exception(message)
#
#
def stopNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType):
try:
print(lineSeperator)
print ('Try to connect to the Node Manager')
connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType);
print ('Connected to the Node Mananger')
print ('Stop the NodeManager')
stopNodeManager();
print('Finished stapping NodeManager')
except WLSTException:
print ('Apparently NodeManager not Started!')
#
def main():
try:
wlsDomainHome = wlsDomainsHome+'/'+wlsDomainName
stopNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType);
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#
main()

You first need to connect to the NodeManager. This is done in connectToNM() that illustrates the use of the user config and key files with the adminUser/adminPwd properties as a fall back. First it tries to connect using the userConfigFile/userKeyFile combination. If that leads to a NameError, then you probably did not provide those, so it retries it on with the adminUser/adminPwd properties.
This construct is reused when trying to connect to the AdminServer to start the rest of the domain.
When it is connected the nodeManager can be stopped, simply with the stopNodeManager() command.
Start/Stop the AdminServerThe startAdmin.py provided below is used to Start the Admin Server. Again, there are  startWeblogic.sh and stopWeblogic.sh scripts in the bin folder of the domain. But those will start the server in the foreground (although you can move them to the background of course). But mainly the servers aren't under control of the NodeManager, what has it's advantages. So the startAdmin.py script:
#############################################################################
# Start AdminServer via NodeManager and try to connect to it.
# Result of this script should be that you are connect to the AdminServer.
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType):
try:
print(lineSeperator)
print('Try to connect to the Node Manager')
try:
nmConnect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the NodeManager adminUser and adminPwd properties')
nmConnect(username=adminUser, password=adminPwd, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType)
print('Connected to the Node Mananger')
except WLSTException:
message='Apparently NodeManager not Started!'
print (message)
raise Exception(message)
#print 'Start Nodemanager'
#startNodeManager(verbose='true', NodeManagerHome=nmHome, ListenPort=nmPort, ListenAddress=nmHost);
#print 'Retry to connect to the Node Manager';
#nmConnect(username=adminUser, password=adminPwd, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType);
#
#
def connectToadminServer(adminUrl, adminServerName):
try:
print(lineSeperator)
print('Try to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
except WLSTException:
print('Apparently AdminServer not Started!')
print('Start AdminServer')
nmStart(adminServerName)
print('Retry to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
#
#
def main():
try:
wlsDomainHome = wlsDomainsHome+'/'+wlsDomainName
print(lineSeperator)
print('Start '+adminServerName+' for domain in : '+wlsDomainHome)
print(lineSeperator)
print ('Connect to the Node Manager')
connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType)
print ('Connect to the AdminServer: '+adminServerName)
connectToadminServer(adminUrl, adminServerName)
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
exit(exitcode=1)
#
main();

The script first uses connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType) to connect to the NodeManager. Just in the same way as in the stopNodeManager.py script.
Then it tries to connect to the AdminServer using connectToadminServer(adminUrl, adminServerName). Now, the connect to the NodeManager might be a little over done. But in the connectToadminServer() function it tries to connect to the AdminServer the same way as to the NodeManager: first with the userConfigFile, userKeyFile combination. But when those properties aren't set, it uses the username password combination. If a connection to the AdminServer fails, the function concludes that it isn't started yet. But since it is still connected to the nodemanager it can start the AdminServer using nmStart(adminServerName). So it is important that in the adminServerName property in the fmw.properties the correct name of the AdminServer is given.

To stop the admin server I have the stopAdmin.py script:
#############################################################################
# Stop AdminServer
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToadminServer( adminUrl, adminServerName):
try:
print(lineSeperator)
print('Try to connect to the AdminServer using user config')
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
except WLSTException:
message='Apparently AdminServer not Started!'
print (message)
raise Exception(message)
#
#
def main():
try:
wlsDomainHome = wlsDomainsHome+'/'+wlsDomainName
print(lineSeperator)
print('Stop '+adminServerName+' for domain in : '+wlsDomainHome)
print(lineSeperator)
print('\nConnect to the AdminServer: '+adminServerName)
connectToadminServer(adminUrl, adminServerName)
#
print('\nShutdown the AdminServer: '+adminServerName)
shutdown(force='true')
#
print('\nFinished stopping AdminServer: '+adminServerName)
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
exit(exitcode=1)
#
main();

This one does a connect to the AdminServer (in the same two layer approach), and when connected it performs a shutdown(force='true'). This is equivalant to the 'Force Shutdown Now' option in the console. Since no server or cluster name is provided the AdminServer is shutdown.
Start&Stop DomainIn my create domain scripts I create clusters for every component. Even when I have only one managed server (for instance in a development environment) I added the servers to a cluster. This makes it easier to expand the domain with multiple nodes. And it makes the management the same for every environment.

So to start and stop the domain I work per cluster. The nice thing then is that when connected you can just issue a start cluster command and the AdminServer uses the different nodeManagers on the hosts to start the servers simultaneously. This reduces the startup time significantly compared to starting the servers of a cluster one by one.

 Now, it might be that one of the servers couldn't be started right away. And this fails the start cluster command. So I added a little code to try to start the different servers one by one. I think it is somewhat over done in most cases, but it does illustrate some functions to figure out the servers to start and to determine their state. So here is my startDomain.py script:
#############################################################################
# Start SOA and OSB domain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7001"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType):
try:
print(lineSeperator)
print('Try to connect to the Node Manager')
try:
nmConnect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the NodeManager adminUser and adminPwd properties')
nmConnect(username=adminUser, password=adminPwd, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType)
print('Connected to the Node Mananger')
except WLSTException:
message='Apparently NodeManager not Started!'
print (message)
raise Exception(message)
#print 'Start Nodemanager'
#startNodeManager(verbose='true', NodeManagerHome=nmHome, ListenPort=nmPort, ListenAddress=nmHost);
#print 'Retry to connect to the Node Manager';
#nmConnect(username=adminUser, password=adminPwd, host=nmHost, port=nmPort, domainName=wlsDomainName, domainDir=wlsDomainHome, nmType=nmType);
#
#
def connectToadminServer(adminUrl, adminServerName):
try:
print(lineSeperator)
print('Try to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
except WLSTException:
print('Apparently AdminServer not Started!')
print('Start AdminServer')
nmStart(adminServerName)
print('Retry to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
#
# Get the Servers of Cluster
def getClusterServers(clustername):
#Cluster config to be fetched from ServerConfig
print(lineSeperator)
print('\nGet Servers from cluster '+clustername)
serverConfig()
cluster = getMBean("/Clusters/" + clustername)
if cluster is None:
errorMsg= "Cluster " + clustername + " does not appear to exist!"
print errorMsg
raise(Exception(errorMsg))
print "Found cluster "+ clustername+ "."
servers = cluster.getServers()
return servers
#
#
def serverStatus(serverName):
serverRuntime=getMBean('/ServerRuntimes/'+serverName)
if serverRuntime is None:
print("Server Runtime for " + serverName + " not found, server apparently SHUTDOWN")
serverState="SHUTDOWN"
else:
print "Found Server Runtime for "+ serverName+ "."
serverState = serverRuntime.getState()
return serverState
#
#
def startClusterServersOneByOne(clusterName):
print(lineSeperator)
print ('Start servers for cluster: '+clusterName)
servers=getClusterServers(clusterName)
# Need to go to domainRuntime to get to the serverRuntimes.
domainRuntime()
#
for server in servers:
print(lineSeperator)
serverName = server.getName()
print('ServerName: '+serverName)
serverState = serverStatus(serverName)
print('Server '+serverName+': '+serverState)
if serverState=="SHUTDOWN":
print ('Server '+serverName+' is not running so start it.')
start(serverName)
elif serverState=="RUNNING":
print ('Server '+serverName+' is already running')
else:
print ('Server '+serverName+' in state '+serverState+', not startable!')
#
print ('\nFinished starting servers.')
#
#
def startClusterServers(clusterName):
print(lineSeperator)
print ('Start servers for cluster: '+clusterName)
#
try:
start(clusterName,'Cluster')
except WLSTException:
print "Apparently Cluster in incompatible state!", sys.exc_info()[0], sys.exc_info()[1]
startClusterServersOneByOne(clusterName)
state(clusterName,'Cluster')
#
print ('\nFinished starting servers.')
#
#
def main():
try:
wlsDomainHome = wlsDomainsHome+'/'+wlsDomainName
print (lineSeperator)
print ('Start Osb Cluster')
print('\nConnect to AdminServer ')
print (lineSeperator)
print ('Connect to the Node Manager')
connectToNM(nmHost, nmPort, nmHome, wlsDomainName, wlsDomainHome, nmType)
print ('Connect to the AdminServer: '+adminServerName)
connectToadminServer(adminUrl, adminServerName)
#
print('Start servers for cluster: '+osbClr)
startClusterServers(osbClr)
#
print('Start servers for cluster: '+soaClr)
startClusterServers(soaClr)
#
print('\nExiting...')
exit()
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
exit(exitcode=1)
#call main()
main()
exit()

It first connects to the AdminServer in the familiar way. If that fails it tries to start it. So far it is copied from the startAdmin.py script. Then it calls the startClusterServers(clusterName) function for each cluster. You need to add each cluster here by hand. I could loop over the clusternames, but you might need to have them started in a certain order. This function first issues the start(clusterName,'Cluster') command. Providing the 'Cluster' parameter as a start type, a complete cluster can be started. If that fails, then apparently one or more servers failed to start. If so it tries to start the servers one by one in the startClusterServersOneByOne(clusterName) function. It gets the servers attached to the cluster using getClusterServers(clustername) function. This is done using the getServers() method of the cluster. This needs to be done in the serverConfig() state. After that for each server the status is fetched by querying the serverRuntime of the server in the serverStatus() function. ServerRuntimes are available in the DomainRuntime() state, so it is needed to switch the state using that function. If the ServerRuntime is not avaiable, the state is considered to be 'SHUTDOWN'. If the server is down a start(serverName) is issued. Lastly in the startClusterServers() function the state(clusterName,'Cluster') is called to print the state of the servers in the cluster.

To Stop the domain I created the stopDomain.py which mainly works in reverse:
#############################################################################
# Stop OSB Domain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-06-27
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7001"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToadminServer(adminUrl, adminServerName):
try:
print(lineSeperator)
print('Try to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
except WLSTException:
message='Apparently AdminServer not Started!'
print (message)
raise Exception(message)
#
# Get the Servers of Cluster
def getClusterServers(clustername):
#Cluster config to be fetched from ServerConfig
print(lineSeperator)
print('\nGet Servers from cluster '+clustername)
serverConfig()
cluster = getMBean("/Clusters/" + clustername)
if cluster is None:
errorMsg= "Cluster " + clustername + " does not appear to exist!"
print errorMsg
raise(Exception(errorMsg))
print "Found cluster "+ clustername+ "."
servers = cluster.getServers()
return servers
#
#
def serverStatus(serverName):
serverRuntime=getMBean('/ServerRuntimes/'+serverName)
if serverRuntime is None:
print("Server Runtime for " + serverName + " not found, server apparently SHUTDOWN")
serverState="SHUTDOWN"
else:
print "Found Server Runtime for "+ serverName+ "."
serverState = serverRuntime.getState()
return serverState
#
#
def stopClusterServers(clusterName):
print(lineSeperator)
print ('Stop servers for cluster: '+clusterName)
#
try:
shutdown(clusterName,'Cluster')
except WLSTException:
print "Apparently Cluster in incompatible state!", sys.exc_info()[0], sys.exc_info()[1]
state(clusterName,'Cluster')
print ('Try to stop servers for cluster: '+clusterName+', one by one')
servers=getClusterServers(clusterName)
# Need to go to domainRuntime to get to the serverRuntimes.
domainRuntime()
#
for server in servers:
print(lineSeperator)
serverName = server.getName()
print('ServerName: '+serverName)
serverState = serverStatus(serverName)
print('Server '+serverName+': '+serverState)
if serverState=="RUNNING":
print ('Server '+serverName+' is running so shut it down.')
shutdown(name=serverName, force='true')
elif serverState=="SHUTDOWN":
print ('Server '+serverName+' is already down.')
else:
print ('Server '+serverName+' in state '+serverState+', not stoppable!')
#
print ('\nFinished stopping servers.')
#
#
def main():
try:
wlsDomainHome = wlsDomainsHome+'/'+wlsDomainName
print (lineSeperator)
print ('Stop Osb Domain')
print (lineSeperator)
print('\nConnect to the AdminServer: '+adminServerName)
connectToadminServer(adminUrl, adminServerName)
print(lineSeperator)
print('First stop servers from cluster '+osbClr)
stopClusterServers(osbClr)
#
print(lineSeperator)
print('\nShutdown the AdminServer: '+adminServerName)
shutdown(force='true')
print ('\nFinished stopping servers.')
#
print('\nExiting...')
exit()
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
exit(exitcode=1)
#call main()
main()
exit()

In the sameway as in starting the servers this script first tries to shutdown the cluster as a whole. The AdminServer takes advantage of the NodeManager to simulaneously stop the servers. The NodeManager first tries to connect to the particular server to hand over a suicide pill. But if it can't reach the server directly it releases it from suffering by doing an explicit OS-Level kill process...
After the shutdown of the servers the AdminServer is shut down.
ConclusionThis concludes this article on starting and stopping your domain. I hope it was enlighting and entertaining. One improvements  would be to reuse the scripts on starting and stopping the AdminServer in the starting and stopping of the Domain. You see that there is some code-overlap. But the way I call them with an explicit property file prevents doing a execfile(). But the properties loaded aren't passed to the execfile() apparently. And it needs a path to the other file. So that's why I for now combined the code into one script.

Create WebLogic users for ServiceBus 12c - Part 2

Wed, 2016-07-13 14:51
Last week I wrote a blog about how to create WebLogic users for ServiceBus 12c. However, I did not now how to assign a particular Application Specific Role to the weblogic user, for particular ServiceBus privileges. I did find out what particular Roles there were (see the blog). But how to assign them I found out just today.

So here it is:
#
#
def grantOSBAppRoleToWlsGroup(osbAppRole, wlsGroup):
#
# Grant OSB AppRole
# http://docs.oracle.com/cd/E23943_01/web.1111/e13813/custom_infra_security.htm#WLSTC1398
# grantAppRole(appStripe, appRoleName,principalClass, principalName)
# appStripe: Specifies an application stripe.
# appRoleName: Specifies a role name.
# principalClass: Specifies the fully qualified name of a class.
# principalName: Specifies the principal name.
#grantAppRole("Service_Bus_Console","Monitor","oracle.security.jps.service.policystore.ApplicationRole","SBMonitor")
#grantAppRole("Service_Bus_Console","Tester","weblogic.security.principal.WLSUserImpl","weblogic")
try:
print('Grant OSB Role: '+osbAppRole+' to WebLogic Group: '+wlsGroup)
grantAppRole("Service_Bus_Console",osbAppRole,"weblogic.security.principal.WLSGroupImpl",wlsGroup)
print('Grant Succeeded')
except:
print('Failed to grant role '+ osbAppRole+' to '+wlsGroup+'.')
print('Check if role not already granted.')

Add this to the createUsers.py script in my previous article.
Then add the following call to add a role to a group:
    #
# Grant AppRole
grantOSBAppRoleToWlsGroup(grpOsbDevOSBAppRole, grpOsbDevName)

This needs the following property in the property file:
# Possible App Roles: MiddlewareAdministrator, Developer, Composer, Deployer, Tester, Monitor, MiddlewareOperator, ApplicationOperator, APICurator
grpOsbDevOSBAppRole=Developer

In the comments I provided the possible values that are described in the docs (see again my previous article).

A description of the grantAppRole can be found here in the 11g docs.

The possible parameters of the function are:


Argument
Definition
appStripeSpecifies an application stripe. For SB12c it is: 'Service_Bus_Console'. You can get it from the pop-list in the EM, WebLogic Domain Menu->Security->Application Roles->Application Stripes Pull Down. (See here).appRoleNameSpecifies a role name. For SB12c this is one of: MiddlewareAdministrator, Developer, Composer, Deployer, Tester, Monitor, MiddlewareOperator, ApplicationOperator, APICuratorprincipalClass Specifies the fully qualified name of a class. Unclear from the docs what to use. But I found (actually on a BI-EE blog):
  • For WebLogic users: weblogic.security.principal.WLSUserImpl
  • For WebLogic groups: weblogic.security.principal.WLSGroupImpl
  • Apparently for Application Roles: oracle.security.jps.service.policystore.ApplicationRole
principalNameSpecifies the principal name.

With this you can enhance the createUsers script to create actual ServiceBus users. For SOASuite or other components you can get the Application Specific Roles by querying the Application Stripe in EM.

Reset your Datasources

Mon, 2016-07-11 05:09
In most SOASuite and Oracle ServiceBus projects the Database Adapter is used. And often it is used to invoke PL/Sql functions and procedures. Actually, it's my favorite interaction method with the database, since the re-entrancy of the database adapter wizard is best there. To do a re-entrant update of a select or DML operation when for instance a column is added, is simply put often quite problematic.

But the thing is with Pl/Sql that when you update a pl/sql package the package state is altered and when calling it from another session you'll get a 'ORA-04068: existing state of packages has been discarded' error. And this is will occur for every connection in the pool of your datasource.

The solution is to reset the datasource. This can be done easily in the WebLogic Administration Console (http://adminserver:port/console). But nowadays you can do it as easily in Enterprise Manager Fusion Middleware Control. This can be quite convenient for developers since often you have EM already open because of your SOASuite tests.

To do this open the Weblogic Domain menu and select the option 'JDBC DataSources':
 Select the DataSource you want to reset, for the example I choose the 'EDNDataSource', but probably you'd not do this for one, but for a custom DataSource:

 Click the Control tab, select the DataSource in the table and click Reset:

Reset will drive WebLogic to recreate all the connections in the DataSource. This prevents popping up the message multiple times.

The functionality in Configuring, Monitoring and Controlling the Datasource  is similar as in the WebLogic Admin console. Only the layout is a little different.

Create Weblogic users for ServiceBus

Thu, 2016-07-07 03:57
At one of my customers there was the question of creating users that needed to log on to EM (Fusion MiddleWare Control) and Service bus console to be able  to deploy SB en WLS Artefacts (queues, datasources etc.) on the Dev and Test environment. And to be able to do some monitoring on Acceptance. Further more they still used weblogic as the only, shared, administrator user. For the latter Oracle recommends to create at least one other user, to prevent you from an accidental admin-lockout. So I'd create named admin-users for each administrator.

Since there were several users to provision on several systems, it would be convenient to have a script to create the users and the particular groups.

As for the Roles there we need to differentiate to WebLogic default roles and and groups, and Service Bus roles.

To be able to log on to the EM, WLS Console and/or Servicebus the user at least needs the WLS Monitor role, which is provided using the Monitors group. But to be able to start and stop servers (that the developers need/want on dev and at this customer also test) you need the Operators group. To be able to create WLS Artefacts, the user need to be in the Deployers group. More on that you can read in chapter '6 Users, Groups, And Security Roles' in the document 'Securing Resources Using Roles and Policies for Oracle WebLogic Server' (12.2.1).

The Servicebus Application Specific Roles are described in the paragraph '11.1.3 Roles' within chapter '11.1 Understanding Oracle Service Bus Application Security' of the document 'Administering Oracle Service Bus'. How to add those to weblogic users and groups is quite clearly described in paragraph '19.1.3.3 Adding a Product-Specific Administration Role to the Enterprise Deployment Administration Group' of the 'Enterprise Deployment Guide for Oracle SOA Suite' (12.2.1), so I'm not going to copy that.

In the following script first administrator-users are created and added to the administrators group.
For every user group that is created (Administrators, OSB Developers  and OSB Testers), the property file (example is provided later) contains three comma seperated lists for the usernames, passwors and the user descriptions. So you can provide as many as you want, provided that the order in the passwords and description lists match the order in the user list. Of course if you have many users, you should consider implementing an LDAP Directory with WebLogic.
For each list of users the script loops over the list and fetches based on an incremented idx-variable the accompanying password and description. It creates the user and adds it as a member to the particular group.

After the administrator-users, the OSB Developers group is created, and for the Dev environment it is added to the WebLogic groups Operators and Deployers. It is done in a loop, based on a list of WebLogic groups this group should be a member of. In EM the group is added to the OSB Application Specific role 'Developer'. I haven't searched for a wls example to do that, but it is a environment specific small activity so I kept that manual.
After the OSB Developers group, the OSB Developer users are created.

Lastly in the same way the OSB Tester group and users are created. The Tester group is only added to the Weblogic Monitors group, and in EM added to the OSB Application Specific role 'Tester'.

So here's the script:
#############################################################################
# Create WebLogic Users and groups
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-07-06
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7001"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def connectToadminServer(adminUrl, adminServerName):
try:
print(lineSeperator)
print('Try to connect to the AdminServer')
try:
connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
except NameError, e:
print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
print('Try to connect to the AdminServer adminUser and adminPwd properties')
connect(adminUser, adminPwd, adminUrl)
except WLSTException:
message='Apparently AdminServer not Started!'
print (message)
raise Exception(message)
#
#
def getRealm(name=None):
cd('/')
if name == None:
realm = cmo.getSecurityConfiguration().getDefaultRealm()
else:
realm = cmo.getSecurityConfiguration().lookupRealm(name)
return realm
#
#
def getAuthenticator(realm, name=None):
if name == None:
authenticator = realm.lookupAuthenticationProvider('DefaultAuthenticator')
else:
authenticator = realm.lookupAuthenticationProvider(name)
return authenticator
#
#
def createUser(authenticator, userName, password, description):
print ("Creating user " + userName)
if authenticator.userExists(userName):
print ("User "+userName+" already exists.")
else:
print ("User "+userName+" does not exist.")
authenticator.createUser(userName, password, description)
print("User "+userName+" created with password "+password+".")
#
#
def createGroup(authenticator, groupName, description):
print ("Creating group " + groupName)
if authenticator.groupExists(groupName):
print ("Group "+groupName+" already exists.")
else:
print ("Group "+groupName+" does not exist.")
authenticator.createGroup(groupName, description)
print("Group "+groupName+" created.")
#
#
def addMember2Group(authenticator, groupName, memberName):
print ("Adding member "+memberName+" to group " + groupName)
if authenticator.isMember(groupName,memberName,true) == 0:
print ("Member "+memberName+" not yet member of the group "+groupName+".")
authenticator.addMemberToGroup(groupName, memberName)
print ("Member "+memberName+" added to the group "+groupName+".")
else:
print ("Member "+memberName+" already member of the group "+groupName+".")
#
#
def main():
try:
print (lineSeperator)
print ('Start Osb Cluster')
print (lineSeperator)
print('\nConnect to AdminServer ')
connectToadminServer(adminUrl, adminServerName)
#
#Create Users
# Get Realm and Authenticator
realm = getRealm()
authenticator = getAuthenticator(realm)
# Administrators
print('\nCreate Administrator users')
administratorList=administrators.split(',')
administratorDescrList=administratorsDesc.split(',')
administratorPasswordList=administratorPasswords.split(',')
#
idx=0
for administrator in administratorList:
administratorDesc=administratorDescrList[idx]
administratorPassword=administratorPasswordList[idx]
print(str(idx)+': Process administrator user '+administrator+' with description '+administratorDesc)
createUser(authenticator, administrator, administratorPassword, administratorDesc)
addMember2Group(authenticator, 'administrators', administrator)
idx=idx+1
#
# OSB Developers
print('\nCreate group '+grpOsbDevName)
createGroup(authenticator, grpOsbDevName, grpOsbDevDesc)
grpOsbDevWlsGrpList=grpOsbDevWlsGrps.split(',')
idx=0
for wlsGroup in grpOsbDevWlsGrpList:
print(str(idx)+': Add '+grpOsbDevName+' to '+wlsGroup)
addMember2Group(authenticator, wlsGroup, grpOsbDevName)
idx=idx+1
#
osbDeveloperList=osbDevelopers.split(',')
osbDeveloperDescrList=osbDeveloperDescriptions.split(',')
osbDeveloperPasswordList=osbDeveloperPasswords.split(',')
#
print('Create OSB Developer users')
idx=0
for osbDeveloper in osbDeveloperList:
osbDeveloperDesc=osbDeveloperDescrList[idx]
osbDeveloperPassword=osbDeveloperPasswordList[idx]
print(str(idx)+': Process OSB Developer user '+osbDeveloper+' with description '+osbDeveloperDesc)
createUser(authenticator, osbDeveloper, osbDeveloperPassword, osbDeveloperDesc)
addMember2Group(authenticator, grpOsbDevName, osbDeveloper)
idx=idx+1
#
# OSB Testers
print('\nCreate group '+grpOsbTestName)
createGroup(authenticator, grpOsbTestName, grpOsbTestDesc)
grpOsbTestWlsGrpList=grpOsbTestWlsGrps.split(',')
idx=0
for wlsGroup in grpOsbTestWlsGrpList:
print(str(idx)+': Add '+grpOsbTestName+' to '+wlsGroup)
addMember2Group(authenticator, wlsGroup, grpOsbTestName)
idx=idx+1
#
osbTesterList=osbTesters.split(',')
osbTesterDescrList=osbTesterDescriptions.split(',')
osbTesterPasswordList=osbTesterPasswords.split(',')
#
print('Create OSB Tester users')
idx=0
for osbTester in osbTesterList:
osbTesterDesc=osbTesterDescrList[idx]
osbTesterPassword=osbTesterPasswordList[idx]
print(str(idx)+': Process OSB Tester user '+osbTester+' with description '+osbTesterDesc)
createUser(authenticator, osbTester, osbTesterPassword, osbTesterDesc)
addMember2Group(authenticator, grpOsbTestName, osbTester)
idx=idx+1
#
print('\nExiting...')
exit()
except NameError, e:
print('Apparently properties not set.')
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
exit(exitcode=1)
#call main()
main()
exit()

And here an example of the property file:
#############################################################################
# Properties for Creating WLS Users
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.0, 2016-07-06
#
#############################################################################
#
# Properties for AdminServer
adminServerName=Adminserver
adminUrl=lxoosb301:7001
# AdminUser
adminUser=weblogic
adminPwd=wlsadmin_O3
#
defaultPassword=welkom123
#
grpOsbDevName=OSBDevelopers
grpOsbDevDesc=OSB Developers
grpOsbDevWlsGrps=Deployers,Operators
#
osbDevelopers=sjaak,maarten
osbDeveloperDescriptions=Sjaak with a Lastname - OSB Developer,Maarten from another father - OSB Developer
osbDeveloperPasswords=sjaak123,maarten456
#
grpOsbTestName=OSBTesters
grpOsbTestDesc=OSB Testers
grpOsbTestWlsGrps=Monitors
#
osbTesters=wim
osbTesterDescriptions=Wim You know - OSBTester
osbTesterPasswords=wimrt789
#
administrators=hans,martien
administratorsDesc=Hans the Admin,Martien van den Akker (extern)
administratorPasswords=hans123,martien456

As you can see the descriptions may have spaces and dashes and other characters in it. But certainly no comma's (','), since that breaks the list. Often the description is used to provide a full name.
Also it contains a references to the admin server and the admin username/passsword. So you need to adapt this part of the property file for each environment.
In a later past I'll show how to create user key files to use encrypted administrator passwords. So after creating the users, remove this file from your system.
To start it you can use the following shell script, or get the running wlst command from:
#!/bin/bash
#############################################################################
# Create users
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-07-06
#
#############################################################################
#
. fmw12c_env.sh
echo
echo Create Users
wlst.sh ./createUsers.py -loadProperties fmw.properties

Rename the fmw.properties reference to the file you named the property file created based on the property file example above.

That (as my other scripts) needs a fmw12c_env.sh script like:
#!/bin/bash
echo set Fusion MiddleWare 12cR2 environment
export FMW_HOME=/ontw/u01/app/oracle/product/fmw12c2
#
echo call setWLSEnv.sh
. $FMW_HOME/wlserver/server/bin/setWLSEnv.sh
export PATH=$FMW_HOME/oracle_common/common/bin:$WL_HOME/common/bin/:$WL_HOME/server/bin:$PATH


OSB 12c Logging part 2

Wed, 2016-07-06 06:52
Two weeks ago, I wrote about how to set the log level voor SB Pipelines (12c) to be able to see the logging of the Log activity in the WebLogic Server logs.

Today I encountered that for a developer at my customer the complete oracle.osb.logging.pipeline logger was missing in the log-configuration. So setting the level from EM (Fusion Middleware Control) following the article above is a little hard.

I could not find why in that case the logger was missing. But I did find a simple solution.
In the paragraph '7.1.4 ODL Log Configuration' of the Administering Oracle Service Bus documentation, I found that  you can change the logging via the EM, wlst and the logging.xml file. This file can be found in ${osb.domain.home}/config/fmwconfig/servers/${osbserver.name}, eg. 'c:\Data\JDeveloper\SOA\system12.2.1.0.42.151011.0031\DefaultDomain\config\fmwconfig\servers\DefaultServer\'.

Go to the end of the file:

...
<logger name='com.sun.xml.ws' level='WARNING' useParentHandlers='true'/>
</loggers>
</logging_configuration>

Copy the last logger and rename it to create an entry for the oracle.osb.logging.pipeline logger:

...
<logger name='com.sun.xml.ws' level='WARNING' useParentHandlers='true'/>
<logger name='oracle.osb.logging.pipeline' level='TRACE:16' />
</loggers>
</logging_configuration>

Set the level and remove the useParentHandlers attribute.
Restart your server and then you should find the option in the EM OSB Log Configuration. If you have multiple OSB servers you'd probably need to update this change for every osb server, since the logging.xml resides in a server specific sub-folder. I haven't tried it to add it for one server and change it to see if it is automatically added to the other server. Would be a nice experiment.

SOA Server (12c) not startable from script-created domain.

Mon, 2016-07-04 04:45
If you're a regular reader of this blog, you've noticed that I've been busy with creating soasuite/osb installations with weblogic domains in a scripted way.

I also optimized my start and stop scripts, that I'll post soon. All this work keeps me from writing on my BPEL book, so for those who are interested in my next episode, I hope you're patient.

But to get to that, it would be nice to have a running SOA Server in my automatically created domain. And unfortunately, that doesn't get up in running mode. Somewhere in the process it fails to start services and switches over to ADMIN mode.

I did some investigation and ran in to the following exception in the server log:

####<Jul 4, 2016 3:30:49 AM EDT> <Error> <Deployer> <darlin-vce-db.darwin-it.local> <SoaServer1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <f2c434e6-7b67-4a8c-9e2f-8b886152cced-0000000a> <1467617449212> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-149205> <Failed to initialize the application "OracleBPMBACServerApp" due to error weblogic.management.DeploymentException: Could not find OSGi framework with name bac-svnserver-osgi-framework
weblogic.management.DeploymentException: Could not find OSGi framework with name bac-svnserver-osgi-framework
at weblogic.osgi.internal.OSGiAppDeploymentExtension.deployBundlesIntoFramework(OSGiAppDeploymentExtension.java:126)
at weblogic.osgi.internal.OSGiAppDeploymentExtension.prepare(OSGiAppDeploymentExtension.java:245)
at weblogic.application.internal.flow.AppDeploymentExtensionFlow.prepare(AppDeploymentExtensionFlow.java:25)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:730)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:45)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:242)
at weblogic.application.internal.EarDeployment.prepare(EarDeployment.java:67)
...

Apparently there was a problem with the OSGi Framework with the name 'bac-svnserver-osgi-framework' that couldn't be found.

OSGi in a domain with both OSB and BPM was a problem in 12cR1. Several blogs are written about that (like this and this one). However, the problem in those blogs is solved soon after the release of 12cR1. In my case something different is happening.

I took a look and it turns out that the 'bac-svnserver-osgi-framework' was not targetted to the SOAServer, although it was targetted to the AdminServer. To check that, navigate to Services->OSGi Frameworks in the Domain Structure:

Here you see the result of my retargetting. If it does not show the SoaCluster or SOA Server, then click on 'bac-svnserver-osgi-framework' and click on the Targets  tab:

At least check the SoaCluster or, if no cluster the SoaServer1. I unchecked the AdminServer because in the earlier posts unchecking the AdminServer was one proposed solution to resolve the OSB-BPM SharedDomain URL issue. But I'm not sure it's needed.

Click Save and activate the changes and, in my case at least, the SoaServer should be startable.


Rename your Weblogic 12c AdminServer

Fri, 2016-07-01 08:55
At one of my customers we create Service Bus Weblogic Domains using a script, driven by a property file. But in one of the environments we encountered that the AdminServer was accidently called 'Adminserver', with a lower-case 's'. Now this is a very minor issue, but the administrator would like to have it fixed, since it is an anomaly regarding with other environments.

This week I got to implement my start-stop scripts (which I will publish later) and so bumbed into this naming fault again.

Renaming it turns out to be walk in the park. So let me show you how I did it. Most of the steps could easily be done in a WLST script, but I trust this is not something you would do on a regular basis, so we do it just in off-line commandline mode.

Before doing anything from the steps below, first shutdown the complete domain, all the weblogic servers, including the AdminServer. And I should advice: backup your domain. This can easily be done by zipping the domain folder.

Then set the fmw environment, that is most conveniently done using a script. The one I always create on a new environment is fmw12c_env.sh:

#!/bin/bash
echo set Fusion MiddleWare 12cR2 environment
export FMW_HOME=/u01/app/oracle/fmw12c2

export SOA_HOME=$FMW_HOME/soa
export OSB_HOME=$FMW_HOME/osb
export MFT_HOME=$FMW_HOME/mft
#
echo call setWLSEnv.sh
. $FMW_HOME/wlserver/server/bin/setWLSEnv.sh
export PATH=$FMW_HOME/oracle_common/common/bin:$WL_HOME/common/bin/:$WL_HOME/server/bin:$PATH

Under Linux you should run this as:
$ . ./fm12c_env.sh

It is important to add the dot in front of the command, to have the settings exported to the calling session. I trust that if you're on windows, you'd be able to create a .bat/.cmd file out of this.

The script put's wlst.sh/.cmd in the path. Start it, which brings you to a prompt like:
wls:/offline> 

In the offline mode, read the domain first with:
wls:/offline> readDomain('/u01/app/work/domains/osb_domain')
Where '/u01/app/work/domains/osb_domain' is the path to the domain.
Navigate to the Servers node:
wls:/offline/osbo3_domain>cd ('/Servers')
wls:/offline/osbo3_domain/Server>ls()
drw- Adminserver
drw- osb_server1
Then navigate to the Adminserver (you could have done that immediately, but you might want to check on your servers first):
wls:/offline/osbo3_domain/Server>cd ('Adminserver')

For the wlst novice: you might have discovered by now that we navigate through (cd()) and introspect (ls())the MBean-hierarchy using filesystem-related commands. The current folder is actually the current MBean at hand, also called the 'Current Managed Object', in short 'cmo'.
So now we're at it, change its name, for instance:
wls:/offline/osbo3_domain/Server/Adminserver>cmo.setName('AdminServer')
To have this change effect you need to write/save/update the domain:
wls:/offline/osbo3_domain/Server/Adminserver>updateDomain()

After that you can quit wlst using the 'exit()' command.

There is still one little issue to solve, before being able to start the AdminServer and the rest of the domain.

In most cases the nodemanager uses the startWebLogic.sh command to start the domain. This one calls the setDomainEnv.sh/.cmd script.

When no Servername is provided as the start parameter, the AdminServer is assumed to be started.
This is evaluated in the setDomainEnv.sh/.cmd in the following lines:
if [ "${SERVER_NAME}" = "" ] ; then
SERVER_NAME="Adminserver"
export SERVER_NAME
fi

So edit the file, locate the lines and change the line 'SERVER_NAME="Adminserver"' to 'SERVER_NAME="AdminServer"' (or the name you used in change the server to).

In this same file, setDomainEnv.sh/.cmd, you might want to disable the start of the Derby Java Database server.
Locate the lines
# Set DERBY_FLAG, if derby is available.

if [ "${DERBY_FLAG}" = "" ] ; then
if [ -f ${WL_HOME}/common/derby/lib/derby.jar ] ; then
DERBY_FLAG="true"
export DERBY_FLAG
fi
fi
And add the following lines just before the 'if':
DERBY_FLAG="false"
export DERBY_FLAG

To be able to start the renamed AdminServer you also might want to move the folder named 'security' from the ${DomainHome}/servers/Adminserver folder (the server folder with the previous name) to the corresponding folder with a new name. This allows the renamed server to reuse the boot.properties file. And remove the old folder, or rename the server folder to the new name.

This concludes the renaming-steps. Now you're good to go to startup the domain again.


ServiceBus 12c: Logging

Fri, 2016-06-24 04:49
As a developer you probably 'log-a-lot' in OSB. (Funny term, perfectly to mock people that have the tendency to excessively add log activities/statements to their code. And hey, if you're being mocked like this: I'm happy to join you, let's make it a 'Geuzennaam').

So as a log-a-lot, I was questioned by a OSB developer this week on a OSB11g->SB12c upgrade that I support, that his logs weren't visible in the server-logs in de 12cR2 SOAQuickStart Integrated Weblogic.

We reviewed the logging settings of the server, all set on Debug.

But it turns out that in the SB Logging configuration in EM all is set to Warning (Inherited).

To check it out, go to http://localhost:7101/em (on a default configured SOA QuickStart Integrated Weblogic domain).

Click on the Target Navigator and Navigate to soa-infra->Service Bus.

Then in the Service Bus menu, navigate to Logs ->  Log Configuration:

Then you need the Log-Levels tab:
Here you see that you can set another level on different SB subsystems. It can be interesting to check out some of those. For instance, I think it was oracle.osb.debug.instancetracking (but I'm not sure, I checked several of them) enable the logging of message contents and variable changes. You see that all the log-level settings are set to Warning, inherited from the level above.

But the one that is of interest for the pipeline-logging is oracle.osb.logging.pipeline:
Set that one on Trace. I used TRACE:16 (FINER). But it turns out that the levels here are different from those in the pipeline log activity and that of those of  the WebLogic Server logs. I haven't got a mapping at hand, but this one let's Debug messages through.

Hit the apply button top right:

This should enable the logging of the Pipeline.

A couple of notes of the automatic generation of a SOA Suite/OSB domain

Fri, 2016-06-24 04:49
Earlier you could have enjoyed my article on the automatic generation of a SOA/OSB domain. Earlier this week I encountered some issues with a domain created at a customer this way.

I got the change to dive into that this week and luckily not only I learned a lot again, but I found the problem as well. I adapted my scripts. I won't repost them completely, I've created a github account, and try to place them there in the near future.

But I'll cover the changes, especially those that caused the problems.

Enrollment
When you create a domain using the config.sh/.cmd wizard, and choose to configure the nodemanager, you'll be asked for the nodemanager user and password. You'll get a per domain nodemanager for free and the domain is enrolled for the nodemanager for you. You might need to adapt the nodemanager.properties in <domain_home>/nodemanager and set the property SecureListener to false. The default is true, while in the Machine definition in the domain the nodemanager/machine is configured to SSL. If you disable the SecureListener, you need to set the property in the wls-console under Machines to Plain instead of SSL. And adapt the listener-port.

The docs on configuring the nodemanager for 12.2.1 can be found here. One of the biggest changes between 11g and 12cR1 is that in 12cR2 you get a per domain nodemanager, with a nodemanager home in the domain home, instead of a nodemanager home in the FMW_HOME, the home of the binaries. Also in the bin folder of the domain you'll find scripts to start and stop the nodemanager.

So for a domain configured in the config-wizard, you need little to do to get the nodemanager working.

But using the scripts in my previous article, you'll find that although a nodemanager password file is created, something is missed for a proper startup of your servers using the nodemanager. You'll find that although the nodemanager starts, connecting to it using nmConnect(), fails with: a message like:
WLSTException: Error occured while performing nmConnect : Cannot connect to Node Manager. : Access to domain 'osb_domain' for user 'weblogic' denied.

You can go to the Domain->security settings in the weblogic console, and change the nodemanager password, but apparently this is not enough. You'll need to explicitly enroll the domain against the nodemanager. To do so:
  • Start the AdminServer, using the startWebLogic.sh/.cmd script in the domain home.
  • Start wlst.sh/.cmd
  • Connect to the AdminServer using:  connect(adminUser, adminPwd, adminURL)
  • Perform an NodeManager Enroll using: nmEnroll(soaDomainHome, nodeManagerHome)
  •  Stop the AdminServer
  • (Re)Start the nodemanager
  • In wlst: Connect to the nodemanager using nmConnect(....)
  • Perform nmStart('AdminServer')
The following enrollDomain.py script might help:
#############################################################################
# Create a SOA/BPM/OSB domain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-06-23
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = 'enrollDomain.py'
#
#Home Folders
soaDomainHome = domainsHome+'/'+soaDomainName
nodeManagerHome = soaDomainHome+'/'+'nodemanager'
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def main():
try:
#
# Section 1: Base Domain + Admin Server
print (lineSeperator)
print ('Enroll '+soaDomainName+' for NodeManager')
print('\nConnect to AdminServer ')
print (lineSeperator)
adminURL=adminListenAddress+':'+adminListenPort
connect(adminUser, adminPwd, adminURL)
#
print('\nPerform nmEnroll')
print (lineSeperator)
#
nmEnroll(soaDomainHome, nodeManagerHome)
#
print ('\nFinished')
#
print('\nExiting...')
exit()
except NameError, e:
print 'Apparently properties not set.'
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#call main()
main()
exit()

It can be called with a shell script (enrollDomain.sh) like:
#!/bin/bash
. fmw12c_env.sh
echo
echo Enroll domain
wlst.sh enrollDomain.py -loadProperties darlin-vce-db-osb.properties

I also adapted the function 'createUnixMachine' in the createSoaBpmDomain.py script:
#
# Create a Unix Machine
def createUnixMachine(serverMachine,serverAddress, serverPort, nmType):
print('\nCreate machine '+serverMachine+' with type UnixMachine')
print (lineSeperator)
cd('/')
create(serverMachine,'UnixMachine')
cd('UnixMachine/'+serverMachine)
create(serverMachine,'NodeManager')
cd('NodeManager/'+serverMachine)
set('ListenAddress',serverAddress)
set('ListenPort',int(serverPort))
set('NMType',nmType)
This allows for the non-default setting of the nodemanager to another listen port and nodemanager type, based on the following properties in the property file:
#
# Server Settings
nmType=Plain
server1Machine=darlin-vce-db
server1Address=darlin-vce-db
server1Port=5555
server2Enabled=false
server2Machine=darlin-vce-db
server2Address=darlin-vce-db2
server2Port=5555
An example of the property file can be found in the earlier article: automatic generation of a SOA/OSB domain.

LoggingIn the example property file in my scripts you can set a location for the logging:
# Logs
logsHome=/u01/app/work/logs

If this is a absolute path, 'nothing is on the hand'. Your domain should start correclty. But if you choose to use a relative path, like 'logs', then starting the AdminServer might succeed (in my case) but starting the managed servers fail with a 'No such file or directory' message:
wls:/nm/osb_domain> nmStart('Adminserver')

Starting server Adminserver ...

Traceback (innermost last):

File "<console", line 1, in ?

File "<iostream", line 188, in nmStart

File "<iostream>", line 553, in raiseWLSTException

WLSTException: Error occurred while performing nmStart : Error Starting server Adminserver : Received error message from Node Manager Server: [Server start command for WebLogic server 'Adminserver' failed due to: [No such file or directory]. Please check Node Manager log and/or server 'Adminserver' log for detailed information.]. Please check Node Manager log for details.

Use dumpStack() to view the full stacktrace :

wls:/nm/osb_domain

Now it turns out that this is caused by the JavaArgs that are generated and set in the script. I found that if you set JavaArgs you need to set redirects for weblogic.Stdout and weblogic.Stderr, like:
'-XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1532m -Dweblogic.Stdout='+logsHome+'AdminServer.out -Dweblogic.Stderr='+logsHome+'AdminServer_err.out'

Where logFolder should be an absolute path. This has to do with the context in which the nodemanager is starting the server. From that context the relative reference apparently does not evaluate to the proper location.
You can however, leave the Java args empty. So I changed my scripts to not use the getServerJavaArgs function, anymore, but get them from the property file. I replaced the xxxJavaArgsBase  with xxxJavaArgs variables. And left them empty. 

The configurator doesn't set the JavaArgs, it leaves them over to the setDomain.sh/.cmd and setStartupEnv.sh/.cmd. If you do so, you can use a relative path, and the servers will start properly.

ServiceBus 12c: Logging

Fri, 2016-06-24 04:49
As a developer you probably 'log-a-lot' in OSB. (Funny term, perfectly to mock people that have the tendency to excessively add log activities/statements to their code. And hey, if you're being mocked like this: I'm happy to join you, let's make it a 'Geuzennaam').

So as a log-a-lot, I was questioned by a OSB developer this week on a OSB11g->SB12c upgrade that I support, that his logs weren't visible in the server-logs in de 12cR2 SOAQuickStart Integrated Weblogic.

We reviewed the logging settings of the server, all set on Debug.

But it turns out that in the SB Logging configuration in EM all is set to Warning (Inherited).

To check it out, go to http://localhost:7101/em (on a default configured SOA QuickStart Integrated Weblogic domain).

Click on the Target Navigator and Navigate to soa-infra->Service Bus.

Then in the Service Bus menu, navigate to Logs ->  Log Configuration:

Then you need the Log-Levels tab:
Here you see that you can set another level on different SB subsystems. It can be interesting to check out some of those. For instance, I think it was oracle.osb.debug.instancetracking (but I'm not sure, I checked several of them) enable the logging of message contents and variable changes. You see that all the log-level settings are set to Warning, inherited from the level above.

But the one that is of interest for the pipeline-logging is oracle.osb.logging.pipeline:
Set that one on Trace. I used TRACE:16 (FINER). But it turns out that the levels here are different from those in the pipeline log activity and that of those of  the WebLogic Server logs. I haven't got a mapping at hand, but this one let's Debug messages through.

Hit the apply button top right:

This should enable the logging of the Pipeline.

Start problems with Nodemanager

Tue, 2016-06-21 02:05
Since my previous long running assignment, I'm involved in a few OSB 11g to 12c upgrade trajects, where I have been working on automatic installs. Hence, my articles about automatic installs, patching and domain configuration.

When I create a new domain, 12cR2 (12.2.1), using my scripts, I'm not able to use the nodemanager of the domain to start the servers. Actually, I can't connect to it. I get it running alright, but connecting to it fails with a message like:
WLSTException: Error occured while performing nmConnect : Cannot connect to Node Manager. : Access to domain 'osb_domain' for user 'weblogic' denied.

When you google the standard solution is to change the nodemanager password on the security tab of the domain in the console.Like in this article. Actually a very good suggestion, but it did not work for me. It turns out that apparently performing an nmEnroll() did the job in that case. What you need to do is:
  • Start the admin server using the startWeblogic.sh script in the domain root.
  • Verify and correct the nodemanager settings under Environment->Machines-><your machine>, ensuring it is inline with the file nodemanager.properties in {domainHome}/nodemanager.
  • Start wlst.sh
  • Connect to the adminserver using: connect({adminuser}, {adminpassword} ,'{adminHost}:{adminPort}')
  • Perform: nmEnroll({domainHome}, {domainHome}/nodemanager)
Here I assume the use of a per-domain nodemanager, where {domainHome}/nodemanager is the nodemanager home within the domain home.

Then I was able to connect to the nodemanager, start the AdminServer and then the OSB Server.


At my customer, they have been struggling in configuring a 'standalone' nodemanager, as they did in the 11g situation. Nodemanager can be started, and connected to. But doning an nmStart of the admin server got:

wls:/nm/osb_domain> nmStart('Adminserver')

Starting server Adminserver ...

Traceback (innermost last):

File "", line 1, in ?

File "", line 188, in nmStart

File "", line 553, in raiseWLSTException

WLSTException: Error occurred while performing nmStart : Error Starting server Adminserver : Received error message from Node Manager Server: [Server start command for WebLogic server 'Adminserver' failed due to: [No such file or directory]. Please check Node Manager log and/or server 'Adminserver' log for detailed information.]. Please check Node Manager log for details.

Use dumpStack() to view the full stacktrace :

wls:/nm/osb_domain

This is also in 12cR2 (12.2.1), with a domain created with the same script. Sharp eyes may notice that the adminserver name is not default: it has a lowercase 's' in stead of a uppercase. They've been fiddling a round with naming of the admin server. What we finally did was to keep the un-default naming, but cleansed the server folder by removing the data, cache and tmp folder. We also removed the logs folder to be able to see if new logs were made from starting from the nodemanager. We configured the per-domain nodemanager and then we did the same as above, performing an nmEnroll() against the domain-nodemanager. After that the 'Adminserver' was startable.

ConclusionI hardly ever had the need to use nmEnroll(), not with a new domain and in 11g at least not even using a seperate nodemanager. From colleagues I did not hear the need to use it in 12c. So why did I need it to solve the sketched problems? I haven't sorted that out, I hope I once get a finger around it. For the moment, take advantage of these experiences.

Pages