Best Practices

OSB Deployment scripts 11.1.1.7 using configtool


These instructions are only valid for OSB Release 11.1.1.7 onwards.

 

How to export desired projects using the configjar utility

 

The configjar tool is located in the OSB installation directory under

C:\oracle\Middleware11117\Oracle_OSB1\tools\configjar

[Check the path relative to your install]

Run the setenv.cmd

And execute the

C:\oracle\Middleware11117\Oracle_OSB1\tools\configjar\configjar.bat -settingsfile C:\OSB_Deployment\OSBPS6_MavenConfigTool\nysolutions_export_projects.xml

 

Where the nysolutions_export_projects.xml file contains the names of the projects and the configuration files and the desired output filename.

 

 

 

The nysolutions_export_projects.xml I used for this post was

 

 

 

    <source>
        <project dir="C:\Users\nitina\osbworkspace\Contracts"/>
        <project dir="C:\Users\nitina\osbworkspace\NYSService1"/>
        <project dir="C:\Users\nitina\osbworkspace\NYSService2"/>
        <project dir="C:\Users\nitina\osbworkspace\NYSService3"/>
        <system  dir="C:\Users\nitina\osbworkspace\nys-conf"/>
    </source>
    <configjar jar="C:\OSB_Deployment\OSBPS6_MavenConfigTool\export\NYSsbconfig.jar">
         <projectLevel includeSystem="true"/>
    </configjar>
    
</configjarSettings>

 

 

 

 

Complete list of options and commands are available in oracle documentation at http://docs.oracle.com/cd/E28280_01/dev.1111/e15866/app_export.htm#BABDJJIA

 

Export Settings File Schema Definition

Below is the schema definition for the export settings XML file. Some of the text below has been wrapped for readability.

<?xml version="1.0"?>
           elementFormDefault="qualified"
           attributeFormDefault="unqualified"
           xmlns:xs="http://www.w3.org/2001/XMLSchema"
          xmlns:tns="http://www.bea.com/alsb/tools/configjar/config">
<xs:element name="configjarSettings" type="tns:configjarSettings"/>
<xs:complexType name="configjarSettings">
   <xs:sequence>
      <xs:element name="source" type="tns:source" />
      <xs:element name="configjar" type="tns:configjar" maxOccurs="unbounded"/>
   </xs:sequence>
</xs:complexType>
<xs:complexType name="source">
   <xs:sequence>
      <xs:choice minOccurs="1" maxOccurs="unbounded">
         <xs:element name="project">
            <xs:complexType>
               <xs:attribute name="dir" type="xs:string" use="required"/>
            </xs:complexType>
         </xs:element>
         <xs:element name="system">
            <xs:complexType>
               <xs:attribute name="dir" type="xs:string" use="required"/>
            </xs:complexType>
         </xs:element>
      </xs:choice>
      <xs:element name="extensionMapping" minOccurs="0">
         <xs:complexType>
            <xs:sequence>
               <xs:element name="mapping" minOccurs="0" maxOccurs="unbounded">
                  <xs:complexType>
                    <xs:attribute name="type" type="xs:string" use="required"/>
                  <xs:attribute name="extensions" type="xs:string"
                                use="required"/>
                  </xs:complexType>
               </xs:element>
            </xs:sequence>
         </xs:complexType>
      </xs:element>
     <xs:element name="fileset" type="tns:contentSet" minOccurs="0"/>
  </xs:sequence>
</xs:complexType>
<xs:complexType name="configjar">
  <xs:sequence>
      <xs:choice minOccurs="1" maxOccurs="1">
         <xs:element name="projectLevel" type="tns:projectLevel"/>
         <xs:element name="resourceLevel" type="tns:resourceLevel"/>
      </xs:choice>
  </xs:sequence>
  <xs:attribute name="jar" type="xs:string" use="required"/>
  <xs:attribute name="overwrite" type="xs:boolean" use="optional"
 default="true"/>
</xs:complexType>
<xs:complexType name="projectLevel">
   <xs:sequence>
      <xs:element name="project" type="xs:string" minOccurs="0"
                  maxOccurs="unbounded"/>
   </xs:sequence>
   <xs:attribute name="includeSystem" type="xs:boolean" use="optional"
               default="false"/>
</xs:complexType>
<xs:complexType name="resourceLevel">
   <xs:sequence>
      <xs:element name="resources" type="tns:contentSet" minOccurs="0"/>
   </xs:sequence>
   <xs:attribute name="includeDependencies" type="xs:boolean" use="optional"
                 default="true"/>
</xs:complexType>
<xs:complexType name="contentSet">
   <xs:sequence>
      <xs:element name="include" type="tns:contentSetPattern" minOccurs="0"
                  maxOccurs="unbounded"/>
      <xs:element name="exclude" type="tns:contentSetPattern" minOccurs="0"
                  maxOccurs="unbounded"/>
   </xs:sequence>
</xs:complexType>
<xs:complexType name="contentSetPattern">
  <xs:attribute name="name" type="xs:string" use="required"/>
</xs:complexType>
</xs:schema>

 

 

Sample configSettings.xml generated using eclipse

 

<?xml version="1.0" encoding="UTF-8"?>
  <tns:source>
    <tns:project dir=""/>
    <tns:extensionMapping>
      <tns:mapping extensions="" type=""/>
    </tns:extensionMapping>
    <tns:fileset>
      <tns:include name=""/>
      <tns:exclude name=""/>
    </tns:fileset>
  </tns:source>
  <tns:configjar jar="" overwrite="true">
    <tns:projectLevel includeSystem="false">
      <tns:project>tns:project</tns:project>
    </tns:projectLevel>
  </tns:configjar>
</tns:configjarSettings>

 

 

Step 2

 

Create a ALSBCustomizationFile.xml which contains all the config settings.

 

Step 3

 

Run the deployment scripts to deploy the NYSsbconfig.jar file generated above…

Update the wlst with environment specific values deployToServer.py

java weblogic.WLST deployToServer.py

 

Where the deployToServer.py file is

from java.util import HashMap
from java.util import HashSet
from java.util import ArrayList
from java.io import FileInputStream
from com.bea.wli.sb.util import Refs
from com.bea.wli.config.customization import Customization
from com.bea.wli.sb.management.importexport import ALSBImportOperation
import sys
#=======================================================================================
# Entry function to deploy project configuration and resources
#        into a ALSB domain
#=======================================================================================
def importToALSBDomain():
    try:
        SessionMBean = None
        print 'Attempting to import :', importJar, "on ALSB Admin Server listening on :", adminUrl
        theBytes = readBinaryFile(importJar)
        print 'Read file', importJar
        sessionName = createSessionName()
        print 'Created session', sessionName
        SessionMBean = getSessionManagementMBean(sessionName)
        print 'SessionMBean started session'
        ALSBConfigurationMBean = findService(String("ALSBConfiguration.").concat(sessionName), "com.bea.wli.sb.management.configuration.ALSBConfigurationMBean")
        print "ALSBConfiguration MBean found", ALSBConfigurationMBean
        ALSBConfigurationMBean.uploadJarFile(theBytes)
        print 'Jar Uploaded'
        if project == "None":
            print 'No project specified, additive deployment performed'
            alsbJarInfo = ALSBConfigurationMBean.getImportJarInfo()
            alsbImportPlan = alsbJarInfo.getDefaultImportPlan()
            alsbImportPlan.setPassphrase(passphrase)
            alsbImportPlan.setPreserveExistingEnvValues(true)
            importResult = ALSBConfigurationMBean.importUploaded(alsbImportPlan)
            SessionMBean.activateSession(sessionName, "Complete import without customization using wlst")
        else:
            print 'ALSB project', project, 'will get overlaid'
            alsbJarInfo = ALSBConfigurationMBean.getImportJarInfo()
            alsbImportPlan = alsbJarInfo.getDefaultImportPlan()
            alsbImportPlan.setPassphrase(passphrase)
            operationMap=HashMap()
            operationMap = alsbImportPlan.getOperations()
            print
            print 'Default importPlan'
            printOpMap(operationMap)
            set = operationMap.entrySet()
            alsbImportPlan.setPreserveExistingEnvValues(true)
            #boolean
            abort = false
            #list of created ref
            createdRef = ArrayList()
            for entry in set:
                ref = entry.getKey()
                op = entry.getValue()
                #set different logic based on the resource type
                type = ref.getTypeId
                if type == Refs.SERVICE_ACCOUNT_TYPE or type == Refs.SERVICE_PROVIDER_TYPE:
                    if op.getOperation() == ALSBImportOperation.Operation.Create:
                        print 'Unable to import a service account or a service provider on a target system', ref
                        abort = true
#                elif op.getOperation() == ALSBImportOperation.Operation.Create:
                else:
                    #keep the list of created resources
                    print 'ref: ',ref
                    createdRef.add(ref)
            if abort == true :
                print 'This jar must be imported manually to resolve the service account and service provider dependencies'
                SessionMBean.discardSession(sessionName)
                raise
            print
            print 'Modified importPlan'
            printOpMap(operationMap)
            importResult = ALSBConfigurationMBean.importUploaded(alsbImportPlan)
            printDiagMap(importResult.getImportDiagnostics())
            if importResult.getFailed().isEmpty() == false:
                print 'One or more resources could not be imported properly'
                raise
            #customize if a customization file is specified
            #affects only the created resources
            if customFile != "None" :
                print 'Loading customization File', customFile
                print 'Customization applied to the created resources only', createdRef
                iStream = FileInputStream(customFile)
                customizationList = Customization.fromXML(iStream)
                filteredCustomizationList = ArrayList()
                setRef = HashSet(createdRef)
                # apply a filter to all the customizations to narrow the target to the created resources
                for customization in customizationList:
                    print customization
                    newcustomization = customization.clone(setRef)
                    filteredCustomizationList.add(newcustomization)
                ALSBConfigurationMBean.customize(filteredCustomizationList)
            SessionMBean.activateSession(sessionName, "Complete import with customization using wlst")
        print "Deployment of : " + importJar + " successful"
    except:
        print "Unexpected error:", sys.exc_info()[0]
        if SessionMBean != None:
            SessionMBean.discardSession(sessionName)
        raise
#=======================================================================================
# Utility function to print the list of operations
#=======================================================================================
def printOpMap(map):
    set = map.entrySet()
    for entry in set:
        op = entry.getValue()
        print op.getOperation(),
        ref = entry.getKey()
        print ref
    print
#=======================================================================================
# Utility function to print the diagnostics
#=======================================================================================
def printDiagMap(map):
    set = map.entrySet()
    for entry in set:
        diag = entry.getValue().toString()
        print diag
    print
#=======================================================================================
# Utility function to read a binary file
#=======================================================================================
def readBinaryFile(fileName):
    file = open(fileName, 'rb')
    bytes = file.read()
    return bytes
#=======================================================================================
# Utility function to create an arbitrary session name
#=======================================================================================
def createSessionName():
    sessionName = String("SessionScript"+Long(System.currentTimeMillis()).toString())
    return sessionName
#=======================================================================================
# Utility function to load a session MBeans
#=======================================================================================
def getSessionManagementMBean(sessionName):
    SessionMBean = findService("SessionManagement", "com.bea.wli.sb.management.configuration.SessionManagementMBean")
    SessionMBean.createSession(sessionName)
    return SessionMBean
# IMPORT script init
try:
    # import the service bus configuration
    adminUser    = "weblogic"
    adminPassword= "weblogic1"
    adminUrl     = "t3://soabpm-vm:7001"
    importJar    = "export/NYSsbconfig.jar"
    customFile   = "ALSBCustomizationFile.xml"
    passphrase   = "osb"
    project      = "NYSService1"
    connect(adminUser,adminPassword,adminUrl)
    domainRuntime()
    importToALSBDomain()
except:
    print "Unexpected error: ", sys.exc_info()[0]
    dumpStack()
    raise

 

That’s it all done…you can then check on the sbconsole that the changes have all been deployed.

Oracle Service Bus   View Configuration Changes

 

Advertisements

Testing Oracle Service Bus components


Testing Approach with OSB

I have been having a thought about what’s the best way forward with automating unit tests with OSB

First of all there are 2 things here…

One is Unit testing and the Second bit is automating them…

UNIT TESTING with OSB

A customer of mine was very keen on unit testing and automation and this is what I explained to them. There isn’t that much of an issue in adding unit tests to all services just that it means we do some additional work mocking services.
For SOAP Services it’s simply a case of writing SOAPUI web services test cases.
However since most of the integrations at the customer weren’t synchronous SOAP Services we will have to write test cases for JMS messages. Again this can be done via SOAPUI or we can also use something like citrus.
Citrus is an integration testing framework written in Java that tests your software application to fit into your customer’s environment. The tool simulates surrounding systems across various transports (e.g. Http, JMS, TCP/IP, SOAP …) in order to perform automatic end-to-end use case testing.

For JMS based Messaging services again we can have SOAPUI test cases as we did for the earlier phases of the project.

Also see http://www.soapui.org/JMS/working-with-jms-messages.html

However the bit where we might struggle or may be have to tweak things a bit is when it comes to proxy chaining which isn’t SOAP or JMS… eg. Local protocol provided by OSB  i.e. proxies that can’t be executed from the outside world…

SO for these we have the following options
1.       Do nothing i.e. don’t unit test them just test with other services… which may not be that bad as they are like internal services.
2.       Change the protocol in Dev environments allowing them to be tested individually however that might also come with other complications like there is a difference in how transactions work in local proxies etc… also the headers are different in SOAP over HTTP from JMS and the same for local.. so further analysis really needs to go into this but most likely we will have to break these down on a case by case basis and would end up with 3-4 different scenarios based on the blueprints.

3.       Also looking at some blogs people are talking about adding SOAP based wrappers for such services. Which we can consider, just that I am not a big fan of adding code that isn’t really part of your project but with careful governance we can take this approach…

 Another aspect you want to consider is how we test transformations

Xquery Transformations:

Eclipse lets you unit test them from the IDE and can also be tested from the OSB console.

Eclipse Xquery

 

xquery test

 

I haven’t tested it using any tool before but can see people have written java code to automate this. http://osbutils.googlecode.com/svn/tags/XQTestFramework-v0.2/src/com/oracle/uk/ocs/osb/xqtest/XQAbstractTest.java might be worth testing it.

In addition another approach commonly used and documented by some fellow bloggers is that we can wrap Transformations and simple services without much overhead which allows them to be easily tested along with other services using SOAPUI for example.

So all in all we can have unit tests for OSB just that we need to identify all the scenarios i.e. soap calls, jms services, adapters, sync, XQuery transforms one way services etc… Also Queues and we should be able to unit test them…
There is a very useful post on Eric’s site regarding how to use Citrus framework which can plug with maven to provide automation of unit tests.
http://www.xenta.nl/blog/2010/04/12/eaioracle-service-bus-testing-with-citrus-framework-part2/
http://www.citrusframework.org/index.html

Part 2 AUTOMATING Unit Tests

If we want to initiate the entire unit tests and plug them with the Continuous integration

I think the biggest challenge is that normally unless you have a dedicated environment which only contains the mocks you would struggle to do that. In a normal integration environment we will need to first either automate creation of a session executing a Customization file which switches the endpoints to mocks and then executes the requests (all this is doable using maven and automation scripts – just not desirable)….. The good thing is with OSB you can undo a session so may be post execution the changes to endpoints in rolled back.
Preferred solution: would be to have a dedicated Unit test environment with all the mocks running on this environment, which can be used with CI. As soon a developer checks the code in the CI kicks in and checks the code out, performs the build, executes all the test cases and provide you the report about the test coverage. This can work with Bamboo as it has worked with Continuum and some other tools which I have tried in the past, I might be looking to hook this will Bamboo in the coming few days for one of our customers.
Test coverage Reports using SOAP UI

run-sample-testsuite

create-sample-report

a web service testing result report created with soapUI Pro

testsuite-coverage

Test Coverage using Citrus

Oracle Service Bus testing with Citrus

Other thing that just came to mind was maybe we can be using dynamic routing to switch between mock or real endpoints based on the input message but in that case the mock business services will have to live on the server as well…again you wont want these to progress through the environments so you will have to have a strong governance model around how to ignore certain files from deployments…. Or may be how to ignore all files sitting in the Unit test folder of the project….  Again more thought needs to go into each of these options…

References: http://eelzinga.wordpress.com/2010/04/12/eaioracle-service-bus-testing-with-citrus-framework-part2/
http://www.citrusframework.org/index.html
http://www.slideshare.net/gschmutz/best-practices-for-testing-soa-suite-11g-based-systems

Continuous Integration with OSB – http://redstack.wordpress.com/2013/05/02/building-osb-projects-with-maven-and-removing-the-eclipse-dependency/

http://soa-java.blogspot.co.uk/2011/12/continuous-integration-for-oracle-osb.html

These are the just the various options I have considered so far for the Testing Approach with OSB and how it plugs into CI… Further analysis may be needed for some of these to ensure they are the best possible solution for your organisation.

I would be interested to know what your views are on some of these?