Duplicate instances when using DB adapter in high availability environment.

Many a times I have noticed that when you use DB Adapters (SOA Suite 11g) for integrating with applications like the E Business Suite in a clustered high availability environment, at times there will be multiple instances for the same request.

The number of instances each entry in the DB would generate would directly correlate to the number of nodes on a cluster.

Such that if there are 3 nodes on the soa cluster then the polling composite deployed on each of the managed servers would be attempting to process this file and you would end up with extra instances of the composite which would result in errors or processing the item multiple times depending on the business process.

There are 2 different solutions for this problem on the blogs and you can chose anyone depending on which one suits your particular scenario.

However I prefer the quick an easy one to use the singleton property of the JCA DB adapter .

Ideally you would want to do this from the source in the composite.xml

There’s a property of Inbound endpoint lifecycle support within Adapters called Singleton.
To enable this feature for high availability environment for a given inbound adapter endpoint, one must add the singleton JCA service binding property in the composite.xml within the <binding.jca> element and set it to a value of true as shows.

Singleton Property in composite.xml

    <binding.jca config="bindin_file.jca">
        <property name="singleton">true</property>
    </binding.jca>

More details about this are available @

http://docs.oracle.com/cd/E23943_01/integration.1111/e10231/life_cycle.htm#BABDAFBH

Runtime Fix

However on an already deployed composite you can go composite dashboard and open the properties tab for the db adapter component from the EM console and add a new property
you will see ‘singleton’ in that list and we can set it to ‘true’

Click Apply and Save.

This should result in the following row in your composite.xml which you can check from the xml definition in the composite dashboard.

<property  name="singleton"  type="xs:boolean"  many="false"  source="PollNYSolutionsJobsEBizReqABCS_ptt:receive[hdr]">true</property>

which is the equivalent of the change in the source code, Obviously you will lose this change if a subsequent deployment happens so make sure this is replicated to the source code as well.

The same can be achieved in OSB using by editing the Configuration Details of the proxy service and set ‘Topic Message Distribution’ to ‘One Copy Per Application’.

Refs:

http://javaoraclesoa.blogspot.co.uk/2012/04/polling-with-dbadapter-in-clustered.html

http://ayshaabbas.blogspot.co.uk/2012/11/db-adapter-singleton-behaviour-in-high.html

SOA 11g – Useful SOA Infra DB Queries [Advanced users]

Sometimes you want to identify all the faults for the messages that were sitting in BPEL Engine Level recovery as undelivered Invokes.

We can run this SQL on the SOA Infra Schema to obtain this info.

select wif.*

FROM WI_FAULT WIF

WHERE wif.cikey IN

(SELECT ci.cikey

FROM Dlv_Message dlv, cube_instance ci

WHERE dlv.State = 0

AND dlv.Dlv_Type = 1

AND dlv.ecid= ci.ecid

AND dlv.conv_id = ci.conversation_id

AND dlv.Receive_Date >TRUNC(Sysdate));

 

Once you have investigated them you can then either recover these instances from the em console application or can utilise the SOA API  to call the

BPELServiceEngine.recoverInvokeMessages() operation by passing the message GUID’s of the instances returned by the above query.

I will share the details in another post soon.

References: Please refer to Oracle A Team blog for the full List of all states from COMPOSITE_INSTANCE, CUBE_INSTANCE, DLV_MESSAGE tables

https://blogs.oracle.com/ateamsoab2b/entry/list_of_all_states_from

 

Performace Tunning with OSB – RouterRuntimeCache and Proxy initialisation

As your project grows and the number of proxy services you have in the project grows significantly (>100) you might notice that the performance of the services degrades heavily.

This can be attributed to the fact that the OSB is now unable to Cache all your services at runtime.

OSB caches proxy service runtime meta-data using a two-level cache with static and dynamic sections. The cache introduces a performance trade off between memory consumption and compilation cost. Note that caching proxy services may help throughput but could impact memory usage.

The static section is an upper-bound Least Recently Used (LRU) cache that is never garbage collected. When a proxy service is bumped from the static section, it is demoted to the dynamic section where the cache can be garbage collected when there is memory pressure.

Fortunately for us there is a fix for this in place already.

The number of proxy services in the static portion of the cache can be tuned by configuring the RouterRuntimeCache appropriately.

It’s done using a system property com.bea.wli.sb.pipeline.RouterRuntimeCache.size. The default value is 100. Note that this is the number of proxies and NOT dependent on the size of each or proxies so the memory it takes can depend on complexity of these proxies and any Xqueries they call.

This setting can be increased to a desired value provided there is sufficient memory for runtime data processing for large number of proxy services, or you will have to increase the system JVM settings accordingly along with the increase to the runtime cache value.

This property value can be set in the setDomainEnv.sh file as an extra java argument as follows:

-Dcom.bea.wli.sb.pipeline.RouterRuntimeCache.size={size}

Example:

EXTRA_JAVA_PROPERTIES=”-Dcom.bea.wli.sb.pipeline.RouterRuntimeCache.size=3000 ${EXTRA_JAVA_PROPERTIES}”
Additionally, you might still notice that the first invocation is slow as we see while trying to test the services, this is due to the fact that these services are being compiled then and any future invocations will be a lot faster due to them being in the static Cache.
There is a very interesting article by Mark Smith illustrating how we can utilise this setting to improve the SLA and ensure that the first invocation does not cause you to fail on your SLA. https://blogs.oracle.com/MarkSmith/entry/osb_proxy_initialization/

Ref: http://docs.oracle.com/cd/E25054_01/core.1111/e10108/osb.htm