little known tips & tricks from across the oracle soa landscape

Ahmed Aboulnaga Technical Director Raastech

Little Known Tips & Tricks from Across the Oracle SOA Landscape

Manchester Central Convention Complex Exchange 8

December 2, 2013 12:45 – 13:45

© Raastech, Inc. 2013 | All rights reserved. Slide 2 of 72 raastech.com

About Me

Ahmed Aboulnaga

18+ years of Oracle middleware experience

Focus on Oracle Fusion Middleware, with emphasis on

SOA, identity management, and portal technologies

Oracle ACE, OCE, OCA

Author / Presenter / Blogger

President of WMOUG / EGOUG


About Raastech

Systems integrator founded in 2009

Headquartered in the Washington DC area

Specializes in Oracle Fusion Middleware

Oracle Gold Partner & Reseller

Oracle SOA Suite Specialized

Certified staff in multiple disciplines

OCE, OCA, ITIL, CISSP, PMP, etc.

Continued contributions to the technology community

Sponsor and volunteer of numerous users groups

Ongoing publishing of books, articles, whitepapers, and blogs

Regular presenters at major conferences


What is this presentation about?

For the experienced Oracle SOA technologist

Random tips and tricks surrounding the following product

sets:

SOA Suite 11g

BAM 11g

OSB 11g

OWSM 11g

OSR 11g

Goal is to share things you may never have heard about

or come across


Agenda

1. Technology Placement & Design Patterns

2. BAM – What to avoid

3. OSB – What to avoid

4. SOA Suite – What to avoid

5. WebLogic Server – What to avoid

6. OSR – What to avoid


Technology Placement

Some overlap between:

Mediator & BPEL

BPEL & OSB

OSB & Mediator

Some customers are BPEL only shops

Some customers are OSB only shops

What to use?

SOA Composite (BPEL + Mediator)

SOA Composite (BPEL)

OSB


BAM Reporting

Design Patterns

Integration

Error Framework

Gateway

Always have a gateway

Virtualization | Security | SLA management

Integration can include any number of technologies

Error framework is a must

Think about transaction reporting (BAM?)


Asynchronous Design Pattern

Mediator

BPEL

BPEL

OSB

OSB

OEG *

BAM



Mediator

BPEL

BPEL

OSB

OSB

OEG *

BAM

First touch point should be a gateway, provides:

• Abstraction/virtualization • SLA monitoring • First line of defense • Could be in DMZ

1



Mediator

BPEL

BPEL

OSB

OSB

2 OEG

*

BAM

JMS queue provides: • Immediate handoff from caller • Message persistence (which also supports recovery)



Mediator

BPEL

BPEL

OSB

OSB

OEG

1-way Async *

BAM

Mediator used for: • Routing • Calls BPEL as 1-way async

(eliminates need for another queue)

3



Mediator

BPEL

BPEL

OSB

OSB

OEG

Synchronous

*

BAM

• Entire sub-process is synchronous • Errors bubbled up to BPEL • BPEL decides how to handle

exceptions • OSB can be used for throttling or

caching

Throttle

Cache

4



Mediator

BPEL

BPEL

OSB

OSB

OEG *

BAM

Create an error management framework:

• Single async service • Maintains errors in a table • Auto-notify by email as required

5



Mediator

BPEL

BPEL

OSB

OSB

OEG *

BAM

Create a message resubmission framework:

• Payloads persisted • Manually republished to

original queue for resubmission

6



Mediator

BPEL

BPEL

OSB

OSB

OEG *

BAM

Use BAM: • To store sensor data • Easily lookup state of transactions

outside of main SOA engines • Keep BAM server on separate

physical servers

7



Mediator

BPEL

BPEL

OSB

OSB

OEG *

Gateway

BAM Reporting

Error Management & Message Resubmission Frameworks

Integration Design Pattern


Technology Placement (for Sensors)

BPEL sensors?

Composite sensors?

BAM sensors?


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff

5. Integration stuff

6. OSR stuff


Understand what BAM is designed for

Not a data repository… use OBIEE instead

Not intended for large amounts of data (think millions of records per data object)

Our mistake:

Instead of composite sensors, dumped huge amount of sensor data into BAM

Why is this a problem?

BAM unable to handle extremely large loads effectively

BAM reports are exceptionally weak

Who nowadays develops web apps that run only in IE?

Best used for aggregated data that needs to be reported against in near real-time


Weak BAM reporting capabilities

No user sorting

No field expansion

Cumbersome navigation

Weak search capabilities


BAM runs only on IE

Yes, I know the history of BAM… but still unacceptable


Don’t forget about BAM purging

People tend to forget about purging BAM Data Objects

If data object is based on a database table:

Simply create a scheduled PL/SQL package to purge the data

Otherwise:

Use BAM Alerts


Avoid BAM RMI Adapter

BAM Adapter comes in 2 flavors; RMI and SOAP


Avoid BAM RMI Adapter

BAM Adapter comes in 2 flavors; RMI and SOAP

In your code:

eis/bam/rmi

eis/bam/soap

Avoid RMI because:

It is not as efficient as SOAP

Bug in BAM Adapter leads to BPEL threads not being released


Understanding BAM inBatch=‘Y’

The BAM Adapter property inBatch batches requests into batches (Could this sentence sound any more messed up?)



BAM Adapter reports ‘success’ 100% of the time, but

pushes the data as predetermined intervals

SOA Composite

Instance

SOA Composite

Instance

SOA Composite

Instance

BAM

inBatch=N

inBatch=N

inBatch=N



BAM Adapter reports ‘success’ 100% of the time, but

pushes the data as predetermined intervals

SOA Composite

Instance

SOA Composite

Instance

SOA Composite

Instance

BAM

inBatch=Y

inBatch=Y

inBatch=Y



Works great, but if there is an error in a single BAM

record:

The entire batch fails

Unknown which record failed

Failure not always in SOA logs but sometimes in BAM logs

BAM UP

BAM Goes DOWN

SOA Suite UP

SOA Suite Goes DOWN

• Requests are batched in SOA • Invocations always success

• Requests are batched in SOA • Invocations always success • Limit to how much can be

batched

• Batched requests are lost • Both are down idiot!


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff


6. OSR stuff


OSB Summary

Pros

Stateless, so performance is superior

Caching, throttling, service virtualization

Can search indexes in payloads (if enabled)

Cons

Cannot reference MDS

Shared custom code required for DVM support

But cannot support multi-input DVMs

Severe limitations in transactional visibility, tracing, and metrics

WSDLs/XSDs must be copied locally


OSB vs. BPEL Performance

1000 transactions (OSB) = 3:32 minutes

1000 transactions (BPEL) = 10:19 minutes

OSB is superior in performance to BPEL (68% better)

Service

OSB Med OSB OSB Med

Service

BPEL Med Med BPEL Med


OSB vs. Composite Tracing


OSB Report Action

OSB performs 2x to 4x faster than BPEL (great!)

OSB sucks in terms of transactional visibility

Consider using the Report Action to capture SOAP

headers and SOAP bodies

Consider creating a web service to log key metadata

(e.g., think BPEL sensors)


OSB Report Action

Can filter by index

Header and body are displayed separately

Enabling header/body on both request/response 4x slower

This is the only OSB functionality that is persisted


OSB Report Action

For all Proxy Services, in the request stage:

1. Add a Report action for $header

2. Add a Report action for $body

3. Specify at least one “Key Name” for both

4. Add the “…Request” string to the name and use the same

value from the body


Using JCA Adapters in OSB

OSB provides native adapters

JCA Adapters must be created in JDeveloper 11g

.JCA and .WSDL manually imported to OSB project

Not supported for all adapters (DB Adapter yes, FTP

Adapter no)


OSB Caching

Result Caching (Coherence) on Business Service

To be used with data that is not frequently updated

Will not re-query backend until it has expired

15% to 25% performance improvement with DB Adapter


OSB on Exalogic – Problems

On OSB installations on Exalogic, the following problems

were experienced:

Client requests get a read time out error

The BEA-380000 error “Request Entity Too Large” appears in

the logs

OSB invocations retry multiple times every 5 minutes

OSB invocations invoke target service twice


OSB on Exalogic – Solution

For all Business Services

1. Under HTTP Transport Configuration

2. Set “Use Chunked Streaming Mode” to “Disabled”

For all Proxy Services

1. View Message Flow

2. Edit Route Node(s)

3. Check on “Quality of Service” and set it to “Exactly Once”


Securing OSB Services with OWSM

1. Click on the Proxy Service

2. Click on the “Policies” tab

3. Select “From OWSM Policy Store” and click “Add”

4. Select the policy “oracle/wss_username_token_service_policy”

5. Update, activate, and submit the changes


Securing OSB services with OWSM

6. When testing, add the following SOAP header to the envelope:

<soap:Header xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

<wsse:Security soap:mustUnderstand="1" xmlns:wsse="http://

docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">

<wsse:UsernameToken wsu:Id="UsernameToken-tk6qMWkQ5hl3Md2INvp16Q22"

xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">

<wsse:Username>oratest</wsse:Username>

<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-

profile-1.0#PasswordText">password123</wsse:Password>

</wsse:UsernameToken>

</wsse:Security>

</soap:Header>


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff


6. OSR stuff


Securing SOA services with OWSM

1. Navigate to the composite, scroll down, and click on the reference

2. Click on the “Policies” tab

3. Attach the policy “oracle/wss_username_token_service_policy”


Optimized calls if using ServiceURL

Ensure that URLs used for composite-to-composite invocations are

identical to the setting in ServerURL

SOA Suite optimizes these calls (uses RMI internally)


Optimized calls if using ServiceURL

Ensure that URLs used for composite-to-composite invocations are

identical to the setting in ServerURL

SOA Suite optimizes these calls (uses RMI internally)

soa_server1 soa_server2

ohs1 ohs2

soadev-vip


Use MDS for all WSDLs

Consider ORAMDS instead of HTTP for composite-to-

external and/or composite-to-composite invocations

Removes the composite loading dependency

(It’s still translated to an optimized call behind the scenes)

<reference name="AsyncStatus" ui:wsdlLocation="oramds:/apps/WSDLs/AsyncStatus.wsdl">

<interface.wsdl interface="http://xmlns.oracle.com/AsyncStatus#wsdl.interface(execute_ptt)"/>

<binding.ws location="oramds:/apps/WSDLs/AsyncStatus.wsdl"

port="http://xmlns.oracle.com/AsynStatus#wsdl.endpoint(AsyncStatus_ep/execute_pt)"

soapVersion="1.1“/>

</reference>


Avoid Mediator Parallel Routing Rules

See this presentation for details why:

http://www.raastech.com/raastech/library/Raastech_2013_Collaborate_SOASuite1

1gPerformance.pdf

This is because of the poor design that Oracle has

chosen for avoiding thread starvation

Edit all “*.mplan” files for your asynchronous Mediator

operations and change the executionType as follows:

OLD: executionType="direct"

NEW: executionType="queued"

http://www.raastech.com/raastech/library/ Raastech_2013_Collaborate_SOASuite11gPerformance.pdf

http://www.raastech.com/raastech/library/ Raastech_2013_Collaborate_SOASuite11gPerformance.pdf


100 messages 98 messages

Wait 1 second

No messages here

Wait 1 second

No messages here

Wait 1 second

No messages here

99 messages

Avoid Mediator Parallel Routing Rules


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff


6. OSR stuff


Singleton property with JMS Topics

For inbound JMS Adapter from a topic, use the

“singleton” property to avoid consumption of multiple

messages in a cluster

For more information:

<service name="JMSConsume" ui:wsdlLocation="JMSConsume.wsdl">

<interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/jms/JMS#wsdl.interface(Consume_ptt)"/>

<binding.jca config="JMSConsume_jms.jca">

<property name="singleton">true</property>

</binding.jca>

</service>

Single message consumption from distributed topic in WebLogic Server 11g http://blog.raastech.com/2012/04/single-message-consumption-from.html

Understanding the "singleton" property with WebLogic JMS topics http://blog.raastech.com/2012/07/understanding-singleton-property-with.html

http://blog.raastech.com/2012/04/single-message-consumption-from.html







http://blog.raastech.com/2012/07/understanding-singleton-property-with.html









Forwarding Policy Replicated



Forwarding Policy Replicated (with Singleton property)



Forwarding Policy Partitioned



Forwarding Policy Partitioned (with Singleton property)


Avoid Distributed Topics

Not possible to have truly highly available JMS destinations

Expectation:

1. With 1 consumer, if code is deployed to all 4 nodes of the cluster, the

message is consumed only once

2. Message is equally available to all nodes of the cluster, so if any node

fails, the message is still available and can be consumed without

manual intervention

Reality:

Point #1 is not possible if you set the forwarding policy to “Replicated”

Point #2 is not possible if the destination's forwarding policy is set to

“Partitioned”

Since these are the only two options available, unable to satisfy both

requirements

See http://blog.raastech.com/2012/07/5-reasons-why-oracle-weblogic-

server.html

http://blog.raastech.com/2012/07/5-reasons-why-oracle-weblogic-server.html













Cannot have durable subscribers on distributed topics

SOA Producer

SOA Consumer

SOA Consumer

SOA Consumer

JMS Topic



Cannot have durable subscribers on distributed topics

SOA Producer

SOA Consumer

SOA Consumer

SOA Consumer

JMS Queue

JMS Queue

JMS Queue


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff


6. OSR stuff


Direct Binding in OSB? Yes or No?

1. Edit the business service

2. For Protocol, select “soa-direct”

3. For Load Balancing Algorithm, select “round-robin”

4. For Endpoint URI, use the format shown in the screenshot


Direct Binding in SOA? Yes or No?

1. Edit the proxy service

2. Under Transport Configuration, modify Protocol to “sb”

3. In the OSB proxy service, Export WSDL via the icon and save it to your local file system

4. Extract the contents of the JAR file to your local file system

5. Create a Direct Binding reference

Enter a name

Ensure that Type is a “Reference”

Ensure that Reference Target is “Oracle Service Bus”

Manually import the WSDL and all referenced WSDLs/XSDs into the project

The Reference Binding Details will be automatically populated


Direct Binding in SOA? Yes or No?


Direct Binding? Yes or No?

No

Avoid direct binding between SOA Suite and OSB

SOAP is a better performant protocol than RMI


PassThroughHeader Property

Do you need to use the “passThroughHeader” property?

Allows elements in the header to pass from one

composite to another

Otherwise, elements in the header that are not explicitly

referenced will be dropped before they are passed to the

next composite.

<Header>

<Security>

<UsernameToken>

<Username>oratest</Username>

<Password>password123</Password>

</UsernameToken>

</Security>

</Header>

Composite 1 Composite 2 Composite 3

<Header>

<Security>

<UsernameToken>

<Username>oratest</Username>

</UsernameToken>

</Security>

</Header>

<Header>

</Header>


PassThroughHeader Property

Do you need to use the “passThroughHeader” property?

Allows elements in the header to pass from one

composite to another

Otherwise, elements in the header that are not explicitly

referenced will be dropped before they are passed to the

next composite.

<component name="UpdateCustomer">

<implementation.mediator src="UpdateCustomer.mplan"/>

<property name="passThroughHeader">true</property>

</component>


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff


6. OSR stuff


Why consider Oracle Service Registry?

Over time, most organizations encounter the following

problems:

No central service registry

Creation of duplicate/redundant services

Difficult to obtain interface specifics on existing services


Service

Oracle Service Registry

Some screenshots from: http://biemond.blogspot.com/2009/12/using-oracle-service-registry-in-soa.html

SOA Suite

OSR

Register Service Configure SOA Suite

Service

Service


Avoid Oracle Service Registry

Do not use at all

Was a hot topic 10 years ago, not so much nowadays

Nobody really does dynamic service lookup

OSR, at best, can be used as a service catalogue

Consider Oracle Enterprise Repository (OER)


Agenda

1. BAM stuff

2. OSB stuff

3. SOA Suite stuff

4. WebLogic stuff


6. OSR stuff

7. Summary


Summary

BAM: Use it for what it is intended for

OSB: Superior performance, poor visibility

SOA Suite: A lot of things to consider; learning about all

aspects of the products takes time and experience

WebLogic Server: Avoid JMS topics

Integration: Avoid direct binding

OSR: Avoid altogether


Contact Information

Ahmed Aboulnaga

Technical Director

[email protected]

@Ahmed_Aboulnaga

little known tips & tricks from across the oracle soa landscape

Technology

bam raastech

bam mediator

recovery raastech

routing calls bpel

major conferences raastech

raastech systems integrator

bam jms queue

transaction reporting