systems for e-business

Systems for E-Business

Integration and Optimization on the Web

Major Issues Majority of organizations are faced with a number of major issues:

Address Web application development

Address Web applications scalability and reliability

Address Web applications integration

Provide framework for Web application integration

Integrate Web and back-end applications

Move from “Web-sites to Web applications to Web transactions”

Selecting the right enabling technologies for Web application development and Integration

CORBA, MTS/COM, Commercial Web Application Servers, Object Transaction Monitors, EJB etc

Enterprise Application Integration and Web Enterprise Application Integration are key

(W)EAI = Middleware?

Web Integration Requirements

Web Server Integration

Enterprise Middleware Integration

Application Development Integration

Integrating back-end databases

Intelligent Data Transformation

Leveraging legacy applications

Different Views on Database Integration on the Web

#1 DBMS/Web Integration: Integrate the relational database into the e-commerce/web architecture by providing a Web-interface to database access and supporting transactions on the Web

#2 Defragmentation: Integrate multiple different data sources and applications into the e-commerce architecture

#3 Function Extension: to create new interactive Composite Applications with a Web interface:

to the user the applications appears to be a new Web applications but behind the scenes, it may invoke one or more mainframe or UNIX/NT applications and databases

Out-of-Scope: Data Consistency Management: This is the ability to provide consistency of data stored in physically disparate databases. DBMS replication as well as message queuing middleware can be used to develop application-specific synchronization processes

Database Integration and Optimization

Prerequisites: CORBA and JEB

CORBA Architecture

Java Enterprise API’s

Client Protocols... HTTP

Hypertext Transfer Protocol

Intended for stateless text transfer

Applets, Active Server Pages

Some vendors put tags into the html stream

resolve at the server to calls to application code running under the application server.

RMI

Remote Method Invocation

Originally intended for Java-only application distribution

Java-to-Java

Objects passed by value

As an API over IIOP

Objects passed by reference

Client protocols ... IIOP

Internet Inter-Orb Protocol (OMG/CORBA)

Intended for CORBA-compliant application distribution


E-architectures for Database Integration

Web Integration Technologies vs Web Integration Patterns

WebIntegration

Servers

Defragmentation“GUI-ification” User InterfaceRedesign

FunctionExtension

WebApplication

Servers

Middleware Web Ext .(at the client)

ObjectTransaction

Monitors

WebScreen

Scrapers

Web-EnabledMessageBrokers

High

Low

FunctionalComplexity

Web Enablement Pattern

WebTerminal

Emulators

Host WebServers

Source: Gartner Group, December 10, 1998

Web Integration(1) via Web Application Servers(WAS)

HTML

Web Browser

HTML

Web Browser

HTML

Web Browser

WAS

BusinessLogic Wrappers

Apps

Apps

Apps

RDBMS

Apps

Examples:

IBM WebPhereOracle Application ServerBEA WebLogicNetscape Application Server

WWW Middleware

Web Application Servers Fill the Gap

Web Servers

Web Application Servers

Firewalls/Security

Universal client Stateless No transactions Weak persistence Simple applications

TP Monitors CORBA OTM Messaging Database Integration Legacy Application

Integration ERP Integration

Web Application Servers Fill the Gap State/session management

Transaction management

Scalability and high performance

Robustness and manageability

Enhanced security

Persistence

Most important … provide necessary integration with back-end infrastructure

Web Application Servers Fill the Gap Web Application Servers (WAS) have to perform multiple roles

Host business logic

Make it accessible

Scalability, integrity, security, responsiveness

Integrated management

Comprehensive run-time

Well defined integration points

Provide a single and consolidated application

development environment

Foundation for EC

Web Application Server Provide Web Application Servers can host a variety of applications including

Java applications, C++ applications, Visual Basic etc.

Integration with back-end infrastructure and middleware

CORBA, MOM and other types of middleware

Enterprise Resource Planning Applications (Peoplesoft, SAP, Baan etc)

Legacy Applications

CICS, IMS, Legacy Databases

Generic Web Application Server Architecture

Group of Server Applications

Load Balancing

Sec

urit

yDispatcher Internet

Extranet

Intranet

HTML + Java

Java

HTTP

HTTP

IIOP

HTML

Packages (SAP, PeopleSoft)

Client/Server applications (C, C++)

Access Components

Management

Native SQLODBCJDBC

Firewall

Web Server

Client Component

CGI,NSAPI,ISAPI

Integrated Development Environment

IIOP

Legacy Applications via:CICS, IMS, MQSeries

WebLogic - Product Architecture

Java & MS COM

Client Apps

WebLogicServer

Common Application Server Components Web Server APIs

NSAPI, ISAPI, CGI, Servlets

Application Manager

Hosts Application Logic

Cgi-bin programs, Java objects, Java Servlets etc.

Manage application instances

Web “Transaction” Manager

Manage Web based “Transactions”

Database “Interaction” Manager

Database connection management

Database connection pooling

Results caching etc

Session/State Manager

Manages state/session information

Generic Web Application Server Functionality

StateMgmt.

SessionMgmt.

WorkloadMgmt.

ProcessMgmt.

RulesEngine

EnterpriseAdapters

Data storeAdapters

MiddlewareAdapters

Business Logic

Web Application Server

Web Server Presentation Logic, HTML Page Server

Web Browser

WAS Application Server Products

Secant: Extreme Server Bluestone: Sapphire/Web Gemstone J Sun: NetDynamics BEA: WebLogic Server Oracle Application Server Sybase: Enterprise Application Server Netscape Application Server IBM: WebSphere Silverstream Software: SilverStream Novera: Jbusiness Persistence: PowerTier Progress: Apptivity Inprise Application Server Forte: SynerJ Apple: WebObjects Information Builders: Parlay

Web Integration(2) via Web-Enabled Message Brokers

HTML

Web Browser

HTML

Web Browser

HTML

Web Browser

Flow Control

Transfor-mation

Manage-ment

Apps

Apps

Apps

RDBMS

Apps

Web Application Servers vs. OTMs

Web Application Server strength:

Integrated development tools for Web-based applications

Some provide rich set of classes / frameworks

Address specific requirements of Web-based applications (e.g. state & session management)

OTM strength:

Support for distributed transactions (ACID properties)

Proven scalability

24 x 7 availability

Most WAS vendors focus on Java/EJB - OTMs provide wider choice of language support and runtime environments

Combining Web Application Servers and OTMs

ServletsWeb

Browser:

Netscapeor

InternetExplorer

WAS

WASApp

NSAPI/ISAPI

ObjectTransaction

Monitor(OTM)

NetworkComputing (NC)

Clients

GUIClients

CGI

WebServer

Web Application Server vs. OTM Evolution

Java

Web Server Integration

HTML

State Management

Session Management

IIOP

Middleware Integration

Back-end Security

Scalability Failover/ Recovery

Transactions

Availability

Result Caching

Web Security

Auditing & Logging

WASWAS OTMOTM

Basic 3-tier CGI/HTTP Client/Server Architecture

RDBMS

HTML

Web Browser

Internet

HTTP

HTTPServer

HTTP

HTMLdocs

AppsSQLAPI

CGI/ISAPI

RDBMS

SQLAPI

Tier 1Tier 1

GUIGUI

Tier 2Tier 2

ApplicationApplicationLogicLogic

Tier 3Tier 3

DataData

Integrating Back-end Databases

Webbrowser

Webserver

WebServer-side

Gateway

staticcontent

documentreportspreadsheetimageaudiovideoanimation

Multi mediadata

CGINSAPIISAPI

dynamiccontent

DatabaseGateway

Universal data access

Universal and uniform access to diverse data sources

not just the popular relational databases

It is estimated that 65% of corporate data are not stored in relational databases

Basic 3-tier CGI/HTTP Architecture

Warehousedatabase

HTML

Web Browser

Internet

Orderpizza

HTTPServer

Orderpizza

HTMLdocs

Apps

Can order be fulfilled?CGI

Order/Entrydatabase

Orderpizza

Tier 1Tier 1 Tier 2Tier 2 Tier 3Tier 3

WorkforceSchedulingdatabase

When canpizza be

delivered?

Pizza can bedelivered @

19:15,but we’re out

of olives

Basic 3-tier CORBA/Java Client/Server Architecture

HTMLJava

Web Browser Internet

CORBAIIOP

WAS

Tier 1Tier 1

GUIGUI

Tier 2Tier 2


Tier 3Tier 3

DataData

CORBAORB

CORBAORB

ServerObjects

CORBAIIOP

RDBMS

CORBA

JDBC

Basic 3-tier CORBA/Java Client/Server Architecture

RDBMSType BHTML/

Java

Web Browser

HTTPServer

HTMLdocs

AppsSQLAPI

CGI/ISAPI

RDBMSType A

SQLAPI

Tier 1Tier 1

GUIGUI

Tier 2Tier 2


Tier 3Tier 3

DataData

CORBAORB

CORBAORB

ServerObjects

RDBMSType C

CORBA

CORBAServer

(5)CORBA

IIOP

(1) Get HTML Page

(2) Get Applet

(3)Load Applet

(4) Invoke CORBA

JDBC

Basic 3-tier CGI/HTTP and CORBA Client/Server Architecture

RDBMS

HTML

Web Browser

Internet

HTTP

CORBAServerwhich

interpretsHTTP

HTTP

HTMLdocs

AppsSQLAPI

CGI/ISAPI

RDBMS

SQLAPI

Tier 1Tier 1

GUIGUI

Tier 2Tier 2


Tier 3Tier 3

DataData

CORBAORB

CORBAORB

ServerObjects

RDBMS

CORBA

CORBAIIOP

HTML/Java

Web Browser

CORBAIIOP

JDBC

HTTP/CGI versus Java/CORBA ORB’s

Feature/function CORBA IIOP HTTP/CGI

Reliable communication Yes Yes, across TCP/IP

State across invocations Yes No

Parameter marshalling Yes - IDL stubs No, CGI program readsenvironment variables

Interface descriptions Yes - via IDL No

Dynamic discovery Yes No

Parameter data typing Yes - Language neutral format No

Performance Fast Slow

Transactions Yes No

Security Yes - via CORBA Yes - via SSL or S-HTTP

Predictions on Java/CORBA Integration

“More than 80 percent of large enterprises will use COM and some aspect of CORBA or IIOP somewhere in their enterprise applications by 2001 (0.8 probability)”

“COM will be functionally complete as an OTM by 1999 and mature enough to support demanding, mainstream, mission-critical applications by 2001 (0.7 probability)

“By 2002, the CORBA and Enterprise JavaBeans programming models will evolve into a common multilingual programming model: Enterprise Beans (0.8 probability)”


Architectures for Database Middlware

SQL Middleware Models Single vendor option

Common SQL Interface option

Open SQL Gateway option

Federated Databases

Integrated Single Vendor

Server Object

OpenClient

Driver

Protocol

dblib

SybaseServer

Driver

Protocol

FAP

TCP/IP

HTML/Java

Web Browser

HTTPServer

CORBAServer

(5)CORBA

IIOP

(1) Get HTML Page

(2) Get Applet

(3)Load Applet

(4) Invoke CORBA

CORBAORB

CORBAORB

HTMLdocs

ServerObject

ServerObject

Middleware Solution#1: Common SQL Interface

ServerObject

JDBC / ODBC

Protocol

DbmsServer A

Driver A

Protocol

ServerObject

ServerObject

Driver A Driver B Driver C

DbmsServer C

Driver C

Protocol

FAP AFAP BFAP C

CORBAORB

HTML/Java

Web Browser

HTTPServer

CORBAServer

(5)CORBA

IIOP

(1) Get HTML Page

(2) Get Applet

(3)Load Applet

(4) Invoke CORBA

HTMLdocs

Middleware Solution #2: Open SQL Gateway

Common SQL API

Protocol

DbmsServer A

GatewayServer

Protocol

ServerObjects

ServerObjects Dbms

Server B

GatewayServer

Protocol

DbmsServer C

GatewayServer

Protocol

GatewayFAP

Gateway Driver

HTML/Java

Web Browser

HTTPServer

CORBAServer

(5)CORBA

IIOP

(1) Get HTML Page

(2) Get Applet

(3)Load Applet

(4) Invoke CORBA

HTMLdocs

CORBAORB

Middleware Solution #3: The Ideal World

Common SQL API

Protocol

DbmsServer A

Protocol

Appl Appl DbmsServer B

Protocol

DbmsServer C

Protocol

StandardSQLFAP

Global Query Control

Standard SQL Driver

HTML/Java

Web Browser

HTTPServer

CORBAServer

(5)CORBA

IIOP

(1) Get HTML Page

(2) Get Applet

(3)Load Applet

(4) Invoke CORBA

CORBAORB

ODBC Architecture

Appl

ODBC Driver Manager

TCP/IP

Appl Appl

SybaseDriver

OracleDriver

MS SQL Driver

ODBC API

Service provider API

DB2 Sybase Oracle

ESQL/DRDA NetLib SQL*Net

•ODBC is a Microsoft extension of the SQL Access Group’s CLI

•It provides the possibility for writing DBMS-independent C code

•A Driver Manager is used to ensure that the right DBMS-driver is loaded automatically at database connect time

•An ODBC driver must provide at the very least ANSI SQL89 functionality

•ODBC 2.0 defines 61 API calls divided into:

•Core (DB connection, execution, commit, rollback, results handling, exception handling)

•Level 1 (access to metadata, BLOB’s, driver specific functions)

•Level 2 (cursor support)

•ODBC is an example of a CLI that supports non-relational sources e.g. MS Excel spreadsheets.

•JDBC support transactions against a DBMS (

Key Points:

JDBC Architecture

JavaAppl

JDBC Driver Manager

Protocol

JavaAppl

JavaAppl

JDBC-ODBC Driver

Driver toSybase

Driver toOracle

ODBC Driver

JDBC API

Service provider API

MSSQL

ServerSybase Oracle

•JDBC is a SQL CLI written entirely in Java

•It provides the possibility for writing DBMS-independent Java code

•A Driver Manager is used to ensure that the right DBMS-driver is loaded automatically at database connect time

•JDBC drivers are ODBC-bridged or native

•A JDBS driver must provide at the very least ANSI SQL92 Emtry Level functionality

•JDBC Interfaces are divided into:

•JDBC Core

•Java Language Extensions

•Java Utility Extensions

•SQL Metadata

•JDBC support transactions against a DBMS (commit, rollback, auto commit)

•JDBC does not support 2-phase commit against multiple databases

Key Points:

NetLib SQL*Net


JDBC Scenarios

JDBC Scenario 1: Invoking a SQL Query

Client

JDBCDriver

Manager

getConnectionConnection

Statement

executeQuery

createStatement

ResultSet

next

getInt

getString

close

close

JDBC Scenario 2: Invoking a Prepared Command

Client


PreparedStatement

setAutoCommit

prepareStatement

setInt

setShort

executeUpdate

commit

close

DriverManager

close

JDBC Scenario 3: Invoking a Stored Procedure

Client

JDBCDriver

Manager


CallableStatement

registerOutputParameter

prepareCall

registerOutputParameter

execute

getXXX

close

close

JDBC 2-tier

JDBCDriver Manager

Driver Driver

DbmsServer

DbmsServer

Tier 1Client

Tier 2DBMS

HTML

JDBC 3-tier (a)

HTML

DbmsServer

DbmsServer

ORBORB

ORBORB JDBC

Driver Manager

Driver

Driver

Tier 1Client

Tier 2JDBC Server

Tier 3DBMS

JDBC 3-tier (b)

HTML

DbmsServer

DbmsServer

ORBORB

ORBORB

JDBCDriver Manager

Driver

Driver

Tier 1Client

Tier 2Web Server

Tier 3DBMS

ServerObjects

Datbase Integration and Optimization

OTM Characteristics - Transactions on the Web

Object Management Architecture

CORBAServices

CommonFacilities

ApplicationObjects

LifecycleEventNamingPersistenceTransactionsConcurrency

ExternalizationSecurityLicensingRelationshipTraderMessagingNotifications

Compound DocsObject Linking

HorizontalInterfaces

Vertical Domain Interfaces

Object Request Broker

Interface Definition Language The Key Idea:

Separate Interface from Implementation

An Interface is a contract between a client and a server

What is CORBA client and CORBA Server?

The contract is specified in Interface Definition Language (IDL)

IDL is a specification language, NOT a programming language

IDL describes only interfaces, not implementations

IDL can be mapped to a variety of programming languages such as C++, JAVA, Smalltalk, C, Ada, Cobol, etc.

IDL also provides Standard Abstractions for Middleware Services

Focus On: Middleware Services

“Standard IDL based interfaces for Classic Middleware Services”Object Request Broker

Trading

Security

Lifecycle

Messaging

Naming

Events

Trans-actions

Interface Definition Language Interface Definition Language (IDL) is a contractual, declarative

language that specifies object interfaces.

IDL defines interfaces that object “speaks”

IDL is a description of the functionality that object has. It typically specifies the attributes and operations belonging to that interface, as well as their signatures.

Interfaces contain operations

An operation’s signature includes the type of “in” and “out” parameters, the type of return value, and exceptions that may be thrown by the operation.

Interface Definition Language IDL provides operating system and programming language

independent interfaces to all services and components that use CORBA.

NOT a programming language.

IDL is part of the design process.

IDL is part of the development process.

IDL - The Key to Integration

IDL interface

The key to the Software Bus

Implementation is Hidden

C++

Smalltalk

Ada

ActiveX

Java

“Service or Contract-oriented View”

C++ Smalltalk

Java

CORBA Interoperability

Solaris MVS HP-UX Alpha NT

NT JavaOS AIX IRIX

C++ COBOL S/Talk

IIOP

Ada

C++ C++JavaVC++

DAIS VisiBroker M3 Orbix

VisiBroker Component Broker Orbix

CORBA Interoperability Internet Inter-ORB Protocol

Provides on-the-wire interoperability between:

CORBA ORBs

Java programs using RMI

RMI over IIOP

CORBA and the Net

CalculationEngine object { CalculateYield Curve (…) CalculateSwap (…) }

CORBA Object C++

AIX

CalculateYield Curve (…) CalculateSwap (…)

Shared Object CNetscape Navigator

ORB

CORBA

Windows NT

ORB ORB

Calculation Engine Object

CalculateYield Curve (…) CalculateSwap (…)

Excel

IIOP

Web Application Development with CORBA

C++

Compile

Link

C++

Compile

Link

Stubs Skels

CORBA

Compile

IDL

Client Servants

Client Server

Write specification for each CORBA

object using IDL

Use the CORBA compiler to generate

Stubs and Skeletons

Write the client application code

Write the server objects code(i.e.

Servants)

Write a container( i.e. Main) that the

Servants can reside in

Compile and link the Server and the

client

Deploy

Run

Web Naming Web Naming is a URL based naming scheme that allows Object References

to be associated with URLs and use standard URL resolution to retrieve object references.

Leverages the ubiquity of the Web to bootstrap CORBA object location.

Can be used in conjunction with CORBA Naming

A Web Naming interface called Resolver provides the Web Naming functionality.

Web Naming powerful for CORBA-enabled Applets.

Server implementations register a IOR with a Web Server using the PUT HTTP method and store the IOR in the Web Servers document root.

Clients retrieve the IOR associated with a URL using HTTP.

Development Tools for Web and CORBA Symantec Visual Café’

Inprise JBuilder

IBM Visual Age

Commercial CORBA Products IONA Technologies Orbix and OrbixWeb

PeerLogic/ICL DAIS

Inprise VisiBroker for C++ and Java

BEA WebLogic Enterprise (formerly M3)

C++ and Java bindings are available

IONA Technologies OrbixOTM

PeerLogic/ICL DAIS

BEA WebLogic Enterprise

Inprise VisiBroker ITS

Support for CORBAServices

Trader

Naming

Security

Events

Orbix DAISWebLogic Enterprise VisiBroker

Events

Transactions

Notification

(Tuxedo based)

(Tuxedo based)(DCE based)

(Encina)

(Tuxedo based)

State and Session Management

Need to maintain state between multiple user interactions

Optimized customer interaction experience through site personalization

Web site tracking

Techniques available to maintain state include

Client-side storage of state

Server-side storage of state information

files or databases

Combination of both

cleanup synchronization issue

A “session-identifier” needs to passed between the browser and the server application

Web-Based Transactions Web based transactions should allow the grouping of HTTP requests

as part of one task.

URL range or specific pages

Web middleware should support Web transactions

Server-Based Transactions

Traditional ACID transactions

TP Monitor, DBMS

Object Transaction Service (OTS) vs. Enterprise Java Beans (EJB)

A legacy integration issue

Web transactions HTTP is stateless i.e. has no

memory of previous requests

Since HTTP makes it difficult to maintain context among different invocations of the database, other means for providing transactional context are required

Developers are using various approaches to overcome this problem (e.g. by passing cookies or hidden HTML fields containing the client’s sessions and transaction id’s back to the client, e.g. Oracle Application Server)

A client/server transaction is inherently a session in which the following occurs:

Start a session on the DBMS server by logging in with name and password

Issue a BEGIN TRANS command

Issue one or more SQL INSERT, DELETE or UPDATE statements

Issue a SQL COMMIT or ROLLBACK statement

End the session by logging out or disconnecting

X/Open Distributed Transaction Processing Model

Application(AP)

Application(AP)

ResourceManager

(RM)

ResourceManager

(RM)

RM API(JDBC)

TransactionManager

(TM)

TransactionManager

(TM)XA

(JavaTransaction

API)

CommuncationsResourceManager(CRM)

CommuncationsResourceManager(CRM)XA+

XATMI, TxRPC

TX(Java

TransactionService)

X/Open DTP and Java Services

Java servletJava servlet

X/OpenResourceManager


JDBC

TransactionManager

(TM)

TransactionManager

(TM)Java

TransactionAPI

(JTA)



XATMI, TxRPC

JavaTransaction

Service(JTS)

OMG Object Transaction Services (OTS)

Transactional ClientTransactional Client

RecoverableServer

RecoverableServer

RM API(SQL)

TransactionService

TransactionService

XA



XATMI, TxRPCTX

Model Interoperability ~ OTS versus DTP

Transactional Application(AP)

ResourceManager

(RM)

JDBC

TransactionalServer

(Coordinator & Terminator)

XA(Java

Transaction API)


XATMI, TxRPC

TX(Object

TransactionServices)

Model Interoperability

TransactionalClient

TransactionalObject


RM API

ORBORB

TransactionServices

X/OpenServer

TransactionManager

CM API

XA

New Application(Objects)

Exsiting Application(X/Open)

OTS versus X/Open TX Interfaces

tx_open( )

tx_close( )

tx_begin( )

tx_rollback( )

tx_commit( )

tx_set_transaction_control()

tx_set_transaction_timeout()

No equivalent

No equivalent

Current::begin()

Current::rollback()

Current::commit()

No equivalent

Current::set_timeout()

TX Interface OTS

OTS versus X/Open AX Interfaces xa_start( )

ax_reg( )

xa_end( )

ax_unreg( )

xa_prepare( )

xa_commit( )

xa_rollback( )

xa_recover( )

No equivalent

xa_forget( )

Receiver::received_request()

Current::resume()

Receiver::sending_reply()

Resource::prepare()

Resource::commit()

Resource::rollback()

No equivalent

Resource::forget()


Optimising Techniques

Tuning JDBC Connection Pools - multiplexing clients on pre-allocated database

connections ~ (e.g. weblogic.Admin class’ create_pool method)

“Cached” connections - reuse database connections for clients

Caching results data in web server or in JDBC Server

Use of stored procedures or prepared commands

Let a transaction service handle transaction context on behalf of transactional client

Let a server object handle JDBC sessions on behalf of client ~ saves message overhead across firewalls and web servers

systems for e-business

Documents

web interface

web transactions

web integration1

new web applications

application code

application distribution

dbmsweb integration

unixnt applications