innovations in grid computing with oracle coherence

1 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.

Insert Information Protection Policy Classification from Slide 8

Innovations in Data Grid Technology with Oracle

Coherence

Bjorn Boe

Solution Architect


reserved.


The following is intended to outline general product use and direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


reserved.


Presentation Objectives

To Understand…

1) What is driving the need for Distributed Caching and In-Memory Data Grid products?

2) How Coherence improves performance, enables linear scalability, and provides availability for shared services

3) Coherence’s Strategic Direction and Latest Features


reserved.


Internet Scale

Problem: How to deal with the flood of customer demand for services that occurs when you expose your infrastructure to a large, potentially unbounded network.

- Or, stated more positively -

Opportunity: How to build infrastructure that allows you to scale business and improve margins by exposing your services to a large, potentially unbounded network.


reserved.


The Internet Scale Problem is Multi-Faceted

• Virtually unlimited number of customers

• Services exposed to partners’ overuse

• Customers’ experience relies on partners’ services

• Global and Real-time


reserved.


Scaling Infrastructure to Meet the Problem

• Applications must act on data in real-time

– Access, query, and aggregate the data

– Trigger business actions based on changes to data

• Must scale predictably and cost-effectively

– Failure to do so results in loss of revenue and cost overruns

– No time to re-architect in Internet time

• Need a new paradigm in data management


reserved.


More. Faster. Cheaper.

•Scale to more users and more data

•Scale predictably and cost-effectively

•Act on data in real-time

•Reduce spending on backend systems

Get on the Grid


reserved.


Oracle Coherence

• Coherence: in-memory data grid

– Distributed caching to applications

– Scalable extreme transaction

processing

– Real-time eventing, query, and

map/reduce aggregations

– Abstraction from back-end data

sources

– High-availability to applications

End Users

Application Servers

Mainframes, Databases, etc.

Application Objects


reserved.


Oracle Fusion Middleware Complete, Open, Integrated, Best-in-Class

Development Tools

User Engagement

Social Web Mobile

Enterprise Management

Identity Management

Content Management

Business Intelligence

Business Process Management

Service Integration

Data Integration

Cloud Application Foundation


reserved.



To Understand…




reserved.


Enterprise Application Scalability Challenges

Service Clients Browsers, supplier and partner clients, application clients, mobile apps, and other service consumers.

Service Implementation Custom applications, BPM processes, service bus endpoints, UI services, and other service providers.

Ease of Scaling

Shared Services RDBMS, cloud services, supplier and partner services, mainframe applications, and so on.

• Grows and scales naturally

• Increasing load • Computationally

intensive work • Highly dependent on

shared services

• Complex and expensive to scale

• High latency, SPOB, and SPOF


reserved.


Traditional Clustering

• Dependent on shared service

latency

• Increase in size of cluster leads to

increase in load on shared service

Service Node 1

Shared Service

Service Node 2

Service Node N

Cached Objects

Cached Objects

Cached Objects

Service Clients

• Inconsistent view of data across

cluster nodes

• No possibility for in-memory

analytics

• Limited in-memory cache due to

• Capacity implications

• Java garbage collection


reserved.


Enter the Coherence Data Grid

• Clustered Caching

– Consistent view of data

across cluster

• Partitioned Caching

– Increase in size of cluster

adds capacity but does not

decrease performance

• Distributed Data Processing

– Enables a scalable platform

for high performance in-

memory analytics

Service Node 1

Ease of Scaling

Service Clients

Shared Services

In-memory Data Grid

Service Node 2

Service Node N


reserved.


How to Integrate with Coherence

Two Options

Custom integration through Coherence API

- or –

Through Existing ‘Switch-On’ OOTB Integrations

Oracle Coherence*Web: HTTP session state

replication with WebLogic, GlassFish, Others

Oracle TopLink Grid: JPA object-relational data

Other Oracle Products that integrate Coherence


reserved.


ActiveCache: Premier WebLogic integration

• Distributed HTTP Session State Replication via WebLogic SPI

• Configuration, Lifecycle and Monitoring of Coherence Clusters

and Servers via WebLogic Admin Console & MBeans

• Dependency Injection in JEE modules

– Configure in web.xml, use annotations to inject named

cache

– Alternative to CacheFactory.getCache() lookup


reserved.


Gartner, Magic Quadrant for Enterprise

Application Servers

This Magic Quadrant graphic was published by Gartner, Inc. as part of a larger research note and should be evaluated in the context of the entire report. The Gartner report is available here: http://www.gartner.com/technology/reprints.do?id=1-17GUO5Z&ct=110928&st=sb

Source: Gartner, Magic Quadrant for Enterprise Application Servers – Massimo Pezzini, Yefim V. Natis, Kimihiko Iijima, Daniel Sholler, Raffaella Faveta – September 26, 2011


reserved.


Coherence Basic Inteface

Custom Integration

• Application uses

Coherence API to

store/access its data

• Application cache

properties are configured

externally through XML

file

• Basic cache access

through NamedCache

Map interface

NamedCache neCache = CacheFactory

.getCache("NetworkElement");

…

NetworkElement ne = new

NetworkElement(“ID-321”, “OC-

196”, “Nortel Networks);

neCache.put(“ID-321”, ne);

…

NetworkElement ne =

neCache.get(“ID-321”);


reserved.


Data Management

Partitioned Caching

• Extreme Scalability: Automatically, dynamically and transparently partitions the data set across the members of the grid.

• Pros: – Linear scalability of data capacity

– Processing power scales with data capacity.

– Fixed cost per data access

• Cons: – Cost Per Access: High percentage

chance that each data access will go across the wire.

• Primary Use: • Large in-memory storage environments

• Parallel processing environments


reserved.


Data Management

Partitioned Fault Tolerance

• Automatically, dynamically and transparently manages the fault tolerance of your data.

• Backups are guaranteed to be on a separate physical machine as the primary.

• Backup responsibilities for one node’s data is shared amongst the other nodes in the grid.


reserved.


Data Management

Near Caching

• Extreme Scalability and Extreme Performance: The best of both worlds between the Replicated and Partitioned topologies. Most recently/frequently used data is stored locally.

• Pros: – All of the same Pros as the Partitioned

topology plus…

– High percentage chance data is local to request.

• Cons: – Cost Per Update: There is a cost

associated with each update to a piece of data that is stored locally on other nodes.

• Primary Use: – Large in-memory storage environments

with likelihood of repetitive data access.


reserved.


Data Source Integration

Read Through / Write Through

• Read Through

– All data reads occur through cache

– If there is a cache miss, the cache will load the data from the data

source automatically

• Write Through

– All data writes occur through cache

– Updates to the cache are written synchronously to the data

source


reserved.


Data Source Integration

Write Behind

• All data writes occur through cache

• Updates to the cache are written asynchronously to the

data source


reserved.


Data Processing

Parallel Query

• Programmatic query mechanism

• Queries performed in parallel across the grid

• Standard indexes provided out-of-the-box and supports implementing

your own custom indexes


reserved.


Data Processing

Continuous Query Cache

• Automatically, transparently and dynamically maintains a view locally

based on a specific criteria (i.e. Filter)

• Same API as all other Coherence caches


reserved.


Data Processing

Invocable Map

• The inverse of caching

• Sends the processing to where the data is in the grid

– Once and only once guarantees

– Processing is automatically fault-tolerant

• Processing can be:

• Targeted to a specific key or collection of keys

• Targeted to any object that matches a specific criteria (i.e. Filter)


reserved.


Data Processing

Events

• Build Complex Event-Driven Apps

• Java Bean Model

• Key-Based

• Filter-Based


reserved.



To Understand…



3) Coherence’s Strategic Direction and Latest Features


reserved.


Release Overview

• Oracle Coherence 3.7.1

– Strategic investment in:

• Exalogic Innovation

• Ease of Use

• Integration Points

– Significant hardening

© 2011 Oracle Corporation


reserved.


Elastic Data Virtual Memory for Your Data

Cache Server 1 Cache

Server 1

Heap Heap

Flash Flash


Server 2

Heap Heap

Flash Flash


Server 3

Heap Heap

Flash Flash

• Revolutionizes the scale of data grids

• Use block storage for cached data

– Optimized for flash

– Works well with NAS, Disk, etc.

• Simplifies capacity planning and deployment

– Configure amount of on-heap data to store

– Overflow data written to block storage

– Data stored in buffers until flushed to journal

– Reduces chances of Out-Of-Memory errors


reserved.


Scalable Distributed Cache to Scalable, Event-Driven Real-Time Applications

Coherence Cluster

Coherence Cluster

Replication

Per Member Cache

Store/Loader API

Data Source

Map API Map API Real-Time Client API Real-Time Client API

Query Filter API

Query Filter API

Aggregation (Map-Reduce)

API


API

Grid Processing API

Grid Processing API

Java / C++ / .NET Java / C++ / .NET

REST (Python, PHP,…)

REST Support Simplified Integration


reserved.


Coherence API Types

Map API Map API Real-Time Client API Real-Time Client API

Query Filter API

Query Filter API


API


API

Grid Processing API

Grid Processing API

Scalable Distributed Cache to Scalable, Event-Driven Real-Time Applications

Coherence Cluster

Coherence Cluster

WAN Replication

WAN Replication

Per Member Cache

Store/Loader API

Data Source

Java / C++ / .NET Java / C++ / .NET

REST (Python, PHP,…)


reserved.


Query Explain Plan

• Evaluate query cost and

index effectiveness

• Quicker time to market of

optimized Coherence

solutions

• Explain Plan

• Provides the estimated cost of

evaluating a filter as part of a

query operation

• Trace • Performs the associated query

• Provides the actual cost of

evaluating a filter as part of a

query operation.



reserved.


Live Objects

New Coherence Incubator Project

• Building blocks for an event-

driven finite-state machine – …that is distributed

– …and scalable

– …and resistant to machine failure

• Architectural possibilities:

– Stated Event Driven Processing

– Ripple Effect


reserved.


Coherence on Exalogic Engineered System Optimized Scalability and Performance in a Box

• Coherence optimized for Exabus

• Pre-configured network/compute nodes

• Elastic Data: Expand Capacity with Flash

• Easy deployment as demand spikes

• Grow from ¼ to multi-rack

4x Throughput,

1/6th Latency!!!


reserved.


Coherence Integrations

Meta-data Services (MDS)

Real-time Decision

SOA Infrastructure

Oracle ADF Oracle GlassFish

Oracle Access Manager

Oracle Data Integration

TopLink

WebLogic Portal

Oracle Service Bus

Service Delivery Platform

WebLogic Server

BI Publisher

Oracle PeopleSoft

WebCenter

Oracle BEAM

ATG

Oracle CEP

Oracle Entitlement Services

Oracle Secure Token Service

Oracle IGBU

Oracle CGBU

ATG Hosted Services Data Cache Simple Clustering

Data Grid Advanced Capabilities

Oracle Identity Federation

Oracle Web Services Manager

SOA Technology Adapters

SOA Business Rules

SOA Human Workflow

SOA BPEL

SOA Coherence Adapter

Oracle Golden Gate


reserved.


GoldenGate/Coherence Integration

• Simplifies Coherence use in shared

database environment

– Propagates DB updates to Coherence

– GoldenGate captures changes to

database tables

– TopLink maps database changes to

cached objects

– Availability—2012

– GoldenGate 11gR2

– Available as patch to WebLogic 12.1.1

and 11.1.1.6/10.3.6

– Oracle database only

Database

Coherence Enabled

Application

Coherence

Read

/Write Th

rou

gh

Custom Packaged

Application

Direct D

atabase U

pd

ates

Update Coherence

Update Cohere

nce

GoldenGate Java Client

TopLink Grid Coherence

Cache Updater

Datab

ase Log Stream

Cap

ture

<entity

class="Address">

<table

name="ADDR”/>

<attributes>

<id name="id">

<generated-

value />

</id>

<version

name="ver“/>

</attributes>

</entity>

JPA Metadata


reserved.


Oracle Enterprise Pack for Eclipse (OEPE)

• Coherence Project Configuration

– Facets

– Library Management

– Descriptor Generation

• Runtime Configuration

– Launch Config Editor

– Run/Deploy/Debug from IDE

• Configuration Support

– tangosol-coherence-override.xml editor

– Validation

– Context Sensitive Help Integration

– Cache Configuration Editor (OEPE 11.1.1.8)



reserved.


Oracle Enterprise Manager 12C

Applications and

Business Services

Platform as a Service

Infrastructure

as a Service

Plan

Setup

Build

Test

Deploy

Monitor

Manage

Meter & Charge

Optimize

DBaaS MWaaS

D P V &

DEEP PERFORMANCE VISIBILITY & ALERTS-BASED MONITORING

R - JVM D REAL-TIME JVM DIAGNOSTICS FOR COHERENCE NODES*

C C D CENTRALIZED CACHE DATA MANAGEMENT

A P AUTOMATED PROVISIONING


reserved.


•Completely customizable performance views – save different views for different monitoring requirements of the enterprise

•Proactive Alerts and notifications based on granular thresholds

•Topology view to show associations and dependencies

•Log monitoring – generate Alerts based on log patters

•Advanced monitoring – push replication, reap sessions, transactional caches, etc

Deep Performance Visibility & Alerts Based Proactive Monitoring

•Real-time threads analysis – find call stack, locks, method local variables, etc

•Real-time heap analysis – heap distribution in different spaces (eden, perm-gen, etc), garbage collection

•Differential heap analysis – pin point heap leaks by comparing two snapshots

Real-time JVM Diagnostics for Coherence Nodes

Oracle Enterprise Manager 12C Monitoring & Diagnostics


reserved.


• Query based data operations

• Central UI for key cache operations – view, update, import, export, purge, add/remove indexes

• Drastically saves time in cache operations and makes it easy for administrators

• Save queries for future references

Centralized Cache Data Management

• Maintain ‘gold image’ in the software library

• Provisioning new cluster or add nodes to an existing cluster

• Support Unicast Addrress as well as WKA based clusters

Automated Provisioning

Oracle Enterprise Manager 12C Administration & Provisioning


reserved.


For More Information

• General Information: http://coherence.oracle.com

• Coherence Training: http://education.oracle.com

• Coherence Discussion Forum: http://forums.oracle.com

• Coherence User Group on Linkedin

• “Oracle Coherence 3.5” by Aleks Seovic

• Email: [email protected]

http://coherence.oracle.com

http://education.oracle.com

http://forums.orace.com

mailto:[email protected]


reserved.


Q & (hopefully) A

innovations in grid computing with oracle coherence

Technology

oracle andor

data management6 copyright

oracle coherence coherence

information purposes

grid7 copyright

shared services10 copyright

sole discretion of oracle

memory data grid products