exalogic workshop overview__hardwarev4

1 | © 2011 Oracle Corporation – Proprietary and Confidential

Fran Navarro

Infraestructura Optimizada Oracle: Exalogic


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remain at the sole discretion of Oracle.


Agenda • Oracle Engineered Systems Strategy

• Workshop Exalogic– Exalogic Hardware– Exalogic Virtualized – Exabus– Exalogic HA + DR– Exalogic – Multi Tenacy

• Exalogic Management


ENGINEERED TO WORK TOGETHER


Before

A NEW ERA: ORACLE ENGINEERED SYSTEMS

HARDWARE AND SOFTWARE ENGINEERED TO WORK TOGETHER

Now


100’s of Components 1 Machine 1000’s of Hours 1 Day

GROUNDBREAKING TIME TO MARKET

FEWER PIECES TO BUY, DEPLOY & MAINTAIN


“By year-end 2015, integrated systems will account

for 35% of total server shipment value.”

THE INTEGRATED SYSTEMS TREND IS CATCHING ON

Gartner Symposium/IT Expo presentation, “Is the Concept of the Server Obsolete – or in Need of Redefining?”, November 2012


Compute Appliances Have a Role Within Gartner's Fabric Continuum

OVCAExa-Systems


Oracle Exalogic Elastic Cloud

EXTREME PERFORMANCE FOR JAVA APPLICATIONS

EXTREME PERFORMANCE FOR ORACLE BUSINESS APPLICATIONS

NATIVELY LEVERAGES EXADATA

BEST CONSOLIDATION PLATFORM


Fusion Middleware Performance on Exalogic

SOA 11gResponse

Time

Standard Hardware Exalogic

9X

Tuxedo 11g Response

time


7X

UCM 11gThroughput


3X10X

0.16ms

1.1ms

58ms

520ms

5,640 tps

17,340 tps

ADF 11gConcurrent

Users


24,000

240,000

10X


Engineered Systems: Scale With Your Business

Quarter Rack

Half Rack

Full Rack

Multi-rack

• Seamless hardware upgrade• Flexible software licensing


Oracle Support for Performance and Availability

• 24/7 support coverage

• Specialized Engineered Systems Support Team

• 2-hour onsite response to hardware issues1

• New updates and upgrades for Database, Server, Storage, and OS software

Complete. Integrated. Proactive. High Availability Services.No Additional Cost.

ORACLE PLATINUM SERVICES

Better support for the complete Oracle stack– Includes higher support levels for Database software

24/7 Oracle remote fault monitoring

Industry-leading response times:– 5 Minute Fault Notification

– 15 Minute Restoration or Escalation to Development

– 30 Minute Joint Debugging with Development

Patch deployment by Oracle engineers

Available now for certified configurations on Oracle SuperCluster1 Covered system must be within an Oracle two-hour service area to receive two-hour response as a standard service.

Oracle engineers perform patching services up to four times per year

http://www.oracle.com/us/support/library/certified-platinum-configs-1652888.pdf


Exalogic Fundamentals

16 | © 2012 Oracle Corporation |

Exalogic Elastic Cloud Overview

• Exalogic Hardware

• Exalogic Virtualized

• Exabus (In deep in Other ppt)

• Exalogic HA + DR

• Exalogic – Multi Tenacy

• Exalogic Management

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.17

X3-2

ExalogicStatus Quo

Applications & Middleware

OS

Virtualization &Cloud Management

Compute

Storage Layer

Networking

Exalogic vs. the Status Quo

Exalogic Hardware


Exalogic X2-2 I Complete, Integrated

StorageDisk array and two storage heads with flash cache provides

shared storage for the Exalogic system software

I/O FabricConverged I/O fabric connects all system components together

and connects the system to the data center networks

Compute4, 8, 16 or 30 servers, each of which can run a single instance of

Operating System or multiple virtual servers


Exalogic X3-2Converged Infrastructure

Enterprise-class, integrated Network Attached Storage ZFS Clustered for high availability 60 TB SAS disk, 4 TB read cache, 292 GB write cache Clones, snapshots, remote replication

Integrated Storage

40 Gb/sec internal I/O backplane 10 Gigabit Ethernet connectivity to datacenter

Internal I/O Fabric and Data Center Connectivity

2 socket, 8-core, 2.9 GHz Intel Xeon processors 256 GB of 1600MHz DRAM 480 cores in a Full Rack; also available in

1/2,1/4,1/8 or multi-rack configurations Redundant SSD, power, cooling, InfiniBand

Compute Power


Exalogic X3-2Seamless Scalability

4 Nodes 8 Nodes 16 Nodes 30 Nodes 240+ Nodes1 TB RAM

800 GB SSD60 TB NAS

2 TB RAM1.6 TB SSD60 TB NAS

4 TB RAM3.2 TB SSD60 TB NAS

7.7 TB RAM6 TB SSD

60 TB NAS

61+ TB RAM48+ TB SSD

480+ TB NAS

Multi-RackFull RackHalf RackQuarter RackEighth Rack


Exalogic X3-2 Hardware ArchitectureSystem Design

Data CenterService Network

Management Network (GbE)

Data CenterMgmt Network

Exalogic X3-2

Ethernet Gateways

StandardOracle

Database

Exab

us

QD

R InfiniB

and I/O B

ackplane

Direct IB Integration:• Exadata• Additional Exalogic

configurations•ZFS Storage Appliance•Backup Media ServersManagement

SwitchStorage

Compute Nodes

…

GbE

Spine Switch

10GbE

Exadata

GbE GbE


Exalogic Integration I Exabus (InfiniBand)

Compute Nodes

InfiniBand Gateways

Management Switch

Compute Nodes

InfiniBand Spine Switch

Exalogic RackInternal I/O Back-plane

– 40 Gb/s per compute node

– Built on QDR InfiniBand

– Fully redundant

– Built in security and QoS

Unique Software/Firmware– Ethernet over InfiniBand bridging

– Exabus RDMA-based APIs

Exabus

Storage System


Exalogic X3-2 I Compute Node

• X3-2 – 1 RU (Rack-Unit), 4, 8, 16 or 30– (2) Intel 2.9 GHz Xeon (8-core) processors– 256 GB 1600 MHz RAM– (2) 100GB SSD/s (RAID1)– (1) Dual-port QDR InfiniBand HCA (PCIe)– ILOM and client O/S access through NET0/eth0 interface

(1GbE)


Exalogic X3-2 I Infiniband Switch / Gateway

• Sun NM2-GW– Full 32-port QDR InfiniBand switch– Fully redundantly deployed in Exalogic (2 or 4)– An “appliance” within Exalogic– 8 10GbE ports – bridged (not switched) to the IB fabric– Serves 2 roles in Exalogic

• Core InfiniBand switching function – all IB-connected components are switched through these

• Connectivity to datacenter 10GbE client network

– ILOM/Mgmt access on Exalogic 1GbE management network


Exalogic X3-2 I Infiniband Switch / Datacenter

• Sun NM2-36p– Full 36-port QDR InfiniBand switch– An “appliance” within Exalogic– Only populated in ½ and Full rack configurations– Only wired and used for multi-rack configurations– Serves the role in Exalogic of forming “fat-tree” fabric

architecture for scaling to multiple racks• High availability is maintained by the 36p switch in each rack

protecting each other (there is only 1 per rack)

– ILOM/Mgmt access on Exalogic 1GbE management network


Exalogic X3-2 I Integrated Storage

• Enterprise-grade NAS– 60TB disk capacity,– 4TB read cache, – 73GB write cache

• ZFS clustering

• Embedded software suite– Clones– Remote replication

Exalogic Sun 7320 ZFS Storage Appliance


Data CenterServiceNetwork

Exalogic Datacenter Integration I Simple View

Exalogic

Big DataAppliance SPARC

SuperClusterExadata

ZFSStorage

Standard TOR/EOR 10GbE Switches

InfiniBand

Data CenterManagement

NetworkGbE

10GbE

Power1PH or 3PH, 15 or 22 kVA, HV or LV

2.98kW-12.54kW


Exalogic Architecture I Fault Tolerance

Compute Nodes

InfiniBand Gateways

Management Switch

Compute Nodes


Exalogic RackDual Power Distribution Units (1+1)

Dual power supplies in each component (1+1)

N+1 cooling (fan) strategy

Redundant IB switches (1+1)

Dual HCA ports in every component (bonded, 1+1)

Redundant storage heads (1+1)

All SSD and HDD RAID 1+

Storage System

GbE

Exabus

Virtualized Exalogic


Virtualized Exalogic I Core Components

Hypervisor– Oracle VM 3.0.3 highly optimized for Exalogic

– Mission-critical server virtualization

Exalogic Control– Infrastructure as a Service (IaaS) management

– Manage users, servers, network

Oracle VM

ExalogicControl


= Virtualized Elastic Cloud Software

Enterprise Manager

Middleware and Business Applications

CoherenceWebLogic Tuxedo

Exabus Integration

Traffic Director

Exabus Integration

Exabus Integration

Exabus Integration

Exalogic Control

Exalogic Elastic Cloud X2-2 Hardware

ExabusOracle VM 3 for Exalogic

Physical Oracle Linux/SolarisOracle Linux Guest OS

Virtualized Exalogic I Elastic Cloud Software


Virtualized Exalogic I Key Use Cases

1. Application Consolidation– Deploy multiple applications on a single Exalogic system

2. Tenant Isolation– Provision secure Exalogic resources to multiple tenants

3. Deployment Simplification– Templates ease the path from test to production, scale up/down

4. Sub-capacity Licensing – Only the cores applications actually use need to be licensed


Near physical performance+ for virtualized applications running on Exalogic

Raw performance advantage enables superior consolidation for transactional applications in a virtualized environment

Virtualized Exalogic I Ultra-Low-Overhead

+ ~2-4% CPU overhead, 0% throughput/latency overhead


Exalogic Control I What Does It Do?

IaaS Management Runtime Engine

Runs directly in the rack

Provision resources – compute network, storage

– Infrastructure as a Service (IaaS) management

Create, deploy, manage, scale and monitor virtual machines

ExalogicControl


Exalogic Control I IaaS Roles

For Datacenter Operations– create, provision, manage and

monitor tenant “virtual data centers”

For Exalogic Tenants– provision users, virtual

machines, virtual networks and storage to applications


New Approach I Virtual Assemblies

VM images and deployment configuration and instructions

Oracle middleware and applications downloadable as ready-to-run assemblies

Develop custom assemblies with authoring tools


Assemblies: Standardization with Flexibility

Capture CompleteApplication Topology

Package IntoSingle Assembly

Metadata

Oracle Virtual AssemblyBuilder Studio

Exalogic – Exabus(in deep in other track)


What is Exabus?

• An assembly of special InfiniBand gateway hardware, device drivers, device firmware, software libraries and configuration files that allow other software ("applications") to make use of the Exalogic Elastic Cloud hardware and ensure the optimal performance and reliability of the system

• Drives the extreme performance of Exalogic

• I/O subsystem inside Exalogic– Offloads expensive CPU resources from I/O and improves overall I/O efficiency– Combination of network stack optimizations (native C++ and Java I/O APIs),

kernel bypass operations (RDMA), hypervisor bypass (SR-IOV) and on-chip network virtualization

– Software is installed on the Exalogic Elastic Cloud Hardware at the time of manufacture


Exalogic Elastic Cloud Software

= Exalogic Elastic Cloud Software

En

terp

rise

Man

ager CoherenceWebLogic Tuxedo

Exabus Integration

Traffic Director

Exabus Integration

Exabus Integration

Exabus Integration

Middleware and Business Applications

Exalogic Elastic Cloud Hardware

Exabus

Physical Oracle Linux/SolarisOracle VM for Exalogic

Oracle Linux Guest OS

Exalogic Control


Application BufferApplication Buffer

TCP IP TransportTCP IP Transport

Kernel

Standard Hardware I/O

20% Buffer Copies40% Transport Processing

40 % Kernel Context Switches

ExabusOptimized Network Virtualization

Exabus

Zero Buffer CopyDirect Memory Access

Kernel Bypass

THROUGHPUT4x

LOWER LATENCY6x

Application Application

Exalogic – HA / DR


HA within a data-centreTypical Needs

High availability within one physical site• Cope with planned and unplanned downtime

Application survives individual failures• Redundancy - no single points of failure for an

application

Service continuity

• Capacity to maintain quality of service during failure


Each rack has redundant Power Supplies, Storage Heads, Storage Disks, Network Cards, Network Switches & Network Gateways

Scales to multiple racks using InfiniBand – max. distance 100 metresRedundant Hardware

Leverage WebLogic, Coherence and Tuxedo clustering capabilities including automatic failover of requests + automated server migration

Adopt ‘Rolling Updates’ best practice for patching without downtimeMiddleware Clustering

Utilise WebLogic’s ‘Active GridLink’ connection pool feature for Oracle RAC Quick detection of database node failure and quick failover of DB connectionsHA Integration with DB

Plan for ‘N+1’ sizing for quantity of compute nodes used for a cluster - ensure even load-balancing of failed over requests to other cluster nodes

Bandwidth of InfiniBand Fabric is designed to easily accommodate maximum peak traffic, even after switch failure

Spare Capacity for Failover

HA within a data centreExalogic Capabilities & Recommendations


DR across two data-centresTypical Needs

Active/Passive data-centre• Primary data-centre runs app during normal ops

• Secondary data-centre is standby to take over if primary data-centre fails

• Assume DB is Active/Passive too

• Assume symmetric sites (same Hw, Sw & capacity)

Minimal data-loss• Recent copy of data needed in secondary site

• Complete data-consistency for secondary site


Mid-Tier

DR across two data-centres

Data-Tier

Active Site Standby Site

Business Data

Transaction Logs

JMS Messages

Mid-Tier

Sw Installations

Sw Configurations

Data-Tier

Global Load Balancer

Mw ServerMw Server

Process State

Recommended Topology

Mw Server

Mw ServerMw Server

Mw Server

ZFS Periodic Replication

Oracle Data Guard

Business Data

Transaction Logs

JMS Messages

Process State

Sw Installations

Sw Configurations

Exalogic Exalogic


The database is the ‘linchpin’ of the system – ensure an Oracle MAA Database Architect is consulted

Store all critical data in the DB: Business data, XA transaction logs, JMS messages, BPEL process state

Let DB dictate DR process

Apply critical data synchronisation at one tier – i.e. the database tier To simplify DR process and to ensure data consistency in 2nd site Oracle Data Guard + apply MAA best practices for data-tier synchronisation

Aim for a single data synchronisation approach

Persistence of software installations, domain configurations & application configurations

Use ZFS periodic replication to enable 2nd site to have fairly recent copies

Leverage Exalogic’s Storage for infrequently

changing files

Is loss of what should be transient non-critical data really important and worth the investment for rare disaster scenarios?

If you really needs this, then use WebLogic WAN HTTP Session State Replication

Use asynchronous session replication if needed

DR across two data-centresSolution Recommendations


Internet/Extranet(via Global LB)

Exalogic Integration I DR Network Topology

Compute Nodes

InfiniBand Gateways

Storage System

Management Switch

Compute Nodes


Exalogic Production Site

10GbE

GbE

Management Network

Exalogic DR

10GbE

GbE

DR Link A

DR Link B

ServiceNetwork


Backup/Disaster Recovery

Exalogic – Multi Tenacy


Multi-TenancyIB Partitions – Dynamic scalability, security & Level of Service

• Security between devices is enforced by switches– Security provisioned based on IO

device groupings called “partitions”

• Level of Service-based traffic separation– Each IO device supports up to 15

Virtual Lanes– Virtual Lanes support per-application

Quality of Service

Partition C

Partition A

Partition B


Multi-TenancyMaximum density, manageability, flexiblity

Compute Node Compute Node Compute Node Compute NodeCompute Node

Partition A Partition CPartition B

WebLogicDomain

A

WebLogicDomain

B

WebLogicDomain

D

WebLogicDomain

B

WebLogicDomain

C

WebLogicDomain

C

Application A Application A

WebLogicDomain

A

• Single application• High Availability• Dedicated CPU/Memory for

maximum performance• Maximum security

• Multiple HA applications or one composite HA application

• Common Level of Service (shared CPU, Memory, failure unit, security)

• Multiple applications

• Single availability

• Maximum density


Compute Node

1SOA1NM 1 SOA2 SOA3 SOA4 SOA5 OSb1 IDM1 IDM2 OPMN1

Compute Node

2SOA6NM 2 SOA7 SOA8 SOA9 SOA10 OSB2 IDM3 IDM4 OPMN2

Compute Node


Compute Node


App 2VerticalSlicing

Compute Node


Compute Node


Compute Node


Compute Node


App 1

• Quarter Rack

• SOA Clusters

• App 1

• App 2


Compute Node


Compute Node


Compute Node


Compute Node


App 1

App 2

Horizontal Slicing

• Dedicated

servers per

Application

• Isolation

• Lower capacity

if one node

down


Multi-TenancyMaximum security and fine grained resource allocation

• Multi-level application isolation– Balance performance, availability,

security and density per Application or Line of Business as required

• Security and resource allocation aligned, separable– Seamless integration with existing

processes and organization

Exalogic Management


Operational Benefits of Engineered Systems

Simpler Architecture. – Single purpose with fewer options and pre-configured

Standardisation– Of components, configuration and manufacture

Self Contained– Everything needed is in the box

Single toolset– All tasks and all teams more productive and more consistent

Easier support– Single vendor with known configurations

Fewer admin jobs to do...and they are easier

• Why is it easier to manage ?Minimal new skills required

• Standard Oracle software• Standard Linux / Solaris and x86

Infrastructure components• Exadata Storage Software / Exalogic

Elastic Cloud software are self managing


Potential Operational Management Benefits• Costs reduce and agility improves as use of Engineered Systems

increases

0 25 50 75 1000

20406080

100120140160

Operational Management CostsI.T. Agility

% of Oracle estate on Engineered Systems

% o

f O

rac

le e

sta

te a

dm

in

cost

s /

agil

ity

before

ES

’s

Engineered systems can be approx 20 – 30% easier to manage than ‘traditional’ AIX / Linux / HP UX / Windows platforms

As ES are used for more of a companies’ Oracle estate, the operational benefits become ever more significant.


E

xalo

gic

Exa

dat

a

Database

Storage

Servers

Network

Operating System

Virtual Machine

Analytics

Middleware

Applications • Engineered Systems are specific pre-

certified combinations of existing Oracle hardware and software. EM 12c is Engineered Systems aware

• Each Engineered system has built in tools and management systems, which are free and integrate with EM 12c.

• A single EM 12c system should be used to centrally manage both Engineered Systems and all other Oracle systems.

The Right ToolsOracle Enterprise Manager 12c provides a single integrated toolset

• Monitoring and alerting ((Performance, Availability and Configurations)

• Provisioning• Testing• S/W Problem diagnosis and resolution• H/W Fault detection and replacement• Performance Tuning• Patching and Upgrading• Backup and Recovery

• Physical and Virtual environments

• Integration with Oracle Support• Integration with other mgmt systems

• Basis for Oracle cloud computing.

Centralised, Standardised, Automated

En

terp

rise

Man

ager


The Right ToolsSingle Shared EM 12c

Exadata runs Db

Exalogic runs Apps

ELAExalogic Admin; Everything except apps

DBMADb Machine Admin; Everything

Applications Admin; Just the apps.

Business Users.

Exi

stin

g T

eam

s.N

etw

ork

/ S

ys A

dmin

/ S

ecu

rity

/ S

tora

geIn

volv

eme

nt d

ecr

eas

ing

over

tim

e.R

ole

for

Ora

cle

ES

rel

ates

mor

e to

pol

icie

s m

anag

emen

t, in

tegr

atio

n an

d st

rate

gic

plan

ning

Cloud ControlDb PacksFMW PacksApps PacksCloud MgmtOps CentreASRExalogic Control


Complete and Integrated Management

CloudControl

OpsCenter

EnterpriseManager

DeployProvisioning of Firmware,

OS, Middeleware, and Applications

Clone and Scale-out

TestFunctional Testing

Load TestingTest Management

MaintainRemote Management

- Telemetry

Phone Home- Proactive Support

My Oracle Support Integration

ManageEnd-to-End Diagnostics

- Service Levels- Root Cause

Configuration Mgmt- Change Tracking

Patch Automation- Firmware, OS


Enterprise ManagerFrom Application Operations to Data Center Operations

Exalogic

Application Operations

CC Agent

Drill BetweenSpecific Views

Enterprise Manager OpsCenter

Data CenterOperations

ILOM

Enterprise Manager Cloud Control


Hardware Schematic overview provides visibility into Network Switches, Storage, and Compute Node ILOMs.

•Support for Oracle Exalogic virtual Configurations

– Understand how your virtual guests map to the underlying Exalogic with the Hardware Schematic overview.

Enterprise Manager Complete Stack Management


Oracle Exalogic Delivery and Support Model Assembled, Tested and Delivered

CustomerPremises

Software and Support

Oracle Hardware and Services

http://www.oracle.com/...

• System Installation Service

• Patching Service

• Startup Pack

• Customer Data and Device Retention

• Operations Management and Solution Support Center

• Deploy Oracle software as needed

• My Oracle Support

• Enterprise Manager automated services

exalogic workshop overview__hardwarev4

Technology

oracle premier

oracle platinum

traditional

integrated

transaction

simpler architecture

infiniband

ethernet connectivity