partner webcast – oracle exalogic elastic cloud: enterprise configuration & deployment guide -...

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 1

CUSTOMER LOGO

“This slide format serves to call attention to a quote from a prominent customer, executive, or thought leader in regards to a particular topic.” Name

Title, Company Name

blogs.oracle.com/IMC

Oracle Exalogic Elastic Cloud: Enterprise Configuration & Deployment

Gokhan Gungor

Oracle ISV Migration Center Consultant

[email protected]

@gokhangungor1


<Insert Picture Here>

Agenda

Introduction to Exalogic

Exalogic Engineered System Architecture

Exalogic Enterprise Deployment

Exalogic Optimization, Monitoring and Management

Q&A


Exalogic Elastic Cloud I Best in Class

Engineered System, Best ROI

Extreme Performance for Java Applications

Extreme Performance for Oracle Business Applications


Up to 5X processor

utilization advantage

for many other

application types: • Java EE and Fusion

Middleware

• Oracle Applications

• OLTP with Oracle Exadata

Exalogic 1.x ROI I Performance

25% processor

utilization

advantage for

standard Linux or

Solaris

applications

25% 50% 2X 3X 5X

Exalogic ROI Standard Hardware ROI

Co

mp

ara

tive

RO

I

Exalogic Processor Utilization Efficiency Advantage

Less hardware, Less software: Easier to Deploy, Manage, Maintain


Enterprise IT Challenges Impacting the Business

Complexity Unique Stack Each

Time

Poor Utilization

Utiliz

ation

Number of Servers


Solution: Integrated Systems Strategy Building Next Generation Datacenters


Exalogic Design Principles

State-of-the-art Sun Oracle hardware components

Dense compute cluster

High performance networking

Balanced system design – CPU/Memory/Networking/Storage

General applicability to mid-tier workloads

Platform for heterogeneous consolidation

Serviceability


Exalogic Logical Description

Exalogic Rack

Compute Node

Shared Storage

Appliance

Compute Node

InfiniBand

Switches

Client

Network

IB

Network Power Distribution Units (PDUs)

Mgmt

Switch

Mgmt

Network

40Gb

10Gb 1Gb

Clients

40Gb Etherne

t

Gatewa

y

Private

IPoIB Oracle Homes

Logs

Binaries

Patches

User Files

FMW/Application

Execution

WLS Domains


Exalogic System Components

Compute Nodes – Physical Servers – Runs middleware/application workloads

InfiniBand Leaf Switch/Gateways – Core InfiniBand switching fabric and 10GbE rack in-/egress

InfiniBand Spine Switch – Multi-rack expansion of IB fabric (Exalogic and Exadata)

ZFS Storage Appliance – High performance shared storage supporting compute nodes

Management Ethernet Switch – Physically discrete management connectivity to all system components

Power Distribution Units – Datacenter power connectivity and distribution to all system components


Exalogic Scalability

Units of expansion are “racks”

– ¼, ½ and Full – and ⅛ Rack also available now

Multiple racks combined to form a cluster on common InfiniBand fabric

– Up to 8 racks with no additional networking hardware outside the racks

– Clusters with more than 8 racks can be built using external switches

Exadata has a similar multi-rack strategy

Exalogic and Exadata racks can be “combined” to share the same

InfiniBand fabric

– Together they form one “machine” from a fabric perspective


Exalogic Configuration Seamless Expansion

Eighth Rack Quarter Rack Half Rack Full Rack Multi-rack

4 Nodes

384 GB RAM

800 GB SSD

60 TB NAS

8 Nodes

768 GB RAM

1.6 TB SSD

60 TB NAS

16 Nodes

1.5 TB RAM

3.2 TB SSD

60 TB NAS

30 Nodes

2.8 TB RAM

6 TB SSD

60 TB NAS

240+ Nodes

23+ TB RAM

48+ TB SSD

480+ TB NAS

Eighth Rack Quarter Rack Half Rack Full Rack Multi-Rack


Exalogic X2-2 Configurations Start small and grow

• Field upgradable to larger configurations

• Delivered and installed by Oracle

Exalogic X2-2 Eighth

Rack

Quarter

Rack

Half Rack Full Rack Expand to 2-8

Racks

Compute Nodes 4 8 16 30 60 – 240

3.06 GHz Xeon Cores 48 96 192 360 720 – 2880

1333 MHz RAM 384 GB 768 GB 1.5 TB 2.9 TB 5.6 – 22.4 TB

FlashFire SSD 128 GB 256 GB 512 GB 960 GB 1.9 – 7.7 TB

ZFS Storage Appliance 1 1 1 1 2 – 8

SAS Disk Storage 60 TB 60 TB 60 TB 60 TB 120 – 480 TB

NM2-GW Switch 2 2 2 4 8 – 32

NM2-36P Switch 0 0 1 1 2 – 8

Weight (Est. in lbs) 880 1000 1500 2100 4200 – 16800


Compute Nodes

SunFire X4170M2 1 RU (Rack-Unit)

CPU: 2 Intel Xeon (Westmere), 6 cores/socket, 12 threads/socket

RAM: 96GB (12 x 8GB 1333MHz LV) – 18 available DIMM slots

Disks: Dual SAS 32GB 2.5” Solid-State Drive (SSD) – Typically in RAID-1 configuration

– 6 other empty 2.5” disk drive bays, No DVD drive

3 PCI-E Slots – Slot 0: Hardware RAID using SAS HBA – PCI-E Slot 0

– Slot 1: Empty

– Slot 2: ConnectX-2 dual-port QDR InfiniBand HCA (Mellanox CX2 based) – for

user application, compute node to storage node, etc.

Network: 4 1GbE ports on motherboard – Only NET0 is connected to the

Cisco Management Switch and NET1-3 ports are not connected – ILOM and management access through NET0/eth0 interface (1GbE)

Rack 1/8 1/4 1/2 Full

Compute Nodes 4 8 16 30

EL X2-2


Term Definition Description

IPoIB IP over InfiniBand Applications connected by an IB fabric communicate using standard

IP address semantics (dual interfaces "bonded“ to be one).

Applications see standard IP stack.

EoIB Ethernet over InfiniBand Applications within an IB fabric communicate with external Ethernet

networks.

Operating System sees an Ethernet interface

SDP Socket Direct Protocol Applications communicate directly with the IB fabric bypassing the

TCP/IP stack in the operating system.

Exalogic Network Protocols


A standards-based network communication

technology often used in supercomputing

Exalogic Networking InfiniBand Fabric

* http://www.openfabrics.org

OpenFabrics Alliance standard

• Open standard *

• Leverages RDMA for more efficient use of CPUs and memory

• Improves application performance

• Cuts hardware costs by increasing CPU efficiency


Exabus – What Is It?

An assembly of InfiniBand Gateway Switches + Device

Drivers + Firmware + Software Libraries + Configuration

Files

Enables applications to make use of the Exalogic

machine

Ensure optimal performance and reliability of the system

Extends and integrates Oracle Linux, Oracle Solaris, the

Open Fabric Enterprise Distribution (OFED) with the

unique hardware design of the Infiniband gateways and

switches in the “I/O backplane” of the Exalogic system

Installed on the Exalogic system at the time of

manufacturing


Storage Overview

Standard Sun ZFS Storage 7320 Appliance read SSD & write cache

options

Same configuration for all available Exalogic rack configurations

Pre-configured and tuned at the factory for performance and reliability

Accessed from Compute Nodes over IPoIB network

Fully redundant – Active-Standby heads – automatic & transparent

failover, RAID-1 storage

Remote File-system (eg. NFS) or Block-device (eg. iSCSI) access –

NFS is recommended for middleware deployment

Snapshots and Block-level Replication capabilities

No features are disabled or removed


Oracle’s Elastic Cloud Foundation Application Grid, OS, Enterprise Manager and hardware

EL X2-2

Installed on premises Factory Assembled and Installed

Ente

rpris

e M

an

ag

er

Exalogic Elastic Cloud Software

Exalogic Elastic Cloud X2-2

Coherence

JRockit and HotSpot

WebLogic

Server Tuxedo

Linux Solaris Performance and Management Optimizations

Applications


Oracle Software Components

All software that runs on the operating systems used in Exalogic

are supported and will run on Exalogic

– Will run “as fast” as they would on same hardware outside Exalogic,

typically faster if dependent on I/O at all

Specific Oracle components have been optimized to yield

maximum performance on the engineered system

– WebLogic Server 10.3.4 (PS3)

JRockit R28.1 (PS3)

JDBC (PS3)

– Coherence 3.6 (PS3)


Enterprise Deployment Overview


EDG Task Roadmap


Tiers

Web Tier – Located the DMZ public zone

– Mandatory proxy: e.g. OHS as the web server front-end

– OHS with mod_wl_ohs plug-in

– Requests are proxied from OHS to WLS running in the app tier

Application Tier – Located in the DMZ secure zone

– WebLogic Managed Servers for running OFM components

– Run on Compute Nodes in the Exalogic Machine

– Optional OHS for load-balancing traffic between WLS instances

Data Tier – Located in the most secured network zone (the intranet)

– Exalogic <-> Oracle DB or RAC over 10Gb Ethernet

– Exalogic <-> Oracle Exadata Database Machine over IPoIB

Exalogic


Reference Topology DB (non-Exadata) with Ethernet


Reference Topology Exadata with InfiniBand


Exalogic Oracle HTTP Server Configuration

OHS deployed

outside of

Exalogic

(DMZ Public Zone)

Exalo

gic

OHS deployed

inside of

Exalogic

to load-balance

WebLogic Clusters

(DMZ Secure Zone)


Exalogic Capacity Planning Guideline

Exalogic x86 machines (Oracle Linux and Oracle Solaris) include

Compute Nodes with 12 processor cores each

Rule-of-thumb 1 JVM Instance/1 Processor Core

Reference topology demonstrates 2 Compute Nodes

Can extrapolate for configuring the other compute nodes


Exalogic Optimization


Extreme Java Performance Oracle Exalogic Elastic Cloud Software

2-3X improvement

in Database OLTP 60% more

Java Operations/sec. – GridLink for Exadata – Run-time connection

load balancing

– XA Affinity

– JDBC over SDP

– Enhanced buffer handling

for InfiniBand

– Optimized multi-core

scheduler

– Cluster IPC

multiplexing over SDP

– Scatter-gather IO

0

50

100

150

200

250

Update Insert Delete

Tx/S

ec

(Higher is better)

Standard WebLogic Server GridLink for Exadata

0

50

100

150

200

250

300

350

400

450

CreateWS Purchase Manage Browse

(Lower latency is Better)

Standard Platform Exalogic

Up to 10X faster

response time


The “Master Switch”: Domain level ExalogicOptimizationsEnabled


The “Master Switch”: Domain level ExalogicOptimizationsEnabled

Domain->Configuration->General

DomainMBean Attribute (“ExalogicOptimizationsEnabled “)

Enables the following MBeans (can be overridden) – SSLMBean.JSSEEnabled=true

– KernelMBean.MuxerClass=weblogic.socket.NIOSocketMuxer

– KernelMBean.scatteredReadsEnabled=true

– KernelMBean.gatheredWritesEnabled=true

– <DS>.JDBCOracleParams.OracleOptimizeUtf8Conversion=true

– <DS>.JDBCOracleParams.OracleEnableJavaNetFastPath=true

– ClusterMBean.SessionLazyDeserializationEnabled=true

Self-tuning thread pool algorithm


Multiple Replication Channels

Not controlled by ExalogicOptimizationsEnabled flag Create Replication Channel

– Environment->Servers-><server>->Protocols->Channels – ClusterMBean.replicationChannel – Cluster->Configuration->Replication

ReplicationChannel NB: Enable Lazy Session Deserialization (controlled by

ExalogicOptimizationsEnabled flag)

Cluster Replication Ports – Environment->Servers-><server>->Configuration->Cluster – ServerMBean.replication-ports – <replication-ports>8001-8010</replication-ports>


SDP Protocol Support

Not controlled by ExalogicOptimizationsEnabled flag

Any network channel can be SDP-enabled, but only tested and

recommended for Replication and JDBC

SDP-enabled JDBC thin driver – -Djava.net.preferIPv4Stack=true

– (PROTOCOL = SDP)

– Oracle Net Services SDP Support

SDP-enabled dynamic listen threads – -Djava.net.preferIPv4Stack=true

– Configure channel MBean SDPEnabled=true

http://download.oracle.com/docs/cd/B28359_01/network.111/b28316/performance.htm


Exalogic Inter-process Communication

60% higher workload, 1/2 latency

Java NIO based Parallel Muxer reduces lock contention

Multiple channel selectors to maintain 3 different lists of open sockets

Non-blocking mechanism

Larger packet sizes (64K for IPoIB and >=32K for SDP) reduces processing overhead

WebLogic WebLogic SDP over InfiniBand

Para

llel

Mu

xe

r

Para

llel

Mu

xe

r

Faster Network

Fewer, Larger Messages

Exalogic Inter-process Communication


Self-Tuning Thread Pool Optimized for Exalogic Compute Node Processors

Thread pool aware of

processor cores

– 2 processors x 6 cores x 2

hyper threads per core =

24 hardware threads

Rapid adjustment to

varying workloads

– Optimal adjustment is 24

threads per time period Exalogic X4170 Compute Node

Self Tuning WebLogic Thread Pool

X86 Dual Processor

6 Cores per Processor with Hyper-Threading

WebLogic Server

Operating System

Hotspot or JRockit

Work

Request

Work

Request

Work

Request

Priority Request Queue

Adjust by 24 __________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

Current Thread Count __________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____


Exalogic Elastic Cloud Management and Monitoring


Integrated Lights Out Manager (ILOM)

Specialized hardware and software for managing and monitoring Sun

hardware devices

– Dedicated Service Processor (SP) present in addition to the normal processor(s)

Provide Browser User Interface (BUI) based and Command Line Interface

(CLI)

Pre-installed on all Compute Nodes, Storage Heads and InfiniBand Switches

– Runs independently of any operating systems

– No SP on IB Switches – ILOM firmware installed in base OS

Accessible from dedicated management IP address

Supports SNMP and IPMI (Intelligent Platform Management Interface) clients


ILOM – Integrated Lights Out Manager

Remote Hardware Monitoring

Monitor System Status and event

logs

Error and Fault Management

Environmental and Power

reporting

Alerting

Indicators - Manage fault LEDs

SNMP Traps, IPMI PETS,

notifications

Hardware inventory & status

NIC Card MAC addresses

Access to system part numbers,

versions and serial numbers

Auditing of ILOM users, and

management user accounts

including access

right control

Monitor system BIOS/POST and

sensors


ILOM System Monitoring and Management CLI Examples

->show /SP/network

# ssh el01cn01-ilom

Userid: root

Password: welcome1

->cd /SYS/T_AMB

->show

-> show /SP/faultmgmt

Login with SSH for

Command Line Interface

Examine the Network Target

Examine System Temperatures

Examine System Faults

Clear a System Fault


ILOM Alert Management


Monitor Exalogic: Enterprise Manager Analyze and correlate data across the entire software stack

Exalogic Application Grid management

– Deployed applications

– WebLogic Domains & Managed Servers

– Coherence clusters

– Physical compute node status

Operational Views

– Operational status

– KPI, Response & Load chart

– Alerts & policy violations

– Configuration changes (24h)

– Component and JVM metrics


Monitor Exalogic: OpsCenter Analyze and correlate data across the entire environment

Hardware monitoring

– Hardware fault monitoring

– Operating System

performance monitoring

– Patch automation

– Configuration and compliance

reporting

© 2010 Oracle Corporation


Exalogic Elastic Cloud Resources


Exalogic Resources

Oracle Exalogic TV

– http://www.youtube.com/exalogicTV

– Direct from the Oracle Exalogic Product Management & Development Team

Oracle Exalogic Elastic Cloud Knowledge Zone

– http://www.oracle.com/partners/en/knowledge-zone/middleware/exalogic-elastic-cloud-

398338.html

– Oracle Partner Network Guided Learning Path, Specialization

Oracle ExaStack Program

– http://www.oracle.com/partners/en/opn-program/oracle-exastack

– Enables Oracle ISV partners to leverage a scalable, integrated infrastructure to deliver

their applications tuned, tested and optimized for high-performance

http://www.youtube.com/exalogicTV

http://www.youtube.com/exalogicTV

http://www.oracle.com/partners/en/knowledge-zone/middleware/exalogic-elastic-cloud-398338.html










http://www.oracle.com/partners/en/opn-program/oracle-exastack






CUSTOMER LOGO

“This slide format serves to call attention to a quote from a prominent customer, executive, or thought leader in regards to a particular topic.” Name

Title, Company Name

twitter.com/oracleimc

partner webcast – oracle exalogic elastic cloud: enterprise configuration & deployment guide -...

Technology