next-gen data center: improving tco & roi in data centers through virtualization and blade...

©2003-2007 IMEX Research All rights Reserved

© 2007 IMEX ResearchAll Rights Reserved

Copying Prohibited

IMEXRESEARCH.COM

Next-Gen Data CenterImproving TCO & ROI in Data Centers thru

Virtualization and Blade Servers

Anil Vasudeva Principal Analyst & President [email protected]


Copying ProhibitedPlease contact IMEX for written permission



Copying ProhibitedChaos in the Enterprise . . .

Server/OS

Network

Application

Storage

Test

Backup

Disaster Recovery

Database

Financials

TM

D/R plan (MF only)

STK Silos Tape

TM

TM

TM

IBM3090-600JMVS/ESAIMS / ADABAS

BU by FDR Upstream

HPUX 10, 11.0, 11.2 MPE Sybase 11.9, 12

Fujitsu DS90UX P/M

8mm BackupLegato to DLT

IBM ADSM3490

HP OmniBack

SunSolaris 3.2, 3.4, 3.5Oracle 7.5, 8.0

IBM RS6000AIX 4.1, 4.2, 4.3DB2/6000

Compaq ProLiant 2500,5500NT 3.5, 4.0 SQL

Cheyenn e to 4mm

Batch

IBM AS/400OS/400

InventoryExchange

CAD/CAME-commerce

Lotus NotesOLTP

CICS

File transfers AIX to HP/UX via Platinum 9.9

FTP between Sun - NT

Database extracts MVS to SP2 via MQ Series 4.4

PeopleSoft

(1) Scales poorly (2) Difficult to manage (3) Reliability is questionable (4) Management costs out of control



Copying ProhibitedDC Infrastructure Nightmares Driving CIOs

ServersServers/Admin

StorageTerabytes/DBA

MIS Alerts Urgent Alerts/Day

SystemAvailability

Servers Utilization

Win 5-10%, SMP 20-35%, MF 30-50%80+%

15-30300+

1TB100TB

20-404-5

HAL- 3 (99.9%)

HAL- 5 (99.999%)

Storage Utilization

30-45% Disk, 20-40% Tape75+%

Application Application/Server

120

Power & Cooling

kW/Rack Capability

TodayTargeted

2-5 kW/Rack15-20 kW/Rack



Copying ProhibitedGenesis of Virtualization

Revenue Growth

IT Costs (Actual)

IT Costs (Budgeted)Profit

CFO vs. CIO - Shocking Observations• IT Infrastructure Investments yet to

achieve TCO/ROI Financial Objectives • Expected Boost in Corporate

Productivity not Visible to CFO/CEOs• Post 2000 Dictum Do More with LessReason – IT Spiral• Web Growth New Apps Mushroom Lo Cost Windows Servers Sprawl (Tier-1)

• Business Growth

More Computing Power> Applications/DB

Servers Sprawl (Tier-2,3)• More Servers

Storage

DC Facilities

IT Support

IT Staff • IT Costs == Business Growth



Copying Prohibited

• WW 5.1 million data centers, Costs $100-175M to build a large DC 2007

– ~$1005/Sqft, $40,000/Rack, $2,500/Server, 2.5U– 82% of installed equipment (Srvr,Stg,Ntwk) has only10% utilizn.– For every $1 invested in new IT infrastructure, $7 spent to maintain– For every $1 in new Server spending, 42c spent on Power & Cooling /2006– Virtual Servers growth outstrip Physical servers by 50% > Rise in managing VM– Blades increasing Power/Rack by 10x Need Power/Cooling, Weight, Solutions to pursue

• Consolidation– IT in Mega Data Centers, Data & Video Info Vaults at SPs

• Remote Offices, Global Reach, Always On– 66% of Enterprise workforce has remote location independent capability– Always On 24/7, Continuous Availability Mandatory

• New Technologies Necessary - a Competitive Weapon– Hi Density Blades, Multicore CPUs, Industry Std Computing Infrastructure, TB Disks,– Fast Networks: 10GbE, MPLS; Global Reach– Convergence (Voice Video, Data): Unified Communications, VoIP– Mobility & Wireless

• Focus – Focus on IT Services and not IT Resources or Infrastructure– Rise of Cloud Computing and SaaS, SOA in its wake– Complex Systems Interdependencies growing exponentially can create system wide instabilities– Automation of Low-Level Risks to free up on Initiatives aligning IT (CIO) to Business (CEO)

Data Center – Cost & Tech Trends



Copying ProhibitedEnd to End IT Infrastructure with HA & Securitywith HA & Security

Enterprise

Tier-3Data Bases

Tier-2Applications

Tier-1Edge Computing

ManagementDirectory Security Policy

Software OS Platform

Caching, Proxy, FW, SSL, IDS, DNS, LB, Web

Servers

IntrDet

IntrDet

Layer 3 Switches (Routers)

Layer 4-7 Switches

Layer 2 Switches

Stg Fibre Switches

Application,HA, File/Print, ERP, Security, SCM, CRM Servers

Database, Middleware, Data Mgmt

Servers

InternetISP

CoreOptical

NetworkingEdge

Access

ISPISP

ISP

IntrDet

Supplier/Partner

Remote Office

Home Networks

DSL

Cable Modem

MP3 VOD

WirelessCellular

xSP

App.Server

VPN

IntrDet

WebServices DBServers

ISP

ISP



Copying ProhibitedFollow SIVA© for a Scalable & Dynamic NGDC

IIntegrationntegration

Integrates physical infrastructure sing Scalable Blades to Optimize CAPSIMS:Cost, Availability, Performance, Scalability, Inter- operability, manageability & Security

VVirtualizationirtualization

Pools Resources, Optimally Provisions them for given Usage/Application to Deliver Business Service, Monitors Usage

AAutomationutomation

Automatically Maintains App Service Level Objectives using Policy Based ILM

SStandardizationtandardizationReduce CAPEX via using Industry Standard Infrastructure - HW, Interfaces, Open Source SW – OS, Middleware, and Shrink Wrap Applications > Reduced OPEX in Support, Training for Delivery of Business Services



Copying ProhibitedConsolidated Data Center

Tier-1Edge

Servers

InternetISP

CoreOptical

NetworkingEdge

Access

ISPISP

ISP

IntrDet

Supplier/Partner

Remote Office

Home Networks

DSL

Cable Modem

MP3 VOD

WirelessCellular

xSP

App.Server

VPN

IntrDet

WebServices DBServers

ISP

ISP

Enterprise

Tier-3Data Base

ServersTier-2 Apps

ManagementDirectory Security Policy

Software OS Platform

Switches: Layer 4-7,Layer 2, 10GbE, FC Stg

FC/ IPSAN

Caching, Proxy, FW, SSL, IDS, DNS,

LB, Web Servers

Application,HA, File/Print, ERP, SCM, CRM Servers

Database, Middleware, Data Mgmt Servers

Tier-1 EdgeApps



Copying Prohibited

SNMP,IETF/ CIM,

SMI-S,SMASH.

Quadrics,Myrinet,SCIInfiniBand,Ethernet / IP,Ethernet IP w/TOE,Ethernet IP w/TOE and RDMA.

SCSI,Fiber Channel,ISCSI.

Ethernet,Wi-Fi.Network Fabric

System FabricM

anag

emen

tFa

bric

Key to Integration: Interconnect Fabrics



Copying ProhibitedBlade Infrastructure:Local Area Grid (LAG© )

MidplaneW/Connectors

To Blades & BackModules

Storage: IP NAS or FC SAN Switch

Management Modules Remote Mgmt+KVM over IP

Cooling N+1Fans/Cooling ModulesPower: N+1Power Supplies

Networking: Gbit Ethernet Switches

BladeControlPanel

Processor Blades(6-24 typically)

MemoryDDR wECC

GbitEthernet

I/FSystemsMonitorModule

Micro-Processors

USB Ports, CD/Floppy I/F

© 2003-06 IMEX Research

~ GbE Switch supportsTrunking/Port AggregationFlow ControlQoS Packet PrioritizationSNMP/RMONIGMP/BOOTP/TFTP……

http://www.dell.com/content/products/compare.aspx/blade?c=us&cs=555&l=en&s=biz



Copying ProhibitedBlade Servers: Vendor Positioning Index(As of Oct 2006 - See IMEX Blade Servers Industry Report 2007 for latest data)

Stra

tegy

(Pot

entia

l)

Delivery (Execution)

IBMHP

Dell

Tatung

Nexcom

Intel

Penguin

FujitsuSiemens

NEC

Sun

Egenera

©20

03-0

7 IM

EX

Res

earc

h

Rackable Sys

Verari

Linux Networx

Appro

HPC Syst

Hitachi



Copying ProhibitedBlades - TCO Savings & ROI

0%

20%

40%

60%

80%

100%

1 2

% C

ontr

ibut

ion

OPEX

CAPEX33%

67%

46%

25%

Blade Servers Rack Servers

3 Year TCO Savings Rack vs. Blade Servers

OPEXStaff/

Support25%Maintenance/

Downtime54%

Facilities/Power21%

CAPEX

Servers46%

SWInfrastructure

22%

Storage Infrastructure

13%

Networking19%

TCO Savings in..

Data: IMEX Research 2004



Copying ProhibitedPower/Cooling Spending to rise dramatically to 40%

$-

$25

$50

$75

$100

$125

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

CAPEX New Servers OPEX New ServersCAPEX Power & Cooling OPEX Power & Cooling

Power & Cooling Spending to rise to 40% of Total DC Spending by 2010En

d U

ser S

pend

ing

in D

ata

Cen

ters



Copying ProhibitedData Center Cooling

Computer Simulation using widely available software (e.g. Fluent Airpack Ansys CFD …) to verify Cooling Designed is the most cost effective before commiting to final implementation.

Source: APC

IT Equipment50%

Where does the power go in Data Centers ?

Source: Emerson Liebert

Many techniques, methodologies and equipments from air cooling to liquid assisted cooling available form a variety of vendors and Consultants ….(Email [email protected] for more info and Assessment of competitive vendor products, consultants and data center power & cooling integrators) Source: IBM 2005

mailto:[email protected]



Copying Prohibited

*IOs per sesond for a required response time ( ms)

(RAID - 0, 3)

500100MB/sec101 505

1K

10 K

100

10

1

Market Segments by Applications

eCommerceeCommerceTransaction Transaction ProcessingProcessing

OLTPOLTPOLTP

Data Warehousing

Visual DB

DSS(RAID - 1, 5, 6)

IOPs

(L

aten

cy)

StreamingStreamingStreamingAudioAudio

VideoVideo

Scientific ComputingScientific Computing

ImagingImaging

NICNICNIC

TP

HPCHPC



Copying ProhibitedState of DB Applications

Scale In

Scale Out

Scal

e U

p

Large Databases by OS

0

10

20

30

40

50

60

70

80

90

100

OLTP DSS#

Larg

e D

Bs

UNIX

Win

z/OS

Linux

Storage Usage vs DB Capacity

0 20 40 60 80

>10 TB

5-10 TB

2-5 TB

1-2 TB

DB

Siz

e

TB

Stg (TB)DB (TB)

Large DB Size Growth by Market Segment

0

100

200

300

400

500

2005 2006 2007 2008 2009 2010

DB S

ize

(TB)

OLTPDSS



Copying ProhibitedThe rapid rise of Clusters in HPC

Rise of Cluster Computing

0

50

100

150

200

250

300

350

1999

e

2000

2001

2002

2003

2004

e

# C

lust

ered

Com

pute

rs in

Top

500

.org



Copying Prohibited

Data rate & capacity

Throughput : : DSL/Cable

100+ Teraflops

Throughput = 100 GB/s

Rendering (Texture & Polygons)

Throughput = 1.2 GB/s

EntertainmentEntertainmentAudio/Video On DemandAudio/Video On Demand

High Performance High Performance ComputingComputing

Commercial Commercial VisualizationVisualization

BioInformaticsBioInformatics DecisionDecisionSupport SystSupport Syst

Dat

a: IM

EX R

esea

rch

& Pa

nasa

s



100+ Teraflops








Dat

a: IM

EX R

esea

rch

& Pa

nasa

s



100+ Teraflops






100+ Teraflops












Dat

a: IM

EX R

esea

rch

& Pa

nasa

s

High Performance Computing

Entertainment Audio/Video OnDemand

Decision-Support Systems

BioinformaticsCommercial Visualization

©2004--07 IMEX Research.com

HPC – From Academia to Wall St. to Hollywood



Copying ProhibitedImplementing Virtualization

Storage Arrays

Application & DB Servers

Client Workstations

SAN GbE or FCSwitches

LANEthernet Switches

At Various LevelsMicroprocessor – Intel VT, AMD-PacificaOS- zOS, pOS, UNIX, Windows, Linux- IBM, HP, Sun, VMWare, Xen, SWSoftFile System- DFSNetworking- MultiportStorage- Host, SAN, Controller- In-Band, Out-of-Band Management



Copying ProhibitedVirtualization Models

Source: Dan Olds

Features• Guest OS - Each application is contained by its own Operating System instance• VZ layer spoofs each OS into believing as if its the only OS on the system• Users can mix and match guest OS’s with various versions of Windows or Linux.Major Players - VMWare, Microsoft, XenSource

Hypervisor Model

Features• A single OS hosts multiple applications.• VZ layer handles resource allocation between applications• VZ layer also provides protection to the host OS so that a misbehaving application does not cause problems for the system as a wholeMajor Players - SWSoft, Sun/Containers

OS Virtualization



Copying ProhibitedWorkloads Consolidation using VZ

Source: Dan Olds

• A single server 1.5x larger than standard 2-way server will handle consolidated load of 6 servers.• VZ manages the workloads + important apps get the compute resources they need automatically w/o operator intervention.• Physical consolidation of 15-20:1 is easily possible• Reasonable goal for VZ x86 servers – 40-50% utilization on large systems (>4way), rising as dual/quad core processors becomes available• Savings result in Real Estate, Power & Cooling, High Availability, Hardware, Management



Copying ProhibitedHW Assisted Virtualization

X86 Instruction Set

Virtualization Extensions

Chipset

or

Man

agem

ent

PacificaVirtualizationTechnology

VTVirtualization

Technology

Direct ConnectArchitectureFSBus

Architecture

AMD64Extensions

Intel EM64 Extensions

VZ Extensions at Processor



Copying ProhibitedStorage Virtualization – Desired Features

Storage

Servers

Clients

SAN

LAN

Storage VZ - Must Have FeaturesScale Non-Disruptively in Capacity• Snapshot Point-In-Time across Stg.devices• Remote Replication across Heterogeneous Stg. Devices• Policy Based Non-Disruptive Data Migration betweenHeterogeneous Stg Systems & Between Stg Tiers

• Centralized Mgmt of all Stg.VZ under Single Image• Support Tiered Storage• Volume Management for Multivendor Stg. Systems• Common Set of Tools: Provisioning, Mgmt & Replication

Storage VZ - Vendors• Cloverleaf, Datacore, EMC• FalconStor, Fujitsu Computer Systems • Hitachi Data Systems• IBM, Network Appliance• StorageAge, Sun• Symantec/Veritas …

0100200300400500600700800900

1000

w /o Stg VZ w Stg VZ

Expe

nses

/Yea

r $K

HW

SANAdmin

SW

Total Svgs - 21%/yr.

- 24%

- 16%

- 19%

Savings achieved through Storage Virtualization



Copying Prohibited

App VZ OS Processor PC Servers StorageAppistry Fedora AMD Altiris Akimbi Syste Cloverleaf Acronis SunData Synapse Novell Intel AppStream AppStream Compellent* Altiris Surgient

OpenVZ Ardence Ardence Datacore BladeLogic VizionCoreRed Hat Checkpoint Egenera EMC* BMC SW VMwareSun Citrix HP FalconStor CA vThere

Fujitsu IBM Fujitsu* CassattFujitsu-SiemeMicrosoft HDS* CirbaHitachi Parallels HP* DunesHP Sun IBM Tivoli EcoraIBM SWsoft IBM* IBMLeoStream Virtual Iron NetApp* MicrosoftNEC VMware Netreon OpswareParallels Xen SANRAD ParallelsPlatform StorageAge PHDMicrosoft Sun/STK* PlateSpinSun Symantec Platform Wyse Vicom Scalent

Tools

Virtualization Players by Category



Copying ProhibitedTCO Savings with Virtualization

$-

$4,000

$8,000

$12,000

$16,000

w/o VZ w VZ

Provisioning

HardwareSAN

NetworkPower & Cooling

DC Real EstateDisaster Recovery

Downtime

Cos

t ove

r 3 y

ears

995 Pre-Virtualization (VZ) Servers 78 VZ Servers

VZ SW & Support

For a copy of TCO

Analysis, Email:

imex@ imexresearch.comOr call (408) 268-0800





Copying Prohibited

•• Server Virtualization (VZ) now a mainstream technologyServer Virtualization (VZ) now a mainstream technology– VZ turning DC core infrastructure upside down, DC Professionals very happy with its future use– VZ means “Doing More for Less” (finally making CFOs get off your back)

•• Issues to be ResolvedIssues to be Resolved– VMs exploding – Managing them a nightmare: needs more tools– Database Performance (one reason HP bought Polyserve)

•• Follow SIVAFollow SIVA©© in executing your DC strategyin executing your DC strategy–– StandardizeStandardize (Windows/Linux, GbE, IP Storage/iSCSI,SATA..)–– IntegrateIntegrate (Blades, Management Tools..)–– VirtualizeVirtualize (Infrastructure-uP,Servers, Storage, Networks,Clients w P2V tools)–– AutomateAutomate (Provide important Apps required resources automatically w/o intervention to OPEX

costs)

•• Create VZ Justification: TCO Reduction of 60Create VZ Justification: TCO Reduction of 60--70% over 3 years, ROI >58%70% over 3 years, ROI >58%– Combined with VZ, Consolidation reduces cost by 65070%– Key advantage through increased utilization HA/BC/DR achievemment

•• Follow VZ in 3 phasesFollow VZ in 3 phases––

Consolidation & Resource Sharing Consolidation & Resource Sharing

HA/BC/DR, HA/BC/DR, WkLdWkLd Balancing Balancing

AutomationAutomation–– Consolidate through VZ and Workload Management, Consolidate through VZ and Workload Management, –– Reduce # systems Footprints & OS instances (Reduce # systems Footprints & OS instances (OS OS LicLic Costs, Costs,

Mgmt Admin Costs)Mgmt Admin Costs)–– Create Workload Mgmt based on Business Policies (Mission CriticaCreate Workload Mgmt based on Business Policies (Mission Critical, & DB l, & DB WkldsWklds))

Summary - Virtualization & Automation



Copying ProhibitedVirtualizing your IT InfrastructureSLA

UsageDept/Owner

AssetsLocation

• Host Name (Mfr/Model/SN,• Platform – OS/Processors/#/Speed/Type

• Pooled Infrastructure Resources by Application Metrics

• Pooled Capacity Provisioning: Processing, Bandwidth, Storage, Repository

• Usage Profiles• Users/Services/Workloads

• Applications (OLTP/BI/HPC/Data Streaming)• Execution: Rules Driven, Adaptive Provisioning

• Services Abstraction, Adaptive Provisioning

VirtualizationUtility - P2V

• Business Priorities• Cost of IT Ops/Charge Back Methods

• Response Time/Availability/Throughput,QoS• Transactions/Sessions/Events/Analysis/Reporting

• Business Services Managed & Charged

For copy of case study on how a major financial institution implemented virtualization email [email protected]



Copying Prohibited

IMEXRESEARCH.COM

Next-Gen Data CenterImproving TCO & ROI in Data Centers thru

Virtualization and Blade Servers

Anil Vasudeva Chief Analyst & President [email protected]


Copying ProhibitedPlease contact IMEX for written permission



Copying ProhibitedFuture: IP Everywhere Based Infrastructure

Networks

StorageServers

TelecomIP

Infrastructure

StandardizationIntegration

Thermionics©2007

VirtualizationProvisioningAutomation

Follow SIVA©2007 – Standardization, Integration, Virtualization & AutonomicsIn your Next Generation Data Center



Copying ProhibitedGo-to-market: Channels of Distribution

Industry Structure

Channels of Distribution

Lg. SysIntegrator

VARS

Sml SI/VAR

Distributor

OEM

VADS

F1000&

Large

Sml

Med

Vertical Markets

Fina

nce

Man

ufac

turin

g

Telecom

Insu

ranc

e

Gov

t.

ISPs

Medical

Manufacturers

Com

pone

nts

-Chi

ps &

Boa

rds

...

HA

+ N

twk

+ Sy

stem

Mgm

t SW

Hubs

NICs

Switches

Routers

Servers

Clusters

SANs

Dis

k &

Tap

e D

rives

& H

ost B

us A

dapt

ers

RAID

VIA IPC

Contllrs

LANs

WANsInternet

Backbone



Copying ProhibitedGlobal IT Spending 2005 by Market

Global IT Spending 2005

4.0%

4.5%

5.0%

5.5%

6.0%

6.5%

7.0%

7.5%

8.0%

8.5%

9.0%

$0 $50 $100 $150 $200

Global IT Spending 2005 $B

cagr

% (2

005-

09)

Utilities

Health Care

Mfg.Equipment

Govt

Retail

Mfg-Process

Telecom

Finance &Insurance

Wholesale

ProfessionalServices

Education

Resources

Transportation

Other



Copying ProhibitedGo-to-market: the SMB Long Tail

LargeMedium

Small Consumer

SMB Spent $150 B 49% of All Spending on IT Infrastructure

Growing at 2x that of Enterprise



Copying Prohibited

$-

$5,000

$10,000

Fact

ory

Rev

$M

2003 2004 2005 2006 2007 2008

WW Blade Servers Market Revenues by Major Target Market

IP T eleco m Enterprises (Lg & SM B )H igh P erfo rmance C o mp. ISP sEmbedded C o mputing

Blade Servers by Market Segments



Copying Prohibited

Data: IMEX Research 2005

Blade Servers Market Opportunity

$-

$2

$4

$6

$8

$10

$12

2003 2004 2005 2006 2007 2008

Fact

ory

Rev

$B

WW Blade Servers Market Opportunity (cum 2004-2008)

Revenues ~ $24 BillionUnits ~ 9 Million



Copying ProhibitedGlobal IT Spending 2005 by Market

Global IT Spending 2005

Global IT Spending 2005 $B

cagr

% (2

005-

09)

Utilities

Health Care

Mfg.Equipment

Govt

Retail

Mfg-Process

TelecomFinance &Insurance

Wholesale

ProfessionalServices

Education

Resources

Transportation

Other



Copying ProhibitedHPC Interconnect – Leaders

GbE for majority of applicationsdominate the Interconnect for HPC.Myrinet for the highlylatency sensitive applications whileInfiniband is rearing up at the Midrange latency applications



Copying ProhibitedFuture: Storage Management on a chip

Host Services Integration StorageProvisioning Layer

SAN Management Layer

DeviceService Layer

File system monitoring

SAN ManagementManagement

ConsoleManagement of

iSCSI HBAsMultiPath IO Supp

and FailoverSecurity

(iSNS, CHAP, SRP)

Virtualization Mirroring Snapshot Fail-Over

iSCSI Target ManagementLVM, Error Handling, SCSI Daemon, API Interoperability

HW Acceleration: TOE, iSCSI Offload, IPsec

Storage provisioning Win, LINUX, Solaris



Copying ProhibitedState-of-the-BladeSystems Vendors 2006

Blade Servers Market Shares By Revenues 2006

HP

IBMDell

Others

FJ/FSC

Blade Servers 2006ASP vs. Shipments

$2,000

$3,000

$4,000

$5,000

$6,000

0 50 100 150 200 250 300

Unit Shipments (000s)

ASP

$

FujitsuOthers (Sun, Hitachi…)

DellIBM

HP