2/27/08
1
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 1
Data Center Consolidation and Virtualization
Bill Dufresne Application Networking Services
CSE-III, CCIE, CISSP
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 2
Data Center Infrastructure must Evolve Extend the Value of the Current Operational Model
Lower Operating Costs Infrastructure Resilience Power and Cooling
Application Delivery Holistic Security Compliance
Enabled by: Consolidation, Virtualization
Improve IT Effectiveness in the New Environment
Event- and Policy-Driven Real-Time Infrastructure Unification of Components, Networks, Communications Streamlined Business Processes, IT as a Service
Enabled by: Integration, Automation
2/27/08
2
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 3
The Network is an Enabler for the New Infrastructure
Storage
Facilities
Servers
Network
FibreChannel Switches and Services
Optical High Speed links
Active-Active Online Transparent Backup
Infrastructure Services
Dynamic Service Provisioning
Accelerated Delivery Service Virtualization
Virtual Machines I/O Virtualization
Applications
Core Switching, SONA
Content Switching, Security
Virtual SANs Storage Virtualization
Server Fabric Switching
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 4
Scaling Ethernet Module
Cisco Catalyst Layer 2 Switch
10 Gigabit Ethernet Gigabit Ethernet
Cisco Catalyst® 6500 Multilayer Switch Cisco Catalyst Layer 3 Switch
Firewall Services
Intrusion Detection
VPN Termination
DOS Protection
Anomaly Detection
Content Caching
SSL Offloading
Server Balancing
File Caching
AON Analysis
App Acceleration
Core
Network Analysis Aggregation
Access
Server Virtualization V
Remote DMA Services
Virtual I/O
Clustering Services
Compute Fabric Services
Fabric Gateway Services
Server Clusters
Infiniband Cisco 3000 Series Fabric Server Switch
Cisco 7000 Series Fabric Server Switch
Virtual Server Link
Virtual Servers
Fibre Channel
Server Farms
Storage/Tape Farms
Edge
Core
Cisco MDS 9500 Multilayer Director
Fabric Routing Services
Data Replication Svcs
Storage Virtualization
Fabric Gateway Services
Virtual Fabrics (VSANs)
A B
Fibre Channel Trunk
A Redundant SAN Fabric
Layers
What Is the Scope of Next Gen DC Architecture?
2/27/08
3
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 5
What is the largest impact to Data Center?
Consolidation is at the forefront of IT change now and in the next 3-5 years. Having no local compute resources at the Campus impacts user experience High Availability between the Campus and DC is critical
The Campus appears as a large Branch location from the Data Center Due to Consolidation, the common services required in the branch translate to the Campus, albeit in a larger fashion
Some services common to the Data Center can be found in the Campus/Branch Security Services, VPN, Firewall, IPS which are all common to the Data Center are also important to the Campus and remote locations
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 7
WAN
Typical Consolidated Enterprise Consolidation Challenges
– Physical Space – Physical Security – Power and Cooling
Application delivery woes – Congested WAN – Bandwidth and latency – Poor productivity
Data protection Safety – Reliable backups – Faster Replication – Effective Storage Tiers
2/27/08
4
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 8
WAAS and Remote Locations
Wide Area Application Services CIFS Caching - Common Internet File Store (Microsoft File Systems)
Servers consolidated to Data Center, users need CIFS Caching to decrease latency-bound access to individual and group files
SMB Print Services Complete removal of MS servers will require local Print Servers to mitigate WAN transit
and latency issues Data Redundancy Elimination Identifying common data chunks in TCP packets, to eliminate data on the WAN
LZ Compression Average 10x compression of data to transit WAN after CIFS Caching and DRE
TCP Flow Optimizations Buffer optimizations to minimize TCP Fall back or Saw Tooth
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 9
Cisco WAAS, QoS, and Enterprise VoIP Cisco WAAS enables enterprise VoIP deployments by easing the
contention for available bandwidth resources and complying with network-based end-to-end QoS
WAN
Without WAAS (QoS only)
WAN
VoIP
Scavenger
ERP VoIP
Scavenger Email ERP
Additional Available Capacity!
With WAAS and QoS
2/27/08
5
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 10
DRE Pattern Matching
DRE Database
NO MATCH NO MATCH NO MATCH NO MATCH Original
Message Encoded Message
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 11
Comparing TCP and TFO
Time (RTT) Slow start Congestion avoidance
cwnd
TCP
TFO
Cisco TFO provides significant throughput improvements over standard TCP implementations
2/27/08
6
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 12
WAN
User Experience Mitigation - WAAS
Campus WAAS scales via WCCP Groups Also provides HA between WAE and WAN Routers
Data Center WAAS Scales via ACE for HA and improved distribution across WAE Farm
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 13
Application Control Engine Application High Availability
– Server Load Balancing Capabilities – Highest Industry throughput 4-16Gbps, 4M connections – Scalable via software licensing – Uses Cisco Common Class-based Policy Language (MQC) – Provides Virtualization 1-250 virtual partitions - scalable via software upgrade
TCP Reuse – Sever Off-load capability to reduce CPU – Requires L7 interception policy
SSL Offload – Server Off-load (80% Server CPU cycles regained) – Up to 15k cps via built-in ASIC – Scalable via software licensing
Enhanced Security Features – Protocol Inspection Engines – RFC Compliance enforcement – Defense in Depth without always requiring additional FW
2/27/08
7
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 14
ACE Virtual Partitioning and App Security in Multi-tier Applications
DataBase servers
LB
LB
LB
Application servers
Front-end servers
Firewalls Front-end Firewalls
ACE with
Application Infrastructure
Control and
Application Security DataBase
servers Application
servers Front-end
servers
FE virtual partition
APP virtual partition
DB virtual partition
Enterprise Network
Enterprise Network
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 15
Admin Context
Context A definition
Context B definition
Resource allocation
Admin management
config
Physical module
Context B
Context A
VIP1 VIP 2
Farm1 Farm2
VIP3 Farm3 Farm4
SSL cert1,2
Domain1 Domain2
Admin
Network/Security
Server Admin
Monitor
Management station
Role
AAA
Contexts, Roles, Domains
2/27/08
8
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 16
Virtualization Provisioning - VFrame DC
Provisioning and Operations Management Platform
– Network and Services Discovery Automates populating the VFrame DC database Recognizes 6500 and 9500 devices in 1.1
– Provisioning Macros Takes advantage of VMWare API’s to minimize server turn-up times Will add support for ACE and FWSM in upcoming software Additional server VM support forthcoming
– Provides Northbound API’s into MoM of choice
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 17
From Silo to Shared Infrastructure
WAN
Campus_1
Campus_2
Campus_n SFS Family
Server Fabric Switching
V
Virtualization Services
Virtual Private Server
Fabric#1
Virtual Private Server
Fabric#2
Virtual Private Server
Fabric#3 (Blade-Based)
MDS 9000 Family
Fabric
Backup
Storage
VSAN
Storage Switching
Mainframe Connectivity
Enterprise Tape Storage
Enterprise Disk Storage
Storage Services
Blade Servers
Network Services
Cisco Catalyst® 6500 Family
Server Farm Switching
Enterprise NAS Storage UNIX/Windows
Servers
UNIX WIN NAS
SSL
VPN
FW
IPS
LB
2/27/08
9
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 18
Increase agility Catch up to pace of
business
App1
Virtualized Storage Pool
Virtualized Server Pool
App2 App3
Virtualized Network and Network Services
Virtualization Better utilization, flexibility,
mobility of applications/data
Reproducible processes
IT resources closely aligned with application and business needs
Automation Policy-based adaptive
service-oriented infrastructure
App Svc.1
App Svc.2
App Svc.3
Service Network 1 Service
Network 2
Service Network 3
Consolidation Improved utilization,
power efficiencies, lower costs
App1
Shared Storage
Standardized Servers
App2 App3
Scalable Data Center Network (LAN+SAN)
Regain IT asset control
Lower operational expenses
Cisco VFrame – Enabling End to End Data Center Management and Provisioning
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 19
Design to Operate Workflow for SOI
Design Service Template
Switch port config VLANs, DHCP, trunks, SVIs
Zones, VSANs, LUNs, NFS volumes
Image mgmt Remote boot VM mappings
VIPs, LB policies
Firewall selection, firewall chaining, firewall rules
Deploy Service Networks
Boot OS / Application
Server I/O
SAN Infrastructure
L4-L7 LANs Discover
Resources
Firewall
Automated failover Policy-based resource optimization
Service maintenance Management integration thru API
Operate
Policies
2/27/08
10
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 20
Cisco VFrame™ Data Center Service Orchestration
Coordinated provisioning and dynamic reuse of physical and virtualized
compute, storage, and network resources
Compute Pool
Hypervisor VM VM
Storage Pool Network Pool
VFrame™ Data Center Operational Cost Savings
Faster and simpler Resource Deployment
Dynamic Management and Provisioning
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 21
VFrame Services with ESX Deployments
ESX
OS
App
OS
App
OS
App
OS
App
VFrame
Virtual Center
VM Creation Image Load Mobility Grid balancing
L2 Network Services 802.1q VLAN Membership L4-L7 Services Associations ESX Boot
SAN Zoning LUN masking LUN mapping
ESX
OS
App
OS
App
OS
App
OS
App
IP Network
Storage Area Network
X86 Server X86 Server
API
2/27/08
11
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 22
ACE
VFrame DC: Sample deployment
Catalyst 6500
SAN
FWSM MDS 9500
NAS
Campus/ WAN/VPN
Campus LAN/WAN
EMC CX-500
NetApp FAS 200
VFrame DC Appliance
Diskless Servers
FC Link
Ethernet – VLAN 249
Ethernet – VLAN 500
Ethernet LOM– VLAN 501
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 23
Infrastructure Design Considerations
Cisco Design Principles become critical to Application and Data Availability
– Services defined at the Aggregation Layer Aggregation Port Count and Services HA are critical
– Service Chassis Design Minimizes IOS dependencies in Aggregation Layer Adds a new set of versioning challenges - Watch Safe Harbor Testing May require PBR configurations in Aggregation Layer
2/27/08
12
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 24
Services Consolidation into Aggregation
Consider consolidating multiple service modules onto ACE Module
– SLB – Firewall – SSL
4/8/16G Fabric Connected Active-Active Designs Higher CPS + Concurrent CPS Single TCP termination, lower latency Note: Feature gap may not permit
migration till future release (examine release notes)
DC Core
Access
Aggregation
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 25
Services Switch outside of Aggregation
Move certain services out of aggregation layer
Ideal for ACE, FWSM, SSL modules
Opens slots in agg layer for 10GE ports
May need QOS or separate links for FT paths
Extend only necessary L2 VLANs to service switches via .1Q trunks (GEC/10G)
Consideration: RHI installs route in local MSFC only, requiring L3 peering with aggregation
DC Core
Service Switch2
(Redundant)
Service Switch1
Access
Aggregation
Access1 Access2
2/27/08
13
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 26
MDS
Disk Utilization – DAS is inefficient and costly
SAN Scalability – Isolated SANs are inefficient and increase TCO
Centralized Backup – Ability to implement effective Tiered Storage Architecture – Serverless Backup scales throughput operations significantly
Additional Services – QoS – SANtap – Compatibility Modes – Encryption/Compression (SAN Extension)
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 27
Island ‘A’
Island ‘B’
Island ‘C’
SAN Islands Have Purpose: At a Cost
SAN islands are built to address several technical and non-technical issues:
– Maintains isolation from fabric events or configuration errors – Provides isolated and controlled management of island infrastructure – Driven by bad experiences of large multi-switch fabrics
However… – Often over-provisioned port count for future growth—wasteful and costly – Widespread issue today—some analysts still recommending islands
2/27/08
14
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 28
Fabric Virtualization—MDS 9000 Family
Each port on the MDS 9000 family exists in a VSAN
Up to 256 VSANs in a single switch (hardware can support up to 4095)
Logical configuration to move a port from one fabric to another
WWN-based VSANs can provide automated VSAN membership
Basis for Virtual Fabric Trunking (VFT) Extended Header (ANSI T11 FC-FS-2 section 10.3)
VSAN ‘A’
VSAN ‘B’
VSAN ‘C’
VSAN ‘D’
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 29
IVR Enabled
IVR Operation Within a Single Switch
Effectively turns any MDS 9000 Family switch into giant fabric router
Enables IVP in any Cisco MDS 9000 Family switch using a license key
Works with all fabric interoperability modes
Enabled through zone creation mechanism
Blue VSAN Shared Storage
Arrays
ANY CISCO MDS 9000 FAMILY SWITCH
Yellow VSAN Blade Server with
Embedded Qlogic Switch (Can Route Individual Blades
Into Different Vsans)
Purple VSAN Brocade Switch
in Native PID_Mode 0
Red VSAN McData Switch in Interop Mode
Orange VSAN Brocade Switch
in native PID_Mode 1
Green VSAN Normal Server with any HBA
2/27/08
15
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 30
MDS IBM BladeSwitches
6 external 4-Gbps Fibre Channel ports 14 internal 4-Gbps Fibre Channel
connecting to blade servers through blade chassis backplane
Offered in 3+7 and 6+14 configurations via port licensing
Integration with Fabric Manager and IBM BladeCenter management tools MDS IBMBladeSwitch
IBM BladeCenter
Enterprise-class capabilities of industry leading SAN-OS
– 16 Virtual SANs (VSAN) – PortChannels – Centralized security services (AAA) – SPAN – Fabric Manager with task-based Wizard
Non-disruptive software upgrade
Full MIB and SMI-S support
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 31
SAN Extension Platform Support
FC over DWDM/CWDM – Cisco ONS 15454, 15540, 15530—DWDM – Cisco MDS 9000 + CWDM SFPs
FC over SONET/SDH – Cisco ONS 15454 with SL-Series modules
FCIP and FICON over IP – Cisco MDS 9000—integrated FCIP, FICON over IP
2/27/08
16
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 32
Data High Availability
Global Site Selector – Application HA via DNS resolution – HA placement at disparate locations – Configs are sync’d across cluster members – Can be used as BIND replacement for internal resolution
ONS – Can be provided as User-owned/operated or Service Provided – Able to transport multiple traffic types over the same media set (E-net, FC, FICON, etc.) – Can be deployed in highly scalable fashion
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 33
WAN
Alternate DC - The Campus - Data Replication via SAN Extension - Use of GSS for Application HA - Usually a subset of Critical Apps only
2/27/08
17
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 34
Business Continuance – Disaster Recovery
End Users
X Primary Data
Center Secondary Data
Center (Campus)
Intranet SP-A
Primary with a Secondary Backup Site Recovering Service Availability after
Failure – Active-Passive Design – two data centers – Highly Available - Data Center Infrastructure – Network fail-over can happen within 10s of
seconds – Application/Server Recovery time is based
on the time it take to complete Data Synchronization of back-end data base, application servers and Web servers
Supported by Cisco’s Solutions – GSS, CSS, CSM, ACE
Data Synchronization
after Failure
SP-B
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 35
Application Optimization – Global Traffic Management Across Multiple Data Centers
End Users
Data Center Campus
Internet SP-A SP-B
Maximizing Resources across Multiple Data Centers
– Active-Active Multiple Data Center Deployments
• Increased End-User Productivity – Increased transaction volumes – Faster download times (Proximity) – Improved transaction flow and
completion rates • Return on investment (ROI) for Secondary Data Center
Preserving Service Availability during Disruption
• Naturally Highly Available • No Guesswork for Data Center Recovery
Supported by Cisco’s Solutions – GSS, CSS, CSM and ACE
Continuous Data Synchronization
2/27/08
18
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 36
Data Centers at Cisco
Deployment Summary
5 enterprise production data centers of 36,000 square feet
Data centers support Cisco business processes
Initial 4-tier model > 4-tier model replaced by vertical and horizontal model (resulted in “silo” challenges) = move to redesign into the Service Oriented Data Center (SODC) model
Benefits
SODC meets three strategic business objectives:
1. Lower TCO
2. Enhance business agility
3. Improve business continuance
Future
3 stages of data center redesign:
1. Consolidation (complete)
2. Virtualization (in progress)
3. Automation (next)
PLEASE SEE NEXT SLIDE
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 37
Data Center Evolution
2005 2004 2006–2007 2008–2010
• Standardization • Virtual machines
• 4-tier silos • Heterogeneous OS • Storage silos • Low utilization
• IP connectivity
• Perimeter security
• Application silos • Distributed
• Server repurposing • VM mobility • Storage
virtualization
• Virtualized network services
• Virtual firewalls
• Optimization
• Infrastructure aligned to application services
• Policy-based management
• Intelligent data management
• Tiered recovery • Usage and SLA-
based funding model
Legacy Data Center
Virtual Data Center
Service Oriented Data Center
Consolidated Data Center
Consolidation Phase Virtualization Phase
Automation Phase
Compute
Storage
Network
Security
Application
• SANs, VSANs • Tiered storage
• Consolidate, centralize
• Consolidated network services
• Secure each application tier
2/27/08
19
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 38
Network World Article 2/20/08 Ten Ways to Make Your Data Center More Efficient – Laura Pickering
1. Measure
2. Consolidate and Virtualize
3. Manage Data Growth
4. Eliminate Overcooling
5. Data Center Physics
6. Continuously Improve Heat Containment
7. Maximize Free Cooling
8. Minimize Electrical Conversion
9. Use Heat Effectively
10. Monitor and Tune Continuously
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 39
Enterprises are Already Seeing Results Network Enabler Results…
Highly Available and High Density Switching Platforms
Director and Fabric-Class FibreChannel Switches with Intelligent Fabric Services
Optical and WAN Networking to extend distance and link facilities
HP announced DC Consolidation of 85 Facilities to 6. Projected over $1B in savings.
Stock Valuation rose 4% on the news.
AIG Reduced the number of servers while driving utilization to >80%
Over a petabyte of online storage added in FY2005 while reducing the storage budget by $10M
TCO per GB of Storage improved by 70%
2/27/08
20
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 40
Clients
Disk SAN
GE/ 10GE
Prim
ary
Dat
a C
ente
r
Server Clusters
Storage Network
Tape
Clustered Servers Resilient IP
GE/ 10GE
Security
IDS
Anomaly Detect/Guard
VPN
Firewall
Application Networking
SSL
ACE
Business Applications
IBM
GSS
Metro Network Optical/Ethernet
ONS 15000 MDS 9216
WAAS
Infiniband
NAS
L2/L3 Network Front-End Network and Apps
WAAS
Campus Backup Campus / Branch
WAN
Disk SAN
WAAS
ACE XML
Cisco Data Center Services
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 41