Moving Enterprise Application Dev/Test to VMware’s

Internal Private Cloud – Architecture,

Implementation and Integration

Thirumalesh Reddy, VMware

Padmaja Vrudhula, VMware

VSVC4948

#VSVC4948

2 2

Executive Summary

Key Lesson Learned

Invest in Agility, and Service Quality and Cost will Improve

AppOps Team Deploy integrated, complex SDLC instances to support 600 developers.

Challenge Process is manual, siloed, slow, unreliable. Reduces developer efficiency. Increases risk.

Two Fundamentally Different Options

1. Fix the “human middleware” on traditional infrastructure

2. Replace and automate on private cloud SDDC

Results From Choice to Replace and Automate on SDDC

Process time – dropped from 4 weeks to 36 hours

Developer productivity – increased 20% or more

Project schedule risk - eliminated

Annual infrastructure and operating costs - reduced by $6M annually

3 3

Session Overview

Agenda

Executive Summary

Fundamental Challenges: VMware IT

Options

Cloud Automation and Management – Architecture, Implementation:

VMware IT

Cost Savings & Delight Customers

Key Takeaways

Session Goals

Automate Workload Provisioning life-cycle

Understand new paradigm of instance refresh – Just re-provision.

Understand resource optimization - Different Service profiles

Dramatically reduce Problem Resolution – Integrated Monitoring from

the outset

4 4

Corporate IT Application Group

Manage portfolio of enterprise

applications used by global

business functions

Team

27 DevOps engineers

Customer

600 developers

Role

Provision 16 different dev/test instances that

include 130+ app components.

Infrastructure footprint

~4000 non-production VMs

~500 production VMs

~290 physical hosts

~4 Petabytes of storage

Enterprise Application Portfolio

SaaS 65

IT tools 50

Business 100

Total 215

5 5

AppOps provisions environments across SDLC

Support 30 to 50 major development projects per year

Team of 27 engineers manually builds each Dev/Test instance

Each project needs Dev/Test multiple times per project

Enterprise App Development Project - 9 months

20 Major Steps

3 to 5 Weeks in Traditional Virtualized Environment

Request for

Dev/Test

Infrastructure

Verification

Hardware

Setup

Build VMs –

new or clone

DNS

Entries

Install,

Setup,

Configure

Workload

Database

Refresh

Latest Code

Deployment

Load

Balancer

Entries

Web Server

Configuration

Firewall

Changes

External

Interface &

Integration

PPM

Tasks

Workload

Monitoring

Setup

Security – VM

access control

Functional

Testing

Environmental

Testing

Production Dev Test UAT Stage Load

Test

6 6

Human Middleware Problem

Variable Quality

Variations in

calendar and

service quality

Schedule Risk

Late projects cause

domino effect

with constrained

resource

Unpredictable

“No we are busy”

is response to

many developer

requests

Disruptive

Developer must

work around

3+ weeks gaps up

to 5x per project

Customer View

7 7

Human Middleware Problem – AppOps team view

Global Team

Management

Project manage

around PTO,

holidays,

variable skills

Capacity

Constrained

Only 4-6 projects

in parallel

Slow and

Error Prone

Many manual steps.

Ticketing systems.

Human error.

Handoffs

Silos. Globally

distributed teams.

Multiple application

experts.

8 8

Two Fundamentally Different Options

Fix

The “human middleware”

on traditional infrastructure

Replace and Automate

End-to-end provisioning

on SDDC Private Cloud

Option 1 Option 2

9 9

Project OneCloud - Explosive Tenant Growth

Corp IT DevOps = Tenant #4

Very low cost per VM

“Cloud first” policy in IT

Corp IT

Ent App

dev/test

Hands

On Labs Hol.vmware.com

Services &

Support Customer environment

reproduction

Sales

Engineering

Demo Pods

VMworld 2013

Management

BU Field

Testing

TechSummit

2013

Tech Ops

Mini R&D

Cloud

vSEL

Private Cloud IaaS Software Defined Data Center

June

2012

Jan.

2013

Today End

2013

2014

Launched

Built on

vCloud Suite

4 tenants

10,000 VMs

9 tenants

38,000 VMs

12 tenants

50,000 VMs

More

services

Timeline

10 10

Application Ops Requirements Vary By Workload Type

…are different than Ops

requirements for this App

Ops requirements for this instance…

Corp IT– App Dev/Test

Provision: Application Stack

Manage: Yes

Duration: 3+ months

Provision: VM

Manage: No

Duration: 3 weeks

One App/Sales – Demo Pod

11 11

Private Cloud IaaS Software Defined Data Center

Bring Your Own - Application Ops

Three-tier Ops Model

Different tenants

Different Application Ops

Application Ops (Provided by Tenant)

Now an infrastructure service consumer. Provisioning.

Monitoring. Configuring. Upgrades. Maintenance.

Many typical ops tasks still required.

Infrastructure Ops (Provided by OneCloud infrastructure team)

Network, storage, compute availability. Deliver to SLA.

Tenant/Service Ops (Provided by OneCloud service team)

Common service definitions, SLA, tenant onboarding, tenant management

Private Cloud IaaS Software Defined Data Center

12 12

Bring Your Own - Cloud Automation and Management

Tenant - needs different service

levels, automation and

management capabilities

IaaS - needs automation and

management capabilities

Service Manager

Decides what goes in service catalog.

Service Catalog

Mechanism to request service.

Policy

Logic used to guide automation.

Cloud Automation and Management

Manage workloads and underlying services.

Tenant 1 Tenant 2 Tenant 3

Private Cloud IaaS Software Defined Data Center

13 13

Two Project Goals

Transition

to Private Cloud

Automate

the Process

Key Dependency

Need SDDC to automate the process

14 14

• Cloud Automation

• Dev, Test instances

• Policy based provisioning

• Storage Mgmt.

• 30+ App Blueprints

• 500+ VM’s

• Non-prod Environments

• One Cloud

Management

Cloud Virtual Infrastructure

Provisioning Automation

Cloud Storage Virtualization

Done

Q3 – Q4 12

• Cloud Automation & Management

• Dev, Test, UAT instances

• Scaling/Upgrades

• Policy based provisioning

• Storage Mgmt.

• Security Mgmt.

• Monitoring & Analytics

• VM Asset Mgmt.

• 50+ App Blueprints

• 2500+ VM’s

• Non-prod Environments

• One Cloud

• VCHS IaaS Validation

Management

Cloud Virtual Infrastructure

Provisioning/Scaling/Upgrade Automation

Monitoring

Cloud Storage Virtualization

Cloud Security

Done

Q1 – Q2 13

Service Catalog

Phased Project Approach

• Stage and load test instances

• Service catalog

• Performance management

• Network, Storage & Security

Virtualization

• 80+ App Blueprints

• 3500+ VM’s

• VCHS IaaS

Management

Cloud Virtual Infrastructure

Monitoring

Cloud Storage Virtualization

Cloud Network Virtualization

Cloud Security

Provisioning/Scaling/Upgrade Automation

Performance Mgmt

Service Catalog

In-progress

Q3 – Q4 13 • Cloud Automation Scaling/Upgrades

• Cloud Storage & Network Mgmt. & Scaling

• Cloud Security Mgmt.

• Cloud Performance Mgmt.

• Usage & Charge-back

• Analytic and Correlations

• 100+ App Blueprints

• 4500+ VM’s

• All Non-prod & Prod Environments

• One Cloud/VCHS IaaS

Cloud Virtual Infrastructure

Management

Monitoring

Cloud Storage Virtualization

Cloud Network Virtualization

Cloud Security

Provisioning/Scaling/Upgrade Automation

Performance Management

Big Data Operational and Biz Analytic and Correlations

Service Catalog

2014

15 15

vCenter Orchestrator

VMware Application Director (VCAC Enterprise)

Dev Test UAT Stage Mgmt.

Cloud Providers

App Blueprints (130+)

VMware vCloud Director / VCAC

VMware vCloud Automation Center

Access Control

Provisioning Policies

Service Catalog

vCAC Workflows

Cloud Administrator

Blueprint

Manager

AppOps/ Biz/Dev Consumer

One Cloud - Private Cloud IaaS Software Defined Data Center

Deployment Plans

Automation and Management – Based on vCloud

vCloud Suite

16 16

Cloud Automation and Management - Extensibility

Load Balancer

(F5/vShield)

IPAM

(Men & Mice)

LDAP

(Lotus)

Config

(GIT Repo)

Other

3rd Party

Plug-Ins

Configure

VCAC Service

Catalogs and

Policies

Configure

Monitor Agents

and Collector

Feeds

Export

App D Services

and Blueprints

Export

Provision and

De-provision

Workflows

Export App D

Update Harness

Container

VMware /

Non-VMware

Components

Plug-In Mgmt.

Service Profile

and Provision

Tasks/Services

Mgmt.

Provisioned

Workloads

Unified Asset

Inventory

Provisioned

Workloads

Monitoring

Mgmt.

Provisioned

Workloads

Monitoring

Provision/

De-provisioning

Audits

Extension 3rd party

components

Private Cloud IaaS Software-defined Data Center

vCloud Suite

…

17 17

Cloud Automation & Mgmt. Platform: Life-cycle Diagram E

xte

nsio

n

vC

lou

d

Dir

ecto

r

Ap

pli

cati

on

Dir

ecto

r (V

CA

C

En

terp

rise)

vC

en

ter

Orc

hestr

ato

r

vC

lou

d

Au

tom

ati

on

Cen

ter

Build Configure Phase

Provision Phase

Monitor & Manage Phase

18 18

Cloud Automation & Management Capabilities

Capability VMware Extension

Service

Catalog vCloud Automation Center - VCAC

Blueprinting Application Director (Part of VCAC Enterprise)

Provision, Scale-out, In-place Upgrade a blue print multiple times with in the same vLAN

Provisioning

Geared towards production ready workloads

Policy Based Automated Provisioning - Balance Costs w.r.t Performance requirements:

• Use Fast Provisioning for Low Cost Envs.

• Use full clones with custom choice of Storage, Networks, Load Balancing etc.

• Highly customizable.

• De-provisioning – Tear down and reclaim all the resources for optimized resource usage

Monitoring Single Pane of Glass - Monitors resource consumption as well as application behavior

Management Comprehensive unified platform for pre-provision set-up, configuration, provision-time

auditing and management, post provisioning workload inventory, monitoring etc.

Use Cases Any Use Case.

Targeted towards Use cases like "Dev. Test, UAT”, “Sales Demos", "Training", Bug

Fixing” etc.

Portability Is completely portable as an OVF and can be setup in-house

Hybrid Cloud Multiple options for Hybrid cloud –

With in house - IPSec with any cloud provider + Private Cloud

CAaMP on Cloud - IPSec with any other cloud + Private Cloud

Χ

Χ

19 19

Policy Driven Business Workloads Provisioning

Self-serviced Portal for consumers to

request Services on-demand from the

published Service Catalog

Request Services for a specific term

more of a leased consumption model

Request Services with the choice of

Service Profiles based on cost &

Performance needs

Policy driven approval process

Fully configured Business Workloads

provisioned based on requests with

out manual intervention

Workloads provisioned with integrated

monitoring provide deep insight &

visibility

Policy driven alerts, notifications

Rabbit MQ Server MQ

Spring Module

Controller

Modules

Lotus LDAP

Server vShield Edge

vCOps

Log Insight

Hyperic

LSPA

Client

Cloud Automation & Management

Access Control

Provisioning Policies

Users

Catalogs

App Director

vC

en

ter

Orc

he

str

ato

r

VC

AC

Se

rvin

gs

Ca

talo

g M

an

ag

er

Scripts/Tasks

Application

Blueprints

Deployment

Tasks

Cloud Provider

Access Control

Provisioning Policies

Users

Catalogs

SDLC Instance

Config

Management

3rd Party

LB

IPAM

Analytics

Async

call

Audits

Postgres DB

Message

Handlers

IPAM

Client LB

Client CM

Client

Extension Module

20 20

Policy Driven Business Workloads De-Provisioning

Self-serviced Portal for extension

of lease term.

Fully automated de-provisioning

once the lease term expires.

Reclamation of resources will

help reduce the future CAPEX

investments.

Rabbit MQ Server MQ

Spring Module

Lotus LDAP

Server vShield Edge

CM

Client

Cloud Automation & Management

Access Control

Provisioning Policies

Users

Catalogs

App Director

vC

en

ter

Orc

he

str

ato

r

VC

AC

Se

rvin

gs

Ca

talo

g M

an

ag

er

Scripts/Tasks

Application

Blueprints

Deployment

Tasks

Cloud Provider

Access Control

Provisioning Policies

Users

Catalogs

SDLC Instance

Config

Management

3rd Party

LB

IPAM

Analytics

Async

call

Purge

Audits

Postgres DB

IPAM

Client LB

Client LDAP

Client

Purge Records

MQ

Purge Message Handlers

Purge Controller Modules

Rea

d V

M P

rofile

s

Extension Module

21 21

Provisioning with VMware Application Director

Cloud Provider (vCAC or vCD)

VM1

VM2 VM3

Guest Cust

Copy Files

Install Web

Start Web

Guest Cust

Copy Files

Install App

Start App

Guest Cust

Copy Files

Install DB

Config DB

Start DB Config App

Config Web

OneCloud IaaS Software Defined Data Center

22 22

Policy Driven Management

Policy Driven Storage Provisioning Leveraging vCloud Stack

In this diagram depending on the Service Profile requested, storage will be provisioned to appropriate storage profile.

Reduced OPEX with automation and reduced CAPEX with workloads provisioned with right resources

Setup different service profile policies driven by the cost and performance needs of the enterprises like Platinum, Gold, Silver etc.

Service Profile = Storage + Network + Monitor + Existing Services

Setup approval and access policies

vSphere

VMware Application Director

Access Control

Provisioning Policies

Users

Catalogs

vCenter Organization

pVDC Tier3 pVDC Tier 2 pVDC Tier 1

Application Blueprints

Cloud Providers

Deployment Profiles

Workflows Policies

vC

en

ter

Orc

hestr

ato

r

VC

AC

Serv

ice C

ata

log

Man

ag

er

vApps provisioned

in Tier 1, Tier 2

and Tier 3 Org

vDCs

Organization vDCs

Organization Network

vApp Network

External Networks

Resource Pools

Datastores

Port Groups or

dvPort Groups

Cloud

System

Admin

23 23

Integrated Monitoring

Monitor Dev, Test, Load or

Production environment

Workload Monitoring

Monitor application layers e.g.

Portal, SOA, EBS

Layer Level Monitoring

View metrics from vCops, Log

Insight in single Graph

Metrics Correlation

Real time monitoring of Key

performance Indicators

Real time view

Log Insight Hyperic vCenter Operations

Management Suite

Allows user to create new

dashboard for the required

metrics

Create New Dashboards

Drill down to resource level from

aggregated view

Drill down

3rd Party

24 24

Results

Phase 1 Phase 2

Cycle Time Hours per dev/test instance

172

36

Today

Phase 1 Phase 2

VMs Transitioned # of Non-production VMs

Phase 1 Phase 2

AppOpps team # of Engineers

Goal – 4000

2,800

2,200

Goal - 5

Start - 27 22

Reduced provision time

95% (4 weeks to 36 hours)

Improved productivity

of 600 developers

20%

Reduced

IT operations costs

$1.5M /year

Able to say

“yes” to developer requests

Reduced the cost of

a VM/month

80% ($133 to $20)

Reduced

infrastructure costs

$4.5M/year

Start – 4 weeks

Goal – 24 hours

Today Today

25 25

VMware IT: Lessons Learned

Build a business case with clear goals and planned ROI and get buy-in from the

CIO/VP level management.

Develop a phased approach with clear scope on the Services to be provided

either in Infrastructure or Applications for the successful implementation.

Develop Architecture using more cohesive integrated stack like vCloud Stack to

build Automation and Mgmt. platform to achieve all the capabilities at lower

costs and time to market.

Define the cost model for the services to serve different cost & performance

needs like Dev, UAT, Mission Critical etc.

Separate Operationalization team from the platform implementation team to

optimal implementation.

Have time dedicated for the Operationalization; new self-service processes,

retuning of skills and also ensure to have QA automation done at the end of

automated provisioning to get maximum ROI and efficiency.

26 26

Key Take-Aways

• Proven vCloud technology. Used by top customers for public and private

clouds.

• Automate, Manage and Monitor your Cloud effectively using vCloud Mgmt.

Stack to reduce provisioning time, costs and improve agility

• Hard to find a better ROI project to reduce OpEx and CapEx

In Summary

What’s Next

• Network virtualization with NSX

• Next level of optimization of Automation and Infrastructure & Application

resource usage

• More visibility into the provisioned environments to monitor, manage & self-

heal.

Other VMworld sessions & booths

27 27

Questions?

28 28

Other VMware Activities Related to This Session

Group Discussions:

VSVC1006-GD

vCloud Suite and SDDC with Tom Stephens

VSVC4948

THANK YOU

Moving Enterprise Application Dev/Test to VMware’s

Internal Private Cloud – Architecture,

Implementation and Integration

Thirumalesh Reddy, VMware

Padmaja Vrudhula, VMware

VSVC4948

#VSVC4948