Virtual Infrastructure 3 Deployment

A VI3 deployment in a Mid-size Company

About Christie Digital• A leader in visual solutions for business, entertainment and

industry

• Head office for Manufacturing & Operations in Kitchener, ON

• Head office for Sales & Field Operations in Cypress, CA

• Numerous AV Industry Awards; won 2 Oscars for technical achievement

CDS: A good problem to have

• Explosive growth: doubled in size in less than 2 years

• Global Information Services (GIS) had to quickly adapt to increasing business needs

• GIS faced the usual suspects:– Provisioning takes too long– Too many “one application, one box” requests – Proliferation of hardware surpassed server room cooling

capacity– Legacy Applications were on older and dying hardware– No time to manage all the requests

GIS Responses to the Business needs

• Next IT investment: IBM BladeCenter + SAN– One blade center – 14 blades – for long term growth– 6 months, had to purchase an additional blade center– 6 months, require yet another blade center

• Have outgrown our initial blade center architecture and required some additional changes, especially for the SAN

• Still had legacy apps on older hardware

Just wasn’t enough

VI3: Stop the Madness

• Virtual Center – recycled older rack mount hardware

• ESX Servers – 2 x IBM Blades, 16 GB RAM, Dual Proc, 4 Core– With shared Storage; created an ESX DRS/HA Cluster

• VM Converter – Migrate physical to virtual

• ESX Patch Repository – IIS Server

• VCB Proxy – Leverage our existing utility Server

Virtual Center SQL 2005 Database

VCB Proxy

VM Converter

ESX Patch Repository

Setting up the Hardware

• Burn in process for 72 hours using IBM diagnostic tools

• Installed ESX manually (only two servers) onto local disk as per:/boot 1000 MB (ext3)

swap 544

/ 5000 (ext3)

/home 5000 (ext3)

/tmp 3000 (ext3)

/var 3000 (ext3)

/vmkcore 110

/vmimages rest of disk

Shared Storage

IBM DS 4800 SAN, Created 3 LUNs

• 1 LUN ~ 100 GB for Templates & ISO’s on SATA

• 2 LUNs ~ 400 GB for VM’s on FC

– Each FC LUN has a “primary” ESX server, but both can see the other LUNs for DRS & HA

– Left 60-70 GB per LUN for snapshots/backups/etc

Authentication

• Set up NTP• Modifying the “wheel” local group using visudo• Added local users to the wheel account• Connected with VI Client and gave wheel group admin access

Active Directory

Domain/admin1

ESX Server

admin1

esxcfg-auth

Patching

patchESX.pl

IIS ServerpatchList.txt

esxupdate -n -r http://IIS/patch#

• Over 20 ESX 3.01 patches have been released– Created IIS repository as per:

http://virtrix.blogspot.com/2007/03/vmware-autopatching-your-esx-host.html

ESX Server

Networking

Each machine had 4 NICs

• 1 NIC for service console– Debated about putting into a management VLAN

• 1 NIC for vmkernel (for Vmotion)

• 2 NICS, load balanced for VM

Network configured from within Virtual Center

“Instant” Provisioning

Current Method

• Un-box, rack and cable

• Configure hardware– If blade, configure SAN

• Walk through vender Server Config Guide

• Install Operating System

• Install Patches

• Configure server for GIS “standards”

Using VMware

• Right-click template, select “create new virtual machine”

• Wait for file copy

• Adjust machine settings as appropriate

Assuming hardware is available, deployment takes approximately 1

business day

Assuming capacity is available, deployment takes approximately

10-20 minutes

VM Provisioning Process

• Created optimized template; excess services uninstalled – http://download3.vmware.com/vmworld/2006/mdc9700.pdf

• 5 GB Template file, used vmkfstools –X to expand– When VM boots, use ISO “gparted”:– http://sourceforge.net/project/showfiles.php?group_id=115843&

package_id=173828

• Have syprep installed on the Windows image; use “sys-unconfig” for Red Hat (ish) installs

Convincing the Powers that be

• Three pronged approach to getting approval

– Get influential techies on your side

– Leverage any publicly announced environment policies

– Play dirty; talk about money

VMware Server – A solution to Dev Machines

“Virtual Lab in a box”• Semi-powerful desktop

– 4 GB of RAM, Dual-Core CPU, RAID Controller, extra NIC

• Use Linux OS (Red Hat, CentOS, etc) as it has a smaller footprint than Windows; use NoMachine for remote admin

• Install VMware Server (free!),

• VM Converter to move templates and VM’s back and forth

Reducing the Carbon Footprint

• CDS ISO 14001 Certified “Environmental Management System Policy”

• VI3 has considerable impact on GIS’s positive contribution to the Policy

VI3 can:

• Reduce the amount of physical hardware (and thus raw materials)

• Reduce energy consumption to power servers

• Reduce cooling requirements, and thus energy consumption

“We are fully committed to environmental solutions …and to meet or exceed applicable environmental laws, regulations and organizational objectives.”

Return on Investment

• In the 2 quarters, 11 known projects that require a server

High-Tier @ $5000 Mid-Tier @ $3000 Low Tier @ $1500

• Still have to add in cost for a blade chassis @ $30 K

$85 K $63 K $46.5 K

Options:

• VMware Infrastructure 3 Investment = $51 K– Plus still have the capacity for 11 more virtual machines!

$55 K $33 K $16.5 K

Lesson’s Learned

– Leverage your VMware Rep!

– Read the fine print…

– VCB is a “backup framework”, not a backup solution

A Windows VMware Admin (free) Tool set

• Putty – required for ssh access to the service console OS

• WinSCP – A GUI scp client to move files (mainly ISO) to and from ESX servers

• Veeam FastSCP – another GUI scp client, but infinitely faster than WinSCP– http://www.veeam.com/veeam_fast_scp.asp

• No Machine – a “RDP” client that easily allow X-windows access to Linux boxes through ssh– http://www.nomachine.com/

Recommended Reading

• RapidApp's Quick Start guide to ESX 3.0http://www.lulu.com/content/712361

• VI3 book; includes free Advance Design 2.5 Guide downloadhttp://www.vi3book.com/

• Mega-link post in user forums

http://www.vmware.com/community/thread.jspa?threadID=81191&start=0&tstart=0