going modular, lessons from 500 modular data center implementations around the world

© 2011 IBM Corporation

Going Modular? Lessons from 500 modular data center implementations around the world

John M. KundtzData Center Services ManagerIBM Global [email protected]

2© 2011 IBM Corporation

Today’s Agenda

Background – what are we seeing

Definition of Modular Data Center

Lessons Learned

Summary

More Information


IT demand

82M Servers installed by 2013

20% Server workloads have been virtualized

650% Storage growth by 2012

<30% Disk storage is used effectively

Cost pressures

75% Of CIOs anticipate a strongly centralized infrastructure in 5 years

2-3% Increase forecast for IT spending

Need for flexibility

70¢ Of every $1 is spent on maintaining existing environment

71% Of data centers are > 7 years old

5-60% Of IT workloads may be cloud-enabled

Change is accelerating …


Future proof existing data center infrastructure

Double IT capacity or reduce operational expenses

by 50%

Rationalize the data center infrastructure across the company

Improve operational efficiencies while reducing operational

expenses by 50%

Flexible design to be responsive to change

Pay as you grow by deferring 40-50% of capital and

operational costs

Integrated management of IT and data center operations Lower operational costs up to 20%

Our research suggests that Clients can respond to these challenges in 4 ways …


Data center capital costs60% costs from mechanical /

electrical systems

Shell 7%

Mechanical20%

Fees24%

Fit-Up Costs

9%

Instrumentation & Controls4%

Power36%

Source: IBM Estimates

Operating costs3- 5X capital costs

Cum

ulat

ive

Cos

t of O

pera

tions

($M

illio

ns)

Year

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Energy Cost

Staffing

Bldg. Maint. & Mgmt.

R E Tax

$100

$150

$200

$250

$50

0

Cum

ulat

ive

Cos

t of O

pera

tions

($M

illio

ns)

Year

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Energy Cost

Staffing


R E Tax

Energy Cost

Staffing


R E Tax

$100

$150

$200

$250

$50

0

Major cost savings is the deferral of 40-50% of the electrical and mechanical capacity until it is required.

Capital cost gets 100% of the focus & are 15% of the total cost.


We have analyzed over 500 global modular data center design and builds in the last 3 years ...

Custom data centers

Scalable modular data centerSCALABLE MODULAR DATA CENTER

Enterprise modular data center data center

ENTERPRISE MODULAR DATA CENTER PORTABLE MODULAR DATA CENTER HIGH DENSITY ZONE


Definitions of modular data centers … Past industry definition of “Modular Data Center”

Modular data center = Shipping container

Modular = subsystems such as UPS systems, rack/row level cooling components

Prefabricated “modules” that provide the ability to easily add space, power and cooling

More flexible definition of modular data center

Plan for unpredictability in requirements

Provide capacity when you need it

Upgrade with no disruption to operations

Maximize flexibility for future technology and computing models

Tailored to each client’s requirements, not a fixed architectural answer

Suitable for mid-size and enterprise clients


1. Data center projects often go off track …

• No governance process to make decisions based on facts

• Operating vs. capital costs are not a standard part of the design and evaluation process

• The real estate, technology and operational management teams are not all equal players

Decisions made on

wants versus needs

• Plans need to reflect unpredictability of business and technology over 10-30 year period

• No clear or detailed statement of requirements

• Clients take on the responsibility by themselves with minimal current data center skills or experience

Unclear control over

how decisions are made

Not finding a trusted

partner from concept to completion

8


Business challenge Server power density trends1: 20x

Data Center challengeSupport expected IT kW/rack power density

2. Plan to maximize capacity & flexibility for technology & computing models, investment is an upfront statement of requirements …

1. ASHRAE Publication: Datacomm Equipment Power and Cooling Applications, 2005, page 24

Source: Page 4, Figure 1.3, Best Practices for Datacom Facility Energy Efficiency Second Ed. (c) 2009.

Kilo

wat

ts/r

ack

0

10

20

30

40

2000 2002 2004 2006 2008 2010 2012 2014

Kilowatts/rack

• Support 5 kw/rack and 32 kW/Rack for high density servers• Design for air cooling as primary; provide liquid cooling without disrupting operations• Able to support 2.5x power density growth by pre-plumbing infrastructure• Leverage Rear Door Heat Exchanger as most cost-effective high density support

10 © 2011 IBM Corporation

3. Plan for unpredictability to manage the lifecycle of a data center. Dodge the next 2-3 retrofits by smart planning for unknowns …

Based on IBM estimates from client engagements

Today 50% of server density is < 5 kw/Rack or less

202060% of server density is > 10-20 kw/rack

Unknown loads. Create a density requirement model which plans for changes over time Unknown capacity. Support 3-5x power density over the 10 year timeframe Predicted load density is a pre-requisite to a better design Design included efficient dual zone capacity ; saving $1M USD per year for client

Kilowatts/rack

Racks loaded at 40% Racks at 90%

Kilowatts/rack


4. A Modular design allows for expansion of power & cooling density with minimal disruption to operations …

0

5

10

15

20

25

30

35

40

1x 3x

Pay now Pay later

• Size electrical and mechanical systems to meet future capacity needs

• Provision for potential water cooling

• Modular UPS design and chiller plant design

• Electrical switchgear sized to grow. Electrical distribution scalable per module

$ (M

illio

ns)

Power Density Growth (# times)

Piping for water cooling

Capital Costs5-10% up front to support 3x growth versus 60% higher cost & disruptive retrofit later

Good design is an insurance policy with a 5-10% premium versus paying 60% later.


5. No single electrical and mechanical approach is cost effective for all sizes and densities …

Example: 20,000 square foot data center, computer room air conditioners

Capital CostsCapital costs for computer room air

conditioners can vary by 5x

Under-floorCooling

CombinedSystems

Close CoupledCooling

$2

1.5

1

.5

0

$ M

illio

ns

Operational CostsOperational cost for computer room air

conditioners can vary by 140%

2

1.5

1

.5

0

Year

1 2 3 4 5 6 7 8 9 10

2.5

$3

$ M

illio

ns

Under-floor Cooling

Combined Systems

Close Coupled

Close Coupled, RDHX

• Although highly efficient, close coupled cooling doesn't always provide the lowest TCO.• Rear door heat exchanger is the lowest cost solution to support high density equipment.• Raised floor cooling is cost effective for low density environment. Source: IBM Estimates


In Summary: Lessons learned from 500 modular data center implementations include …

• A good modular design is an insurance policy with a 5% premium versus 60% later.

• There are major cost savings is the deferral of electrical and mechanical capacity until it is required.

• No single electrical and mechanical approach is cost effective for all sizes and power densities.

Decisions made on

wants versus needs

• Insure your data center strategy can meet business and IT requirements over the long term.

• Best investment is an upfront statement of requirements.

• Plan for unpredictability. Dodge the next 2-3 retrofits by smart planning for unknowns.

• There is not a one-size-fits-all strategy.

• Find a partner who doesn’t use a fixed architectural approach.

Not finding a trusted

partner from concept to completion

13

Unclear control over

how decisions are made


Scalable modular data center.

Enterprise modular data center.

Portable modular data center.

High densityzone.

• Turnkey center for 500-2,500 sq ft.

• Implement in 8-12 weeks.

• Standardized design starting with modules as small as 5,000 sq ft.

• Save up to 50% operational costs.

• Rapidly deploy in 12-14 weeks.

• Without impacting operations.

Watch a video at :http://www-03.ibm.com/systems/data/flash/dynamicinfrastructure/datacenterdesignsolutions/

15-25% lower TCO than traditional data centers

Defer 40-50% of capex and opex cost

Fully functional data center

35% lower cost than site retrofit

IBM has designed over 500 modular data centers in the last 3 years to be responsive to change …

Design new infrastructure for flexibility with modularity IBM’s data center familyTM solutions align to your business and cost objectives.


IBM site and facilities services Extend the life of an existing data center

Data center strategy Data center consolidation and relocation

Modular data centers Scalable modular data center (SMDC)

Enterprise modular data center (EMDC)

Portable modular data center (PMDC)

For more information on “going modular”