uptime institutetier standard for data centers facts vs. … · uptime institute tier standard for...

Uptime InstituteTier Standard for Data Centers

Facts vs. Myths and Misconceptions

Presented byKenneth (“Ken”) Brill, Founder

Uptime [email protected]

2 © 2011 Uptime Institute

Agenda Introduction to Tiers Benefits and misconceptions Impact of IT availability trends on required

facility Tier level Certifications Training More Tier information


Uptime Institute Is the ONLY Authority for Tier Standard Certification

4 © 2011 Uptime Institute4

Uptime’s International Tier Standard Has Two Parts Topology Tier I Tier II Tier III Tier IV

Operational Sustainability (new 2010) Gold Silver Bronze

Chronology Topology introduced in 1996 with update in 2008 Operational Sustainability introduced July, 2010


This Table Was Eliminated in 2008 to Reduce Confusion & Misconceptions

TIER I TIER II TIER III TIER IVBuilding Type Tenant Tenant Standalone StandaloneStaffing None 1 Shift 1+ Shifts 24 by

ForeverUsable for critical load 100%N 100%N 90%N 90%NInitial build-out UPS output watts/ft² (typ)

20-30 40-50 40-60 50-80

Ultimate UPS output watts/ft² (typical)

20-30 40-50 100-150 150+

Class A uninterruptible cooling

No No Maybe Yes

Support space to raised floor ratio

20% 30% 80-90+% 100+%

Raised floor height (typical) 12” 18” 30-36” 30-36”Floor loading lbs/ft² (typical) 85 100 150 150+Utility voltage (typical) 208,480 208,480 12-15 kV 12-15 kVSingle points-of-failure Many +

human errorMany + human error

Some + human error

Fire+ EPO + human error

Annual site caused IT downtime (actual field data)

28.8 hours 22.0 hours 1.6 hours 0.8 hours

Representative site availability

99.67% 99.75% 99.98% 99.99%

Months to implement 3 3 to 6 15 to 20 15 to 20Year first deployed 1965 1970 1985 1995


All Drawings Have Been Eliminated


Common Myths & Misconceptions Two utility sources are required for Tier IV Tier III has no single points-of-failure Fault Tolerance of Tier III is higher than Tier II Site location can preclude a topology rating MTBF calculation determines Tier Tier is determined by equipment count Tier III+, Tier III.6, etc., have meaning Tier IV eliminates human error


Common Myths & Misconceptions Two utility sources are required for Tier IV Tier III has no single points-of-failure Fault Tolerance of Tier III is higher than Tier II Site location can preclude a Tier rating MTBF calculation determines Tier Tier is determined by equipment count Tier III+, Tier 111.6, etc. have meaning Tier IV eliminates human error


Data Center Site Infrastructure Tier Standard: Topology


Tier Topology Certifications Tier IV – Fault Tolerance Constructed facility 5 Complete 0 In-progress

Design documents 14 Complete 3 In-progress

Tier III – Concurrent Maintenance Constructed facility 10 Complete 8 In-progress

Design documents 33 Complete 23 In-progress

As of March 1, 2011


Data Center Site Infrastructure Tier Standard: Operational Sustainability


Site Certification:Topology + Operational Sustainability Tier III Gold Certifications awarded UnitedHealth Care, Next Technology Center Target Corporation, Target Technology Center

Brooklyn Park, MN Target Corporation, Target Technology Center

Elk River, MN Fujitsu Services, London North Data Centre

London, UK Others are in progress


Uptime’s Tier Standard System Provides an unbiased review of a single site or

a comparison of two sites or a portfolio of sites Assists owners in maximizing the life-cycle value of

their investment Assists purchasers of third-party services in

quantifying what they are getting Encourages operations to maximize uptime,

minimize risks, and achieve efficiency Incents and measures processes optimizing energy

efficiency Quantifies site infrastructure management

effectiveness


Tier Standard Does NOT Compete With or Replace Other Standards & Codes Tier Standard exclusions: Construction and building codes Environmental regulations Safety policies and practices Security (physical and data) Personnel policies Energy efficiency International Standards Organization® (ISO ®)


Tier Standard Was Developed By and For “Owners” with “Skin in the Game” BMO Financial Group Boeing Delta Airlines Fidelity Investments Fujitsu Services GE HSBC Johnson and Johnson JP Morgan Chase Kaiser Permanente Lowe’s

Morgan Stanley Nationwide Mutual

Insurance Progressive Insurance Saudi ARAMCO Social Security

Administration State Farm Insurance Target United Health US Bank VISA


Tier Standard Topology DefinitionsTier I Basic CapacityTier II Redundant ComponentsTier III Concurrent Maintenance - applies to

Each and Every component and path on a planned basis*

Tier IV Fault Tolerance - anticipates a single, mechanical or electrical system or equipment event and subsequent consequential impact(s)** Excludes EPO (North America only), fire, single-corded IT equipment, incorrectly installed dual-corded IT equipment, sabotage, rules violations, and/or mistakes. See Standard for additional details and exclusions.


Tier Standard Topology Ratings Are Vendor and Solution Agnostic Encourages innovative engineering Allows for client equipment preferences Not a proscriptive checklist, design menu, or a

recipe


Availability Trends - Today and Future Do you expect prime time and/or availability

requirements to go up or down in the future? Is your compensation tied to an Availability

Service Level Agreement? Is planned and unplanned Facilities downtime

included or excluded from your SLA? Do you have a financially enforceable SLA with your

Facility provider (internal or external) that reflects IT’s commitments to users?


Tier I & II Sites Require IT Downtime Tier I and II sites require total shutdown of all IT

hardware during facility maintenance, i.e., all IT hardware dependent upon UPS power Critical power distribution Cooling backbones

Some capacity units (Tier II) MAY be maintained without impacting IT (but often with considerable risk of unplanned downtime)


UtilityBatteryGenerator

THREE POWER PATHS

ONE POWER PATH

COMPUTER HARDWARE

SINGLE POINTS–OF–FAILURE• UPS system level failure• Major circuit breakers (2-20)• Minor circuit breakers (20-500)• Plugs and receptacles (21-505)• Electrical connections (258-6180)• Human error• False EPO

Single Power Path


Utility UtilityBatteryGenerator

THREE POWER PATHS

TWO POWER PATHS

COMPUTER HARDWARE

Generator Battery

23

1

SINGLE POINTS–OF–FAILURE• Go to UPS failure• Human error

•False EPO•A/B circuiting errors

Dual Power Path

45


Planned or Unplanned Site Downtime Can Trigger a Corporate Heart Attack!

Customers/UsersNetwork Nodes

ApplicationsOperating System

HardwareSite

10,000s100s100s10s to 100s10s to 100s1


Fault Tolerance Is Justified By Eliminating Unplanned User Downtime

Site Restoration = ¼ to 1hr

Hardware Restore = ½ to 1hr

Software Restore = ¾ to 2hr

Telecom Restore = 0 to ¾hr

TOTAL OUTAGE = 1½ to 4¾hr


Concurrent Maintainability Is Justified by Eliminating Planned User Downtime Tier I sites require 2 total shut downs per year 2 x 14 hours = 28 hours of IT downtime per year Plus the impact of any unplanned failures

Tier II sites require 3 total shut downs every two years 3 x 14 hours/2 = 21 hours of IT downtime per year Plus the impact of any unplanned failures

Tier III and Tier IV sites do NOT require planned shut downs**Be aware that many “self proclaimed” Tier III Concurrently Maintainable) and Tier IV (Fault Tolerant) sites are not!


Tier Standard: Topology Requirements Are Deceptively Simple Active capacity components Distribution paths Compartmentalization Concurrent Maintainability Fault Tolerance


Topology Requirements by Tier Level Tier I Tier II Tier III Tier IV

Active Capacity Components to Support IT Load

N N+1 N+1 N after any

failureDistribution Paths

1 1 1 active, 1 inactive

2 active

Compartmentalization No No No YesConcurrent Maintainability No No Yes YesFault Tolerance (1 event) No No No YesContinuous Cooling,Continuous Duty EnginesAmbient Temperature

LoadYes

YesYes

Worst


Accredited Tier Designer (ATD) Intense, three-day curriculum

with concluding test Exclusively for professional or

chartered engineers responsible for infrastructure center design

Focuses on practical applicationof Tier topology

Examples drawn from real-life designs submitted for Design Certification

Addresses, in technical detail, the misconceptions that lead to needless cost and complexity


A Worldwide Growing Community of ATD ProfessionalsAustralia 2Belgium 1Brazil 5Canada 3China 2Costa Rica 7Czech Republic 1Egypt 1England 6France 1Greece 2India 1Hong Kong 1Kenya 1Luxembourg 1Mexico 3Malaysia 1

Netherlands 1Russia 15Saudi Arabia 2Slovakia 1Spain 3South Africa 1Taiwan 6Thailand 1Turkey 1United Emirates 3United States 30Venezuela 1

As of September 31, 2010


Site Certification Process:Tier Certified Design Documents Uptime, in-office, Principal-

level review of 30%, 60%, 85%, 100% design document package

Topology deficiencies and potential design enhancements

Conference call with owner and design team to discuss deficiencies report

Compliance review of revised drawings

Award letter and foil


Site Certification Process: Tier Certified Constructed Facility On-site inspection by Uptime

team during commissioning Identify discrepancies between

certified design drawings and as-built construction

Observe delivered functionality Topology deficiencies and

other enhancements Conference call with owner

team Award letter, foil, and plaque


Tier Standard:Operational Sustainability (New – 2010) Operational Sustainability is defined as the

behaviors and risks beyond design Topology that impact the ability of a data center to meet business objectives over the long term.


Abnormal Incident Reports (AIRs) By: Root Cause

2009 Failures

(Previous Years)

2009 Incidents

(Previous Years)

Manufacturer 5 19% (25%) 211 52% (43%)

Design 1 4% (3%) 20 5% (5%)

External Environment - 0% (14%) 10 2% (4%)

Installation 4 15% (7%) 25 6% (7%)

Operations 16 62% (51%) 144 35% (41%)

Total Root Cause 26 100% (100%) 410 100% (100%)


“Gold” Operational Sustainability Behaviors Prevent 70% of AIRs Failures Failures are never the result of one factor Between 5 and 10 things (typically 7)

contribute to a failure Management is the leading cause (4 out of 7)

A failure will consistently occur every time the 7 things line up

Swiss cheese or dominos analogy


Operational Sustainability: Categories Management & operations (immediate ability to improve)

Staffing and organization Maintenance Training Planning, coordination, and management

Building characteristics (limited ability to improve) Features Infrastructure Operating conditions Pre-operational

Site location (limited ability to eliminate or even mitigate) Natural disasters Man-made disasters Utility robustness and redundancy


Accredited Tier Specialist (ATS) (New 2010) Intense three-day curriculum with concluding

test For data center professionals (i.e., for non-

engineers) Includes both Topology and Sustainability

concepts ATS accredited individuals now in 12

countriesAs of March 1, 2011


Conclusions Tier Standards for Data Centers Topology + Operational Sustainability For owner/operators and buyers of hosting services Together, assure life-cycle business value Downtime frequency, duration, and recoverability CapEx and OpEx effectiveness Life-cycle (resale) value

Operational Sustainability practices often have greater impact on actual IT availability than Topology


More Information Tier Standard: Topology at

http://uptimeinstitute.org/content/view/302/281/#Tier-Classification

Tier Standard: Operational Sustainability athttp://uptimeinstitute.org/content/view/302/281/#Operational-Sustainability

Current world-wide listing of ATD engineers at http://atd.uptimeinstitute.com/award_list.htm

Certified sites or certifications in process at http://professionalservices.uptimeinstitute.com/tiercert.htm

Upcoming ATD and ATS seminars at http://atd.uptimeinstitute.com/schedule.htm

Tier Myths and Misconceptions at http://professionalservices.uptimeinstitute.com/myths.htm

http://uptimeinstitute.org/content/view/302/281/�




http://atd.uptimeinstitute.com/award_list.htm�

http://professionalservices.uptimeinstitute.com/tiercert.htm�

http://atd.uptimeinstitute.com/schedule.htm�

http://professionalservices.uptimeinstitute.com/myths.htm�


Questions