10/8/10 1

Natural Low Energy Cooling Conceptual Design

Ultra Efficient HPC Data Center

Project Funding

  Canada's Advanced Research and Innovation Network (CANARIE)

  Canada California Strategic Innovation Partnership (CCSIP) •  ISTP Canada

•  University of California

•  McGill University

Site Selection

  Three candidate locations in Quebec •  McDonald Campus of McGill University in St. Anne de Bellevue

•  Campus of the Institut de recherche d’Hydro-Quebec (IREQ) in Varennes

•  IREQ campus in Shawinigan

  McDonald Campus of McGill University selected as site for project

  All enjoy   Cold climate

  Renewable energy resource (hydroelectric)

  Inexpensive electricity

The System Approach: An Overview

  Goals: Most Efficient Class One Data Center

  Climate Evaluation

  Define Loads and How to Best Serve Them •  Water cooled equipment

•  Medium temperature chilled water (65F, 75F)

  Optimize Heat Rejection for Climate and Loads •  Evaporative free cooling – Primary cooling

•  Seasonal ice storage – Top up cooling

  Backup Approach

  Results

Goals

  Provide ASHRAE TC9.9 Class 1 Datacenter

  No compressor based cooling

  Lower construction cost

  Lower operating cost

  Best efficiency

  Environmentally friendly

  Construction materials

  Recycle heat, water

$0

$1,000,000

$2,000,000

$3,000,000

$4,000,000

$5,000,000

$6,000,000

$7,000,000

$8,000,000

$9,000,000

$10,000,000

San Diego (1.35 PUE) at $0.09/kWh

Montreal (1.06 PUE) at $0.05/kWh

$5 Million/yr Annual Savings Target

Proposed Annual Electrical Costs Comparison

Aerial Perspective – “Farm” at McDonald Campus

Stormwater Detension

Office, Shipping/ Receiving

20,000 SF Phase 1 8 MW IT Load

Electrical Infrastructure

Cooling Ice Pond

Cooling Towers and Mechanical Infrastructure

VA Hospital

Fuel Tanks

Climate: Free Cooling Analysis with 65F CHWS

0.000

0.002

0.004

0.006

0.008

0.010

0.012

0.014

0.016

0.018

0.020

0.022

0.024

0.026

0.028

0.030

-30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

Hum

idity

Rat

io (l

bs H

2O p

er lb

s dr

y ai

r)

Dry Bulb Temperature (F)

Data Source: Government of Canada - National Climate Data & Information Archive

Data Set: WMO #71627, Montreal/Pierre Elliott Trudeau Airport, Typical Year

Elevation: 118 feet

Air Pressure: 14.633224 psia

Full Free Cooling

7446 hrs/yr

Partial Free Cooling 1234 hrs/yr

Auxillary Cooling 80 hrs/yr

Climate: Free Cooling Analysis With 75F CHWS

0.000

0.002

0.004

0.006

0.008

0.010

0.012

0.014

0.016

0.018

0.020

0.022

0.024

0.026

0.028

0.030

-30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

Hum

idity

Rat

io (l

bs H

2O p

er lb

s dr

y ai

r)

Dry Bulb Temperature (F)

Data Source: Government of Canada - National Climate Data & Information Archive

Data Set: Montreal/Pierre Elliott Trudeau Intl Airport, Typical Year

Elevation: 118 feet

Air Pressure: 14.633224 psia

Full Free Cooling

8646 hrs/yr

Partial Free Cooling 114 hrs/yr

Auxillary Cooling 0 hrs/yr

Load Strategies

  Climate analysis shows higher temperature chilled water offers many more hours of free cooling

  Highly concentrated heat loads •  A single high density rack

can put off as much waste heat as a VW Beetle (40kW)

•  Air exiting racks typically exceeds 90F

Load Heat Collection Strategies

  Higher temperature Chilled Water Supply (CHWS) offers many more hours of free cooling: Design to use 75F and 65F CHWS •  Direct water based cooling most efficient

•  Hot aisle / cold aisle for minority of load

Primary Cooling Strategies: Medium Temp. Cooling Water and Free Cooling

Primary Cooling Strategies: Medium Temp. Cooling Water and Free Cooling

  Design to cool with 65F/18C and 75F/24C water •  90% of 65F load served with cooling tower provided free cooling;

99.3% of 75F load

•  590,000 ton-hrs (2,100 MWh) Top up Cooling Required

Supplemental Cooling: Seasonal Ice Storage Slush Pond System

  Fill in winter with plowed snow collection

  Melt water cools data center

Slush Pond

  Paved collection basin, 75,000 ft3 (2,100 m3)   Drive-in slope on one side for plow loading

  Lightweight, waterproof insulating cover or roof to protect from warm rains

  Extensive drain system to collect meltwater

  Berms for sides, or dig into ground

Sundsvall, Sweden, Snow Storage - Empty

Sundsvall, Sweden, Snow Storage - Full

Snow dump overruns in Montreal, 2008-09

Slush Pond System – Pumping and Filtration

  Mature waste water handling technology   Remove gravel, wood, grit from melt water

  Remove oils and road chemicals prior to release as required

  Filter

  Select heat exchangers for highly corrosive fluid   Maintain complete separation between pond water and

building loop water

  Integrate settling tank to also serve as emergency storage

Key approaches

  Keep storage pond simple   Leverage local snow removal program if possible

  Collect snow dumpage fees?

  Provide appropriate maintenance   Provide for pile grooming, drain clearing, filter cleaning, etc

  Use in lieu of chillers to save cost   Consider emergency chiller rental for backup

  Design properly   Simple concept but careful design required

Office Approaches

  Much lower load

  Design for comfort and optimal use of medium temperature water

Backup – Do Not Invest in Chillers 'Just in Case'!

Pay for it only

when (if) you

ever need it

Design for

portable

air-cooled

chillers

to connect in

an emergency

Results

Transformer Loss; 0.5%

UPS Loss; 0.6%

PDU Loss; 1.0%

Data Center Lights; 0.2%

Racks; 94.0%

Fans; 1.5%

CRAH Fans; 0.0%

Humidifier; 0.0%

CHW Plant; 2.1%

McGill-USCD HPC Data Center PUE Itemization

Power Usage Effectiveness (PUE) = Total Energy / Rack Energy = 1.06

Results

Supply Temperatures

Hours of Free Cooling / year PUE

Annual Energy Use Mechanical Cooling Needed Water Usage

Air Cooled

Water Cooled Energy Cost

( $0.058/kWh) Hours per Year

Additional Load at Extreme Weather (wetbulb = 68.7°F)

Evaporation + Carry Over

°C °F °C °F hrs/yr % of yr MWh/yr $ tons gallons

23.9 75.0 23.9 75.0 8,532 97% 1.06 74,567 $4,325,000 228 0 30,100,000