Optimizing High Density Cooling Solutions for Data Centers

Source: Gartner 2006 Data Center study 180 respondents

• Rack Power of 10 kW per enclosure can results from deployment of high density blade servers.

• 90% of data centers are operating at 1.8kW per enclosure• Limiting factors when considering a data centre builds are:-

• Power reticulation• Assume a 43U cabinet, with 6 x 7U Blade servers each consuming

3kW. This means 18kw Power draw per rack, which also implies 18kW cooling required per rack!

• This system would typically be mission critical ,thus redundancy would also be required.

• Costs are huge!

• The limiting factors presently are:

• Cooling • The system described would require 2,500cfm cool air at the in-

take, and thus the same at the outlet (hot) isle per rack• A typical 600x600 floor tile can achieve 500 cfm from a grated

ventilated floor tile, and only 300cfm from a louver ventilated tile.• This means that 5 floor tiles for every enclosure in a high density

configuration – just not feasible!• If the air reticulation cannot remove the hot air fast enough, it

WILL re-circulate back to the blade server, causing it to overheat.

These work and designs are right!Problem is not the design and the layout, its how the air is directed and the volumeof air need to cool cabinets reaching 4kW-15kW per enclosure.

Design

Χ Cooling abilityAir VOLUME becomes the limiting factor

• Will you have enough cooling with existing or upgrading cooling equipment?

• If so, try maximizing your infrastructure before embarking on new designs and development!

• Will you need High Density Solutions? • Do you need ALL the High Density Now?

• If not can you upgrade easily when you need to?

• What are your options?

Load Spreading – Provide the room with the average power and cooling requirements and spread the load across a greater area i.e. use more enclosures in a bigger data centre.

Borrowed cooling – Allow high density racks to borrow underutilized cooling from adjacent racks

Supplemental cooling – Add additional freestanding cooling units to specific racks within the datacenter.

Whole room cooling – Provide additional capacity to allow for peak expected enclosure density. This is typically a very expensive and environmentally damaging route.

• Load Spreading – Provide the room with the average power and cooling requirements and spread the load across a greater area i.e. use more enclosures

• Borrowed cooling – Allow high density racks to borrow underutilized cooling from adjacent racks

• Supplemental cooling – Add additional freestanding cooling units to specific racks within the datacenter.

• Whole room cooling – Provide additional capacity to allow for peak expected enclosure density. This is typically a very expensive and environmentally damaging route.

• Dedicated high density Areas – Within the standard data centre, provide a limited area that has high cooling capacity.

• In Row Supplemental Cooling• In open hot / cold aisle format• In closed loop single row

• Hot Aisle Thermal Management• Cold Aisle Containment• Self Contained Closed Loop

(POD)

These can be retrofitted when required

• Advantages• Fits into Traditional SmartRac (Standard High

Density Cabinets) configuration• Scalable – add additional coolers when required• Cost effective• Eliminated Hot Spots

• Disadvantages• Not as efficient in cooling as the environment is not

contained• Cold air not easy to manage/direct

Closed Loop Single In-Row Cooling

• In Row Supplemental Cooling• Hot Aisle Thermal Management• Cold Aisle Containment• Self Contained POD

• Advantages• Heat removal capacity• Entire room flooded with cold air – better in the event of A/C

failure• Disadvantages

• No A/C redundancy• Very high temperatures in hot aisle• High noise levels• Temperature between rack bottom and top not constant• Cannot accommodate fire suppression in hot aisle• A/C shut down required for servicing• Use of EC fan technology reduced.

1. The void between the raised floor and the ground is flooded with cold air from the CRAC

2. The positive pressure in the floor allows the cold air to filter into the contained aisle through vented floor tiles

3. This cold air is then pulled through the equipment inside the rack

4. The air is now warmer due to the heat generated by the equipment. The hot air is channelled upward in the plenum door by twelve fans.

5. The hot air continues upward, through the chimney, until it is exhausted into the ceiling void

• Advantages• Cold air contained and easier to manage• Temperature between rack bottom and top more constant• Fire suppression available for cold aisle – less extinguishant

required• A/C redundancy

• Disadvantages• Environment will heat up quickly in the event of A/C failure• Prone to hotspots in the event of fan failure (fans must be

hot swappable). Not in the case with EC fan technology.

• Advantages• Combines advantages of hot and cold aisle containment

– Fire suppression easily accommodated– A/C redundancy– More consistency in temperatures across the rack

• Exhausted air is re-circulated back into a/c system– Higher cooling capacity– Greater control over humidity

• Secure• No effect on surrounding Environment• Stand alone units. No need for raised flooring

• Disadvantages• Prone to hotspots in the event of fan failure (excluding EC fan technology)• Environment will heat up quickly in the event of A/C failure• Need to build in Redundancy.

Integrated cooling solution for rack mounted electronics devices and server rack, suitable for heat load up to 35 kW each rack.

Complete range of models designed for all size data centre, high efficiency and the reliable design of Modrac cooling units reduce strongly the total cost of ownership and improve the return of investments.

• Modular solution, suitable to install inside 43U, 300mm rack

• Same module for open and closed architecture

• 96mm Air filter for open loop version

• Supply connection from bottom or top

• 2 or 3 way modulating CW valve

• High efficiency due to the air direct cooling (max DT)

• Small footprint

• EC fan available – fans slow down and use less power, again saves costs!!

• Controlled by microprocessor Modrac C2020

• Mod-bus connection via RS 485 (opt)

• MIB interface for BMS monitoring (opt)

• Wide range of solutions, up to 35kW

• Built in redundancy

• Manage thermal load locally, eliminate hotspots

• Hot swappable fan units

In-Row cooling module is provided with high efficiency coils, a strong air flow it’s generated by high reliable and efficient radial fans, this ensures an effective cooling of servers in all operations conditions.

The operation on ModRac’s In-Row cooler is managed by Modrac C2020 microprocessor electronic control

Type size kW Win Wflow l/ s WPD kPa Ain Aflow mc/ h n° fanA4 12.3 0.59 35 1500 3B2 20.6 0.98 56 2800 5C4 31.5 1.51 58 4200 5A4 13.9 0.67 42 1500 3B2 21.8 1.03 62 2800 5C4 34.7 1.64 65 4200 5

45closed

10

7

Type size kW Win Wflow l/ s WPD kPa Ain Aflow mc/ h n° fanA1 10,5 0,5 29 3000 3A5 15,6 0,75 37 3600 5B1 20,3 0,97 56 4800 5A1 12,6 0,59 32 3000 3A5 18,3 0,87 45 3600 5B1 24,2 1,14 53 4800 5

30open

10

7

Controlling the various operating modes• Compressor operation with enabling signal to the external motor-condensing unit (only direct expansion models)• Hotspot mode (enabled by an external thermostat) : fans rotate at maximum speed to prevent excessive temperatures in particularly critical areas of the room.• Dehumidification

Step by step operator guidance via keypad• USER, to change set points, access alarm history and counters of working hours.• SERVICE, to start-up and make maintenance on the unit.• MANUFACTURER, for factory setting.

Flash EPROM for easy configuration and software updates• Central configuration of units via laptop• Hardware key for uploading and downloading software without a laptop andfor copying the configuration to other units

Individual forwarding of alarms• Via bus system/BMS system (optional)• Via voltage free contacts (standard) 9 off-load contacts are available. Alarms can be assigned a high or low priority.

Energy-saving mode• Different kinds of capacity modulation (with valve opening and fan speed) aimed to energy saving.

MonitoringRemote Control

Configuration

Email/SMSAudible AlarmHTTP/SNMP

The thermal load removed from the server rack has to be transferred outdoors. To do this, units are supported by chiller units, either in chilled water or DX state.

Units can have all the advantages of freecooling chiller to reduce the energy consuption. When outside tempreatures are below 18 deg C, we use the ΔT, to cool, thus keeping the condenser off and saving energy and costs.Green datacentres!