forecasting data center power requirements: tips from the trenches frank bereznay moses chu ibm

Forecasting Data Center Power Requirements:

Tips from the Trenches

Frank BereznayMoses Chu

IBM

Agenda

Overall Facility Utilization / Usage Profiles Measuring Power Usage at the Equipment

Level Reporting and Forecasting Techniques Summary

Facility Overview

Two Story Facility Built in the early 1990’s to accommodate mainframe class

equipment. 2 UPS Systems

Primary UPS (A/B) 2,400 KW Secondary UPS (C/D) 600 KW Added later

3,500 Items Connected to UPS Systems Equipment Mix Described as typical for today’s Data

Center. Newer blade farms creating hot spots for A/C

systems

Initial Measurements

There were a lot of opinions about the overall utilization profile for the UPS systems, but nobody had hard empirical data.

A low tech ‘clip board’ agent was installed and activated at the UPS panels at four hour intervals.

Initial MeasurementsUsage Profile for Primary UPS System

85.0%

85.3%

85.5%

85.8%

86.0%

86.3%

86.5%

86.8%

87.0%

0 4 8 12 16 20

Hour of the Day

Sun Mon Tue Wed Thur Fri Sat

Initial MeasurementsUsage Profile for Secondary UPS System

79.5%

80.0%

80.5%

81.0%

81.5%

82.0%

0 4 8 12 16 20

Hour of the Day

Sun Mon Tue Wed Thur Fri Sat

Usage Profile by Equipment Type

WinTel Servers52%

Non-WinTel Server8%

Mainframe3%

Storage21%

Network - Data10%

Telecom1%

Other5%

Measuring Power Usage at the Equipment Level

Not as easy as it sounds. Engineers were reluctant to open up the back

side of PDU cabinets to attach reading devices.

Power cables coming out of the cabinet were not labeled.

Similar situation at equipment end of power run.


Looked for some equipment documentation on actual power usage. Generally speaking, the documentation only

provided rating factors for the circuit breakers. Blade servers were the exception, they provided

actual power calculators. Looked online (and the CMG proceedings)

for articles providing this kind of information. Came up empty.


Pursued two efforts in parallel Captured before and after power readings at the

UPS level for any change that projected a 5KW or greater impact

Worked with staff at a newer Data Center that had the capability to provide circuit level power usage profiles

Developed spreadsheet of findings and created some initial forecasts


Initial forecasts tended to underestimate the impact of power events. Part of the learning curve for this type of analysis.

A ‘life cycle’ usage profile for equipment types began to emerge from the dual measurement efforts being conducted.


Power Usage Profile for WinTel Servers

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

Power Up Idle TypicalLoad

HeavyLoad

Idle PowerDown

Measurement Points


Power Usage Profile for Midrange and Mainframe Servers

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%


HeavyLoad

Idle PowerDown


Power Usage Profile for Storage Equipment

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%


HeavyLoad

Idle PowerDown

Reporting and Forecasting Techniques

All data capture and analysis was done in EXCEL, nothing fancy.

Experimented with a variety of reporting techniques before we came up with the final set of reports.


Project Name / Identifier Equipment Type Qty KW Install date Week of Status 7-Jul 14-Jul 21-Jul 28-Jul 4-AugBlade Farm Upgrade L20p G4 64 0.3 07/11/08 07/07/08 16.0 Retire Development Regatta IBM 9119-590 -1 20.3 07/13/08 07/07/08 (20.3) New Prod Storage Frame DMX3-24 2 34.7 07/16/08 07/14/08 69.4 Retire Reporting Servers ML530 G2 -36 0.6 07/24/08 07/21/08 (20.2)New OLAP Reporting Servers ProLiant ML530 8 0.7 07/25/08 07/21/08 5.5Retire Prod Storage Frames DMX3-24 -2 34.7 07/30/08 07/28/08 (69.4)New zSeries Development Host IBM 2094-S38 1 12.7 08/06/08 08/04/08 12.7New Devl Storage Frames DMX3-24 2 34.7 08/08/08 08/04/08 69.4 16.0 69.4 5.5 0.0 82.1 (20.3) 0.0 (20.2) (69.4) 0.0 (4.3) 69.4 (14.7) (69.4) 82.1Utilization Threshold 2,400 Baseline Peak KW Usage 2,045 Projected KW Consumption 2040.7 2110.1 2095.4 2026.0 2108.1Projected Peak Utilization 85.0% 87.9% 87.3% 84.4% 87.8%

Weekly IncreaseWeekly DecreaseWeekly Change

Sample Table to Capture Power Events and Forecast UPS Utilization


Sample Graphics Based Reporting Format

(80.0)

(60.0)

(40.0)

(20.0)

0.0

20.0

40.0

60.0

80.0

100.0

7-Jul

14-Jul

21-Jul

28-Jul

4-Aug

KW

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Weekly Increase Weekly Decrease Projected Peak Utilization


Physical Capacity of UPS System in KW 2400 Utilization Threshold for UPS System 85%

Utilization Threshold of UPS System in KW 2040

Monthly Peak Utilization as of July 7 2045

Available KW -5

Project Name / Identifier Equipment Type Qty KW Install date Week of Status Total KWAvailable

KWBlade Farm Upgrade L20p G4 64 0.3 07/11/08 07/07/08 16.00 (21.00)Retire Development Regatta IBM 9119-590 -1 20.3 07/13/08 07/07/08 (20.30) (0.70)New Prod Storage Frame DMX3-24 2 34.7 07/16/08 07/14/08 69.40 (70.10)Retire Reporting Servers ML530 G2 -36 0.6 07/24/08 07/21/08 (20.20) (49.90)New OLAP Reporting Servers ProLiant ML530 8 0.7 07/25/08 07/21/08 5.52 (55.42)Retire Prod Storage Frames DMX3-24 -2 34.7 07/30/08 07/28/08 (69.40) 13.98 New zSeries Development Host IBM 2094-S38 1 12.7 08/06/08 08/04/08 12.67 1.31 New Devl Storage Frames DMX3-24 2 34.7 08/08/08 08/04/08 69.40 (68.09)

Sample Tabular Report to Track Available KW for UPS System

Summary

Primary goal of the paper was to share some lessons learned.

Power analytics are fairly straightforward. The data can be elusive and difficult to get.

The paper contains a table of power factors developed over time.

Date Updated Vendor Category Model Configuration Watts Source of Metric Notes / Comments

EMC Storage Frame DMX 4500 14,400 UPS Measure

Actual power usage varies by storage configuration - See SI staff for an estimate.

EMC Storage Frame DMX1000 325 SI Planning Docs Actual power usage varies by storage

EMC Storage Frame DMX2000 500 SI Planning Docs


EMC Storage Frame DMX3 (DMX1500) 20,000 UPS Measure


EMC Storage Frame DMX3-2462 TB / 98 GB 228 Disks 18,500

Secondary DC Measure - 2/28/08

A linear regression was done on DMX3 frames. Use 2,351 + 204 * TB of Stg to get Watts estimate.










5/7/2008 HP Windows Server DL360 174Data Captured for SCE Credits

HP Windows Server ProLiant DL740 G1 1,175 BDNA HP Windows Server ProLiant DL760 G1 1,555 UPS MeasureHP Windows Server ProLiant ML370 1,294 UPS MeasureHP Windows Server ProLiant ML530 690 UPS MeasureIBM Mainframe Server IBM 2094-S28 12,672 IBMIBM Mainframe Server IBM 2094-S38 12,672 IBMIBM Midrange Server 7040 Model 681 8,000 UPS MeasureIBM Midrange Server IBM 7013-590 19,200 BDNA IBM Midrange Server IBM 7017-S80 2,120 BDNA IBM Midrange Server IBM 7017-S85 2,120 BDNA IBM Midrange Server IBM 7026-6M1 650 UPS MeasureIBM Midrange Server IBM 7026-B80 460 BDNA IBM Midrange Server IBM 7026-H50 1,122 BDNA IBM Midrange Server IBM 7026-M80 1,122 BDNA IBM Midrange Server IBM 7028-6C4 783 BDNA

The End

Questions??? Article References

Power Monitoring 101 http://viewer.bitpipe.com/viewer/viewDocument.do?ac

cessId=10352520 Avoid Common Pitfalls when Calculating Data

Center Power Loads http://searchdatacenter.techtarget.com/tip/0,289483,si

d80_gci1363455_mem1,00.html?track=NL-455&ad=718537USCA&asrc=EM_NLN_8862833

forecasting data center power requirements: tips from the trenches frank bereznay moses chu ibm

Documents