A S HR A E J O U R NA L as hr ae. or gA U G U ST 2 0 1 5 6 8COLUMNDATA CENTERSDonald L. Beaty, P.E., is president and David Quirk, P.E., is vice president of DLB Associates Consulting Engineers, in Eatontown, N.J. Beaty is publications chair and Quirk is the chair of ASHRAE TC 9.9. BY DONALD L. BEATY, P.E., FELLOW ASHRAE; DAVI D QUI RK, P.E., MEMBER ASHRAEDriver for Energy Savings: Carrot or Stick?In the past decade, signicant strides have been made in realizing energy savings in data centers. The question is whether these strides are the product of the carrot or the stick. Further, is it better to use the carrot or the stick within your data center to achieve your energy-efciency improvements and goals? It is important to begin by looking at the history behind and drivers for more efcient data centers. The governance of efciency improvements in data centers is not a simple one. Not all methods of governance are created equal, nor will they produce the desired results and/or change in human behavior. Data centers are facilities driven by complicated behavioral motives that often run in opposition to energy improvements.Many governance approaches to saving energy in data centers exist; some are better than others, depending on the type of data center business model. Roughly, they can all be grouped into the following categories, each of which will be expanded upon in this column: Awareness programs (TC 9.9 & The Green Grid); Economic drivers and nancial incentive programs; Corporate PR or recognition programs; Technology improvement programs; and Legislative, regulatory, codes & standards programs.This column will examine the history of data center energy-efciency governance and whether the carrot or the stick is driving the right behavior and subsequent energy improvements.BackgroundThe gasoline shortages in the early 1970s have gener-ally been credited as being the original driver toward energy efciency. This was followed by increasing prices and an economic motive for energy conservation. Of course, this was all happening before the rst PC or rst server was invented. A second signicant global driver was the realization that CO2 levels in the atmosphere were correlated with a global warming trend. International efforts to con-trol greenhouse gas emissions started with the United Nations Framework Convention on Climate Change in 1992 and continued with the Kyoto Protocol in 1997. By this time, PCs were a part of most households, and the Internet, along with data centers, had started to grow extremely rapidly. Data center experts, concerned about the increased aggregate energy use of data cen-ters, started meeting in the late 1990s, and many of these players were instrumental in starting ASHRAEs TG9.HDEC, High Density Electronic Equipment, in 2002, which ultimately became TC 9.9, Mission Critical Facilities, Data Centers, Technology Spaces and Electronic Equipment, in 2004.By the beginning of the millennium, the trend toward more efcient data centers was clearly underway.Data Center Trade-OffsEnergy savings in a data center does not come natu-rally. Many objectives and behavioral dynamics compete with energy objectives. Some of those key challenges can be summarized as follows: Rapidly changing technology (challenges to regulat-ing due to a mismatch in life cycles; months or single years for data center technology changes vs. decades for codes and standards). The mission in mission critical facilities results in a lot of costly provisioning of redundant components, equipment, infrastructure pathways, controls strategies, ThisarticlewaspublishedinASHRAEJournal,August2015.Copyright2015ASHRAE.Postedatwww.ashrae.org.Thisarticlemaynotbecopiedand/or distributed electronically or in paper form without permission of ASHRAE. For more information about ASHRAE Journal, visit www.ashrae.org.A U G U ST 2 0 1 5 as hr ae. or gA S HR A E J O U R NA L 6 9COLUMNDATA CENTERSetc., to ensure the facility is resilient to almost all forms of failures/outages. Reliable operations are paramount (failures are not an option, which leads to behavior-driven actions to avoid any forms of escalations and eliminate all un-knowns). An example helps clarify the challenges associ-ated with driving energy efciency in data centers. Imagine your role is the operations manager of a data center and the following typical scenario occurs. Temperatures are monitored 24/7/365 and are alarmed with escalations to your boss, your bosss boss, and so on. At 3 a.m. Saturday morning on a holiday, one CRAH/CRAC unit fails and the lag unit doesnt start, resulting in a high temperature that gets alarmed; the escalations begin. Questions ensue from your manage-ment on how this could happen and how to prevent it from happening again.This scenario plays out all the time in data centers everywhere and drives behavioral approaches to HVAC operations that are not necessarily in line with energy savings. Imagine a similar scenario in which the chillers failed to restart after the switchover from economizer mode. Another would be the freezing of a cooling tower used in economizer mode for a water-cooled chiller plant. The human response is to disable the economizer opera-tion to avoid future escalations and the responsible individual(s) potentially having to nd another job. This is the reality of data center operations, and this reality must be taken into account when determining what form of governance to select to drive energy sav-ings within a given data center company and facility. Awareness ProgramsAwareness programs have included, but are not lim-ited to, The Green Grids publications, The EPAs Report to Congress, and ASHRAE TC 9.9s publications.The formation of The Green Grid in 2007 brought an era of awareness of energy efciency in data centers through the development of simple metrics. It created and published the power usage effectiveness (PUETM) metric, as discussed in Data Center Energy Metric in the January 2013 ASHRAE Journal. Today, the PUE is per-haps the most widely recognized energy-efciency met-ric for comparing the energy efciency of data centers worldwide. In an EPA report to Congress in 2007 (PL 109-431), PUE had already gained common acceptance, and examina-tion of a sufcient number of data centers indicated a downward trend in PUE (from 1.95 in the 2003 sample to 1.63 in the 2005 sample). By this time, drivers were certainly in place to encour-age increased energy efciency through awareness programs. An update to this EPA report is due by the end of this year and it will be interesting to see how the PUE trend continues.ASHRAE TC 9.9 has been instrumental in dissemi-nating and publishing information that can be used to implement improved data center energy efciency. One enabler of increased energy efciency, the relax-ation of temperature and humidity environmental conditions, has been published in TC 9.9s Thermal Guidelines for Data Processing Environments, currently in its 3rd edition (ASHRAE 2012), with a 4th edition scheduled for publication by the end of 2015. Two TC 9.9 books were specically written with energy efciency in mind. These are Best Practices for Datacom Facility Energy Efciency (2nd Edition, 2009) and Green Tips for Data Centers (2010). Also, the primary metric used to assess data center infrastructure efciency is described in detail in the book PUE: A Comprehensive Examination of the Metric (ASHRAE and The Green Grid, 2013). A book on measuring energy consumption in datacom facilities, Real-Time Energy Consumption Measurements in Data Centers, was published by ASHRAE in 2010. Finally, a publication that examines metrics for a subset of information technology equipment (ITE), computers used for storage, has just been pub-lished: Server Efciency: Metrics for Computer Servers and Storage (ASHRAE 2015).Awareness programs are extremely agile and can more readily keep pace with the rapid changes in the data center industry. Typically these programs spur energy-efciency efforts as a function of their broad reach and acceptance and, therefore, set the stage for either the carrot or the stick to be implemented on a company-by-company basis.Legislative, Regulatory, Codes & Standards ProgramsThe primary legislative driver for data centers in the U.S., as with most buildings, is the International Code Council (ICC) codes. The International Energy A S HR A E J O U R NA L as hr ae. or gA U G U ST 2 0 1 5 7 0Conservation Code (IECC) is the version that has its roots developed from and references to ASHRAE Standard 90.1. These codes are adopted by the states. The rst edition of ASHRAE Standard 90.1 (Standard 90-1975, Energy Conservation in New Building Design) was published 40 years ago. Today, Standard 90.1 undergoes con-tinuous updates, with major new published editions occurring on a three-year cycle. Standard 90.1 excluded process loads (which includes data centers) until Standard 90.1-2010 was published. Prior to then, the building shell and ofce areas were covered under Standard 90.1, but the cooling pro-cess for the computer equipment was considered a process load and, thus, exempt from the main provi-sions of the standard.With the inclusion of data centers under the Standard 90.1 umbrella, certain provisions of the standard, such as the requirement for econo-mizers for data center cooling, are now included in the energy code as requirements under the prescriptive path.By using the stick, not the carrot, these programs get results through mandatory and prescriptive force. They typically involve a much longer cycle for development and imple-mentation. As a result, they often run in conict with a rapidly chang-ing industry like data centers. Economic Drivers and Financial Incentive ProgramsEconomic drivers are the most traditional driver of energy ef-ciency, and few energy-efciency decisions are typically made without at least some analysis of the trade-off between capital and operating costs. Governmental entities (and utilities) with a vested interest in lowering energy consumption often facilitate energy efciency through nancial incentives. At the federal level, this has typi-cally taken the form of a federal tax credit. At the state level, incentive programs have been organized that include both prescriptive incentives for equipment with efciency levels that exceed target levels (such as the currently adopted energy code), and more custom applications. The custom applications are most typically used for retrots of existing facilities. In these cases, the baseline is often the as-is efciency of the data center, and energy calculations can be performed to determine the change in energy efciency from the existing condition. The custom incentive can then be paid based on the difference between the exist-ing and the post-implementation energy use. Colocation data centers tend to adopt these programs much more than other types of data centers, such as those under the U.S. federal government. In short, certain busi-ness models are more highly incen-tivized to use these programs for data center energy savings.These programs use the carrot approach. In doing so, they are often successful at gaining momen-tum and adoption because the bosses are the ones signing up for the programs. This in turn changes their behavior and approach if a subsequent failure occurs downstream.Corporate PR or Recognition Programs Corporations whose mission includes environmental stewardship have several options they can take to decrease the environmental impact of building and operating buildings, and to obtain public recognition for their efforts. The U.S. Green Building Council (USGBC) introduced LEED for Data Centers as part of LEED Version 3 in 2009, and many data centers have received LEED ratings through this program. Data center own-ers and operators can also use the U.S. Department of Energys (DOE) Energy Star Program to demonstrate low energy use. The U.S. Environmental Protection Agency (EPA) Portfolio Manager added the data center building type in 2010. Data centers that score COLUMNDATA CENTERSAdvertisement formerly in this space.A U G U ST 2 0 1 5 as hr ae. or gA S HR A E J O U R NA L 7 1in the top 25% in terms of energy efciency can earn the Energy Star rating. The EPA also has Energy Star ratings for servers, which encour-ages manufacturers to design ef-cient IT equipment, as well as opera-tors to buy them to minimize energy consumption.A major component of President Barack Obamas Climate Action Plan includes the DOEs Better Buildings Challenge, whose goals include reducing data center energy consumption. By joining the program, companies commit to reduce their data center energy use by at least 20% over the next 10 years. The DOE estimated that if all data centers in the U.S. became 20% more energy efcient, the countrys energy consumption as a whole would go down by more than 20 billion kWh by 2020. About 200 organizations have joined the Better Buildings Challenge thus far. According to the DOE, they have already completed upgrades to more than 9,000 facilities.Like the incentive programs, these programs use the carrot approach with similar adoption and accep-tance at the management level.Technology Improvement Programs Another enabler of more ef-cient data centers is an impressive increase in energy-efcient data center cooling products. Data center cooling technology is one of the most rapidly improving technologies in the building environment.Over the past 15 years, chillers and other air-conditioning equipment have become more efcient. Many companies have come out with products, many with innovative Advertisement formerly in this space.A S HR A E J O U R NA L as hr ae. or gA U G U ST 2 0 1 5 7 2economizer cycles (such as indirect adiabatic cooling) that have very low annualized cooling PUEs. These technologies are being increasingly adopted by industry, especially in new construction. New technologies on the electrical distribution infrastructure have included dc power distribution and eco-mode uninterruptible power supplies.In addition to collaborators such as ASHRAE and The Green Grid, work is underway by industry/university cooperative research center programs (I/UCRC), partly funded by the National Science Foundation. One of these pro-grams (I/UCRC ES2, Energy-Smart Electronic Systems), is actively looking at technologies to save energy. This is in addition to the work done by corporate research departments.The acceleration and adoption of these programs are often dependent upon select industry companies (with the right business models and nancial capabilities), national labs, or other academic or government entities to prove or demonstrate these new technologies. Still to ComeLooking forward, the efciency of ITE equipment and the infrastruc-ture that serves this equipment will continue to increase, and ASHRAE TC 9.9 will continue to play a major role as a driver in this shift. A new development is a new energy standard focused solely on data centers. The intent is to replace the prescriptive-based requirements currently in ASHRAE Standard 90.1-2010 and 2013 editions. This new proposed standard, ASHRAE Standard 90.4P, Energy Standard for Data Centers and Telecommunications Buildings, has undergone one public review as of this writing, with a second public review slated for summer 2015. Standard 90.4P is primarily a performance-based standard. To support a performance-based stan-dard, accurate energy models are needed. Due to the many customized types of infrastructure equipment found in data centers, ASHRAE TC 9.9 is also, in conjunction with other committees, proposing the develop-ment of additional energy modeling research through an ASHRAE Work Statement (WS). Some of the concepts recom-mended in the WS, such as a server model that can adjust both airow and power as a function of inlet air temperature, have already been modeled by the U.S. DOE and incor-porated into EnergyPlus in the past year. Other concepts will be added to Energy Modeling Toolkits once this research moves forward.Final ThoughtsA myriad of drivers in the data center and ITE industry continue to create more efcient data centers, mostly through the use of carrot programs. Some programs work better for driving actual energy improvements for certain types of data center business models than others (e.g. incentive for coloca-tion). Combinations of programs have helped to further drive improvements.The industry is committed to reducing energy consumption, while at the same time retaining the very high level of system reliability that is demanded for these facili-ties. However, no single program is appropriate for all cases.To date, the industry energy needle has been moved signi-cantly by the carrots, not the sticks. Awareness, incentive, recognition, and technology advancement pro-grams have made a signicant dif-ference in the industrys focus since the late 1990s. These programs have enabled agil-ity to the industrys rapidly changing landscape. They have also enabled the necessary adjustments in the mind-set and the associated work ows to modify human behaviors that have long since worked in oppo-sition to energy savings. COLUMNDATA CENTERSAdvertisement formerly in this space.