Under YoUr Floor?

What’s Hiding

PLUS: Migration to 40/100 Gigabit,Outside Plant Documentation andPreparing for Gigabit Wi-Fi

BICSInewsM a g a z i n enovember/december 2013 l volume 34, number 6

Under YoUr Floor?

What’s Hiding

by Matthew P. O’Hare, RCDD, LEED AP

l o w -v o l t a g e d e s i g n i n t h e d a t a C e n t e r

If using a raised floor, the design of the pathway system for the low-voltage cable plant remains one of the most critical aspects of the sub-floor environment.

Electrical and mechanical systems are often considered the backbone of the data center and therefore carry a higher level of importance than other systems. The key design aspects for these systems include proper sizing for “day 1” and the future, redundancy and resiliency in a failure scenario and operational efficiency under normal conditions. However, there is a third critical component in data center design that is often overlooked during the early concept and planning stages. Designers and engineers who are experienced in low-voltage systems infrastructure realize the importance of the communications cable plant and what is required, operationally and physically, from this spider web of cable. This includes the primary and redundant layouts of pathways and cables which, when designed properly, enhances the reliability and functionality of the space. The inherent tradeoff when designing low-voltage systems is that a massive amount of cable must be routed throughout the plant which, if not done properly, can have a dramatic effect on the overall efficiency of the space.

the PhYsiCal Cable Plant

The applications and devices in today’s data centers are running faster than ever. In an effort to keep up with current and emerging technology, cable manufacturers have had to increase the performance of their cabling systems. Focusing on the physical aspect of the cable itself, one of the changes we have seen is an increased outside diameter (OD) on the cable due to an increase in insulation thickness, specific pair twisting configurations or the addition of separating members within the jacket. Another change implemented to enhance the performance of the cable has been to

increase the thickness of the copper wire. For example, category 5e cable ranged from 24 to 26 AWG, while category 6A ranges from 22 to 24 AWG. The type of cable used in the initial design of the low-voltage infrastructure is commonly based on current and anticipated technology growth. However, one needs a crystal ball to really see what the technology and cable requirements will be five or seven years down the road. It is therefore important to follow standards and best practices that allow for expansion within the pathways and spaces. Companies who choose to design the intra-data center cable plant often don’t take this future-proofing concept into consideration. In addition, sometimes the available initial investment capital may not provide for future accommodations. Many in the industry have experienced sites where the initial cable plant conveyance system was properly planned, but accommodations were not made for the cabling technology evolution. This can be a logistical nightmare for moves, adds and changes (MACs). Another serious failure in design is when a raised-floor system is used for both cable conveyance and a supply air plenum and the low-voltage cable plant is designed as an afterthought—or worse yet, designed after the commissioning of the mechanical plant. There is a common belief that cable can just be hidden under the raised floor; however, many fail to realize the choking effect this can have on the heating, ventilation and air conditioning (HVAC) systems.

the PathwaYsSome data center professionals

are of the mindset that the raised-access floor system is (or should be) obsolete and no longer required in a data center

&designdeployment

Matthew O’Hare, RCDD, LEED AP, is a team leader within the mission critical group at Executive Construction, Inc., where he focuses on the construction of the physical plant with a specialty in low-voltage systems and site commissioning. He can be reached at mohare@ecibuild.com.

l o w -v o l t a g e d e s i g n i n t h e d a t a C e n t e r

Another serious failure in

design is when a raised-

floor system is used for

both cable conveyance

and a supply air plenum

and the low-voltage cable

plant is designed as an

afterthought—or worse

yet, designed after the

commissioning of the

mechanical plant.

due to the containment systems that are now available. However, if using a raised floor, the design of the pathway system for the low-voltage cable plant remains one of the most critical aspects of the sub-floor environment. Typical low-voltage communica-tions pathways in the data center range from dedicated conduits (primarily for carrier or high-security use), innerduct/fabric-duct, basket tray or a six-sided cable trough, typically used in a plenum environment. When specifying which solution to use, it is very important to understand and follow the National Electric Code (NEC®), Article 645: Information Technology Equipment. This article states that should the data center space meet the minimum requirements, the use of non-plenum rated (e.g., communications riser [CMR]) cable under the raised floor is allowed. Non-plenum cable can potentially save 30 percent on the cost of the cable. The requirements in the NEC® Article 645 are as follows:n The data center space is protected by an emergency power off (EPO) system.n The data center space is served by a separate and dedicated HVAC system.n The data center space has smoke detection under the raised floor.n The data center space will house listed IT equipment.n The data center space will only be occupied by maintenance and operational staff to support the equipment.n The data center space is separated from other spaces by fire-rated construction.

The caveat to this is the reference in the NEC® article to the mechanical code, which clearly states to use plenum cable in any air plenum environment. Oftentimes, the use plenum or non-plenum cable is up

to the local authority having jurisdic-tion (AHJ). They typically approve the use of non-plenum cabling materials, but in some cities, plenum cable is required no matter what.

Fill ratio CalCUlations

With the approved application of Article 645, a basket tray would typically be the type of product used in new data center construction—either overhead or below the raised floor. The size of the tray and the fill ratios must be understood, especially below the floor. For example, consider a data center where a cabinet row consists of 30 cabinets with a perpendicular split aisle in the middle. Assume that each cabinet will receive 48 category 6A cables for running 10 gigabit Ethernet (GbE). Industry standards state that the allowable outside diameter (OD) for a category 6A cable is 9 millimeters (mm [0.354 inches (in)]). The latest generations of category 6A cable have an OD of approximately 7.6 mm (0.30 in), which is being used in this example. Per standard recommendations, the basket tray

should initially be filled 25 percent of the way, allowing another 25 percent for expansion. This expansion requires some basic area and fill calculations. Finding the usable area of the basket tray is based on the following calculation: A(u) = (L x W) x 25%

Figuring a 508 mm x 150 mm (20 in x 6 in) tray, the usable area ends up to be 30 square inches (in2). Now we need to find the cross sectional area of the cable using the formula. We first need to determine the radius (r), which is equal to half of the diameter. Therefore, in a 7.6 mm (0.30 in) cable, the radius would be 3.6 mm (0.15 in). Now we can find the cross sectional area of a cable using the following calculation: A= π x 0.152

Based on the calculation, the cross sectional area of a single 7.6 mm (0.30 in) category 6A cable is 0.706 in2. We can then determine the quantity of cables by dividing the usable area of the basket tray by the area of the cable. In our example,

In this underfloor basket tray infrastructure designed and installed for expansion, note the perforated air supply tiles on the opposite side of the tray. Tray systems should not be directly mounted (cantilevered) to raised floor pedestals to avoid deflection and reduced

capacity of the floor system because it can possibly void the flooring warranty. Photo courtesy of Executive Construction, Inc.

the usable area of 30 in2 is divided by the cross sectional area of the cable at 0.0706 in2, resulting in a total quantity of 424 category 6A cables for the initial installation in a 508 mm x 150 mm (20 in x 6 in) cable tray. Referring back to our example of 30 cabinets in a row with a perpendicular split aisle in the middle and 48 cables per cabinet, we would need to accommodate 720 cables on either side of the row. That would require installing one level of 508 mm x 150 mm (20 in x 6 in) tray and one level of 400 mm x 150 mm (16 in x 6 in) tray below the floor perpendicular to the center aisle on both sides. Imagine if these cables were routed along the center aisle to a central core row. We would need four levels of tray down the center aisle to accommodate the 1440 cables to that one row of 30 cabinets. That is precisely why most data center designs call for interconnect cabinets at the end or middle of the row that connect the row back to the core via optical fiber, eliminating having massive amounts of copper cable traversing throughout the data center.

biM and CFd Modeling

With the aforementioned example, one can quickly realize how the low-voltage cable plant can severely impact the HVAC systems and the overall design of the sub-floor environment. The mechanical engineer will incorporate the information provided regarding the cable plant to help determine how much volume of space is required to maintain proper static pressure under the raised floor. Placement of the basket tray is one key factor to the subfloor design. Typically, the tray should be routed down the hot aisle or underneath the cabinets. If placed in the cold aisle, the cable plant will act as an air dam,

preventing cool air from reaching the intake side of the equipment. The same air blocking effect can happen if the traversing aisles routing to the core row are clogged with several layers of basket tray. To help alleviate this problem, the design of the basket tray pathway system should resemble a grid that allows the cables to traverse up and down aisles and route around the immediate perimeter of the cabinet rows to help keep the vertical profile down to a minimum. Building information modeling (BIM) is a useful tool used to design the pathways and visualize the impacts on other aspects of the data center physical plant. The electronic tools used in BIM vary based on the different software suites available, but the premise and outcome are typically the same. BIM creates a 3D electronic drawing that can be viewed 360 degrees (i.e., X, Y and Z planes). The advanced technology construction industry dictates that a 3D BIM model be created for all aspects of the design (e.g., architectural, electrical, mechanical, fire protection/suppression, low-voltage systems) of the data center. This allows for identifying conflicts between the

different systems, above and below the raised floor, prior to construction. When these electronic modeling layers are brought together, they will help verify constructability and coordinate the design. Most often, the cost of creating the BIM model is recouped by the money saved from reducing repeat work and change orders due to conflicts in the field. Having a fully coordinated BIM model will also speed up the raised-floor construction process, allowing fabrication to commence and supports to be placed earlier in the construction schedule. Often times, the mechanical, electrical, plumbing/fire protection (MEP/FP) contractors can save time and money by using the fully coordinated BIM model as spool sheets and fabricating right from the model, rather than waiting to verify the layouts in the field. Another tool that should be used when designing the data center is computational fluid dynamics (CFD) modeling. The CFD model provides an analysis of hot and cold air flow through the data center whitespace, identifying hot and cold spots at varying elevations based on dictated system loads. With the proper input

Underfloor view of the fully coordinated 3D BIM model. Photo courtesy of Ascent, LLC.

from designers and engineers, the airflow model will also show where adjustments need to be made to the systems under the floor and overhead to maximize cabinet cooling efficiency. It could also show where cabinet rows should start or where perforated tile adjustments should be made to avoid the vacuum effect of high-velocity air moving under perforated tiles (most notably closest to the air supply) in a raised-floor air plenum environment. When using CFD modeling, it is important to provide as much information about the data center plant as possible to gain the highest accuracy for the model. Oftentimes, the tile cuts within the cabinets are overlooked when preparing the model. Though they typically have brush closures to prevent air from escaping, they do allow a small amount to pass through. This adds up quickly when you are designing a 10,000 ft2 data center with 300 cabinets that each has two brushed cable openings.

site seleCtionThe information gathered

about the low-voltage plant will also assist with the general construction design for either an upgrade or new construction. Ceiling-to-deck heights should be maximized to prevent the systems installed below the raised floor from restricting the HVAC systems. With large, multitier basket tray plants, it is recommended that the branch electrical infrastructure be installed overhead. Installing it below the raised floor adds another layer of air blockage to the mechanical systems. The area above the cabinets will also have supplementary support systems (e.g., lighting, fire alarm, fire sprinklers/suppression, containment). For example, a four-tier basket tray system typically requires a 1220 mm (48 in) high raised floor to maintain proper airflow. If you accommodate high-density 52 RU cabinets that are 2.4 meters (m [8 ft]) high, allow 1.2 m (4 ft) in elevation for the systems above the cabinets and another 1.83 m (6 ft) above the ceiling for other infrastructure and hot air return. In this scenario,

the total slab-to-deck height would be 6.7 m (22 ft). If there is no “ceiling” required, the slab to deck can be reduced; however, it is still important to leave enough headroom for air return and other previously mentioned systems.

ConClUsionThe low-voltage cable

plant designer—whether the client’s or the design firm’s BICSI Registered Communications Distribution Designer (RCDD®)—should be one of the first involved stakeholders in planning and designing the data center. Remember, BIM and CFD are keys to implementing a successful construction project when it comes to the efficiency of the data center space. Sadly, the cable plant is often one of the last systems considered, resulting in lost efficiency and money. In today’s world of high-speed data and advancing technology, systems are growing and changing at a lightning pace. Proper planning structured around industry standards will provide maximum flexibility when looking toward the future. n

CFD modeling depicts sectional air flow analysis in a hot aisle containment scenario. Conduit routing and brushed openings were included in the model. Photo courtesy of Ascent, LLC.