hacking the conventional data center infrastructure
TRANSCRIPT
![Page 1: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/1.jpg)
Hacking the Conventional Data Center Infrastructure
Presented by: Kevin Brown
Confidential Property of Schneider Electric
![Page 2: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/2.jpg)
Commonly Stated Benefits of Open Compute
• Simplification of servers
• Consolidation of server power supplies
• Eliminate centralized UPS and move to the rack with Li-ion
• Less downtime
• Less cost
Page 2 Confidential Property of Schneider Electric |
![Page 3: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/3.jpg)
Critical questions arise about what is upstream of the rack…
• If I adopt Open Compute, what does my power architecture look like?
• Can I get 2N (i.e. tier 3) redundancy levels like I have today?
• How do I support traditional and Open Compute IT loads in the same data center?
Page 3 Confidential Property of Schneider Electric |
? Answering these questions will broaden adoption…
![Page 4: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/4.jpg)
Important to understand the cost trade-offs
• Analyzed capital cost (material cost only)
• Compared traditional and Open Compute specific architectures
• From MV switchgear down to (and including) IT power supply
• Key assumptions:
• OCP PSU/BBU/shelf costs based on design by Schneider Electric
• Traditional server PSU costs & sizing based on various IT vendors/suppliers
• Models based on 9.6MW data center, 10 kW/rack
• Costs normalized to $/Watt
Based on the content of white paper 228, Analysis of Data Center Architectures Supporting Open Compute Project (OCP)
Page 4 Confidential Property of Schneider Electric |
![Page 5: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/5.jpg)
Traditional vs. OCP-based designs
$2.77/watt
$1.53/watt
$2.08/watt
Traditional 2N
Open Compute Specific 1N
Open Compute Specific 2N
We think most will want to maintain 2N (Tier 3) redundancy
“Tiers” in a nutshell
Tier 1: Single power path to IT load; single points of failure Tier 3: Dual (redundant) power paths to IT load, concurrently maintainable
![Page 6: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/6.jpg)
Page 6 Confidential Property of Schneider Electric |
Traditional 2N power architecture today… Typical 2N design for traditional IT loads
• 2N power paths from utility to load, Tier 3(ish)
• Adds complexities (“Belts & suspenders”):
– Load bank
– Ties
– Additional UPS output breakers
• Concurrently maintainable
![Page 7: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/7.jpg)
Page 7 Confidential Property of Schneider Electric |
Open Compute Specific 1N architecture An example of a cost-reduced architecture to support OCP loads
• Aligns with the simplicity and cost-reduction mindset of OCP
• Open Compute servers with one PSU path
• Single path to the IT load
– with rack-based battery backup
– no centralized UPS
• Minimal breaker count
![Page 8: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/8.jpg)
Traditional 2N vs. Open Compute Specific 1N
Page 8 Confidential Property of Schneider Electric |
0.87
0.22 0.15 0.34 1.53
0.03
2.77
Traditional 2N
Redundancy Rack system Open Compute
Specific 1N
Power supplies
Batteries UPS
45% capex savings… but where do these savings come from? $
/ wat
t
![Page 9: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/9.jpg)
Traditional 2N vs. Open Compute Specific 1N
Page 9 Confidential Property of Schneider Electric |
0.87
0.22 0.15 0.34 1.53
0.03
2.77
Traditional 2N
Open Compute
Specific 1N
Power supplies
Batteries UPS Rack system Redundancy
31% savings is from redundancy differences… $
/ wat
t
![Page 10: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/10.jpg)
Traditional 2N vs. Open Compute Specific 1N
Page 10 Confidential Property of Schneider Electric |
0.87
0.22 0.15 0.34 1.53
0.03
2.77
Open Compute
Specific 1N
Power supplies
Batteries UPS Rack system
Redundancy Traditional 2N
1% cost adder for rack system $
/ wat
t
![Page 11: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/11.jpg)
0.87
0.22 0.15 0.34 1.53
0.03
2.77
Redundancy Traditional 2N
Batteries UPS Rack system
Open Compute
Specific 1N
Power supplies
Traditional 2N vs. Open Compute Specific 1N
Page 11 Confidential Property of Schneider Electric |
8% savings from eliminating upstream UPS…
$ / w
att
![Page 12: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/12.jpg)
Traditional 2N vs. Open Compute Specific 1N
Page 12 Confidential Property of Schneider Electric |
0.87
0.22 0.15 0.34 1.53
0.03
2.77
UPS Rack system
Redundancy Traditional 2N
Open Compute
Specific 1N
Power supplies
Batteries
5% cost adder for Li-ion batteries
$ / w
att
![Page 13: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/13.jpg)
Traditional 2N vs. Open Compute Specific 1N
Page 13 Confidential Property of Schneider Electric |
0.87
0.22 0.15 0.34 1.53
0.03
2.77
Open Compute
Specific 1N
Power supplies
Batteries UPS Rack system
Redundancy Traditional 2N
12% savings from power supplies
$ / w
att
![Page 14: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/14.jpg)
• Redundancy • Concurrent maintainability
2N (or tier 3) is still important to many data centers
Page 14 Confidential Property of Schneider Electric |
![Page 15: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/15.jpg)
Utility A
OC
32A
400A
400A
3200A
RPP
1250 A
1250 A
Utility B
32APod level
Rack level
400A
400A
3200A
RPP
1250 A
1250 A
PDU
ServerPSUBBU
ServerPSUBBU
PDU
PSU
PSU
1600A
Cooling
1600A
Cooling
MV level
LV level
G G G G G
630 A
G G GG
Open Compute Specific 2N architecture 2N simple design to support OCP IT loads
Page 15 Confidential Property of Schneider Electric |
reduces complexity by eliminating unnecessary cross-ties, additional breakers
saves cost with one battery path, one raw utility path
trusts the redundant server power supplies
Compared to traditional 2N, this architecture….
![Page 16: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/16.jpg)
Cost difference of Traditional 2N vs. Open Compute Specific 2N
Page 16 Confidential Property of Schneider Electric |
2.080.22
0.01
2.77
0.160.22
0.40
Open Compute
Specific 2N
Power supplies
Batteries UPS Rack system
Architecture simplification
Traditional 2N
$ / w
att
25% capex savings…
![Page 17: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/17.jpg)
Cost difference of Traditional 2N vs. Open Compute Specific 2N
Page 17 Confidential Property of Schneider Electric |
2.080.22
0.01
2.77
0.160.22
0.40
Power supplies
Open Compute
Specific 2N
Batteries UPS Rack system
Architecture simplification
Traditional 2N
$ / w
att A simplified 2N design costs $2.37/watt… (that accounts for more than half of the 25% savings)
![Page 18: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/18.jpg)
Cost difference of Traditional 2N vs. Open Compute Specific 2N
Page 18 Confidential Property of Schneider Electric |
2.080.22
0.01
2.77
0.150.22
0.40
Open Compute
Specific 2N
Power supplies
Batteries UPS Rack system
Architecture simplification
Traditional 2N
Rack, UPS, battery, and PSU differences represent the remaining…
$ / w
att
![Page 19: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/19.jpg)
What if I have a mix of Open Compute and traditional IT loads?
Page 19 Confidential Property of Schneider Electric |
![Page 20: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/20.jpg)
$2.77
$1.53 $2.08 $2.15
0 to 0.23
Traditional 2N
Open Compute
Specific 1N
Open Compute
Specific 2N
Simplified 2N
Premium for traditional power supplies (PSUs) depends on mix of loads
A design that accommodates both types of loads
Assumes $0.46 traditional PSU cost per watt of IT load $
/ wat
t
![Page 21: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/21.jpg)
Page 21 Confidential Property of Schneider Electric |
• Flexible architecture that allows for mix of traditional IT loads and OCP loads
• UPS is upstream to support both traditional and OCP loads
• To minimize cost, one power path with UPS, one without
• OCP loads have dual PSUs without BBUs
Example of a Simplified 2N design that accommodates both types of loads
![Page 22: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/22.jpg)
$2.08 $2.08 $2.08
$0.07 $0.07 $0.07
$0.12 $0.23
100% OCP loads
50% OCP loads
0% OCP loads
9% premium
Premium for traditional PSUs*
Impact of load mix on cost premium
Premium for simplified architecture that accommodates both load types
Cost of Open Compute Specific 2N design
* Assumes $0.46 traditional PSU cost per watt of IT load
3% premium
15% premium
$ / w
att
![Page 23: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/23.jpg)
PSU cost is a highly sensitive variable...
Page 23 Confidential Property of Schneider Electric |
$0.077 per PSU
watt
2 PSUs per
server X
3 PSU
oversizing X =
$0.46 per IT load
watt
Our baseline assumption for traditional PSUs:
Sensitivity analysis:
We varied the PSU oversizing down to 2 and up to 4.5, or $0.32 to $0.70 / IT load watt
![Page 24: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/24.jpg)
$2.08 $2.08 $2.08
$0.07 $0.07 $0.07
$0.04 to
$0.25
$0.09 to
$0.47
100% OCP loads
50% OCP loads
0% OCP loads
Premium varies significantly as traditional PSU cost changes
Sensitivity analysis
3% premium
8% to 26% premium
5% to 15% premium
We assume a more right-sized PSU for OCP • 12 kW of PSU watts x 2 PSUs = 24 kW of PSU for
10kW of IT load
• PSU oversizing = 1.2
$ / w
att
![Page 25: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/25.jpg)
Freely available resources to help with planning decisions
Page 25 Confidential Property of Schneider Electric |
Reference Designs • Designs to support OCP • One-line diagrams, bill of materials, layout drawings • www.schneider-electric.com/datacenterdesigns
TradeOff Tools • OCP vs. traditional cost comparison tool • Li-ion vs. VRLA TCO tool • www.tools.apc.com
White Papers • WP228, Analysis of Data Center Architectures
Supporting OCP Designs • WP229, Battery Technology for Data Centers:
VRLA vs. Li-Ion • www.whitepapers.apc.com
![Page 26: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/26.jpg)
Key take-aways…
1. To increase adoption, critical questions about power infrastructure must be addressed
2. Today’s traditional 2N architectures have opportunity for simplification and cost reduction
3. Simplified 2N represents a small premium (3%) over OCP-specific 2N, and gives flexibility for mixed-loads
Page 26 Confidential Property of Schneider Electric |
![Page 27: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/27.jpg)
Page 27 Confidential Property of Schneider Electric |
Lithium-ion Batteries
![Page 28: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/28.jpg)
Benefits of lithium-ion batteries over VRLA batteries for UPS applications
• Higher energy density (70-260 kWh/kg vs. 15-50 kWh/kg)
• Double the life span (10-15 years vs. 4-6 years)
• 1/3rd smaller footprint
• Weighs 2/3rds less
• Faster re-charge time (1/2–1 hour vs. 6-12 hours)
• Improved cycle life (# of charge/discharge cycles) - >1000 cycles vs. <400 cycles
• Specified to operate at higher temperatures (40C) without impact on cycle life
Page 28 Confidential Property of Schneider Electric |
![Page 29: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/29.jpg)
Li-ion batteries and Open Compute
• Li-ion technology has enabled the rack-based approach
• BUT… important to separate the discussion of battery type from where the battery goes…
Page 29 Confidential Property of Schneider Electric |
Where energy storage resides determines benefits:
• Centralized storage (upstream of rack)
– Flexibility to support mix of OCP and traditional IT
– Isolation of battery problems from IT equipment
– Frees up U space in the IT rack
– Minimize stranded capacity of batteries
– Less batteries to manage
– Filtering and transient protection from upstream UPS
• Decentralized storage (at the rack)
– Fault isolation
– Incremental deployment
– Maintainability
![Page 30: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/30.jpg)
Questions?
Page 30 Confidential Property of Schneider Electric |
![Page 31: Hacking the Conventional Data Center Infrastructure](https://reader033.vdocuments.mx/reader033/viewer/2022042907/58cf0ccb1a28ab5f2b8b5d9d/html5/thumbnails/31.jpg)