intelligent placement of datacenters for internet services Íñigo goiri, kien le, jordi guitart,...
Post on 22-Dec-2015
220 views
TRANSCRIPT
![Page 1: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/1.jpg)
Intelligent Placement of Datacenters for Internet Services
Íñigo Goiri, Kien Le, Jordi Guitart,Jordi Torres, and Ricardo Bianchini
1
![Page 2: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/2.jpg)
Motivation
• Internet services require thousands of servers• Use multiple “mirror” datacenters
– High availability and fault tolerance– Low response time
• Spend millions building and operating datacenters• Consume enormous amounts of brown energy
2
![Page 3: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/3.jpg)
Datacenter construction costs
• Each datacenter costs >$100M to construct– The smaller datacenters are rated at ~25MW
• Examples:– Microsoft DCs in Virginia & Chicago: $500M each
3
![Page 4: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/4.jpg)
Energy costs and carbon emissions
Company #Servers Energy/year (MWh)
Energy cost/year
CO2/year (Metric tons)
eBay 16K 0.6 x 105 $3.7M 0.4 x 105
Akamai 40K 1.7 x 105 $10M 1.0 x 105
Rackspace 50K 2 x 105 $12M 1.2 x 105
Microsoft >200K >6 x 105 >$36M >3.6 x 105
Google >500K >6.3 x 105 >$38M >3.8 x 105
Sources: [Qureshi’09], EPA
4
![Page 5: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/5.jpg)
Intelligent Placement of Datacenters
Goal: Manage the monetary and environmental costs
• Define framework• Model costs and datacenter characteristics• Define optimization problem• Create solution approaches
• Collect cost and location-related data• Create placement tool
5
![Page 6: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/6.jpg)
Outline
• Motivation• Placing datacenters• Evaluation• Conclusion
6
![Page 7: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/7.jpg)
Selecting datacenter locations
• Model datacenter placement– Network latencies– Availability
7
![Page 8: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/8.jpg)
Selecting datacenter locations
• Model datacenter placement– Network latencies– Availability
• CAPEX costs– Distance to electricity and networking infrastructure– Land and construction (maximum PUE)– Power delivery, cooling, backup equipment– Servers and networking equipment
8
![Page 9: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/9.jpg)
Selecting datacenter locations
• Model datacenter placement– Network latencies– Availability
• CAPEX costs– Distance to electricity and networking infrastructure– Land and construction (maximum PUE)– Power delivery, cooling, backup equipment– Servers and networking equipment
• OPEX costs– Maintenance and administration– Electricity and water prices (average PUE)
9
![Page 10: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/10.jpg)
Selecting datacenter locations
• Model datacenter placement– Network latencies– Availability
• CAPEX costs– Distance to electricity and networking infrastructure– Land and construction (maximum PUE)– Power delivery, cooling, backup equipment– Servers and networking equipment
• OPEX costs– Maintenance and administration– Electricity and water prices (average PUE)
• Incentives (taxes)
10
![Page 11: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/11.jpg)
Selecting datacenter locations
• Model datacenter placement– Network latencies– Availability
• CAPEX costs– Distance to electricity and networking infrastructure– Land and construction (maximum PUE)– Power delivery, cooling, backup equipment– Servers and networking equipment
• OPEX costs– Maintenance and administration– Electricity and water prices (average PUE)
• Incentives (taxes)
11
![Page 12: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/12.jpg)
Formulating the problem• Goal
– Minimize CAPEX and OPEX
• Constraints– Response times < MAX LATENCY for all users– Min consistency delay between 2 DCs < MAX DELAY– Min system availability > MIN AVAILABILITY
• Output– Number of servers at each location– Minimum cost
12
![Page 13: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/13.jpg)
Solving the (non-linear) problem
• Linear Programming– Does not support non-linear costs
• Brute force– Too slow
• Simple heuristics– May not produce accurate results efficiently
13
![Page 14: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/14.jpg)
Our approach for solving the problem
• Evaluate each potential solution– Quickly via Linear Programming (LP)
• Consider neighboring configurations– Simulated annealing (SA)
• Cost optimization process– Combine SA and LP
14Current solution Near neighbor
LP
SA
LP
![Page 15: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/15.jpg)
Our approach for solving the problem
15
LP
SA
LP
LP
SA
LP
SA
$13.8M/month
$9.2M/month $10.7M/month
$10.3M/month
![Page 16: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/16.jpg)
Summary of our approach
• Generate a grid of tentative locations• Collect data about each location• Define datacenter characteristics• Instantiate optimization problem• Solve optimization problem
16
![Page 17: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/17.jpg)
Tool demo
• We built a tool that– Embodies the problem– Input data for the US– Multiple solution approaches
Short video at:http://www.darklab.rutgers.edu/DCL/dcl.html
17
![Page 18: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/18.jpg)
Outline
• Motivation• Placing datacenters• Evaluation• Conclusion
18
![Page 19: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/19.jpg)
Comparing locations for60k-server DC
0100020003000400050006000700080009000
Austin Bismarck Los Angeles
New York Orlando Seattle St. LouisCost
(tho
usan
d do
llars
per
mon
th)
Servers Land Building Connection Energy Water Staff Networking
19
![Page 20: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/20.jpg)
Interesting questions
• How much does…… lower latency cost?… higher availability cost?… faster consistency cost?… a green DC network cost?… a chiller-less DC network cost?
20
![Page 21: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/21.jpg)
Cost of 60k-servergreen DC network
21Green DC network costs $100k/month more, except when latency <70ms
![Page 22: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/22.jpg)
Cost of a 60k-serverchiller-less DC network
0
2
4
6
8
10
12
14
30 50 70 90 110
Cost
(in
mill
ion
dolla
rs)
Maximum latency (milliseconds)
Chiller-less
Traditional
22Chiller-less DC network is cheaper but it cannot achieve low latencies
![Page 23: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/23.jpg)
Conclusions
• First scientific work on smart datacenter placement– Proposed framework and optimization problem– Proposed solution approach– Characterized many locations across the US– Built a tool to automate the process– Answered many interesting questions
• Results show that smart placement can save millions• Work enables smaller companies to reap the benefits
23
![Page 24: Intelligent Placement of Datacenters for Internet Services Íñigo Goiri, Kien Le, Jordi Guitart, Jordi Torres, and Ricardo Bianchini 1](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649d785503460f94a5a4fd/html5/thumbnails/24.jpg)
Intelligent Placement of Datacenters for Internet Services
Íñigo Goiri, Kien Le, Jordi Guitart,Jordi Torres, and Ricardo Bianchini
24