eedc intelligent placement of datacenters
TRANSCRIPT
![Page 1: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/1.jpg)
Execution Environments for Distributed Computing
Intelligent Placement of Datacenters for Internet
Services
EEDC
343
30
Master in Computer Architecture, Networks and Systems - CANS
Homework number: 6
Members: Roger Rafanell [email protected]
![Page 2: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/2.jpg)
2
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
2
![Page 3: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/3.jpg)
3
Introduction: Datacenter construction costs
Each datacenter costs >$100M to construct– The smaller datacenters are rated at ~25MW
Examples:– Microsoft DCs in Virginia & Chicago: $500M each
3
![Page 4: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/4.jpg)
4
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
4
![Page 5: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/5.jpg)
5
Motivation
Internet services require thousands of servers
Use multiple “mirror” datacenters– High availability and fault tolerance
– Low response time
Spend millions building and operating datacenters
Consume enormous amounts of brown energy!!
![Page 6: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/6.jpg)
6
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
6
![Page 7: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/7.jpg)
7
Intelligent placement of datacenters
Goal: Manage the monetary and environmental costs
Define framework Model costs and datacenter characteristics Create solution approaches Collect cost and location-related data Create placement tool
7
![Page 8: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/8.jpg)
8
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
8
![Page 9: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/9.jpg)
9
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
9
![Page 10: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/10.jpg)
10
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
OPEX costs– Maintenance and administration
– Electricity and water prices (average PUE)
10
![Page 11: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/11.jpg)
11
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
OPEX costs– Maintenance and administration
– Electricity and water prices (average PUE)
Incentives (taxes)
11
![Page 12: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/12.jpg)
12
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
OPEX costs– Maintenance and administration
– Electricity and water prices (average PUE)
Incentives (taxes)
12
![Page 13: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/13.jpg)
13
The problem formulation
Goal– Minimize CAPEX and OPEX
Constraints– Response times < MAX LATENCY for all users– Min consistency delay between 2 DCs < MAX DELAY– Min system availability > MIN AVAILABILITY
Output– Number of servers at each location– Minimum cost
13
![Page 14: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/14.jpg)
14
Solving the (non-linear) problem
Linear Programming– Does not support non-linear costs
Brute force– Too slow
Simple heuristics– May not produce accurate results efficiently
14
![Page 15: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/15.jpg)
15
Our approach for solving the problem
Evaluate each potential solution– Quickly via Linear Programming (LP)
Consider neighboring configurations– Simulated annealing (SA)
Cost optimization process– Combine SA and LP
15
Current solution Near neighbor
LP
SA
LP
![Page 16: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/16.jpg)
16
Our approach for solving the problem
16
LP
SA
LP
LP
SA
LP
SA
$13.8M/month
$9.2M/month $10.7M/month
$10.3M/month
![Page 17: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/17.jpg)
17
Summary of our approach
1) Generate a grid of tentative locations
2) Collect data about each location
3) Define datacenter characteristics
4) Instantiate optimization problem
5) Solve optimization problem
17
![Page 18: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/18.jpg)
18
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
18
![Page 19: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/19.jpg)
19
Comparing locations for 60k-server DC
0100020003000400050006000700080009000
Austin Bismarck Los Angeles
New York Orlando Seattle St. LouisCost
(tho
usan
d do
llars
per
mon
th)
Servers Land Building Connection Energy Water Staff Networking
19
![Page 20: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/20.jpg)
20
Interesting questions
How much does…… lower latency cost?
… higher availability cost?
… faster consistency cost?
… a green DC network cost?
… a chiller-less DC network cost?
20
![Page 21: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/21.jpg)
21
Cost of 60k-server green DC network
21
Green DC network costs $100k/month more, except when latency <70ms
![Page 22: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/22.jpg)
22
Cost of a 60k-server chiller-less DC network
0
2
4
6
8
10
12
14
30 50 70 90 110
Cost
(in
mill
ion
dolla
rs)
Maximum latency (milliseconds)
Chiller-less
Traditional
22
Chiller-less DC network is cheaper but it cannot achieve low latencies
![Page 23: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/23.jpg)
23
Conclusions
First scientific work on smart datacenter placement– Proposed framework and optimization problem
– Proposed solution approach
– Characterized many locations across the US
– Built a tool to automate the process
– Answered many interesting questions
Results show that smart placement can save millions Work enables smaller companies to reap the benefits
23
![Page 24: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/24.jpg)
24
Future work
To extend with data from Europe
Include tax incentives
Test the tool with data from real services
24
![Page 25: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/25.jpg)
25
Questions
![Page 26: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/26.jpg)
26
Specially thanks to the real authors of the work …
Íñigo Goiri, Kien Le, Jordi Guitart,Jordi Torres, and Ricardo Bianchini
26
![Page 27: EEDC Intelligent Placement of Datacenters](https://reader031.vdocuments.mx/reader031/viewer/2022013011/55814b5ed8b42ab9548b47b0/html5/thumbnails/27.jpg)
27
Energy costs and carbon emissions
Company #ServersEnergy/year
(MWh)Energy
cost/yearCO2/year
(Metric tons)
eBay 16K 0.6 x 105 $3.7M 0.4 x 105
Akamai 40K 1.7 x 105 $10M 1.0 x 105
Rackspace 50K 2 x 105 $12M 1.2 x 105
Microsoft >200K >6 x 105 >$36M >3.6 x 105
Google >500K >6.3 x 105 >$38M >3.8 x 105
Sources: [Qureshi’09], EPA
27