phoenix: a weight-based network coordinate system using matrix factorization
DESCRIPTION
Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization. Yang Chen Department of Computer Science Duke University [email protected]. Outline. Background System Design Evaluation Perspective Future Work. Background. Internet Distance. 50ms. Alice. Bob. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/1.jpg)
Phoenix: A Weight-Based Network Coordinate System
Using Matrix Factorization
Yang ChenDepartment of Computer Science
Duke [email protected]
![Page 2: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/2.jpg)
Outline
• Background• System Design• Evaluation• Perspective Future Work
2
![Page 3: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/3.jpg)
BACKGROUND
3
![Page 4: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/4.jpg)
Internet Distance
• Round-trip propagation / transmission delay between two Internet nodes
What?
• Strong indicator of network proximity• Relatively stable
Why?
• Measurement tool “Ping” is with major operating systems
How?
4
50ms
Alice Bob
![Page 5: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/5.jpg)
Use Cases
• Knowledge of Internet distance is useful for…– P2P content delivery (file sharing/streaming)– Online/mobile games– Overlay routing– Server selection in P2P/Cloud– Network monitoring
5
![Page 6: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/6.jpg)
Scalability
• Huge number of end-to-end paths in large scale systems
SLOW and COSTLY when the system becomes large!6
N nodes measurements
![Page 7: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/7.jpg)
Network Coordinate (NC) Systems
7
(5, 10, 2) (-3, 4, -2)
Distance Function
22ms
• Scalable measurement: N2 NK (K << N)• Every node is assigned with coordinates• Distance function: compute the distance between
two nodes without explicit measurement
AliceBob
[Ng et al, INFOCOM’02]
![Page 8: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/8.jpg)
Deployments
8
They are all using Network Coordinate Systems!
![Page 9: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/9.jpg)
Basic models
• Euclidean Distance-based NC (ENC)– Modeling the Internet as a Euclidean space– Systems: Vivaldi [Dabek et al., SIGCOMM’04], GNP [Ng et al,
INFOCOM’02], NPS [Ng et al., USENIX ATC’04], PIC [Costa et al.,
ICDCS’04]…• Matrix Factorization-based NC (MFNC)
– Factorizing an Internet distance matrix as the product of two smaller matrices
– Systems: IDES [Mao et al., JSAC’06], Phoenix, …
9
![Page 10: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/10.jpg)
Modeling the Internet as a Euclidean space
• In a d-dimensional Euclidean space, each node will be mapped to a position
• Compute distances based on coordinates using Euclidean distance
10
d=3
![Page 11: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/11.jpg)
Triangle Inequality Violation
Czech Republic
Slovakia
Hungary
5.6 ms
3.6 ms
29.9 ms
A Triangle Inequality Violation (TIV) example in GEANT network
29.9 > 5.6+3.6
11
Lots of TIVs in the Internetdue sub-optimal routing!!
Predicted distances in Euclidean space must
satisfy triangle inequality
[Zheng et al, PAM’05]
![Page 12: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/12.jpg)
Correlation in Internet Distance Matrices
Duke UNC Yale Aachen Oxford Toronto THU NUS
Duke - 3 24 107 122 37 219 252
UNC 3 - 24 106 109 38 219 253
12
Internet paths with nearby end nodes are often overlap!!
Rows in different Internet distance matrices are large correlated (low effective rank)[Tang et al, IMC’03], [Lim et al, ToN’05], [Liao et al, CoNEXT’11]
Distance measurement using PlanetLab nodes
![Page 13: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/13.jpg)
Factorization of an Internet Distance Matrix
13
N ro
ws
N columnsd columns
[Mao et al., JSAC’06]
![Page 14: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/14.jpg)
Matrix Factorization-Based NC
• Each node i has an outgoing vector Xi and an incoming vector Yi
• Distance function is the dot product.14
N ro
ws
N columnsd columns
No triangle inequality constrain in this model!
![Page 15: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/15.jpg)
SYSTEM DESIGN
15
![Page 16: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/16.jpg)
Goals
• Substantial improvement in prediction accuracy
• Decentralized and scalable• Robust to dynamic Internet
16
![Page 17: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/17.jpg)
Workflow of Phoenix
System Initialization
Peer Discovery
Scalable Measuremen
t
Coordinates Calculation
17
System Initialization
Peer Discovery
Scalable Measurement
Coordinates Calculation
![Page 18: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/18.jpg)
System Initialization
• Early nodes (N<K): Full-mesh measurement• Compute coordinates of early nodes by minimizing the overall discrepancy
between predicted distances and measured distances
18
Measured DistancePredicted Distance
(X1,Y1) (X2,Y2)
(X3,Y3) (X4,Y4)
Nonnegative matrix factorization: [D. D. Lee and H. S. Seung, Nature, 401(6755):788–791, 1999.]
![Page 19: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/19.jpg)
Dynamic Peer Discovery
19
Tracker
H2 H3 H5 H3 H4 H6
H2 H3 H4 H5 H6 H1 H3 H4 H5 H6
Gossip among nodes
• N>K, all nodes become ordinary nodes
![Page 20: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/20.jpg)
Reference Node Selection
20
• Every new node randomly selects K existing nodes as reference nodes
![Page 21: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/21.jpg)
Measurement and Bootstrap Coordinates Calculation
21
Measured DistancePredicted Distance
• Node Hnew computes its own coordinates by minimizing the overall discrepancy between predicted distances and measured distances (Non-negative least squares)
(X1,Y1)(XK,YK)(X2,Y2)
(Xnew,Ynew)
![Page 22: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/22.jpg)
Accuracy of Reference Coordinates
Node 1
Node 2
Node 3
…
Node N
0 20 40 60 80 100 120 140
Predicted DistanceMeasured distance
22
(XA,YA)
Distance between Node A and every other node
Node A
![Page 23: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/23.jpg)
Accuracy of Reference Coordinates (cont.)
Node 1
Node 2
Node 3
…
Node N
0 20 40 60 80 100 120
Predicted DistanceMeasured Distance
23Distance between Node B and every other node
(XB,YB)
Misleading the nodes referring to Node B!!
Node B
![Page 24: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/24.jpg)
Referring to Inaccurate Coordinates
24
(X1,Y1)(XK,YK)(X2,Y2)
(Xnew,Ynew)
Error Propagation: Hnew may mislead nodes refer to it
Minimize the impact
of RK
Give preference to accurate reference
coordinates
![Page 25: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/25.jpg)
Heuristic Weight Assignment
R1
R2
R3
…
RK
0 20 40 60 80 100 120 140 160
Predicted Distance
Measured distance
25
Bootstrap Coordinates
Distance between Hnew and every reference node
Enhanced Coordinates
Updating coordinates regularly
![Page 26: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/26.jpg)
EVALUATION
26
![Page 27: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/27.jpg)
Evaluation Setup
• Data sets– PL: 169 PlanetLab nodes– King: 1740 Internet DNS servers
• Metric– Relative Error (RE)
27
![Page 28: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/28.jpg)
Evaluation: Relative Error
28
90th Percentile Relative Error
Phoenix Phoenix (Simple)
Vivaldi IDES
0.63 0.91 0.83 0.89
![Page 29: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/29.jpg)
Evaluation (cont.)
• Other findings through evaluation– Robust to node churn– Fast convergence– Robust to measurement anomalies– Robust to distance variation
29
![Page 30: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/30.jpg)
FUTURE WORK
30
![Page 31: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/31.jpg)
Perspective Topics
• NC systems in mobile-centric environment– Access latency, host mobility, host churn
• Scalable Prediction of other important network parameters– Available bandwidth, shortest-path distance in
social graph
31
![Page 32: Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization](https://reader036.vdocuments.mx/reader036/viewer/2022062417/56815ed3550346895dcd6628/html5/thumbnails/32.jpg)
Software
• NCSim– Simulator of Decentralized Network
Coordinate Algorithms– http://code.google.com/p/ncsim/
• Phoenix– Original Phoenix simulator in IEEE TNSM
paper– http://www.cs.duke.edu/~ychen/Phoenix_TNS
M_2011.zip
32