Scaling Online Social Networks (OSNs)

Download Scaling Online Social Networks (OSNs)

Post on 31-May-2015

1.426 views

Category:

Technology

1 download

Embed Size (px)

DESCRIPTION

The final presentation of a semester project. Course: Implementation of Distributed Systems (KTH Royal Institute of Technology)

TRANSCRIPT

<ul><li> 1. Scaling Online Social Networks (OSNs)Presented by: Maria Stylianou Coworker: Anis Uddin Supervisor: arnas GirdzijauskasKTH - Royal Institute of TechnologyImplementation of Distributed Systems December 6th, 2012</li></ul> <p> 2. Outline Motivation Current Algorithms SPAR JA-BE-JA Contributions Challenges Solution Evaluation &amp; Conclusions 2 3. Outline Motivation Current Algorithms SPAR JA-BE-JA Contributions Challenges Solution Evaluation &amp; Conclusions 3 4. Pandoras box Online Social NetworksSource: http://technorati.com/social-media/article/social-networks-theyre-what-every-local/Motivation-Algorithms-Contribution-Evaluation 4 5. Easy to maintain... Online Social NetworksSource: http://mastersofmedia.hum.uva.nl/2009/09/14/a-review-of-taken-out-of-context/Motivation-Algorithms-Contribution-Evaluation 5 6. ...or not! Online Social NetworksSource: http://mastersofmedia.hum.uva.nl/2009/09/14/a-review-of-taken-out-of-context/Motivation-Algorithms-Contribution-Evaluation 6 7. Scaling ApproachesVertical Scaling Horizontal Scaling Full Replication Adding servers Data Locality Clean &amp; Disjoint Partitions But: But: Expensive Saturation Not applicable in OSNsMotivation-Algorithms-Contribution-Evaluation7 8. Scaling ApproachesVertical Scaling Horizontal Scaling Full Replication Adding servers Data Locality Clean &amp; Disjoint Partitions But: But: Expensive Saturation Not applicable in OSNs InefficientMotivation-Algorithms-Contribution-Evaluation8 9. Existing Solutions for OSNsRelational Databases Key-Value Stores Motivation-Algorithms-Contribution-Evaluation 9 10. Existing Solutions for OSNsRelational Databases Key-Value StoresInefficient Motivation-Algorithms-Contribution-Evaluation 10 11. Outline Motivation Current Algorithms SPAR JA-BE-JA Contributions Challenges Solutions Evaluation &amp; Conclusions 11 12. SPARSocial Partitioning &amp; Replication middle-ware Transparent OSN scalability avoids Data Locality performance Load Balancingbottlenecks Fault Tolerance Stability Replication Overhead MinimizationMotivation-Algorithms-Contribution-Evaluation 12 13. SPAREvents Nodes Add/Remove Edges Add/Remove Servers Add/Remove Motivation-Algorithms-Contribution-Evaluation 13 14. SPAR AlgorithmM2 6 5 1 Create Edge (1,6)253 14 5M1 6Master NodeReplica NodeM3Motivation-Algorithms-Contribution-Evaluation 14 15. SPAR AlgorithmM2 6 5Create Edge (1,6)251C1: Create 6 in M1Create 1 in M33 146 5M1 6Master Node 1Replica NodeM3Motivation-Algorithms-Contribution-Evaluation 15 16. SPAR Algorithm M265 Create Edge (1,6)2 1C2: Move 1 to M331452 Master NodeM16 1 3 Replica Node M3 4 Motivation-Algorithms-Contribution-Evaluation 16 17. SPAR AlgorithmM2 6 5Create Edge (1,6)251C3: Move 6 to M13 14 6M1Master NodeReplica NodeM3Motivation-Algorithms-Contribution-Evaluation 17 18. JA-BE-JA Distributed Partitioning Algorithm K-way Partitioning Load Balancing Gossip LearningMotivation-Algorithms-Contribution-Evaluation 18 19. JA-BE-JA - Policies Sampling Swapping Local Energy Function Select neighbors Reach minimum Random Simulated Annealing Select from random Escape from local walk optima Hybrid Local &amp; RandomSource: http://socialnetworking.lovetoknow.com/Growth_of_Online_Social_Networking_in_Business Motivation-Algorithms-Contribution-Evaluation19 20. Outline Motivation Current Algorithms SPAR JA-BE-JA Contributions Challenges Solution Evaluation &amp; Conclusions 20 21. Challenges Global ViewPartition Managerrequirement Single Pointof FailureSPARSPARReplicationOverhead Motivation-Algorithms-Contribution-Evaluation 21 22. Our SolutionGlobal ViewPartition Manager requirement Single Pointof FailureSPARLocal ViewDistributed &amp;Partition JA-BE-JAManager Replication OverheadMotivation-Algorithms-Contribution-Evaluation22 23. Our Solution (wait for it...) Client RequestsSPAR Data StoreServers Motivation-Algorithms-Contribution-Evaluation23 24. Our Solution Client RequestsSPAR&amp;JA-BE-JA JA BE JA Data StoreServers Motivation-Algorithms-Contribution-Evaluation24 25. Outline Motivation Current Algorithms SPAR JA-BE-JA Contributions Challenges Solution Evaluation &amp; Conclusions 25 26. Implementation SPAR SPAR-JA This is SPARJA!Motivation-Algorithms-Contribution-Evaluation 26 27. Datasets Facebook Graphsby Stanford Network Analysis Project #nodes: 150 #edges: ~3000 #nodes: 224 #edges: ~6000 #nodes: 786 #edges: ~60000Source: http://snap.stanford.edu/ Motivation-Algorithms-Contribution-Evaluation 27 28. Datasets Synthesized Graphs using our own Graph Generator Graph Visualization Tool #nodes: 1000, #degree: 10 https://gephi.org/Randomized Clustered Highly Clustered Motivation-Algorithms-Contribution-Evaluation28 29. ExperimentsReplication Overhead on Different Datasets#k-replicas: 0 (fault tolerance) #Servers: 4Synthesized Graphs10000 edgessynth-r: Randomizedsynth-c: Clusteredsynth-hc: Highly ClusteredFacebook Graphsfcbk-1: ~3000 edgesfcbk-2: ~6000 edgesfcbk-3: ~60000 edges Motivation-Algorithms-Contribution-Evaluation29 30. ExperimentsReplication Overhead vs Replication Factor K=0 K=2 Motivation-Algorithms-Contribution-Evaluation 30 31. ExperimentsReplication Overhead on both algorithmsFault ToleranceK=2synth-hc:- Highly Clustered- Synthesized Graph- 10000 edgesMotivation-Algorithms-Contribution-Evaluation 31 32. ExperimentsReplication Overhead on both algorithms Fault Tolerance K=2 fcbk-3: - 3rd facebook graph - 60,000 edgesMotivation-Algorithms-Contribution-Evaluation32 33. Conclusions SPAR + JA-BE-JA = SPAR-JA Highly clustered nodes Achieves fault tolerance by-default Better than SPAR in case of high clusterization Future Work More datasets Bigger datasets Motivation-Algorithms-Contribution-Evaluation 33 34. Scaling Online Social Networks (OSNs)Presented by: Maria Stylianou Coworker: Anis Uddin Supervisor: arnas GirdzijauskasKTH - Royal Institute of TechnologyImplementation of Distributed Systems December 6th, 2012</p>