hermes: dynamic partitioning for distributed social ... ?· hermes: dynamic partitioning for...

Download Hermes: Dynamic Partitioning for Distributed Social ... ?· Hermes: Dynamic Partitioning for Distributed…

Post on 23-Jul-2018




0 download

Embed Size (px)


  • Hermes: Dynamic Partitioning for DistributedSocial Network Graph Databases

    Daniel NicoaraUniversity of Waterloo

    Shahin KamaliUniversity of Waterloo

    Khuzaima DaudjeeUniversity of Waterloo

    Lei ChenHKUST

    ABSTRACTSocial networks are large graphs that require multiple graphdatabase servers to store and manage them. Each databaseserver hosts a graph partition with the objectives of bal-ancing server loads, reducing remote traversals (edge-cuts),and adapting the partitioning to changes in the structureof the graph in the face of changing workloads. To achievethese objectives, a dynamic repartitioning algorithm is re-quired to modify an existing partitioning to maintain goodquality partitions while not imposing a significant overheadto the system. In this paper, we introduce a lightweightrepartitioner, which dynamically modifies a partitioning us-ing a small amount of resources. In contrast to the exist-ing repartitioning algorithms, our lightweight repartitioneris ecient, making it suitable for use in a real system. Weintegrated our lightweight repartitioner into Hermes, whichwe designed as an extension of the open source Neo4j graphdatabase system, to support workloads over partitioned graphdata distributed over multiple servers. Using real-worldsocial network data, we show that Hermes leverages thelightweight repartitioner to maintain high quality partitionsand provides a 2 to 3 times performance improvement overthe de-facto standard random hash-based partitioning.

    1. INTRODUCTIONLarge scale graphs, in particular social networks, perme-

    ate our lives. The scale of these networks, often in millionsof vertices or more, means that it is often infeasible to store,query and manage them on a single graph database server.Thus, there is a need to partition, or shard, the graph acrossmultiple database servers, allowing the load and concurrentprocessing to be distributed over these servers to providegood performance and increase availability. Social networksexhibit a high degree of correlation for accesses of certaingroups of records, for example through frictionless sharing[15]. Also, these networks have a heavy-tailed distributionfor popularity of vertices. To achieve a good partitioningwhich improves the overall performance, the following ob-

    c 2015, Copyright is with the authors. Published in Proc. 18th Inter-national Conference on Extending Database Technology (EDBT), March23-27, 2015, Brussels, Belgium: ISBN 978-3-89318-067-7, on Distribution of this paper is permitted under the terms of theCreative Commons license CC-by-nc-nd 4.0.

    jectives need to be met:

    The partitioning should be balanced. Each vertex of thegraph has a weight that indicates the popularity of thevertex (e.g., in terms of the frequency of queries to thatvertex). In social networks, a small number of users (e.g.,celebrities, politicians) are extremely popular while a largenumber of users are much less popular. This discrepancyreveals the importance of achieving a balanced partitioningin which all partitions have almost equal aggregate weightdefined as the total weight of vertices in the partition.

    The partitioning should minimize the number of edge-cuts.An edge-cut is defined by an edge connecting vertices intwo dierent partitions and involves queries that need totransition from a partition on one server to a partitionon another server. This results in shifting local traversalto remote traversal, thereby incurring significant networklatency. In social networks, it is critical to minimize edge-cuts since most operations are done on the node that rep-resents a user and its immediate neighbors. Since these 1-hop traversal operations are so prevalent in these networks,minimizing edge-cuts is analogous to keeping communitiesintact. This leads to highly local queries similar to thosein SPAR [27] and minimizes the network load, allowing forbetter scalability by reducing network IO.

    The partitioning should be incremental. Social networksare dynamic in the sense that users and their relationsare always changing, e.g., a new user might be added, twousers might get connected, or an ordinary user might be-come popular. Although the changes in the social graphcan be much slower when compared to the read trac [8],a good partitioning solution should dynamically adapt itspartitioning to these changes. Considering the size of thegraph, it is infeasible to create a partitioning from scratch;hence, a repartitioning solution, a repartitioner, is neededto improve on an existing partitioning. This usually in-volves migrating some vertices from one partition to an-other.

    The repartitioning algorithm should perform well in termsof time and memory requirements. To achieve this e-ciency, it is desirable to perform repartitioning locally byaccessing a small amount of information about the struc-ture of the graph. From a practical point of view, thisrequirement is critical and prevents us from applying ex-isting approaches, e.g., [18, 30, 31, 6] for the repartitioningproblem.

    The focus of this paper is on the design and provision ofa practical partitioned social graph data management sys-tem that can support remote traversals while providing an

    25 10.5441/002/edbt.2015.04

  • eective method to dynamically repartition the graph usingonly local views. The distributed partitioning aims to co-locate vertices of the graph on-the-fly so as to satisfy theabove requirements. The fundamental contribution of thispaper is a dynamic partitioning algorithm, referred to aslightweight repartitioner, that can identify which parts ofgraph data can benefit from co-location. The algorithm aimsto incrementally improve an existing partitioning by decreas-ing edge-cuts while maintaining almost balanced partitions.The main advantage of the algorithm is that it relies on onlya small amount of knowledge on the graph structure referredto as auxiliary data. Since the auxiliary data is small andeasy to update, our repartitioning algorithm is performantin terms of time and memory while maintaining high-qualitypartitionings in terms of edge-cut and load balance.

    We built Hermes as an extension of the Neo4j1 open sourcegraph database system by incorporating into it our algo-rithm to provide the functionality to move data on-the-flyto achieve data locality and reduce the cost of remote traver-sals for graph data. Our experimental evaluation of Hermesusing real-world social network graphs shows that our tech-niques are eective in producing performance gains and workalmost as well as the popular Metis partitioning algorithms[18, 30, 6] that performs static oine partitioning by relyingon a global view of the graph.

    The rest of the paper is structured as follows. Section 2describes the problem addressed in the paper and reviewsclassical approaches and their shortcomings. Section 3 in-troduces and analyzes the lightweight repartitioner. Section4 presents an overview of the Hermes system. Section 5presents performance evaluation of the system. Section 6covers related work, and Section 7 concludes the paper.

    2. PROBLEM DEFINITIONIn this section we formally define the partitioning problem

    and review some of the related results. In what follows, theterm graph refers to an undirected graph with weights onvertices.

    2.1 Graph PartitioningIn the classical (, )-graph partitioning problem [20], the

    goal is to partition a given graph into vertex-disjoint sub-graphs. The weight of a partition is the total weight of ver-tices in that partition. In a valid solution, the weight of eachpartition is at most a factor 1 away from the averageweight of partitions. More precisely, for a partition P of agraph G, we need to have !(P )

    Pv2V (G)

    !(v)/. Here,

    !(P ) and !(v) denote the weight of a partition P and vertexv, respectively. Parameter is called the imbalance load fac-tor and defines how imbalanced the partitions are allowedto be. Practically, is in range [1, 2]. Here, = 1 impliesthat partitions are required to be completely balanced (allhave the same aggregate weights), while = 2 allows theweight of one partition to be up to twice the average weightof all partitions. The goal of the minimization problem is toachieve a valid solution in which the number of edge-cuts isminimized.The partitioning problem is NP-hard [13]. Moreover, there

    is no approximation algorithm with a constant approxima-

    1Neo4j is being used by customers such as Adobe and HP[3].

    tion ratio unless P=NP [7]. Hence, it is not possible to intro-duce algorithms which provide worst-case guarantees on thequality of solutions, and it makes more sense to study thetypical behavior of algorithms. Consequently, the problemis mostly approached through heuristics [20] [12] which areaimed to improve the average-case performance. Regardless,the time complexity of these heuristics (n3) which makesthem unsuitable in practice.

    To improve the time complexity, a class of multi-level al-gorithms were introduced. In each level of these algorithms,the input graph is coarsened to a representative graph ofsmaller size; when the representative graph is small enough,a partitioning algorithm like that of Kernighan-Lin [20] isapplied to it, and the resulting partitions are mapped back(uncoarsened) to the original graph. Many algorithms fit inthis general framework of multi-level algorithms; a widelyused example is the family of Metis algorithms [19, 30, 6].The multi-level algorithms are global in the sense that theyneed to know the whole structure of the graph in the coars-ening phase, and the coarsened graph in each stage shouldbe stored for the uncoarsening stage. This problem is par-


View more >