NeuGraph:Parallel Deep Neural Network Computation on Large Graphs

Lingxiao Ma†, Zhi Yang†, Youshan Miao ‡, Jilong Xue ‡, Ming Wu ‡, Lidong Zhou ‡, Yafei Dai†

† Peking University

‡ Microsoft Research

USENIX ATC’19 Lightning Talk, Renton, WA, USA

NeuGraph: Parallel Deep Neural Network Computation on Large Graphs 2

Image Object Detection Speech Recognition

Self-Driving Personal Assistant

User-Item Graph Knowledge Graph

Recommendation Question Answering

Neural Networks

Input Feature Vector

Input: Graph Data

Graph Neural Networks

* Figures from Internet

NeuGraph: Parallel Deep Neural Network Computation on Large Graphs

Graph Neural Networks (GNN)

3

- Information propagation via Graph

- Information transformation via Neural Networks

VertexNN Transformation

EdgeNN Transformation

NeuGraph: Parallel Deep Neural Network Computation on Large Graphs

Challenges in Processing GNNs

4

large, sparse, and irregular

Scalability ProblemEfficiency Problem

small and regular

Graphs

* Figures from Internet

NeuGraph: Parallel Deep Neural Network Computation on Large Graphs

NeuGraph- Bridge graph and dataflow models to support efficient and scalable GNN processing

5

Kernel Optimization

Streaming Optimization

Dataflow Generation

SAGA-NN Abstraction Programming GNN apps

Handle Scalability

Handle Efficiency

Compatibility

W

E0,0V0 V1

V0

SAG SAG

A0 matmul ReLU

ApplyVertex

E1,0weights params.

Existing Dataflow System

NeuGraph: Parallel Deep Neural Network Computation on Large Graphs

NeuGraph- Bridge graph and dataflow models to support efficient and scalable GNN processing

6

- Performance- Outperform state-of-the-art frameworks (e.g., TensorFlow and DGL) on small graphs

- Scale to large real-world graphs with GPUs

2019 USENIX Annual Technical ConferenceTrack II: Graph Processing Frameworks

11:55 AM-12:15 PM, on July 11th

NeuGraph