graph matching networks for learning the similarity of ...11-16... · - good at exact matches, not...
TRANSCRIPT
![Page 1: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/1.jpg)
Yujia Li, Chenjie Gu, Thomas Dullien*, Oriol Vinyals, Pushmeet Kohli
Graph Matching Networksfor Learning the Similarity of Graph Structured Objects
*
![Page 2: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/2.jpg)
Graph Matching Networks — Yujia Li
Graph structured data appear in many applications
Molecules Scene Graphs*
Image credit: *Johnson et al. Image Retrieval using Scene Graphs. **Brockschmidt et al. Generative Code Modeling with Graphs
Programs**Binaries
![Page 3: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/3.jpg)
Graph Matching Networks — Yujia Li
Graph structured data appear in many applications
Molecules Scene Graphs*
Image credit: *Johnson et al. Image Retrieval using Scene Graphs. **Brockschmidt et al. Generative Code Modeling with Graphs
Programs**Binaries
Drug Discovery Semantic Image Retrieval Code Search
Software Vulnerabilities
![Page 4: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/4.jpg)
Graph Matching Networks — Yujia Li
Finding similar graphs
Query Graph
Candidate Graphs
Nodes and edges can have attributes
The notion of “similarity” varies across problems
Reasoning about both the graph structure and the semantics
Graph structures vary a lot
![Page 5: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/5.jpg)
Graph Matching Networks — Yujia Li
The binary function similarity search problem
EXE
00000000: 7f45 4c46 0201 0100 .ELF....00000008: 0000 0000 0000 0000 ........00000010: 0300 3e00 0100 0000 ..>.....00000018: 4005 0000 0000 0000 @.......00000020: 4000 0000 0000 0000 @.......00000028: 7819 0000 0000 0000 x.......00000030: 0000 0000 4000 3800 [email protected]: 0900 4000 1e00 1d00 [email protected]: 0600 0000 0400 0000 ........00000048: 4000 0000 0000 0000 @.......00000050: 4000 0000 0000 0000 @.......
containsvulnerability?
![Page 6: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/6.jpg)
Graph Matching Networks — Yujia Li
The binary function similarity search problem
EXE
00000000: 7f45 4c46 0201 0100 .ELF....00000008: 0000 0000 0000 0000 ........00000010: 0300 3e00 0100 0000 ..>.....00000018: 4005 0000 0000 0000 @.......00000020: 4000 0000 0000 0000 @.......00000028: 7819 0000 0000 0000 x.......00000030: 0000 0000 4000 3800 [email protected]: 0900 4000 1e00 1d00 [email protected]: 0600 0000 0400 0000 ........00000048: 4000 0000 0000 0000 @.......00000050: 4000 0000 0000 0000 @.......
containsvulnerability?
binary analysis
000000000000064a <f>: 64a: 55 push %rbp 64b: 48 89 e5 mov %rsp,%rbp 64e: 89 7d fc mov %edi,-0x4(%rbp) 651: 83 7d fc 00 cmpl $0x0,-0x4(%rbp) 655: 7e 09 jle 660 <f+0x16> 657: 8b 45 fc mov -0x4(%rbp),%eax 65a: 0f af 45 fc imul -0x4(%rbp),%eax 65e: eb 06 jmp 666 <f+0x1c> 660: 8b 45 fc mov -0x4(%rbp),%eax 663: 83 c0 01 add $0x1,%eax 666: 5d pop %rbp 667: c3 retq
push %rbpmov %rsp,%rbpmov %edi,-0x4(%rbp)
mov -0x4(%rbp),%eaximul -0x4(%rbp),%eaxjmp 666 <f+0x1c>
mov -0x4(%rbp),%eaxadd $0x1,%eax
pop %rbpretq
cmpl $0x0,-0x4(%rbp)jle 660 <f+0x16>
graph sizes in our dataset: from 10 to 103
![Page 7: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/7.jpg)
Graph Matching Networks — Yujia Li
The binary function similarity search problem
EXE
00000000: 7f45 4c46 0201 0100 .ELF....00000008: 0000 0000 0000 0000 ........00000010: 0300 3e00 0100 0000 ..>.....00000018: 4005 0000 0000 0000 @.......00000020: 4000 0000 0000 0000 @.......00000028: 7819 0000 0000 0000 x.......00000030: 0000 0000 4000 3800 [email protected]: 0900 4000 1e00 1d00 [email protected]: 0600 0000 0400 0000 ........00000048: 4000 0000 0000 0000 @.......00000050: 4000 0000 0000 0000 @.......
containsvulnerability?
binary analysis
similar
search in a library of binarieswith known vulnerabilities
not similar
![Page 8: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/8.jpg)
Graph Matching Networks — Yujia Li
The binary function similarity search problem
EXE
00000000: 7f45 4c46 0201 0100 .ELF....00000008: 0000 0000 0000 0000 ........00000010: 0300 3e00 0100 0000 ..>.....00000018: 4005 0000 0000 0000 @.......00000020: 4000 0000 0000 0000 @.......00000028: 7819 0000 0000 0000 x.......00000030: 0000 0000 4000 3800 [email protected]: 0900 4000 1e00 1d00 [email protected]: 0600 0000 0400 0000 ........00000048: 4000 0000 0000 0000 @.......00000050: 4000 0000 0000 0000 @.......
containsvulnerability?
binary analysis
similar
search in a library of binarieswith known vulnerabilities
not similar
![Page 9: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/9.jpg)
Graph Matching Networks — Yujia Li
Most existing approaches
Mostly hand-engineered algorithms / heuristics with limited learning:
Graph hashes (graph → descriptor): widely used in security applications
- human-designed hash functions that encode graph structure- good at exact matches, not so good at estimating similarity
Graph kernels (pair of graphs → similarity): popular in various graph-level prediction tasks
- human-designed kernels as a measure of similarity between graphs- the design of kernels is important for performance
![Page 10: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/10.jpg)
Graph Matching Networks — Yujia Li
Different graph similarity estimation paradigms
Graph embedding
Graph → descriptor
Measure distance on descriptors
Fast hashing based retrieval
Graph matching
Compute distance jointly on the pair of graphs
More computation for better accuracy
![Page 11: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/11.jpg)
Graph Matching Networks — Yujia Li
Graph similarity learning
Learn a similarity (or distance) function
d( , ) → small
d( , ) → large
![Page 12: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/12.jpg)
Graph Matching Networks — Yujia Li
Graph similarity learning
Learn a similarity (or distance) function
d( , ) → small
d( , ) → large
Supervised learning on labeled pairs or triplets
t = +1 ⇒ G1, G2 similar ⇒ d(G1, G2) ↙
t = -1 ⇒ G1, G2 not similar ⇒ d(G1, G2) ↗
G1, G2 similar, G1, G3 not similar
⇒ d(G1, G2) ↙ d(G1, G3) ↗
![Page 13: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/13.jpg)
Graph Matching Networks — Yujia Li
Learning graph embeddings with Graph Neural Nets
d(G1, G2) = Euclidean/Hamming distance(embed(G1), embed(G2))
![Page 14: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/14.jpg)
Graph Matching Networks — Yujia Li
Learning graph embeddings with Graph Neural Nets
d(G1, G2) = Euclidean/Hamming distance(embed(G1), embed(G2))
Input Graph Message Passing Aggregate over Graph
embed( ) =
![Page 15: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/15.jpg)
Graph Matching Networks — Yujia Li
Graph embedding model details
Messages:
Node updates:Aggregation:
sum pooling, attention pooling etc.
![Page 16: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/16.jpg)
Graph Matching Networks — Yujia Li
Graph Matching Networks
h1, h2 = embed-and-match(G1, G2)
d(G1, G2) = Euclidean/Hamming distance(h1, h2)
![Page 17: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/17.jpg)
Graph Matching Networks — Yujia Li
Graph Matching Networks
h1, h2 = embed-and-match(G1, G2)
d(G1, G2) = Euclidean/Hamming distance(h1, h2)
Attention:
Weighted difference:
![Page 18: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/18.jpg)
Graph Matching Networks — Yujia Li
Graph Matching Networks
h1, h2 = embed-and-match(G1, G2)
d(G1, G2) = Euclidean/Hamming distance(h1, h2)
Total cross-graph message
Effectively: match node i to the closest node in the other graph and take the difference.
![Page 19: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/19.jpg)
Graph Matching Networks — Yujia Li
Other variants
Other variants of GNNs for embedding:
- e.g. Graph Convolutional Networks (GCNs), which is a simpler variant without modeling edge features
Siamese networks:
- instead of using Euclidean or Hamming distance, learn a distance score through a neural net
- d(G1, G2) = MLP(concat(embed(G1), embed(G2)))- learn the embedding model and the scoring MLP jointly
![Page 20: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/20.jpg)
Graph Matching Networks — Yujia Li
Graph Matching
Similarity score
Graph Embedding
Similarity score
Siamese Network
Similarity score
![Page 21: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/21.jpg)
Graph Matching Networks — Yujia Li
Experiments
Graph edit distance learning
Data: synthetic graphs
Similarity: small edit distance → similar
Control-flow graph based binary function similarity
search
Data:compile ffmpeg with
different compilers and optimization levels.
Similarity:binary functions associated
with the same original function → similar
Mesh graph retrieval
Data:mesh graphs for 100 object classes (COIL-DEL dataset)
Similarity:mesh for the same object
class → similar
![Page 22: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/22.jpg)
Graph Matching Networks — Yujia Li
Synthetic task: graph edit distance learning
Training and evaluating on graphs of size n, and edge density (probability) p
Measuring pair classification AUC / triplet prediction accuracy.
Learned models do better than WL kernel.
Matching model better than embedding model.
![Page 23: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/23.jpg)
Graph Matching Networks — Yujia Li
Results on binary function similarity search
Hand-engineered baseline (graph hashing + locality sensitive hashing) vs GNN embedding vs GMN.
Graph topology only vs jointly over structures and features.
![Page 24: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/24.jpg)
Graph Matching Networks — Yujia Li
Results on binary function similarity search
1) learned approaches better than hand-engineered solution2) matching better than embedding alone3) joint modeling of structure and features better than structure alone4) performance better with more graph propagation steps
![Page 25: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/25.jpg)
Graph Matching Networks — Yujia Li
More ablation studies
GMNs consistently better than alternatives.
Siamese vs matching: fusing two graphs early better than only at the end.
![Page 26: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/26.jpg)
Graph Matching Networks — Yujia Li
Learned attention patterns
We never supervise the cross-graph attention, but the model still learns some interesting attention patterns.
![Page 27: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/27.jpg)
Graph Matching Networks — Yujia Li
Learned attention patterns
When the two graphs are identical, the learned attention pattern may (not always) correspond to node matching.
After 10 messagepassing steps
Model trained on the edit distance learning task.
![Page 28: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/28.jpg)
Graph Matching Networks — Yujia Li
Learned attention patterns
Otherwise the attention pattern is less interpretable.
After 10 messagepassing steps
Model trained on the edit distance learning task.
![Page 29: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/29.jpg)
Graph Matching Networks — Yujia Li
Conclusions and future directions
Takeaways:
- graph similarity can be learned.- learned graph embedding models are good and efficient models for this.- graph matching networks are even better.
Future directions:
- make cross-graph attention and matching more efficient- explore new architectures that can utilize the new capability of learned
graph similarity
![Page 30: Graph Matching Networks for Learning the Similarity of ...11-16... · - good at exact matches, not so good at estimating similarity Graph kernels (pair of graphs → similarity):](https://reader034.vdocuments.mx/reader034/viewer/2022050404/5f8138f2722267104f70f798/html5/thumbnails/30.jpg)
Yujia Li, Chenjie Gu, Thomas Dullien*, Oriol Vinyals, Pushmeet Kohli
Graph Matching Networksfor Learning the Similarity of Graph Structured Objects
*