a tour in optimal transport
TRANSCRIPT
![Page 1: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/1.jpg)
A TOUR IN OPTIMAL TRANSPORT Michiel Stock
@michielstockKERMIT
![Page 2: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/2.jpg)
![Page 3: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/3.jpg)
![Page 4: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/4.jpg)
DESSERTS MADE BY TINNE
![Page 5: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/5.jpg)
PORTIONS PER KERMIT MEMBER
![Page 6: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/6.jpg)
PREFERENCES FOR DESSERTS
![Page 7: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/7.jpg)
FORMAL PROBLEM DESCRIPTION
r: vector containing portions of dessert per person (general: n-dim.)
c: vector containing portions of each dessert (general: m-dim.)
M: a cost matrix (negative preference)
U(r, c) = {P 2 Rn⇥m>0 | P1m = r, P |1n = c}
Polyhedral set containing all valid partitions:
Solve the following problem:
dM (r, c) = minP2U(r,c)
X
i,j
PijMij
Minimizer is the optimal distribution!P ?ij
![Page 8: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/8.jpg)
d�M (r, c) = minP2U(r,c)
hP,MiF � 1
�h(P )
OPTIMAL TRANSPORT WITH ENTROPIC REGULARIZATION
Cost:
Entropic regularization:
Tuning parameter:
hP,MiF =X
i,j
PijMij
h(P ) = �X
i,j
Pij logPij
�
Constrain solution to possess a minimal ‘evenness’
![Page 9: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/9.jpg)
GEOMETRY OF THE OPTIMAL TRANSPORT PROBLEM
M
P ?
P ?�
rc|
U(r, c)
![Page 10: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/10.jpg)
DERIVATION OF THE SOLUTION
@L(P,�, {a1, . . . , an})@Pij
|P⇤ij= 0
Lagrangian of the problem:
Choose constants to satisfy constraints!P ?ij = e�ai�bj�1e��Mij
= ↵i�je��Mij
L(P,�, {a1, . . . , an}, {b1, . . . , bm}) =X
ij
PijMij +1
�
X
ij
Pij logPij
+
nX
i=1
ai(ri �X
j
Pij) +
mX
j=1
bj(cj �X
i
Pij)
@L(P,�, {a1, . . . , an})@Pij
= Mij +logPij
�+
1
�� ai � bj
![Page 11: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/11.jpg)
THE SINKHORN-KNOPP ALGORITHM
Init
Until convergedScale rowsScale columns
P = e��M
![Page 12: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/12.jpg)
SOLUTION (HIGH LAMBDA)
Solution is very good approximation of unregularized OT problem!
total average
preference:
36
![Page 13: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/13.jpg)
SOLUTION (LOW(ER) LAMBDA)
Every person has to try a bit of everything!
total average
preference:
29.6
![Page 14: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/14.jpg)
APPLICATIONS
➤ Matching distributions
➤ Interpolation
➤ Domain adaptation
➤ Color transfer
➤ Comparing distributions
➤ Modelling complex systems
![Page 15: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/15.jpg)
MATCHING AND INTERPOLATING DISTRIBUTIONS
![Page 16: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/16.jpg)
DOMAIN ADAPTATION WHEN DISTR, TRAIN AND TEST DIFFER
![Page 17: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/17.jpg)
IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS
![Page 18: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/18.jpg)
IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS
![Page 19: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/19.jpg)
COMPARING DISTRIBUTIONS (WITH METRIC/COST)
➤ Comparing two distributions with cost
➤ Comparing two sets of objects with pairwise similarity
No equal number of
bins required!
![Page 20: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/20.jpg)
d�M (r, c) = minP2U(r,c)
hP,MiF � 1
�h(P )
OPTIMAL TRANSPORT AS ENERGY MINIMISATION
OT can be seen as a physical system of interacting parts
Energy of the system
Physical constrains (i.e. mass balance)
Inverse temperature
Entropy of system
![Page 21: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/21.jpg)
Interacting systems with competition.
![Page 22: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/22.jpg)
COMPUTATIONAL FLUID DYNAMICS
Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv
![Page 23: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/23.jpg)
LEARNING EPIGENETIC LANDSCAPES
Reconstruction of developmental landscapes by optimal-transport analysis of single-cell gene expression sheds light on cellular reprogramming. doi: https://doi.org/10.1101/191056
![Page 24: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/24.jpg)
IN SUMMARY
➤ OT is a simple framework for thinking about distributions
➤ Powerful tool for modelling complex systems (constraints + competition)
➤ Efficient solvers: O(n^2) (when using entropic regularization)
![Page 25: A tour in optimal transport](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5a6d62967f8b9af2418b5485/html5/thumbnails/25.jpg)
REFERENCES
Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv
Courty, N., Flamary, R., Tuia, D. and Rakotomamonjy, A. (2016). Optimal transport for domain adaptation
Cuturi, M. (2013) Sinkhorn distances: lightspeed computation of optimal transportation distances