![Page 1: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/1.jpg)
Swarat ChaudhuriRice University
Biplab DekaRakesh Kumar
UIUC
Principled Approximations in Probabilistic Programming
![Page 2: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/2.jpg)
The computing stack (approximation)
AlgorithmsCompiler and runtime
Architecture
The APPROX view: with probabilitiesand approximations!
![Page 3: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/3.jpg)
The computing stack
AlgorithmsCompiler and runtimeArchitecture
The APPROX view: with probabilitiesand approximations!
Probabilistic program
Program synthesis
Hardware for sampling
![Page 4: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/4.jpg)
DPMM Clustering
• In each iteration, update the cluster assignments of data points one at a time
• For each data point, compute the probability of it belonging to all existing clusters and an unseen cluster
• Sample from this probability distribution
Gaussian Dirichlet Process Mixture Model (DPMM)
Gibbs Sampling Algorithm
Cluster 3 Cluster 2
Cluster 1
Cluster 2
Cluster 1
Cluster 2
Cluster 1
New Cluster
Model
![Page 5: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/5.jpg)
Sampling in Hardware
Prefix Sum Comparators
PRNG
Encoder
CLK
Prob
abili
ties
Sam
ple
32b 35b 1b
3b
32b
Idea: exploit errors!
![Page 6: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/6.jpg)
Robustness to hardware faultsStuck-at Faults Transient Faults
Sampler
Clustering using
DPMM
Deka, Biplab. “On Fault Tolerance of Hardware Samplers”. Masters Thesis, University of Illinois at Urbana Champaign, 2014
![Page 7: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/7.jpg)
Approximating compilers
Traditional compiler:• Input: Deterministic program• Goal: Executable semantically equivalent to source • Method: Syntax-guided translation
Approximating compiler:• Input: Probabilistic program• Goal:
– Satisfy basic boolean invariants– Minimize quantitative error
• Method: Program synthesis
![Page 8: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/8.jpg)
Probabilistic programs
heightMan = Gaussian(177,64);heightWoman = Gaussian(164,64);assume(heightWoman > heightMan);return heightMan
Source: Tutorial on Infer.NET by John Winn and Tom Minka
Addition: assertions, angelic nondeterminism
More complex example: clustering
![Page 9: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/9.jpg)
Probabilistic programming ++
• Random variables X: range over distributions
• Deterministic variables y– Either holes or temporaries
• Functions f(X1,..., Xk, y1,..., yk)– Can map random variables to deterministic ones
• Expectation, probability
• Assertions • Pareto-optimality goals
![Page 10: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/10.jpg)
Example
X = Gaussian(??, 10);assume (X < 10);c = Pr(X > 0);assert (c > 0.7);minimize (c);
One synthesis algorithm in [CCS14]Based on probabilistic abstract interpretation
Hole
[CCS14] Chaudhuri, Clochard, Solar-Lezama. Bridging boolean and quantitative synthesis using smoothed proof search. POPL 2014.
![Page 11: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/11.jpg)
Use in approximation
Holes = degree of approximation
Assertions = invariants, hard bounds
Declarative error minimization
Deterministic temporaries track resource
consumption
Synthesis = Compilation
![Page 12: Principled Approximations in Probabilistic Programming](https://reader036.vdocuments.mx/reader036/viewer/2022083006/568139d9550346895da18ea1/html5/thumbnails/12.jpg)
Thank you!
Questions?