fuzzi: a three level logic for differential privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11....
TRANSCRIPT
![Page 1: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/1.jpg)
Fuzzi: A Three Level Logic for Differential Privacy
Hengchu Zhang, Edo Roth, Andreas Haeberlen,Benjamin C. Pierce, Aaron Roth
University of Pennsylvania
![Page 2: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/2.jpg)
Differential Privacy is Useful
2
![Page 3: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/3.jpg)
Differential Privacy is Useful
is close to
3
![Page 4: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/4.jpg)
Privacy Parameters
Parameter 𝜺 bounds the multiplicative difference in probability
4
![Page 5: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/5.jpg)
c := c1; c2 | if e then c1 else c2 | while e do c end | x = e | x[e1] = e2 | x $= laplace(e, width)
5
Differential Privacy in an imperative programming
language?
![Page 6: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/6.jpg)
Fuzzi and its Three Levels
Advanced Probabilistic Couplings for Differential Privacy
Barthe et al. 2016.
manualproofs
apRHL
Type System
abstraction
automation
Type System for differentially private, imperative programs
6
Base LogicLanguage Semantics
![Page 7: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/7.jpg)
partition
An Example Fuzzi Programpartition(income, income_groups, ...);
t_part = income_groups[0];bsum(t_part, low_income_sum, 1000.0);low_income_sum += laplace(1000.0);
t_part = income_groups[1];bsum(t_part, mid_income_sum, 5000.0);mid_income_sum += laplace(5000.0);
t_part = income_groups[2];bsum(t_part, high_income_sum, 10000.0);high_income_sum += laplace(10000.0);
100.0, 205.0, 1000.0, 2500.0, 99999.0, 10000.0, ...
100.0, 205.0, ...
1000.0, 2500.0, ...
99999.0, 10000.0, ...
sum
100000.0
50000000.0900000000.0 (899990000.0)
laplace noise
125759.1
50075392.6
900042943.8 (90025315.9)
7
90025315.9 (900042943.8)
![Page 8: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/8.jpg)
// income :1 {float}// epsilon=0.0, delta=0.0income_groups = partition(income, ...);
// income_groups :1 [{float}]// epsilon=0.0, delta=0.0t_part = income_groups[0];income_sum = bsum(t_part, 1000.0);
// low_income_sum :1000.0 float// epsilon=1.0, delta=0.0income_sum = laplace(income_sum, 1000.0);
An Example Fuzzi Program
partition
100.0, 205.0, 1000.0, 2500.0, 99999.0, 10000.0, ...
100.0, 205.0, ...
1000.0, 2500.0, ...
99999.0, 10000.0, ...
sum
100000.0
50000000.0900000000.0 (899990000.0)
laplace noise
125759.1
50075392.6
90025315.9 (900042943.8)
8
![Page 9: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/9.jpg)
Fuzzi Type System
9
![Page 10: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/10.jpg)
Type System as an Interface to apRHL
10
P(M)
prog
Q(N)
Hoare Logic
P(M, M’)
prog, prog’
Q(N, N’)
Relational
P(M, M’)
prob-prog, prob-prog’
Q(N, N’)
ApproximateRelationalHoare Logic
𝝐, 𝛿
![Page 11: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/11.jpg)
Type System as an Interface to apRHL
11
P(M, M’)
c, c
Q(N, N’)
apRHL
𝝐, 𝛿
![Page 12: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/12.jpg)
Packaging Manual Proofs for Mechanisms
12
P(M, M’)
prob-prog, prob-prog’
Q(N, N’)
apRHL
𝝐, 𝛿
bmap
partition
.
.
.
![Page 13: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/13.jpg)
Evaluation
Differentially Private Dataset
Logistic Regression 0.84 (11.02, 10e-6) MNIST
Ensemble of Logistic Regression
0.82 (20.0, 0.0) MNIST (partitioned)
Naive Bayes 0.69 (7.70, 0.0) Spambase
K-Means 0.55 - 0.9, median 0.69 (21.0, 0.0)
Iris
13
![Page 14: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/14.jpg)
1. Privacy Type System⠀ Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy.
Reed and Pierce. 2010.⠀ Linear Dependent Types for Differential Privacy.
Gaboardi et al. 2013.⠀ A Framework for Adaptive Differential Privacy.
Winograd-Cort et al. 2017.⠀ Duet: An Expressive Higher-order Language and Linear Type System for Statically Enforcing Differential Privacy.
Near et al. 2019.
2. Manual Proofs ⇔ Automated Type System⠀ RustBelt: Securing the Foundations of the Rust Programming Language.
Jung et al. 2017.14
⠀ Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy.
⠀ Reed and Pierce. 2010.
Linear Dependent Types for Differential Privacy.
Gaboardi et al. 2013.
⠀ A Framework for Adaptive Differential Privacy. Winograd-Cort et al. 2017.
⠀ Foundational Proof-Carrying Code.⠀ Appel. 2001.
⠀ A very modal model of a modern, major, general type system.
⠀ Appel et al. 2007.
Semantics of Types for Mutable State.⠀ Ahmed. 2004.
RustBelt: Securing the Foundations of the Rust Programming Language. Jung et al. 2017.
⠀ Distance Makes the Types Grow Stronger: A Calculus for Differential Privacy.
⠀ Reed and Pierce. 2010.
Linear Dependent Types for Differential Privacy.
Gaboardi et al. 2013.
⠀ A Framework for Adaptive Differential Privacy. Winograd-Cort et al. 2017.
⠀ Foundational Proof-Carrying Code.⠀ Appel. 2001.
⠀ A very modal model of a modern, major, general type system.
⠀ Appel et al. 2007.
Semantics of Types for Mutable State.⠀ Ahmed. 2004.
RustBelt: Securing the Foundations of the Rust Programming Language. Jung et al. 2017.
⠀ Fuzzi: A Three Level Logic for Differential Privacy.
⠀ Zhang et al. 2019.
![Page 15: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/15.jpg)
Conclusion1. We propose a high-level sensitivity type system for tracking differential privacy
a. We establish soundness through straightforward embedding into apRHL;b. The type system is expressive enough for verification conditions of manual differential privacy proofs in
apRHL.
2. We show how to push manual proof results of DP back into sensitivity type systema. We develop manual proofs of bag-map, bag-sum, partition, advanced composition.
3. We evaluate Fuzzi by implementing 4 textbook machine learning algorithmsa. We build a prototype of Fuzzi in Haskellb. We translate Fuzzi program into Python3 for execution
15
![Page 16: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/16.jpg)
Fuzzi: A Three Level Logic for Differential Privacy
Hengchu Zhang, Edo Roth, Andreas Haeberlen,Benjamin C. Pierce, Aaron Roth
University of Pennsylvania
![Page 17: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/17.jpg)
A Privacy Type System for Simple While Programs
17
Plus
![Page 18: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/18.jpg)
A Privacy Type System for Simple While Programs
18
Plus
Laplace
![Page 19: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/19.jpg)
1. Compositional✓ Given f1 -DP, and f2 -DP✓ Running f1 followed by f2 is -DP
2. Robust to post-processing✓ Further analysis on the results of f does not weaken its DP
guarantees
Properties of Differential Privacy
19
![Page 20: Fuzzi: A Three Level Logic for Differential Privacyahae/talks/fuzzi-icfp2019.pdf · 2019. 11. 18. · Fuzzi and its Three Levels Advanced Probabilistic Couplings for Differential](https://reader034.vdocuments.mx/reader034/viewer/2022051923/6010f33bfeaee1760c19ac87/html5/thumbnails/20.jpg)
Differential Privacy is Subtle
On the Privacy Properties of Variants on the Sparse Vector TechniqueChen and Machanavajjhala. 2015.
Understanding the Sparse Vector Technique for Differential PrivacyMin et al. 2016.
20