security and privacy of machine learning - ian …machine learning staff research scientist google...
TRANSCRIPT
![Page 1: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/1.jpg)
SESSION ID:
#RSAC
Ian Goodfellow
SECURITY AND PRIVACY OF MACHINE LEARNING
Staff Research ScientistGoogle Brain@goodfellow_ian
![Page 2: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/2.jpg)
(Goodfellow 2018)
#RSAC
Machine Learning and Security
2
yy
hh
xx
W
wyyh1
h1
x1
x1
h2
h2
x2
x2
Machine Learning for Security
Malware detection Intrusion detection …
Security against Machine Learning
yy
hh
xx
W
wyyh1
h1
x1
x1
h2
h2
x2
x2
Password guessing Fake reviews …
![Page 3: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/3.jpg)
(Goodfellow 2018)
#RSAC
Security of Machine Learning
3
yy
hh
xx
W
wyyh1
h1
x1
x1
h2
h2
x2
x2
![Page 4: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/4.jpg)
(Goodfellow 2018)
#RSAC
An overview of a field
4
This presentation summarizes the work of many people, not just my own / my collaborators
Download the slides for this link to extensive references
The presentation focuses on the concepts, not the history or the inventors
![Page 5: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/5.jpg)
(Goodfellow 2018)
#RSAC
Machine Learning Pipeline
5
X ✓x
y
Training data
Learning algorithmLearned parameters
Test input
Test output
![Page 6: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/6.jpg)
(Goodfellow 2018)
#RSAC
Privacy of Training Data
6
X ✓ X
![Page 7: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/7.jpg)
(Goodfellow 2018)
#RSAC
Defining (ε, δ)-Differential Privacy
7
(Abadi 2017)
![Page 8: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/8.jpg)
(Goodfellow 2018)
#RSAC
Private Aggregation of Teacher Ensembles
8
(Papernot et al 2016)
![Page 9: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/9.jpg)
(Goodfellow 2018)
#RSAC
Training Set Poisoning
9
xX ✓ y
![Page 10: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/10.jpg)
(Goodfellow 2018)
#RSAC
ImageNet Poisoning
10
(Koh and Liang 2017)
![Page 11: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/11.jpg)
(Goodfellow 2018)
#RSAC
Adversarial Examples
11
X ✓
x
y
![Page 12: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/12.jpg)
(Goodfellow 2018)
#RSAC
Model Theft
12
X ✓x
y✓
![Page 13: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/13.jpg)
(Goodfellow 2018)
#RSAC
Model Theft++
13
X ✓x
y✓Xx
![Page 14: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/14.jpg)
(Goodfellow 2018)
#RSAC
Deep Dive on Adversarial Examples
14
...solving CAPTCHAS and reading addresses...
...recognizing objects and faces….
(Szegedy et al, 2014)
(Goodfellow et al, 2013)
(Taigmen et al, 2013)
(Goodfellow et al, 2013)
and other tasks...
Since 2013, deep neural networks have matched human performance at...
![Page 15: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/15.jpg)
(Goodfellow 2018)
#RSAC
Adversarial Examples
15
![Page 16: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/16.jpg)
(Goodfellow 2018)
#RSAC
Turning objects into airplanes
16
![Page 17: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/17.jpg)
(Goodfellow 2018)
#RSAC
Attacking a linear model
17
![Page 18: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/18.jpg)
(Goodfellow 2018)
#RSAC
Wrong almost everywhere
18
![Page 19: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/19.jpg)
(Goodfellow 2018)
#RSAC
Cross-model, cross-dataset transfer
19
![Page 20: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/20.jpg)
(Goodfellow 2018)
#RSAC
Transfer across learning algorithms
20
(Papernot 2016)
![Page 21: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/21.jpg)
(Goodfellow 2018)
#RSAC
Transfer attack
21
Train your own model
Target model with unknown weights, machine learning
algorithm, training set; maybe non-differentiable
Substitute model mimicking target
model with known, differentiable function
Adversarial examples
Adversarial crafting against substitute
Deploy adversarial examples against the target; transferability
property results in them succeeding
![Page 22: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/22.jpg)
(Goodfellow 2018)
#RSAC
Enhancing Transfer with Ensembles
22
(Liu et al, 2016)
![Page 23: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/23.jpg)
(Goodfellow 2018)
#RSAC
Transfer to the Human Brain
23
(Elsayed et al, 2018)
![Page 24: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/24.jpg)
(Goodfellow 2018)
#RSAC
Transfer to the Physical World
24
(Kurakin et al, 2016)
![Page 25: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/25.jpg)
(Goodfellow 2018)
#RSAC
Adversarial Training
25
0 50 100 150 200 250 300
Training time (epochs)
10�2
10�1
100
Tes
tm
iscl
ass
ifica
tion
rate Train=Clean, Test=Clean
Train=Clean, Test=Adv
Train=Adv, Test=Clean
Train=Adv, Test=Adv
![Page 26: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/26.jpg)
(Goodfellow 2018)
#RSAC
Adversarial Training vs Certified Defenses
26
Adversarial Training: Train on adversarial examples This minimizes a lower bound on the true worst-case error Achieves a high amount of (empirically tested) robustness on small to medium datasets
Certified defenses Minimize an upper bound on true worst-case error Robustness is guaranteed, but amount of robustness is small Verification of models that weren’t trained to be easy to verify is hard
![Page 27: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/27.jpg)
(Goodfellow 2018)
#RSAC
Limitations of defenses
27
Even certified defenses so far assume unrealistic threat model
Typical model: attacker can change input within some norm ball
Real attacks will be stranger, hard to characterize ahead of time (Brown et al., 2017)
![Page 28: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/28.jpg)
(Goodfellow 2018)
#RSAC
Clever Hans
28
(“Clever Hans, Clever Algorithms,”
Bob Sturm)
![Page 29: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/29.jpg)
(Goodfellow 2018)
#RSAC
Get involved!
29
https://github.com/tensorflow/cleverhans
![Page 30: SECURITY AND PRIVACY OF MACHINE LEARNING - Ian …MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian (Goodfellow 2018) #RSAC Machine Learning and Security 2 y h](https://reader035.vdocuments.mx/reader035/viewer/2022070910/5f9c1c3e658bfe20356c4248/html5/thumbnails/30.jpg)
(Goodfellow 2018)
#RSAC
Apply What You Have Learned
30
Publishing an ML model or a prediction API? Is the training data sensitive? -> train with differential privacy
Consider how an attacker could cause damage by fooling your model Current defenses are not practical Rely on situations with no incentive to cause harm / limited amount of potential harm