physical adversarial examples for object detectors...current state of physical attacks 5 what [s the...
TRANSCRIPT
![Page 1: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/1.jpg)
Physical Adversarial Examples for Object Detectors
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, Tadayoshi Kohno, Dawn Song
![Page 2: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/2.jpg)
Deep Neural Networks are Useful
Speech Recognition,
Machine XLATAutomated Game Playing
Fast Brain Lesion SegmentationImage Courtesy: Nvidia/Imperial College
2
![Page 3: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/3.jpg)
Deep Neural Networks are Useful, But Vulnerable
“panda”57.7% confidence
“gibbon”99.3% confidence
Image Credit: OpenAI
=+ ε
Goodfellow et al., Explaining and Harnessing Adversarial Examples, arXiv 1412.6572, 2015
fθ(x) = y fθ(x’) ≠ y fθ(x’) = y* where x’ = x + δ
3
![Page 4: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/4.jpg)
Deep Neural Networks are Useful, But Vulnerable
“panda”57.7% confidence
“gibbon”99.3% confidence
Image Credit: OpenAI
=+ ε
4
Can we physically & robustly perturb real objects, in ways that cause misclassifications in a DNN?
![Page 5: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/5.jpg)
Current State of Physical Attacks
5
What’s the dominant object in this image?
Classification
Eykholt et al., Robust Physical-World Attacks on Deep Learning Visual Classification, CVPR 2018
Kurakin et al., Adversarial Examples in the Physical World, arXiv 1607.02533, 2016Athalye et al., Synthesizing Robust Adversarial Examples, ICML 2018Brown et al., Adversarial Patch, arXiv 1712.09665Sharif et al., Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition, CCS 2016
Our prior work
![Page 6: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/6.jpg)
Different types of Deep Learning Models
6
What’s the dominant object in this image?
Classification
What are the objects in this scene, and where are they?
Object Detection
What are the precise shapes and locations of objects?
Semantic Segmentation
Focus of this paper
![Page 7: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/7.jpg)
Challenges in Attacking Detectors
7
The location of the target object within the scene can vary widely
Detectors process entire scene, allowing them to use contextual information
Not limited to producing a single labeling, instead labels all objects in the scene
We will start with an algorithm to attack classifiers and modify it to attack detectors
![Page 8: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/8.jpg)
Review: Robust Physical Perturbations (RP2)
8
A distance function The target class
Perturbation/Noise MatrixLp norm (L-0, L-1, L-2, …)
Loss Function
Challenge: How can we make the perturbations robust to changing environmental conditions?
![Page 9: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/9.jpg)
9
Modeling the Effects of the Environment
• Sample from the distribution Xv by:• Taking real images varying physical conditions (e.g., distances and angles)
• Using synthetic transformations
![Page 10: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/10.jpg)
Optimizing Spatial Constraints
10
Subtle Poster
Camouflage Sticker
Approximate vandalism
Example Masks
![Page 11: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/11.jpg)
How To Choose A Mask?
11
We had very good success with the octagonal maskPossibility: Mask surface area should be large or should be focused on “sensitive” regions
Use L-1
![Page 12: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/12.jpg)
Process of Creating a Useful Sticker Attack
12
L-1 Perturbation Result Mask Sticker Attack!
![Page 13: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/13.jpg)
Adapting RP2: Translational Invariance
13
...
![Page 14: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/14.jpg)
Adapting RP2: Adversarial Loss Function
14
P(object)CxCywh
P(Stop sign)P(person)
P(cat)…
P(vase)
5 bounding boxes
80 x 1, 80 classes5 x 1
S x S grid cells
Output of YOLO, 19 x 19 x 425 tensor
Input scene
Prob. of object being class ‘y’
Minimize the probability of “Stop” sign among all predictions
![Page 15: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/15.jpg)
Poster and Sticker Attack
15
![Page 16: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/16.jpg)
Evaluation Method & Data
16
• Record a video while moving towards a sign
• Sample video frames
• Count number of frames in which Stop sign was NOT detected
White-box Black-box
![Page 17: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/17.jpg)
17
Poster Attack on YOLO v2
![Page 18: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/18.jpg)
18
Sticker Attack on YOLO v2
![Page 19: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/19.jpg)
Black-box transfer
toFaster-RCNN
![Page 20: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/20.jpg)
Creation Attacks
• Cause the detector to hallucinate
• A meaningless-to-humans object but detected as an attacker-chosen class
20
Threshold after which we stop optimizing for box confidence, set to 0.2
Y is the class that the attacker wants the detector to see
YOLO labels this as a Stop sign
![Page 21: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/21.jpg)
Takeaways
• Adversarial examples generalize to varied environmental conditions
• With modest changes to our prior work on attacking classifiers, adversarial examples generalize to the richer class of object detection models• We introduced a new type of adversarial loss (disappearance, creation)
• We also introduced the Translational Invariance property
• Our evaluation based on the state-of-the-art YOLO v2 detector shows that physical attacks are possible up to distances of ~30 feet
• Do these attacks have system wide effects?
21
![Page 22: Physical Adversarial Examples for Object Detectors...Current State of Physical Attacks 5 What [s the dominant object in this image? Classification Eykholt et al., Robust Physical-World](https://reader030.vdocuments.mx/reader030/viewer/2022041005/5ea9309eaf8c2c23fa1478d0/html5/thumbnails/22.jpg)
Thank you!
• Adversarial examples generalize to varied environmental conditions
• With modest changes to our prior work on attacking classifiers, adversarial examples generalize to the richer class of object detection models• We introduced a new type of adversarial loss (disappearance, creation)
• We also introduced the Translational Invariance property
• Our evaluation based on the state-of-the-art YOLO v2 detector shows that physical attacks are possible up to distances of ~30 feet
• Do these attacks have system wide effects?
22
Earlence Fernandes, [email protected], earlence.com