skill squatting attacks on amazon alexa · 2019-12-18 · skill squatting attacks on amazon alexa...
TRANSCRIPT
Skill Squatting Attacks on Amazon Alexa
Skill Squatting Attacks on Amazon Alexa
Deepak Kumar University of Illinois
Riccardo Paccagnella University of Illinois
Paul Murley University of Illinois
Joshua Mason University of Illinois
1
Eric Hennenfent University of Illinois
Adam Bates University of Illinois
Michael Bailey University of Illinois
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar2
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Speech recognition systems frequently make errors, even in normal use
3
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar4
“Alexa, turn onthe living room
lights.”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar5
“Alexa, turn onthe living room
lights.”
“Okay, playingLiving on a Prayer,
Lullaby Edition”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar6
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar7
Alexa Skills
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar8
Alexa Skills
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar9
Alexa Skills
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar10
Skills: Apps, but for Alexa
(
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar11
Alexa
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar12
Alexa
Skills……
cat forkscat fastcat facts
…………
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar13
Alexa
Skills……
cat forkscat fastcat facts
…………
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar14
Alexa
Skills……
cat forkscat fastcat facts
…………
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar15
Alexa
Skills……
cat forkscat fastcat facts
…………
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar16
Alexa
Skills……
cat forkscat fastcat facts
…………
cat fact
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar17
Alexa
Skills……
cat forkscat fastcat facts
…………
cat fact
“A group of cats is called a clowder!”
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
1. Alexa makes mistakes 2. Skills are the new apps
18
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
1. Alexa makes mistakes 2. Skills are the new apps
19
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
1. Alexa makes mistakes 2. Skills are the new apps
20
What could go wrong?
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar21
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar22
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar23
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar24
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Can Alexa errors be leveraged to cause harm to end users?
25
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Skill Squatting Attacks
An attacker can leverage predictable errors in Alexa to route users to skills that they didn't intend to go to
26
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar27
Alexa
Skills……
cat forkscat fastcat factscat fax
………
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar28
Alexa
Skills……
cat forkscat fastcat facts
…………
Skills……
cat forkscat fastcat factscat fax
………
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar29
Alexa
Skills……
cat forkscat fastcat facts
…………
Skills……
cat forkscat fastcat factscat fax
………
cat fact
“A group of cats is called a murder!”
“Alexa, tell me some cat facts!”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
How can you tell which errors are predictable?
30
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Send speech samples to Alexa, figure out where it goes wrong
31
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Speech Corpus
32
• Leveraged the NSP Dataset
• 60 speakers, 188 unique words each (11,460 audio samples)
• Speakers were representative of 6 US dialect regions
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Measuring Interpretation Errors
We sent each speech sample to Alexa 50 times, providing us 573,000 transcriptions across the 60 speakers
33
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Predictable Errors
34
Word PredictionSail SaleRip Rap
Outshine OutshyneLung Lang
Accelerate XcelerateMill No
Preferably PreferrablyEarthy FiCalm ComCoal Call
Outdoors Out DoorsLoud Louder
Word PredictionSuperhighway Super Highway
Wet WhatMain MaineBoil BoyleSell CellFull Four
Dime TimeBean BeenDull Doll
Sweeten SwedenLuck LockCon Khan
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Predictable Errors
35
Word PredictionSail SaleRip Rap
Outshine OutshyneLung Lang
Accelerate XcelerateMill No
Preferably PreferrablyEarthy FiCalm ComCoal Call
Outdoors Out DoorsLoud Louder
Word PredictionSuperhighway Super Highway
Wet WhatMain MaineBoil BoyleSell CellFull Four
Dime TimeBean BeenDull Doll
Sweeten SwedenLuck LockCon Khan
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Can we use our predictable errors to route users to unintended skills?
36
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Validating the Skill Squatting Attack• Split speakers into two sets: “training” set and the “testing” set
• For each word with predictable error, we built two skills: the word, and the predictable error
• Skill A: Wet
• Skill B: What
• Sent the testing set through to Alexa, observed how many times skill B was triggered instead of skill A
37
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
A Brief, Ethical Note….
• We validated this attack strictly in a developer environment, no real skills were targeted or tested in the wild
• This is a fundamental limitation, but it’s what we thought was the right thing to do
38
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Successfully squatted 25 of 27 (93%) predictable errors
at least once
(
39
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
I would never want a cat fact. Why does this matter?
40
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar41
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar42
Alexa
“Alexa, ask Amex to pay Bailey $100” Skills
……
Amex………………
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar43
Alexa
“Alexa, ask Amex to pay Bailey $100” Skills
……
Amex………………
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar44
Alexa
“Alexa, ask Amex to pay Bailey $100” Skills
……
Amex………………
“You need to log in. I’ve sent a card to your phone.”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar45
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar46
Alexa
“Alexa, ask Amex to pay Bailey $100” Skills
……
AmexAm X.
……………
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar47
Alexa
“Alexa, ask Amex to pay Bailey $100” Skills
……
AmexAm X.
……………
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar48
Alexa
“Alexa, ask Amex to pay Bailey $100” Skills
……
AmexAm X.
……………
“You need to log in. I’ve sent a card to your phone.”
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar49
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar50
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Do different regions exhibit unique predictable interpretation errors?
51
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar52
Predictable Errors by Region
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar53
Predictable Errors by Region
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar54
Predictable Errors by Region
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar55
Predictable Errors by Region
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar56
Predictable Errors by Region
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Spear Skill Squatting Attacks
• An attacker can leverage accent-specific predictable errors in Alexa to route distinct groups of users to skills that they didn't intend to go to
57
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Validating the Spear Skill Squatting Attack
58
Squatted Pair Region Target % Overall % Significant?Tool/Two South 34.0% 14.1% Yes
Dock/Doc West 97.4% 81.6% NoMighty/My T. West 20.0% 4.1% Yes
Exterior/Xterior New England 42.9% 22.5% YesMeal/Meow New England 55.6% 34.3% YesWool/Well Midland 50% 32.4% No Pal/Pow Midland 65.9% 37.7% Yes
Accuser/Who’s There Midland 26.0% 4.9% Yes
Pin/Pen Midland 26.3% 10.0% YesMalfunction/No
Function Mid-Atlantic 36.0% 27.5% No
Fade/Feed Mid-Atlantic 59.0% 14.7% Yes
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Validating the Spear Skill Squatting Attack
59
Squatted Pair Region Target % Overall % Significant?Tool/Two South 34.0% 14.1% Yes
Dock/Doc West 97.4% 81.6% NoMighty/My T. West 20.0% 4.1% Yes
Exterior/Xterior New England 42.9% 22.5% YesMeal/Meow New England 55.6% 34.3% YesWool/Well Midland 50% 32.4% NoPal/Pow Midland 65.9% 37.7% Yes
Accuser/Who’s There Midland 26.0% 4.9% Yes
Pin/Pen Midland 26.3% 10.0% YesMalfunction/No
Function Mid-Atlantic 36.0% 27.5% No
Fade/Feed Mid-Atlantic 59.0% 14.7% Yes
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Validating the Spear Skill Squatting Attack
60
Squatted Pair Region Target % Overall % Significant?Tool/Two South 34.0% 14.1% Yes
Dock/Doc West 97.4% 81.6% NoMighty/My T. West 20.0% 4.1% Yes
Exterior/Xterior New England 42.9% 22.5% YesMeal/Meow New England 55.6% 34.3% YesWool/Well Midland 50% 32.4% NoPal/Pow Midland 65.9% 37.7% Yes
Accuser/Who’s There Midland 26.0% 4.9% Yes
Pin/Pen Midland 26.3% 10.0% YesMalfunction/No
Function Mid-Atlantic 36.0% 27.5% No
Fade/Feed Mid-Atlantic 59.0% 14.7% Yes
Successfully squatted 8 out of 11 spear squattable pairs
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Limitations
• Scale + Representativeness of the dataset
• Skill behavior outside of a development environment
61
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Limitations
• Scale + Representativeness of the dataset
• Skill behavior outside of a development environment
62
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Takeaways
• New medium, same problems
• “Typosquatting” in the land of IoT
• Opaque ML for decision making is still nascent
• Interface quirks can and will be exploited to cause abuse
63
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
Takeaways
• New medium, same problems
• “Typosquatting” in the land of IoT
• Opaque ML for decision making is still nascent
• Interface quirks can and will be exploited to cause abuse
64
Tracking Certificate Misissuance in the Wild ▪︎ Deepak Kumar
Moving Forward
65
• Working with Amazon to fix these issues in their platform
• Investigating the widespread harms of skill squatting
• Investigating IoT trust relationships
• Do users intrinsically trust voice-based devices more than online?
Tracking Certificate Misissuance in the Wild ▪︎ Deepak Kumar
Moving Forward
66
• Working with Amazon to fix these issues in their platform
• Measuring the widespread harms of skill squatting
• Investigating IoT trust relationships
• Do users intrinsically trust voice-based devices more than online?
Tracking Certificate Misissuance in the Wild ▪︎ Deepak Kumar
Moving Forward
67
• Working with Amazon to fix these issues in their platform
• Measuring the widespread harms of skill squatting
• Investigating IoT trust relationships
• Do users intrinsically trust voice-based devices more than online?
@_kumarde
Skill Squatting Attacks on Amazon Alexa ▪︎ Deepak Kumar
backup
68