safety bot guaranteed -- shmoocon 2017
TRANSCRIPT
January 15, 2017
SAFETY BOT
GUARANTEED
RICH SEYMOUR
Who Am I?
Data Scientist at Endgame
Got a PhD in Materials Science doing
nanoscale simulations on supercomputers
Play bass in a band (bleach bones)
@rseymour on twitter
2
Outline
Bots
Safety
Their powers combined!
3
Outline
Bots
Safety
Their powers combined!
4
BOTS
the hype cycle peaks?
Talking about chatbots
Image and many of these bot
slides courtesy of bot master
and coworker:
Bobby Filar @filar
6
Popular Bots
Amazon – Alexa
Apple - Siri
Facebook - M
Google - Google Home
Microsoft – Cortana
7
Bots are Changing Lives
8
Bots are Changing Lives Pt 2
9
Bots Boiled Down
Bots take words and turn them into actions.
“Alexa, turn on the lamp” “Ok”
“Open the pod bay doors HAL”
10
Voice Recognition and Speech Synthesis
We don’t need
these pieces to
have a bot
11
Text to
Speech
bot
Speech
to Text
Click-Through vs. Conversational Interfaces
12
+ 6 more steps
vs
Id like to order a cheese pizza.
Great! What size pizza would
you like?
Medium
Okay. I have an order for a
medium cheese pizza. Is that
all?
Yes. That is all.
Causes of the Bot Revolution?
Platform Diversity
• Messaging
• Intelligent assistants
• Team collaboration software
App Stagnation
• More users on messaging platforms
• Hard to gain traction w/in App Stores
Ease-of-Use
• Simplifies UI/UX
• Allows user to maintain focus on current screen/activity
• Provide walkthroughs
• Eliminates need to know query languages
13
Cortana
Bot Development Kits (BDKs)
The rise of bots is largely due to an increase in BDKs
Companies like Chatfuel, Wit.ai, and API.ai
• Provide simple UI for development
• Closed-domain, Rule-based, goal-oriented bots
• No programming skills required
How popular have these frameworks become?
• Wit.ai purchased by Facebook (Jan 2015)
• API.ai purchased by Google (Sep 2016)
Major companies pushing out frameworks too!
• Microsoft Bot Framework
• Amazon Alexa Skills Kit
14
What bots are made of.
GENERATIVE RETRIEVAL
16
OPEN DOMAIN CLOSED DOMAIN
LONG CONVERSATION SHORT CONVERSATION
Types of Bots
Active Research Surprisingly Easy
GOOD BOTS
DoNotPay Bot
• Bot helps people fight parking tickets
• Wildly successfully in 3 major cities
Esterbot
• QA bot tied to a LinkedIn account or resume.
• Recruiters talk to a bot before they get to you
BAD BOTS
Microsoft TayBot
• Great idea gone terribly
awry
• Learning from internet
conversations sometimes
leads to being…
the worst.
17
Good Bots and Bad Bots
18
GENERATIVE RETRIEVAL
PROS
• Bots of the future!
• Excellent recall ability
• Ability to learn as exposure to
user increases
• Simple to implement
• Smaller datasets
• No grammatical mistakes
CONS
• Requires a lot of data
• Complex models large
vocabularies massive training
time for model building
• Prone to grammatical errors
• Really, really inappropriate…
right TayBot?
• Not intelligent
• Limited “memory”
• Works best in closed-domain, short
conversation scenarios
Which is best?
Bot Workflow aka Bot Vocabulary
Session
Identity
Dialogue Script
Context
Intents
Entities
Action
19
Pattern Matching without Context (Eliza 1964-1966)
20
Context & Intent
Context (teal)
• Current state
• Steers conversation
• Ensures all req’d parameters are
collected to perform Action
Intent (red)
• Maps Utterance (blue) specific
Action
• What does the user want?
21
Entities
Entities are parameters needed to satisfy Intents and execute Actions.
Entities can be anything
• File hash
• IP address
• Cities
Permutations of an entity allow for diversity of vocabulary
• Regular Expressions
• Synonyms22
Training Data
Few bot developers have necessary
training data
BDKs provide access to Domains or
pre-defined Knowledge packages:
• Reservations
• Small Talk
• Weather
BDKs log user interactions to constantly
update training set
23
Natural Language Processing & Machine Learning
BDKs have built in ML/NLP capabilities
NLP used to handle incoming data
• Tokenization
• Word Boundary & Word Sense
• Synonym generation
• Entity extraction
ML used to classify user input to an Intent and map
to an output
• Supervised Learning
• Features: n-gram or word-based vectors
• Model constantly updated as bot gets used
24
Input NLP
Output ML
features
NLP 101 Direct Matching (No ML)
Show flights from Washington to Denver on Monday
Show flights from Washington to Denver on Monday
Show flights from Washington to Denver on Monday
VB NNS IN NNP IN NNP IN NNP
if utterance contains show_vb AND flights_nns:
Utterance
Tokenization
Parts-of-SpeechTagging (POS)
Show flights from Washington to Denver on MondayNamed EntityRecognition (NER) Location Location Date
SHOW_FLIGHTS(Washington, Denver, Monday)
Perform Action
25
NLP 101 ML-Based
Show flights from Washington to Denver on Monday
Show flights from Washington to Denver on Monday
Utterance
Tokenization
Show flights from Washington to Denver on MondayNamed EntityRecognition (NER)
Location Location Date
0 1 1 0 1 0 1 0 1 1 0 0 1Bag-of-Words(Feature Vector)
SHOW_FLIGHTS
CHANGE_FLIGHTS
BOOK_FLIGHTS
Intent Classification SHOW_FLIGHTS(location, location, date)
SHOW_FLIGHTS(Washington, Denver, Monday)Perform Action
26
Getting started with this Language Stuff
I recommend spaCy available
at spacy.io
27
How can bots help Security?
Some Ideas
Guidance and Training
29
Is the command line a chatbot?
Guidance and Training
30
disown
Guidance and Training
31
Search
32
Search (but not security twitter, let’s say your logs)
33
“What can I help you with today?”
“Show me all of the processes running that started in the last hour”
“Ok which endpoints?”
Search (but not security twitter, let’s say your logs)
34
“What can I help you with today?”
“Show me all of the processes running that started in the last hour”
“Ok which endpoints?”
Search (but not security twitter, let’s say your logs)
35
“What can I help you with today?”
“Show me all of the processes running that started in the last hour”
“Ok which endpoints?”
“Everyone!”
“Ok”
Search results ensue
Question and Answer Bots
36
Q&A Bots
37
Chat Ops and Checklists
38
Hubot – GitHub *
ScoreBot - Netflix
* https://www.youtube.com/watch?v=NST3u-GjjFw
Chat Ops and Checklists
39
Ignorance
Ineptitude
Chat Ops and Checklists
40
Chat Ops and Checklists
41
But checklists are
the worst!
Chat Ops and Checklists
42
Well maybe bake the checklists into the bots?
Outline
Bots
Safety (get shmoo balls out)
How To!
43
Safety
Safety!
45
Just what is Safety?
How is Safety different from
Security?
46
Just what is Safety?
How is Safety different from
Security?
Safety generally means
preventing people getting hurt47
Just what is Safety?
Safety ideas have historically
been borrowed by Security.
48
Domino Theory -- Heinrich 1931
49
Domino Theory -- Heinrich 1931
50
Bigotry
included
Others have followed
51
Root Cause Analysis
Fault Tree Analysis
Swiss-cheese model
Engineering a Safer World
52
Engineering a Safer WorldSystems Thinking Applied to Safety
Nancy G. Leveson2012
Systems Theoretic Process Analysis
(STPA)
Engineering a Safer World
53
Engineering a Safer WorldSystems Thinking Applied to Safety
Nancy G. Leveson2012
Systems Theoretic Process Analysis
(STPA)
Systems Thinking Applied to Safety
54
Bertalanffy
Wiener
Systems Thinking Applied to Safety
55
Bertalanffy – Systems Theory
Wiener - Cybernetics
Safety Defined
56
Hazard: a system state or set of conditions that combined with worst case environmental conditions will lead to an accident
Accident: an event that results in an unacceptable level of loss
Safety: freedom from accidents
The Driving Force of Systems Safety
57
Accidents today happen while
the each component of the
system is working
The Driving Force of Systems Safety
58
The accidents we see now are:
Component Interaction
Accidents
The Driving Force (continued)
59
“Preventing future accidents requires shifting from a focus on preventing failures to the broader goal of designing and implementing controls that will enforce the necessary constraints.”
What it doesn’t do
60
It doesn’t use Risk %s
It doesn’t catalog possible failure
It doesn’t blame operator error
It doesn’t triage
Purpose
61
What does your system do?
Losses
62
Unacceptable.
Not component failures.
Real. Bad. Stuff.
Hazards
63
What could make you suffer a
loss?
Threats?
Hazards
64
What could make you suffer a loss?
Threats?
Check out William Young’s STPA-Sec work
Constraints
65
How can we stop hazards from
becoming losses.
How can these in turn be
violated. (iterate)
The minimal system
66
The minimal system
67
Yup, just
boxes
and
arrows!
68
Nancy Leveson’s
outline of possible
“control flaws leading
to hazards”
Engineering a Safer
World
p 93
Very High Level
69
Maybe this is a useful process?
70
Maybe this is a useful process?
71
STPA takes a high level view, early in the design
72
What if we had a bot to help us
go through this process?
Outline
Bots
Safety
How To! (SafetyBot!)
73
SafetyBot
STPA takes a high level view, early in the design
75
What if we had a bot to help us
go through this process?
What if you had ~ 2 weeks to
write such a bot!
GENERATIVE RETRIEVAL
76
OPEN DOMAIN CLOSED DOMAIN
LONG CONVERSATION SHORT CONVERSATION
Types of Bots
Active Research Surprisingly Easy
Choose a language you’ve never used!
77
SafetyBot
78
context
Context
manager
Node
entity
Node
entity
context
Node
entity
Current
context
Next
context
The context manager is the main
loop
Prints out bot talk and pulls in
user input.
That is then passed to the current
‘node’ for processing / storage
SafetyBot
Live Demo
Thanks thanks thanks thanks thanks!
80
Questions?