safety bot guaranteed -- shmoocon 2017

January 15, 2017

SAFETY BOT

GUARANTEED

RICH SEYMOUR

Who Am I?

Data Scientist at Endgame

Got a PhD in Materials Science doing

nanoscale simulations on supercomputers

Play bass in a band (bleach bones)

@rseymour on twitter

2

Outline

Bots

Safety

Their powers combined!

3

Outline

Bots

Safety

Their powers combined!

4

BOTS

the hype cycle peaks?

Talking about chatbots

Image and many of these bot

slides courtesy of bot master

and coworker:

Bobby Filar @filar

6

Popular Bots

Amazon – Alexa

Apple - Siri

Facebook - M

Google - Google Home

Microsoft – Cortana

7

Bots are Changing Lives

8

Bots are Changing Lives Pt 2

9

Bots Boiled Down

Bots take words and turn them into actions.

“Alexa, turn on the lamp” “Ok”

“Open the pod bay doors HAL”

10

Voice Recognition and Speech Synthesis

We don’t need

these pieces to

have a bot

11

Text to

Speech

bot

Speech

to Text

Click-Through vs. Conversational Interfaces

12

+ 6 more steps

vs

Id like to order a cheese pizza.

Great! What size pizza would

you like?

Medium

Okay. I have an order for a

medium cheese pizza. Is that

all?

Yes. That is all.

Causes of the Bot Revolution?

Platform Diversity

• Messaging

• Intelligent assistants

• Team collaboration software

App Stagnation

• More users on messaging platforms

• Hard to gain traction w/in App Stores

Ease-of-Use

• Simplifies UI/UX

• Allows user to maintain focus on current screen/activity

• Provide walkthroughs

• Eliminates need to know query languages

13

Cortana

Bot Development Kits (BDKs)

The rise of bots is largely due to an increase in BDKs

Companies like Chatfuel, Wit.ai, and API.ai

• Provide simple UI for development

• Closed-domain, Rule-based, goal-oriented bots

• No programming skills required

How popular have these frameworks become?

• Wit.ai purchased by Facebook (Jan 2015)

• API.ai purchased by Google (Sep 2016)

Major companies pushing out frameworks too!

• Microsoft Bot Framework

• Amazon Alexa Skills Kit

14

What bots are made of.

GENERATIVE RETRIEVAL

16

OPEN DOMAIN CLOSED DOMAIN

LONG CONVERSATION SHORT CONVERSATION

Types of Bots

Active Research Surprisingly Easy

GOOD BOTS

DoNotPay Bot

• Bot helps people fight parking tickets

• Wildly successfully in 3 major cities

Esterbot

• QA bot tied to a LinkedIn account or resume.

• Recruiters talk to a bot before they get to you

BAD BOTS

Microsoft TayBot

• Great idea gone terribly

awry

• Learning from internet

conversations sometimes

leads to being…

the worst.

17

Good Bots and Bad Bots

18

GENERATIVE RETRIEVAL

PROS

• Bots of the future!

• Excellent recall ability

• Ability to learn as exposure to

user increases

• Simple to implement

• Smaller datasets

• No grammatical mistakes

CONS

• Requires a lot of data

• Complex models large

vocabularies massive training

time for model building

• Prone to grammatical errors

• Really, really inappropriate…

right TayBot?

• Not intelligent

• Limited “memory”

• Works best in closed-domain, short

conversation scenarios

Which is best?

Bot Workflow aka Bot Vocabulary

Session

Identity

Dialogue Script

Context

Intents

Entities

Action

19

Pattern Matching without Context (Eliza 1964-1966)

20

Context & Intent

Context (teal)

• Current state

• Steers conversation

• Ensures all req’d parameters are

collected to perform Action

Intent (red)

• Maps Utterance (blue) specific

Action

• What does the user want?

21

Entities

Entities are parameters needed to satisfy Intents and execute Actions.

Entities can be anything

• File hash

• IP address

• Cities

Permutations of an entity allow for diversity of vocabulary

• Regular Expressions

• Synonyms22

Training Data

Few bot developers have necessary

training data

BDKs provide access to Domains or

pre-defined Knowledge packages:

• Reservations

• Small Talk

• Weather

BDKs log user interactions to constantly

update training set

23

Natural Language Processing & Machine Learning

BDKs have built in ML/NLP capabilities

NLP used to handle incoming data

• Tokenization

• Word Boundary & Word Sense

• Synonym generation

• Entity extraction

ML used to classify user input to an Intent and map

to an output

• Supervised Learning

• Features: n-gram or word-based vectors

• Model constantly updated as bot gets used

24

Input NLP

Output ML

features

NLP 101 Direct Matching (No ML)

Show flights from Washington to Denver on Monday

Show flights from Washington to Denver on Monday

Show flights from Washington to Denver on Monday

VB NNS IN NNP IN NNP IN NNP

if utterance contains show_vb AND flights_nns:

Utterance

Tokenization

Parts-of-SpeechTagging (POS)

Show flights from Washington to Denver on MondayNamed EntityRecognition (NER) Location Location Date

SHOW_FLIGHTS(Washington, Denver, Monday)

Perform Action

25

NLP 101 ML-Based

Show flights from Washington to Denver on Monday

Show flights from Washington to Denver on Monday

Utterance

Tokenization

Show flights from Washington to Denver on MondayNamed EntityRecognition (NER)

Location Location Date

0 1 1 0 1 0 1 0 1 1 0 0 1Bag-of-Words(Feature Vector)

SHOW_FLIGHTS

CHANGE_FLIGHTS

BOOK_FLIGHTS

Intent Classification SHOW_FLIGHTS(location, location, date)

SHOW_FLIGHTS(Washington, Denver, Monday)Perform Action

26

Getting started with this Language Stuff

I recommend spaCy available

at spacy.io

27

How can bots help Security?

Some Ideas

Guidance and Training

29

Is the command line a chatbot?

Guidance and Training

30

disown

Guidance and Training

31

Search

32

Search (but not security twitter, let’s say your logs)

33

“What can I help you with today?”

“Show me all of the processes running that started in the last hour”

“Ok which endpoints?”

Search (but not security twitter, let’s say your logs)

34

“What can I help you with today?”

“Show me all of the processes running that started in the last hour”

“Ok which endpoints?”

Search (but not security twitter, let’s say your logs)

35

“What can I help you with today?”

“Show me all of the processes running that started in the last hour”

“Ok which endpoints?”

“Everyone!”

“Ok”

Search results ensue

Question and Answer Bots

36

Q&A Bots

37

Chat Ops and Checklists

38

Hubot – GitHub *

ScoreBot - Netflix

* https://www.youtube.com/watch?v=NST3u-GjjFw

Chat Ops and Checklists

39

Ignorance

Ineptitude

Chat Ops and Checklists

40

Chat Ops and Checklists

41

But checklists are

the worst!

Chat Ops and Checklists

42

Well maybe bake the checklists into the bots?

Outline

Bots

Safety (get shmoo balls out)

How To!

43

Safety

Safety!

45

Just what is Safety?

How is Safety different from

Security?

46

Just what is Safety?

How is Safety different from

Security?

Safety generally means

preventing people getting hurt47

Just what is Safety?

Safety ideas have historically

been borrowed by Security.

48

Domino Theory -- Heinrich 1931

49

Domino Theory -- Heinrich 1931

50

Bigotry

included

Others have followed

51

Root Cause Analysis

Fault Tree Analysis

Swiss-cheese model

Engineering a Safer World

52

Engineering a Safer WorldSystems Thinking Applied to Safety

Nancy G. Leveson2012

Systems Theoretic Process Analysis

(STPA)

Engineering a Safer World

53

Engineering a Safer WorldSystems Thinking Applied to Safety

Nancy G. Leveson2012

Systems Theoretic Process Analysis

(STPA)

Systems Thinking Applied to Safety

54

Bertalanffy

Wiener

Systems Thinking Applied to Safety

55

Bertalanffy – Systems Theory

Wiener - Cybernetics

Safety Defined

56

Hazard: a system state or set of conditions that combined with worst case environmental conditions will lead to an accident

Accident: an event that results in an unacceptable level of loss

Safety: freedom from accidents

The Driving Force of Systems Safety

57

Accidents today happen while

the each component of the

system is working

The Driving Force of Systems Safety

58

The accidents we see now are:

Component Interaction

Accidents

The Driving Force (continued)

59

“Preventing future accidents requires shifting from a focus on preventing failures to the broader goal of designing and implementing controls that will enforce the necessary constraints.”

What it doesn’t do

60

It doesn’t use Risk %s

It doesn’t catalog possible failure

It doesn’t blame operator error

It doesn’t triage

Purpose

61

What does your system do?

Losses

62

Unacceptable.

Not component failures.

Real. Bad. Stuff.

Hazards

63

What could make you suffer a

loss?

Threats?

Hazards

64

What could make you suffer a loss?

Threats?

Check out William Young’s STPA-Sec work

Constraints

65

How can we stop hazards from

becoming losses.

How can these in turn be

violated. (iterate)

The minimal system

66

The minimal system

67

Yup, just

boxes

and

arrows!

68

Nancy Leveson’s

outline of possible

“control flaws leading

to hazards”

Engineering a Safer

World

p 93

Very High Level

69

Maybe this is a useful process?

70

Maybe this is a useful process?

71

STPA takes a high level view, early in the design

72

What if we had a bot to help us

go through this process?

Outline

Bots

Safety

How To! (SafetyBot!)

73

SafetyBot

STPA takes a high level view, early in the design

75

What if we had a bot to help us

go through this process?

What if you had ~ 2 weeks to

write such a bot!

GENERATIVE RETRIEVAL

76

OPEN DOMAIN CLOSED DOMAIN

LONG CONVERSATION SHORT CONVERSATION

Types of Bots

Active Research Surprisingly Easy

Choose a language you’ve never used!

77

SafetyBot

78

context

Context

manager

Node

entity

Node

entity

context

Node

entity

Current

context

Next

context

The context manager is the main

loop

Prints out bot talk and pulls in

user input.

That is then passed to the current

‘node’ for processing / storage

SafetyBot

Live Demo

Thanks thanks thanks thanks thanks!

80

Questions?