dn 2017 | attention models for chatbot technology | igor mikhalev | firmshift
TRANSCRIPT
ROLLEN 2017
reason.ai — attention models for chatbot technology
1 1
Attention models for chatbot technology
“a medium extra hot latte macchiato with skimmed milk and a double shot of
espresso?”
HOW WOULD YOU PREFER ORDERING
2
But during the industrial age we needed to learn using machine
interfaces for driving cars, washing clothes, ordering stuff online
and getting coffee.
We’re nearing the moment when we can start using our own language
again.
And give machines the responsibility to learn to understand us.
A brand gets to have a hyper-relevant, goal-oriented and personal 1:1 conversation with customers through a channel they like and
anytime they like it.
“Resilient toy” approach
• User’s primary goal is to break the system
• Users employ a set of strategies to achieve that
Users’ strategies for breaking the toy
User: I have a red car. Bot: OK. User: What color is my car? Bot: Maybe it is green?
Forgetfulness
Contradiction
User: I have two brothers.Bot: OK.User: I don’t have brothers.Bot: OK.
“Bad intent” detection
• Contradiction • Forgetfulness • Topic Irrelevance • Word sense misinterpretation • Syntactic misinterpretation • Named entity confusion • Word play • Repetition
But also:
• Phatic • Cooperative
Assumption: no background knowledge
“What’s the radius of the Earth?”
Not allowed when ordering a pizza
Building on this assumption
• Premises are introduced by the user and get discharged (utilized) during dialog
• We can use natural deduction and resource logic to model that pattern
Syntactic categories represent classes of context, which should be glued into a coherent whole
Lambek calculus
A proof net for LCGA proof net for LCG is a frame with linkage, satisfying a set of well-formedness constraints (acyclicity, coloring, depending on the formulation)
Attention Matrix
syntactic categories
sequents
sequent decomposition
linkage
embeddings
sequences
seq2seq
attention
Smarter dialog management
•Spot the user’s utterance that the current response violates/misinterprets
•Assign a score (probability) that a hypothetical answer breaks the dialog