boosting a chatterbot understanding with a weighted...
TRANSCRIPT
OverviewPrevious Work
Our ContributionSummary
Boosting a Chatterbot Understanding with aWeighted Filtered-Popping Network Parser
Javier M. Sastre1,2,3 Jorge Sastre3 Javier García4
1IGM, Université Paris-Est2Grup Transducens, DLSI, Universitat d’Alacant
3iTEAM, Universitat Politècnica de València4Telefónica I+D
4th Language & Technology Conference
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
The MovistarBot
Conversational agent (chatterbot)
Developed in collaboration with TelefónicaR&D
Accessible through MSN Messenger
Dialog based on short text messages inSpanish
Sells mobile services
Provides entertaining conversation
Current status: private beta
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
Sending SMSs
Spanish sentence
quisiera enviar unmensaje
me gustaría enviar unmensaje al móvil 555
envía hola Paco al 555
mándame este mensajeal 555: hola Paco
etc.
Literal translation
I would like to send amessage
I would like to send amessage to the mobile 555
send hello Paco to the 555
send (for me) this messageto the 555: hello Paco
etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
Searching for Downloads
Spanish sentence
¿qué juegos tienes?
me gustaría bajarme elPacman
quiero canciones deMichael Jackson
enséñame las fotos quetengas de gatos
etc.
Literal translation
which games do you have?
I would like to download thePacman (game)
I want songs of MichaelJackson
show me the photos thatyou have of cats
etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
Downloading
Spanish sentence
descárgame elcontenido 7
el 7 es el contenido quequiero
etc.
Literal translation
download (for me) thecontent 7
the 7 is the content that Iwant
etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
Searching for Alerts
Spanish sentence
muéstrame las alertas
enséñame las alertas defútbol
¿qué alertas de fútboltienes?
quiero ver las alertasque tengas disponiblesde fútbol
etc.
Literal translation
show me the alerts
show me the alerts ofsoccer
¿which alerts of soccer doyou have?
I want to see the alerts youhave available of soccer
etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
Subscribing to Alerts
Spanish sentence
quiero suscribirme a la 7
alta alerta 7
etc.
Literal translation
I want to subscribe to the 7
subscribe alert 7
etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The MovistarBotSupported Services
Information Service
Spanish sentence
¿cómo puedo ver la teleen mi móvil?
¿qué es la tarifa planade música?
quiero recargar mitarjeta
etc.
Literal translation
how can I watch TV in mymobile?
what is the flat fee ofmusic?
I want to recharge my (SIM)card
etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
Base Robot
The Base Robot
Developed by Telefónica R&D
Based on AIML [Wallace, 2004]
Command-based service requests (e.g.: sms phonemessage)
Keyword-based recognition of requests in Spanish redirectto command use
Keyword-based general conversation rules forentertainment
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
The Robot Extensions
Developed by us
Spanish-to-command translation of request sentences
Simplification of unrecognized sentences: to infinitive formDivided into two parts:
linguistic data→ grammars & dictionariescomputer programs→ NLP engine & tools
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
Grammars
Local-grammars [Gross, 1997]: exact sentence recognitionRecursive transition networks (RTNs) [Woods, 1970] withoutput:
XML tags determining the service and argumentsWeights determining the best interpretation
Powerful linguistic operators: <V+Trans_msg:Y2:Y3>
RTNs are intuitive graphical objects
RTNs can be reused by other RTNs
RTNs can be constructed with the Unitex (open-source)system [Paumier, 2006]
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
A Local-Grammar Example
<aplicación.N>
<juego.N>
<software.N>
<videojuego.N>
<canción.N>
<música.N>
<tono.N>
<dibujo.N>
<imagen.N>
<foto.N>
<logotipo.N>
<juegos> </juegos>
<musica> </musica>
<fotos>
</fotos>
<archivo.N>
<fichero.N>de
mp3
java
<E>
de
en
<fondo.N> de<escritorio.N>
<pantalla.N>
ocio
recreo
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
The Dictionary
Unitex’s DELAF Spanish dictionaryOpen-source (LGPLLR licensed)Description of inflected formsMore than 600,000 entries
Entry example:envía,enviar.V+Trans_msg:3Ps:2Ys
Addition of new entries
Addition of new semantic classes
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
Computer Programs
Application of RTNs based on a filtered-popping network(FPN) parser [Sastre, 2009a]
Case & diacritic mark insensitiveness
Automatic grammar weighting procedureDictionary tools for:
Dictionary compression and decompression [Revuz, 1992]Linguistic-aware entry extraction:<V+Trans_msg>Linguistic-aware edition of semantic classes:+Trans_msg <enviar.V>+<mandar.V>+ etc.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
Implementation Details
C++ standard & Standard Template Library
Robust code
Developed with Eclipse CDT & g++
Open-source
*nix & Windows compatible
Tomcat servlet interfacing the engine over the Internet
Local use by single user
Remote use by multiple users
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
System Workflow
The user types a sentence in his Messenger
The NLP engine translates or simplifies itThe chatterbot applies its own AIML rules:
complete service requests→ launch serviceincomplete service request→ ask for missingargumentsnot a service request→ apply generalconversation rules
Robot & NLP engine independence
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
System Workflow
The user types a sentence in his Messenger
The NLP engine translates or simplifies itThe chatterbot applies its own AIML rules:
complete service requests→ launch serviceincomplete service request→ ask for missingargumentsnot a service request→ apply generalconversation rules
Robot & NLP engine independence
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
System Workflow
The user types a sentence in his Messenger
The NLP engine translates or simplifies itThe chatterbot applies its own AIML rules:
complete service requests→ launch serviceincomplete service request→ ask for missingargumentsnot a service request→ apply generalconversation rules
Robot & NLP engine independence
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
System Workflow
The user types a sentence in his Messenger
The NLP engine translates or simplifies itThe chatterbot applies its own AIML rules:
complete service requests→ launch serviceincomplete service request→ ask for missingargumentsnot a service request→ apply generalconversation rules
Robot & NLP engine independence
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
The Robot ExtensionsLinguistic DataComputer Programs
Performance
Testing machine: GNU/Linux Debian laptop, Intel PentiumCentrino 2GHz, 2GB RAM
First version (only SMS service and no weights): 2000sentences per second
Last version: 200 sentences per second
Increasing global ambiguity penalizes performance
Possible solution: top-ranked-only interpretation extractor
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
Summary
Robust & powerful NLP engine
Open-source
Exact analysis of texts
Argument extraction
Successfully applied to the MovistarBot use case
Easy to integrate with other applications
Expected to suit larger applications with a top-ranked-onlyinterpretation extractor
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
References I
Blanco, X., 2000.Les dictionnaires électroniques de l’espagnol (DELASs etDELACs).Lingvisticæ Investigationes, 23(2):201–218.
Gross, M., 1997.The construction of local grammars.In Emmanuel Roche and Yves Schabes (eds.), Finite-StateLanguage Processing. Cambridge, MA, USA: MIT Press,pages 329–352.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
References II
Paumier, S., 2006.Unitex 1.2 User Manual .Université de Marne-la-Vallée.http://www-igm.univ-mlv.fr/ unitex/Unitex-Manual.pdf .
Revuz, D., 1992.Minimisation of acyclic deterministic automata in lineartime.Theoretical Computer Science, 92(1):181–189.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser
OverviewPrevious Work
Our ContributionSummary
References III
Sastre, J. M., 2009a.Efficient parsing using filtered-popping recursive transitionnetworks.Lecture Notes in Computer Science, 5642:241–244.
Wallace, R., 2004.The elements of AIML style. ALICE AI Foundation.http://www.alicebot.org .
Woods, W. A., 1970.Transition network grammars for natural language analysis.Communications of the ACM, 13(10):591–606.
Javier M. Sastre, Jorge Sastre, Javier García Boosting a Chatterbot Understanding with a WFPN Parser