toyoaki nishida kyoto university · prologue communicative intelligence super intelligence people....
TRANSCRIPT
Artificial Intelligence Adv., April 9, 2014
Copyright © 2014, Toyoaki Nishida, Atsushi Nakazawa, Yoshimasa Ohmoto, Yasser Mohammad, At ,Inc. All Rights Reserved.
1. Introduction
Toyoaki NishidaKyoto University
Conversational Informatics: a field of study that centers on understanding and augmenting conversation
Prologue
Communicative Intelligence
Super Intelligence People
Challenge: A robot that can participate in conversation
Long-term goal
Eye gaze
Hand gesture PostureFacial expression
AskingNegotiating
Proposing
ConvivialitySocial networksTrust
Conversation is a complex business
Long-term goal: primordial soup of conversation
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Application
Platform Evaluation
Content production Model building
Analysis
Theory
Measurement
Conversational interactions
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Understanding and Designing Interaction
What is AI?What is AI?
Artificial intelligence
Artificial mind
Example→ Enthiran (The Robot), 2010
AI in the moviesYear Title Who Wikipedia
1968 2001: A Space Odyssey HAL9000 http://ja.wikipedia.org/wiki/2001%E5%B9%B4%E5%AE%87%E5%AE%99%E3%81%AE%E6%97%85
1977 Star Wars C‐3PO, R2‐D2 http://ja.wikipedia.org/wiki/%E3%82%B9%E3%82%BF%E3%83%BC%E3%83%BB%E3%82%A6%E3%82%A9%E3%83%BC%E3%82%BA%E3%83%BB%E3%82%B7%E3%83%AA%E3%83%BC%E3%82%BA
1982 Blade Runner Replicants http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%AC%E3%83%BC%E3%83%89%E3%83%A9%E3%83%B3%E3%83%8A%E3%83%BC
1984 The Terminator The terminator http://ja.wikipedia.org/wiki/%E3%82%BF%E3%83%BC%E3%83%9F%E3%83%8D%E3%83%BC%E3%82%BF%E3%83%BC_(%E6%98%A0%E7%94%BB)
1987 RoboCop RoboCop (cyborg) http://ja.wikipedia.org/wiki/%E3%83%AD%E3%83%9C%E3%82%B3%E3%83%83%E3%83%97
1993 War Games WOPR: War Operation Plan Response
http://ja.wikipedia.org/wiki/%E3%82%A6%E3%82%A9%E3%83%BC%E3%83%BB%E3%82%B2%E3%83%BC%E3%83%A0_(%E6%98%A0%E7%94%BB)
1994 Disclosure Angel http://ja.wikipedia.org/wiki/%E3%83%87%E3%82%A3%E3%82%B9%E3%82%AF%E3%83%AD%E3%83%BC%E3%82%B8%E3%83%A3%E3%83%BC_(%E6%98%A0%E7%94%BB)
1998 Bicentennial Man Andrew, a new NDR‐114 robot http://ja.wikipedia.org/wiki/%E3%82%A2%E3%83%B3%E3%83%89%E3%83%AA%E3%83%A5%E3%83%BCNDR114
1999 The Matrix Computer http://ja.wikipedia.org/wiki/%E3%83%9E%E3%83%88%E3%83%AA%E3%83%83%E3%82%AF%E3%82%B9_(%E6%98%A0%E7%94%BB)
2001 A.I. Artificial Intelligence David , achild Mecha http://ja.wikipedia.org/wiki/A.I.
2002 Minority ReportInsect robots(user interface is interesting,too)
http://ja.wikipedia.org/wiki/%E3%83%9E%E3%82%A4%E3%83%8E%E3%83%AA%E3%83%86%E3%82%A3%E3%83%BB%E3%83%AA%E3%83%9D%E3%83%BC%E3%83%88
2004 I, ROBOT V.I.K.I . , Sonny http://ja.wikipedia.org/wiki/%E3%82%A2%E3%82%A4,%E3%83%AD%E3%83%9C%E3%83%83%E3%83%88
2009 ATOM Astro boy Astro boy http://ja.wikipedia.org/wiki/ATOM_(%E6%98%A0%E7%94%BB)
2009 Avatar (tele‐existence) http://ja.wikipedia.org/wiki/%E3%82%A2%E3%83%90%E3%82%BF%E3%83%BC
2009 Surrogate (tele‐existence) http://ja.wikipedia.org/wiki/%E3%82%B5%E3%83%AD%E3%82%B2%E3%83%BC%E3%83%88_(%E6%98%A0%E7%94%BB)
2014 Transcendence (Technical singularity) http://ja.wikipedia.org/wiki/%E3%83%88%E3%83%A9%E3%83%B3%E3%82%BB%E3%83%B3%E3%83%87%E3%83%B3%E3%82%B9
[Nishida 2012b]
Year AI ICT1940~ 1936: Turing Machine, 1947: von Neumann Computer, 1948: Information
Theory, by C. Shannon and W. Weaver, 1948: Cybernetics by Wiener
1950~ 1952‐62: Checker program by A.Samuel1956: Dartmouth Conference 1957: FORTRAN by J.Backus
1960~ 1961: Symbolic Integration program SAINT by J.Slagle1962: Perceptron by F.Rosenblatt1966: The ALPAC report against Machine Translation by R. Pierce1967: Formula Manipulation System Macsyma by J.Moses1967: Dendral for Mass Spectrum Analysis by E.Feigenbaum
1961: Mathematical theory of Packet Networks by L. Kleinrock1963: Interactive Computer Graphics by I.Sutherland
1968: Mouse and Bitmap display for oN Line System (NLS) by D.C.Engelbart1969: ARPA‐net
1970~ 1971: Natural Language Dialogue System SHRDLU, by T.Winograd1973: Combinatorial Explosion problem pointed out in The Lighthill report1974: MYCIN by T.ShortliffeMid 1970’s: Prial Sketch and Visual Perceptron by D.Marr1976: Automated Mathematician (AM) by D.Lenat1979: Autonomous Vehicle Stanford Cart by H.Moravec
1970: ALOHAnet1970: Relational Database Theory by E.F.Codd1972: Theory of NP‐completeness by S.Cook and R.KarpMid 1970’s: Alto Machine by A.Kay and A.Goldberg1976: Ethernet1979: Spreadsheet Program Visicalc by D.Bricklin
1980~ 1982: Fifth Generation Computer Project1984: The CYC Project by D.LenatMid 1980’s: Back‐propagation algorithm was widely used1985: the Cybernetic Artist Aaron by H.Cohen1986: Subsumption Architecture by R.Brooks1989: An Autonomous Vehicle ALVINN by D.Pomerleau
1982:TCP/IP Protocol by B.Kahn and V.CerfMid 1980’s: First Wireless Tag Products1987: UUNET started the Commercial UUCP Network Connection Service1988: Internet worm (Morris Worm)1989: World Wide Web by T.Berners‐Lee1989: The number of hosts on the Internet has exceeded 100,000.
1990~ 1990: Genetic Programming by J.R.KozaEarly1990’s: TD‐Gammon by G.TesauroMid 1990’s: Data Mining Technology1997: DeepBlue defeated the World Chess Champion G.Kasparov1997: The First Robocup by H.Kitano1999: Robot pets became commercially available
1992: The number of hosts on the Internet has exceeded 1,000,000.1994: Shopping malls on the Internet1994: W3C was founded by T. Berners‐Lee1997: Google Search1998: XML1.0(eXtensible Markup Language) by W3C1998: PayPal
2000~ 2000: Honda Asimo
2004: The Mars Exploration Rovers (Spirit & Opportunity)
2001: Wikipedia.2003: Skype / iTunes store2004: Facebook2005: YouTube / Google Earth2006: Twitter2007: Google Street View
2010~ 2010: Google Driverless Car / Kinect2011: IBM Watson Jeopardy defeated two of the greatest champions2012: Siri
History of AI research in contrast with ICT
1960 Prof. Toshiyuki Sakai’s group at Kyoto
1961 Sonotype: phonetic typewriter by Shuji Doshita
1970 The world’s first face recognition system by Takeo Kanade
1979 Japanese Word Processor JW-10 by Toshiba
1997 1st Robocup games and conference at Nagoya, Japan
1979 IJCAI 79 (Tokyo)
1997 IJCAI 97 (Nagoya)
1982-1992 Fifth Generation Project1986 JSAI (Japanese Society for AI)
1990 1st PRICAI (Nagoya, Japan)
1970’s Research groups for AI were formed in Tokyo, Osaka, Kyushu
1960’s Image Understanding and NLP by Makoto Nagao
Successful Topics of AI
Large-scale Search Knowledge-based Systems Language, Speech, Vision Planning Machine Learning and Data Mining Using AI in Creating Works of Art
AARON (1985)
The Mars Exploration Rovers (2003)
Experiments in Musical Intelligence (1987)
DeepBlue (1997)http://www.research.ibm.com/deepblue/
http://marsrovers.jpl.nasa.gov/mission/spacecraft_surface_rover.html
http://www.kurzweilcyberart.com/
http://arts.ucsc.edu/faculty/cope/http://artsites.ucsc.edu/faculty/cope/mp3page.htm
IBM Watson (2011)http://www-03.ibm.com/innovation/us/watson/index.shtml
ALVINN (An Autonomous Land Vehicle in a Neural Network) on Navlab (1989)http://dl.acm.org/citation.cfm?id=89891
Recent Trends
1997: Deep Blue defeated G. Kasparov… Source: IBM Deep Blue, wikipedia
1997: Official opening of Robocup (H. Kitano)… Source: robocup.org, robocup.or.jp, Wikipedia
1997: Mars Pathfinder… Source: NASA Mars Pathfinfer, Wikipedia
1999: Robot pet SONY AIBO… Source: SONY AIBO, Wikipedia
1999: OpenCV Project… Source: OpenCVWiki,wikipedia
2000: Honda ASIMO… Source: Honda ASIMO, Wikipedia
2004: Mars Exploration Rovers… Source: NASA‐JPL
2010: Google Driverless Car… Source: Wikipedia, Sebastian Thrun’s home page, 3P
2010: Kinect… Source: xbox.com, Wikipedia
2011: IBM Watson defeated two Jeopardy! champions… Source: IBM Watson, Wikipedia
2011: iPhone Siri… Source: Apple, Wikipedia
2011: Google Voice Search… Source: Google
2012: Zen Takemiya Masaki with 4 stones … Source: Sig ECS, UEC2012: Google Glass Project
… Source: Project Glass2012: NTT DOCOMO’s Shabette Concier
… NTT DOCOMOOther: Smarter than You Think (New York Times)
Landmarks
‐ From philosophy to science‐technology
‐ Toy problems to the real world
‐ Powerful tools
‐ Grand challenges and competitions
Traditional AI
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Dark side of super intelligenceDark side of super intelligence
Technology abuse
Responsibility flaw
Moral in crisis
Over‐dependency on technology
(Technological) singularity(Technological) singularity
The day when machine intelligence surpasses human intelligence.
IEEE Spectrum June 2008 issuehttp://spectrum.ieee.org/biomedical/ethics/signs‐of‐the‐singularity
“The AI Scenario: We create superhuman artificial intelligence (AI) in computers.”
“The IA Scenario: We enhance human intelligence through human‐to‐computer interfaces‐‐that is, we achieve intelligence amplification (IA).”
The fear of utopia
So what?So what?
Dismiss as nonsense.
Neo‐Luddism.
Invent a better solution.
Endow AI with empathic capabilities.
Communicative Intelligence for Bridging People and CI
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Communicative Intelligence
PeopleSuper Intelligence
Towards Empathic Agents
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
DeepBlue Entertain with a game
IBM Watson Entertain with a game
Siri Conversation partner
(AI that can pass entrance exam) Effective and affective tutor
Traditional AI:High competence
Future AI:High empathy
The ability to understand others’ emotions and/or perspectives and, often, to resonate with others’ emotional states.
An affective response that is identical, or very similar, to what the other person is feeling or might be expected to feel given the context: a response stemming from an understanding of another’s emotional state or condition.
or …
[Eisenberg 2010]
Empathy
The more common ground is shared, the more empathy will be gained.
... the universe of discourse, first‐person view, knowledge and skills, the communication style and rituals, the value system, ...
[Nishida 2013c]
Sharing hypothesis
A road to empathic agents
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Empathy
Sharing hypothesis
Common ground Conversational intelligence
Primordial soup of conversation
Engagement
Research platform
(a) IMADE – A real-world interaction measurement, analysis and design environment
[Sumi 2010a; Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Research platform
(b) ICIE – immersive collaborative interaction environment
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
[1] C. Nitschke, 2013.
Synthesized Real World
Google Street View Navigation on Immersive 360° Display [1]Youtube, KyotoUniNishidaLab http://youtu.be/V‐9SKpcMrzk
[1] C. Nitschke, D. Lala, 2013.
Synthesized Real World
Remote Meeting and Interaction in Immersive Shared Environment [1]Youtube, KyotoUniNishidaLab http://youtu.be/GD4XlH_nOyo
[1] D. Lala, Y.F.O. Mohammad, T. Nishida. Unsupervised gesture recognition system for learning manipulative actions in virtual basketball. Proc. ICHAI (2013).
Virtual Basketball
Virtual Basketball [1]Youtube, KyotoUniNishidaLab http://youtu.be/ZtjSRjHBgUs
Immersive WOZ environment
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Immersive WOZ environment
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Learning by imitation
[Mohammad 2009]
Learning by imitation
[Mohammad 2009]
Constrained Motif Discovery: • Given a time series X(t)
find recurring patterns of length between L1 and L2 using distance function D subject to the constraint P(t), where P(t) is an estimation of the probability that a motif occurrence exists near time step t.
[Mohammad 2009]
P(t)
unlikely
likely
Change point discovery
Learning by imitation
Future
Change angle
GH
Past Futuret
;...; 1H t seq t n seq t 1 ;...;G t seq t seq t n
1
1
1
ˆ
f
f
Ti i i
l
i ii
l
ii
s t t t
csx
c
t
TtVtStUtH )()()()( Find optimal lP
ggT uutGtG )()(Find optimal lF
11and,)( jjjFg
ii liut
fTll
Tll
i litUUtUUt ,)()()(
)()()()()(ˆ)(~ tttttxtx PFPF
Learning by imitation
Robust Singular Spectrum Transform
[Mohammad 2009]
[Yano 2012]
3D conversation capture
Collaborative annotation system
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Corneal Imaging
Corneal reflections. (a) The cornea. (b) The corneal limbus is the surface shape discontinuity, where the cornea dissolves into the sclera. (c) A reflected office environment. (d) Focus on iris texture instead of corneal reflections. (e),(f) Examples of corneal images and corresponding scene images.
The cornea of the human eye acts like a mirror that partially reflects incident light from a person’s environment over a large field of view (>200 degree), which can be well noticed when looking at a person’s eye. These corneal reflections can be extracted from an image of the eye by modeling the eye‐camera geometry as a catadioptric (mirror + lens) imaging system.
[Nakawaza-Nitschke 2013]
Corneal Imaging
(c) First prototype of a corneal imaging camera.[Nakawaza-Nitschke 2013]
Experimental settings for evaluating the facilitative agent
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Conversation quantization as conceptual foundation
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
A: That’s Diamond Head.
C: I went there with my family in 1985.
B: I know. That is a volcano.
Conversation quantization as conceptual foundation
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Ground
A: <pointing M>That’s Diamond HeadB: <nod> I know. That’s is a volcano.C: <gaze at M>I wend there with my family in 1985
Interaction
A: participant
C: participantB: participant
M: mountain
DiscoursePreceding / Succeeding / Related
(b) Conversation quantum(a) Conversation scene
(c) (Hypothetical) observer
Producing a conversation quantum
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Schema
Conversation quantum
Schemata dictionary
Segmentation and transcription
A: That’s Diamond Head.
C: I went there with my family in 1985.
B: I know. That is a volcano.
InteractionA: <pointing M>That’s Diamond HeadB: <nod> I know. That’s is a volcano.C: <gaze at M>I wend there with my family in 1985
Discourse
Preceding / Succeeding / Related
A: participant
C: participantB: participant
M: mountain
Ground
Schemata‐based recognition
Consuming a conversation quantum
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
P: That’s Diamond Head.
Q: I know. That is a volcano.
Conversation quantum
Schemata‐based recognition
Schemata dictionary
Agent controller
Dialogue manager
Schema
InteractionA: <pointing M>That’s Diamond HeadB: <nod> I know. That’s is a volcano.C: <gaze at M>I wend there with my family in 1985
Discourse
Preceding / Succeeding / Related
A: participant
C: participantB: participant
M: mountain
Ground
P: That’s Diamond Head.
1. This course centers on conversations.2. Why conversations? ‐> Foundation of thought and communication3. Why do people converse with each other? ‐> As a part of social
interaction, for creating and maintaining stories, for fun.4. The complexity of conversations: ‐> Multi‐level, multi‐layered, polysemy,
polymorphism, coordination of multi‐modality.5. People are not only proficient in expressing ideas but also skillful in
interpreting utterances, thereby they learn from each other.6. Building empathic agents helps people effectively communicate with
each other by conversations.7. Conversational informatics addresses science and engineering of
communicative intelligence.8. Conversational intelligence can be counted as an important branch of
artificial intelligence.9. We will draw on a data‐intensive approach. 10. Conversation quantization as a foundation of data‐intensive approach
to conversational informatics.
Summary
Agenda
Credits:Will be awarded based on one or more reports on subjects given at the class.
Calendar (tentative)
1. Introduction (April 9th) 2. History of Conversational System Development (April 16th) 3. Methodologies for Conversational System Development (April 23rd) 4. Smart Conversation Space (April 30th)5. Measuring, Analysis and Modeling (May 7th) 6. From Observation to Interaction (May 14th) 7. Application of Simulation and Imitation for Interaction Learning (May 21st)8. Cognitive Design for Discussion Support (May 28th)9. Computer Vision Techniques for Conversational Interaction‐1 (June 4th)10. Computer Vision Techniques for Conversational Interaction‐2 (June 11th)11. Affective Computing (June 25th) 12. Language Use – 1 (July 2nd) 13. Language Use ‐ 2 (July 9th)14. Speaking Turn Taking System (July 16th)
References
[Eisenberg 2010] Eisenberg, N., Eggum, N., & Di Giunta, L. Empathy-related responding: Associations with prosocial behavior, aggression, and intergroup relations. Social Issues and Policy Review, 4(1), 143–180, 2010.
[Nishida 2012a] Toyoaki Nishida: The Best of AI in Japan - Prologue. AI Magazine 33(2): 108-111, 2012http://www.aaai.org/ojs/index.php/aimagazine/article/view/2358/2288
[Nishida 2012b] Toyoaki Nishida: Artificial intelligence research in the second half century, Journal of Information Processing and Management 55(7): 461-471, 2012 (in Japanese)http://dx.doi.org/10.1241/johokanri.55.461
[Nishida 2013a] Toyoaki Nishida: What’s AI, Journal of Japanese Society for Artificial Intelligence 28(2): 326-335, 2013 (in Japanese)[Nishida 2013b] Toyoaki Nishida: Conversation Quantization as a Foundation of Conversational Intelligence. DNIS 2013: 230-245, 2013.
http://dx.doi.org/10.1007/978-3-642-37134-9_18[Nishida 2013c] Nishida, T. Toward mutual dependency between empathy and technology. AI SOC, 28(3), 277–287, 2013.[Nishida-Nakazawa-Ohmoto-Mohammad 2014] Toyoaki Nishida, Atsushi Nakazawa, Yoshimasa Ohmoto, Yasser Mohammad, Conversational
Informatics – A Data-Intensive Approach with Emphasis on Nonverbal Communication --, Springer, in press.[Nitschke 2013] Christian Nitschke, Atsushi Nakazawa and Toyoaki Nishida, " I see what you see: Point of Gaze Estimation from Corneal Images",
Asian Conference on Computer Vision (ACPR2013), 2013.[Sumi 2010a] Sumi, Y., Yano, M., & Nishida, T. Analysis environment of conversational structure with nonverbal multimodal data. In International
Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (pp. 44:1–44:4). New York, NY, USA: ACM, 2010.
[Mohammad 2009] Yasser Mohammad, Toyoaki Nishida, Shogo Okada. Unsupervised Simultaneous Learning of Gestures, Actions and theirAssociations for Human-Robot Interaction, in: Proc. IROS 2009: The 2009 IEEE/RSJ International Conference on Intelligent RObots and Systems, pp. 2537-2544, 2009.
[Yano 2012] Yano M. Construction of 3-dimensional recording environments for multi-party conversation with RGB-depth sensors. Master thesis, Department of Intelligence Science and Technology, Kyoto University (in Japanese)