architectures and standards for ivas at the social cognitive systems group h. van welbergen, k....
TRANSCRIPT
ARCHITECTURES AND STANDARDS FOR IVAS AT THE SOCIAL
COGNITIVE SYSTEMS GROUP
H. van Welbergen, K. Bergmann, H. Buschmeier, S. Kahl,
I. de Kok, A. Sadeghipour, R. Yaghoubzadeh, S. Kopp
MAIN RESEARCH THEMES
How can cognitive systems be designed as intelligent, socially adept interaction partners that allow a fluent and coordinated interaction with humans
Developing methods to model the behavioral, preceptual motor and cognitive mechanisms of embodied human-like communication and cooperation
Application and evaluation in IVAs
Scenarios: Embedding IVAs in mouse-keyboard interaction Virtual coaches Virtual assistents for the elderly and cognitively impaired Cognitive models that investigate semantic speech/gesture
coordination Computational models of dialog coordination based on linguistic
feedback
CURRENT ARCHITECTURE AND STANDARDS
Our components: A multimodal memory component realized as a spreading
activation model of semantic coordination for speech-gesture production (Bergmann et al., 2013)
Incremental NLG based on SPUD (Buschmeier et al., 2012) A Behavior Planner for iconic gestures (Bergmann and Kopp,
2009) A BML 1.0 Realizer for incremental and adaptive behavior
(van Welbergen et al. 2014) An IS-based incremental dialog manager, capable of handling
uncertain input (yet unpublished)
Several external components Often multiple alternatives available, allowing trade-offs
between recognition/synthesis quality, reactivity and control
CURRENT ARCHITECTURE AND STANDARDS
Combining components We follow the SAIBA reference architecture
BML 1.0 for realization, extensions for incrementality and adaptivity
No standardized communication (via FML) between the Intent and Behavior Planner
Components Run on various programming languages, OSs, computers Allow incremental processing IPAACA
Implements the Incremental Unit architecture (Schlangen and Skantze, 2011)
Embeds it in a message oriented middleware (RSB; Wienke and Wrede 2011)
FUTURE ARCHITECTURES AND STANDARDS: LONG TERM Base our architecture on universal (less problem
specific) and cognitively motivated principles
Ongoing work: Fully incremental production an recognition and
production, e.g. in attentive speaking (Buschmeier et al 2012)
Representation and decision making under uncertainty Using uncertainty as valuable information rather than as a
mere nuisance Priming and imitation learning of gesture (Sadeghipour
and Kopp, 2011)
FUTURE ARCHITECTURES AND STANDARDS: SHORT TERM
SAIBA provides us: A common terminology for large scale IVA components A standardized interface for behavior realization
We would like to enhance it to a full reference architecture for IVAs
Our requirements Handle both input processing and output generation Coordination between these at multiple levels Incremental input and output processing Representing memory Representing uncertainty
FUTURE ARCHITECTURES AND STANDARDS
Asap(Kopp et al., 2014)
SUGGESTIONS FOR DISCUSSION: A NEW REFERENCE ARCHITECTURE FOR IVAS Our requirements
Handle both input processing and output generation Coordination between these at multiple levels Incremental input and output processing Representing memory Representing uncertainty
Requirements of others Representing and interacting with ‘the world’ Generalization to multi-agent/multi-human
SUGGESTIONS FOR DISCUSSION: A NEW REFERENCE ARCHITECTURE FOR IVAS Desired outcome:
Prioritized list of requirements, plan to incrementally embed them
Definition of shared terminology Architecture sketch, identification of shared interfaces Agenda for shared interfaces between modules
Workshops/plans of progress for BML, FML, PML, …
SUGGESTIONS FOR DISCUSSION: SHARING AND COMBINING SMALLER COMPONENTS Many interesting ‘small’ IVA components are
presented every year In isolated projects/experiments
How can these be embedded in the larger effort of developing a full IVA?
We are especially interested in doing this for the Behavior Planner The challenge is not only in implementing individual
parts that are potentially useful but also in combining the components
SUGGESTIONS FOR DISCUSSION: SHARING AND COMBINING SMALLER COMPONENTS Desired outcome:
(Start of) design pattern catalogue for the design of (parts of) a behavior planner E.g. MAS, blackboard, MOM, IU-architecture, subsumption
architecture Ideas to share smaller components
List of good candidate components for sharing Inspired by best practices from related fields (e.g. ROS:
Quigley et al., 2009) Can we develop standardized components using challenges
for each of them?
SUGGESTIONS FOR DISCUSSION: CHALLENGES
Goal: Foster the development and comparison of reusable IVA
components
Different from the goals of IVA’s GALA Provide a forum to present the state-of-the art in
technology of virtual humans and innovative applications
Aimed to stimulate (under)graduate student participation
Disseminate demos in an ever evolving gallery for study and reuse In practice the “dissemination” was limited to a video
SUGGESTIONS FOR DISCUSSION: CHALLENGES
Desired outcome: GALA post-mortem Best practices of successful challenges in related fields
RoboCup@Home, Blizzard TTS, Give NLG, INTERSPEECH, … Requirements/guidelines for a challenge Plan to embed challenge in a workshop/conference
(IVA?) Example design (sketch) of a first challenge
THANKS FOR YOUR ATTENTION
Questions?
SUGGESTIONS FOR DISCUSSION: CHALLENGES
Challenges in related fields: RoboCup@Home, Blizzard TTS, Give NLG, INTERSPEECH, …
Some interesting ideas from these: Shared set of ‘resources’, fosters comparable system building
E.g. recorded voices to build off (Blizzard), shared environment to navigate through (Give, RoboCup@Home), shared corpus (INTERSPEECH)
Given by the organizers of the challenge Predefined set of automatically obtained evaluation criteria (GIVE,
INTERSPEECH) Baselines (INTERSPEECH)
‘Sharing’ subjects (Blizzard, RoboCup@Home) Both the challenge and the submissions are full-fledged, peer-
reviewed papers (INTERSPEECH challenges, Blizzard challenges) Mandatory (but delayed) publishing of source code and technical
documentation (RoboCup@Home)
SUGGESTIONS FOR DISCUSSION: SHARING AND COMBINING SMALLER COMPONENTS Inspiration from related fields: ROS (Quigley et al.,
2009) Provides over 3000 robotics components Provides the ‘plumbing’ of robotic systems
Connecting components: Publish subscribe, rpc, actions, distributed parameter setting,
IDL’s for messages, starting components, central time, … Debugging connections: visualization, recording/playback
Building and distributing components: Tools for building and distributing ROS-packages
We have reinvented the wheel on several of these… Some adhering to conventions required