© löckelt, becker, pfleger, alexandersson; dfki edilog 2002 workshop jan alexandersson (tilman...

Jan Alexandersson

(Tilman Becker, Markus Löckelt, Norbert Pfleger)

German Research Center for Artificial Intelligence – DFKI GmbHStuhlsatzenhausweg 3, Geb. 43.8 - 66123 Saarbrücken

phone: (+44) 681 302 5347Email: janal@dfki.de

http://www.dfki.de/~janal

Overlay – a non-monotonic operation for discourse processing

What is this Talk about• Dialogue systems must deal with utterances relating

elliptically to previous dialogue

User: „half past five“

System: „at what time would you like to start recording?“

User: „I would like to start recording at half past five“

• ... or utterances only partially related to the previous discourse

System: [Showing a list of films] „Here is a list of films“

User: „what films are on TV tonight?“

User: ``Thats a boring program. I‘d rather go to the movies´´

User: ``Thats a boring program. I‘d rather go to the movies tonight´´

What is this Talk about?

``...an expensive large portrait of the queen by Wainright hanging in the library´´

``and one of the princess over the mantel´´

• There are similar elliptical phenomena like one-anaphora

``and an expensive large portrait of the princess hanging over the mantel´´

Overview

• SmartKom

• Domain Model

• Overlay

• Discourse Modelling

• Action Planning

• Processing Partial Utterances

• Conclusion

What is the challenge?

• General tasks for a dialogue manager– Enrichment:

• Given incomplete information, enrich it from other sources in order to proceed.

– Validation: • Validate hypotheses against the current discourse state

Our Approach

• One general operation – Overlay– Domain representation:

• (Typed) Feature Structures

– Non-monotonic:• Always succeeds

– Computes a score:• Reflecting the structural consistency of the two

arguments of overlay

SmartKom: Task-Oriented Dialogue System

MMDialogue

Back-Bone

Home:Consumer Electronics

Public:Cinema,

Phone, Fax,

Mail, Biometrics

Mobile:Car andPedestrianNavigation

Application

SmartKom-Mobile

SmartKom-Public

SmartKom-Home/Office

Module Overview on the SmartKom Control GUI

The True Story

The DFKI Dialogue Back-Bone

Communication pools

Main data flow

Context information

Analysers

ExternalServices

ModalityFusion

DiscourseModelling

ActionPlanning

PresentationManager

Generators• Speech

• Gesture

• Speech

• Graphics

• Gesture

theater: MovieTheater

movie: Movie

reservationNumber: PositiveInteger

Domain Model• Used for communication in the back-bone

• Frame-based ontology; representation as Typed Feature Structures in M3L (XML)

name: Stringdirector: Person

cast: PersonListyearOfProduction: PositiveInteger…

address: Addressseats: SeatStructure…

CinemaReservation

• Application objects composed of subobjects

• Slots: Feature paths meaningful for the dialogue (entities that can be talked about / referenced to); e.g. movie:director:lastName in a CinemaReservation object

• Slots can recursively contain other slots

firstName: StringlastName: String…

Discourse Modeling• Lattice of intention hypothesis sequences

from Modality Fusion

• Enrichment and Validation

– compare and enrich with a selected number of discourse states:

• fill in consistent information

• compute a score

– for each hypothesis - background pair:• Overlay (covering, background)

• Overlay gives a score representing how well the covering fits the background.

• Scores of overlay, speech interpretation etc. give overall score

• One augmented hypothesis sequence with best score is passed on to Action Planning

Covering:Background:

Intentionlattice

Selectedaugmentedhypothesis

sequence

Example for Overlay with TFS

User: What films are on TV tonight?

System: [presents list of films]

User: That‘s a boring program, I‘d rather go to the movies.

How do we inherit “tonight” ?

The Domain Model

String:...

Time:entEntertainm

beginTime

...Cinema:

ePerformanccinema

...Channel:

Broadcastchannel

A named entertainment at

some time

A named TV program at some

time on some channel

A named Movie at some time at some cinema

Unification Simulation

Films on TV tonight

Broadcast

anychanneltonightTimebeginTime

Go to the movies

:ePerformanc

anycinemaTimebeginTimeFail – type clash

String:...

Time:entEntertainm

beginTime

...Cinema:

ePerformanccinema

...Channel:

Broadcastchannel

""Time:

ePerformanc

anycinematonightbeginTime

Overlay Simulation

String:...

Time:entEntertainm

beginTime

...Cinema:

ePerformanccinema

...Channel:

Broadcastchannel

Go to the moviesFilms on TV tonight

Broadcast

anychanneltonightTimebeginTime

Time:ePerformanc

anycinemabeginTime

..."":

ePerformanc

tonightTimebeginTime

Assimilation

Background

Covering

``Formal´´ Definition Overlay

• Let– co be covering

– bg be background

• Step 1:– Assimilate(co,bg)

• Step 2:– Overlay(co,assimilate(co,bg))

• If co and bg are frames: recursion

• If co is empty: use bg

• If bg is empty: use co

• If conflict: use co

Domain Models with Multiple Inheritance

• Assimilate(co,bg) – Compute the set of minimal upper

bounds (MUB)

– Specialize the MUBs

– Unify the specialized MUBs

• Overlay remains untouched

MUB MUB

Processing Partial User Contributions

• Our Dialogues contain a lot of partial contributions– ``Three´´

– ``Three o‘clock´´

– ``At three o‘clock´´

– ``At three [] o‘clock´´

– ``Later´´

– ``Information []´´– ...

• Requirement– Advanced discourse modelling and action planning

– Bidirection communication DiM - AP

Discourse Modelling - Three-Tiered Context Representation

DO1 DO2

DO3 DO9

Modality layer

Discourse layer

System: Here [pointing gesture] I show you a list of films running in Heidelberg.

show heidelberglist

LO2 LO3. . .

Domain layer DomainObject1

ticket first

DO11 DO12

reserve

LO4 LO5 LO6

DomainObject2

User: Reserve a ticket for the first!

Action Planner – Regression Planning

• AP processes plan operators– Uniform description of mixed initiative with user or applications based on

Games and Moves. Example:• User: request – System: response

• System: request – Application: response

• Processing steps

1. Constructs a plan from plan operators– A Plan specifies a partial order of games

2. Interprets the plan– Execute the games

• Expectations are published during execution of the plan– Guide the interpretation of input

Action Planning - Regression planning

provides(Y)needs(D)

condition(D=x)

Goalprovides()

needs(X,Y)provides(X)needs(A,B)

provides(C)needs()

provides(B)needs(D)

provides(A)needs(C)

provides(F,G)needs(H)

provides(D)needs(E)

For Example: Slot D=y

Current Stateprovides(?)needs(?)

successorstates

AP - Example Plan: VCR

Goal: VCR_record

VCR_checkMedium

VCR_getChannel

VCR_getStartTime VCR_getEndTime

VCR_rewindMedium

Action Planning

• Initiation of sub-dialogues to gather necessary information

• Order proposed by the system, but the user may not follow the trail

• Several cases of user responses to requests:– Providing a matching answer,

– Supplying information not (yet) asked for,

– Changing already established information

– Acting uncooperatively or changing goals

Expectations

• When AP initiates a sub-dialogue, it expects a certain answer

• An Expectation structure is published to help the analysis modules– Expected Slots: corresponding to the request

– Possible Slots: plausible in context

– Filled Slots: already provided

– The currently active Application

• Allows for a fine-grained distinction between types of possible utterances

• Discourse Modelling can employ different rankings for each category

Integration of Partial

• Partial Utterances are analyzed as subobjects– validation and enrichment include the integration into the

preceding context (to achieve a coherent discourse)

– we consider three types of partial utterances: 1. The system has the initiative and the user responds elliptically to a

system request

2. An elliptical user utterance does not correspond to a request but can be interpreted in context

3. Misinterpretation or non-cooperative user behaviour

Integration of Partial - Possible Situations

User InitiativeSystem Initiative

Expected slot N A

expected,

unify,

plan continues

Possible slot

plausible,

unify,

replanning

plausible,

unify,

replanning

Filled slot

plausible,

overlay,

replanning

plausible,

overlay,

replanning

Other implausible – recover strategies

User contribution

DiM processing

AP processing

DiM - Processing User Responses

• User provides expected slots– Bridging:

• a new application object is created (of same type as the current application object)

• for each subobject in the hypotheses its type is compared to the expected slots

– matching subobjects are integrated into the new application object

– remaining subobjects are interpreted as plausible partial utterances (next slide)

– Overlay:• The extended application object is overlayed over the application object

currently in focus

Processing Plausible Partial Utterances

• User provides possible slots– processed like expected slots (but get a score penalty)

• User changes filled slots– search for a discourse object DO of the same type in the discourse

context

– create application object AO1 of the same type as the application object AO2 within which DO was mentioned

– overlay AO1 over AO2

– gets a score penalty based on position in discourse history

• If integration fails, all subobjects are passed on unchanged (integration fails)

Example: Expected and Possible Slots• (1) User: I want to record a film.

(2) System: When should I start recording?

(3) User: 1:30pm on channel two.

• Analysis of (1) contains the setting of a goal VCR_record (empty application object of type VCR introduced)

• AP triggers presentation (2) and publishes an expectation containing: – VCR as the type of the current application object

– VCR_startTime as the (only) expected slot

– some possible slots (e.g., VCR_endTime and VCR_channel)

– and maybe some filled slots (e.g. VCR_mediumPresent)

• Analysis of (3) contains two subobjects– the expected slot is filled by “1:30pm”

– but also one possible slot is filled by “channel two”

Example: Filled Slots

(1) User: What is currently running on TV?

(2) System: The following [Smartakus points at a list of programmes] programmes are currently running.

(3) User: And tonight?

Example: Filled Slots

show program currently

avMediumavMedium

timebeginTime

broadcast

ctdomainObje

hypothesis

Modality layer

Discourse layer

Domain layer

DO1 DO2

LO1 LO2

""tonight

avMediumavMedium

tonight

timebeginTime

broadcast

ctdomainObje

hypothesis

tonight

Conclusion

• Uniform, flexible and robust mechanism for interpreting possible partial utterances

• Pragmatical classification of partial utterances from the point of view of the action planner

• Modality independent

• Approach integrates seamlessly with other aspects of discourse processing, e.g. anaphora resolution

• Implemented and running in, e.g., a large multimodal dialogue system

Papers

• Jan Alexandersson and Tilman Becker. Overlay as the Basic Operation for Discourse Processing in a Multimodal Dialogue System. In: Proceedings of the IJCAI Workshop ``Knowledge and Reasoning in Practical Dialogue Systems,´´ Seattle, 2001.

• Norbert Pfleger, Jan Alexandersson, and Tilman Becker. Scoring Functions for Overlay and their Application in Discourse Processing. In Proceedings of ``KONVENS 2002,´´ Saarbrücken, Germany, 2002.

• Markus Loeckelt, Tilman Becker, Norbert Pfleger and Jan Alexandersson Making Sense of Partial. In: Bos, Foster & Matheson (eds): ``Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002),´´ Edinburgh, UK, Pages 101-107.

• Jan Alexandersson and Tilman Becker. The Formal Foundations Underlying Overlay. Submitted to IWCS5

Thank you very much for your attention!Merci infiniment de votre attention!Babelfish.altavista.com

© löckelt, becker, pfleger, alexandersson; dfki edilog 2002 workshop jan alexandersson (tilman...

Documents

aufgaben und rolle klären: qualifizierungsmodul für...

living water viktor schauberger and the secrets of natural...

notice · becker, bill becker, cynthia k becker, daniel r...

geschützte zwingernamen im verein für deutsche ... ·...

session 49 gunnar alexandersson

wendige pfleger - selbst.de

per alexandersson thesis

becker school graphics-casestudy€¦ · becker school...

facebook fanpage: erik becker criminologia canal no youtube:...

becker control valve products - tri-state meter...

alexandersson - living water - viktor schauberger and the...

session 31 gunnar alexandersson

tove alexandersson order sheet - suuntajyvaskyla.fi · tove...

becker professional education - acca study material order...

session 42 gunnar alexandersson

reid & hughes assessment, becker+becker, 2012

mikael alexandersson gÖteborgs universitet

geburtsregister...

die schwester - der pfleger nr. 04.2011

session 33 gunnar alexandersson