Adaptive Dialogue Systems - Interaction with Interact

Kristiina Jokinen and Antti Kerminen and Mauri KaipainenMedia Lab, University of Art and Design HelsinkiHameentie 135 C, FIN-00560 Helsinki, Finland{kjokinen|akermine|mkaipain}@uiah.fi

Tommi Jauhiainen and Graham WilcockDepartment of General Linguistics, University of Helsinki

FIN-00014 University of Helsinki, Finland{tsjauhia|gwilcock}@ling.helsinki.fi

Markku Turunen and Jaakko HakulinenTAUCHI Unit, University of Tampere

FIN-33014 University of Tampere, Finland{mturunen|jh}@cs.uta.fi

Jukka Kuusisto and Krista LagusNeural Networks Research Centre, Helsinki University of Technology

P.O.9800 FIN-02015 HUT, Finland{krista|jkuusist}@james.hut.fi

Abstract

Technological development has madecomputer interaction more commonand also commercially feasible, andthe number of interactive systems hasgrown rapidly. At the same time, thesystems should be able to adapt to var-ious situations and various users, so asto provide the most efficient and help-ful mode of interaction. The aim ofthe Interact project is to explore nat-ural human-computer interaction andto develop dialogue models which willallow users to interact with the com-puter in a natural and robust way. Thepaper describes the innovative goals ofthe project and presents ways that theInteract system supports adaptivity ondifferent system design and interactionmanagement levels.

1 Introduction

The need for flexible interaction is apparent notonly in everyday computer use, but also in vari-ous situations and services where interactive sys-

tems can diminish routine work on the part ofthe service provider, and also cater for the userswith fast and tailored access to digital infor-mation (call centers, help systems, interactivebanking and booking facilities, routing systems,information retrieval, etc.).

The innovative goal of the Finnish Interactproject is to enable natural language interac-tion in a wider range of situations than has beenpossible so far, and in situations where its usehas not been functional or robust enough. Thismeans that the systems should support rich in-teraction and also be able to learn and adapttheir functionality to the changing situation. Italso implies that the needs of special groups willbe taken into account when designing more nat-ural interactive systems. Within the current sys-tem, such scenarios can e.g. include an intelli-gent bus-stop which allows spoken and text in-teraction concerning city transportation, with asign language help facility.

The project addresses especially the problemof adaptivity: the users are situated in mo-bile environments in which their needs, activitiesand abilities vary. To allow the users to expresstheir wishes in a way characteristic to them and

Philadelphia, July 2002, pp. 64-73. Association for Computational Linguistics. Proceedings of the Third SIGdial Workshop on Discourse and Dialogue,

the situation, interaction with the system shouldtake place in a robust and efficient manner, en-abling rich and flexible communication. Natu-ral language is thus the preferred mode of in-teraction, compared to graphical interfaces forexample. Adaptivity also appears in the tech-niques and methods used in the modelling ofthe interaction and the system’s processing ca-pabilities. An important aspect in this respectis to combine machine learning techniques withrule-based natural language processing, to in-vestigate limitations and advantages of the twoapproaches for language technology.

In this paper we focus on adaptivity whichmanifests itself in various system properties:

• agent-based architecture

• natural language capability

• self-organising topic recognition

• conversational ability.

The paper is organized as follows. We firstintroduce the dialogue system architecture. Wethen explain how the modules function andaddress the specific design decisions that con-tribute to the system’s adaptivity. We concludeby discussing the system’s capabilities and pro-viding pointers for future work.

2 Agent-based architecture

To allow system development with reusablemodules, flexible application building and easycombination of different techniques, the frame-work must itself be designed specifically to sup-port adaptivity. We argue in favour of a sys-tem architecture using highly specialized agents,and use the Jaspis adaptive speech applicationframework (Turunen and Hakulinen, 2000; Tu-runen and Hakulinen, 2001a). Compared to e.g.Galaxy (Seneff et al., 1998), the system supportsmore flexible component communication. Thesystem is depicted in Figure 1.

2.1 Information Storage

The Jaspis architecture contains several featureswhich support adaptive applications. First ofall, the information about the system state is

kept in a shared knowledge base called Informa-tion Storage. This blackboard-type informationstorage can be accessed by each system compo-nent via the Information Manager, which allowsthem to utilize all the information that the sys-tem contains, such as dialogue history and userprofiles, directly. Since the important informa-tion is kept in a shared place, system compo-nents can be stateless, and the system can switchbetween them dynamically. Information Stor-age thus facilitates the system’s adaptation todifferent internal situations, and it also enablesthe most suitable component to be chosen tohandle each situation.

2.2 Flexible Component Management

The system is organized into modules whichcontain three kinds of components: managers,agents and evaluators. Each module containsone manager which co-ordinates component in-teraction inside the module. The present archi-tecture implements e.g. the Input/Output Man-ager, the Dialogue Manager and the Presenta-tion Manager, and they have different prioritieswhich allow them to react to the interaction flowdifferently. The basic principle is that whenevera manager stops processing, all managers canreact to the situation, and based on their prior-ities, one of them is selected. There is also theInteraction Manager which coordinates applica-tions on the most general level.

The number and type of modules that can beconnected to the system is not limited. The In-teraction Manager handles all the connectionsbetween modules and the system can be dis-tributed for multiple computers. In Interactwe have built a demonstration application onbus-timetable information which runs on severalplatforms using different operating systems andprogramming languages. This makes the systemhighly modular and allows experiments with dif-ferent approaches from multiple disciplines.

2.3 Interaction Agents and Evaluators

Inside the modules, there are several agentswhich handle various interaction situations suchas speech output presentations and dialogue de-cisions. These interaction agents can be very

Figure 1: The system architecture.

specialized, e.g. they deal only with speechrecognition errors or outputs related to greet-ings. They can also be used to model differ-ent interaction strategies for the same task, e.g.different dialogue agents can implement alterna-tive dialogue strategies and control techniques.Using specialized agents it is possible to con-struct modular, reusable and extendable inter-action components that are easy to implementand maintain. For example, different error han-dling methods can be included to the system byconstructing new agents which handle errors us-ing alternative approaches. Similarly, we cansupport multilingual outputs by constructingpresentation agents that incorporate languagespecific features for each language, while imple-menting general interaction techniques, such aserror correction methods, to take care of errorsituations in speech applications in general (Tu-runen and Hakulinen, 2001b).

The agents have different capabilities and theappropriate agent to handle a particular situa-tion at hand is selected dynamically based onthe context. The choice is done using evalua-tors which determine applicability of the agentsto various interaction situations. Each evaluatorgives a score for every agent, using a scale be-tween [0,1]. Zero means that an agent is notsuitable for the situation, one means that anagent is perfectly suitable for the situation, val-

ues between zero and one indicate the level ofsuitability. Scaling functions can be used to em-phasize certain evaluators over the others Thescores are then multiplied, and the final score, asuitability factor, is given for every agent. Sincescores are multiplied, an agent which receiveszero from one evaluator is useless for that situ-ation. It is possible to use different approachesin the evaluation of the agents, and for instance,the dialogue evaluators are based on reinforce-ment learning.

Simple examples of evaluators are for instancepresentation evaluators that select presentationagents to generate suitable implicit or explicitconfirmations based on the dialogue history andthe system’s knowledge of the user. Another ex-ample concerns dialogue strategies: the evalua-tors may give better scores for system-initiativeagents if the dialogue is not proceeding well withthe user-initiative dialogue style, or the evalua-tors may prefer presentation agents which givemore detailed and helpful information, if theusers seem to have problems in communicatingwith the application.

Different evaluators evaluate different aspectsof interaction, and this makes the evaluationprocess highly adaptive itself: there is no singleevaluator which makes the final decision. In-stead, the choice of the appropriate interactionagent is a combination of different evaluations.

Evaluators have access to all information in theInformation Storage, for example dialogue his-tory and other contextual information, and it isalso possible to use different approaches in theevaluation of the agents (such as rule-based andstatistical approaches). Evaluators are the keyconcept when considering the whole system andits adaptation to various interaction situations.

2.4 Distributed Input and Output

The input/output subsystem is also distributedwhich makes it possible to use several input andoutput devices for the same purposes. For ex-ample, we can use several speech recognitionengines, each of which with different capabili-ties, to adapt the system to the user’s way oftalking. The system architecture contains vir-tual devices which abstract the actual devices,such as speech recognizers and speech synthesiz-ers. From the application developers viewpointthis makes it easy to experiment with differentmodalities, since special agents are used to addand interpret modality specific features. It isalso used for multilingual inputs and outputs,although the Interact project focuses on Finnishspeech applications.

3 Natural Language Capabilities

The use of Finnish as an interaction languagebrings special problems for the system’s nat-ural language understanding component. Theextreme multiplicity of word forms prevents theuse of all-including dictionaries. For instance,a Finnish noun can theoretically have around2200, and a verb around 12000 different forms(Karlsson, 1983). In spoken language thesenumbers are further increased as all the differ-ent ways to pronounce any given word come intoconsideration (Jauhiainen, 2001). Our dialoguesystem is designed to understand both writtenand spoken input.

3.1 Written and spoken input

The different word forms are analyzed usingFintwol, the two-level morphological analyzerfor Finnish (Koskenniemi, 1983). The forms arecurrently input to the syntactic parser CPARSE

(Carlson, 2001). However, the flexible sys-tem architecture also allows us to experimentwith different morphosyntactic analyzers, suchas TextMorfo (Kielikone Oy 1999) and ConexorFDG (Conexor Oy 1997-2000), and we planto run them in parallel as separate competingagents to test and compare their applicabilityas well as the Jaspis architecture in the giventask.

We use the Lingsoft Speech Recognizer for thespoken language input. The current state of theFinnish speech recognizer forces us to limit theuser’s spoken input to rather restricted vocab-ulary and utterance structure, compared to theunlimited written input. The system uses fullword lists which include all the morphologicalforms that are to be recognized, and a strictcontext-free grammar which dictates all the pos-sible utterance structures. We are currently ex-ploring possibilities for a HMM-based languagemodel, with the conditional probabilities deter-mined by a trigram backoff model.

3.2 Language analysis

The task of the parsing component is to mapthe speaker utterances into task-relevant do-main concepts which are to be processed bythe dialogue manager. The number of domainconcepts concerning the demonstration system’sapplication domain, bus-timetables, is rathersmall and contains e.g. bus, departure-timeand arrival-location. However, semanticallyequivalent utterances can of course vary in thelexical elements they contain, and in written andespecially in spoken Finnish the word order inalmost any given sentence can also be changedwithout major changes on the semantic level un-derstood by the system (the difference lies in theinformation structure of the utterance). For in-stance, the request How does one get to Malmi?can be realised as given in Table 1.

There are two ways to approach the problem:on one hand we can concentrate on finding thekeywords and their relevant word forms, on theother hand we can use more specialized syntac-tic analyzers. At the moment we use CPARSEas the syntactic analyzer for text-based input.The grammar has been adjusted for the demon-

Kuinka paasee bussilla Malmille?Miten paasee Malmille bussilla?Kuinka Malmille paasee bussilla?Malmille miten paasee bussilla?Milla bussilla paase Malmille?Malmille olisin bussia kysellyt.Paaseeko bussilla Malmille?

Table 1: Some alternative utterances for Kuinkapaasee Malmille bussilla? ’How does-one-get to-Malmi by bus?

stration system so that it especially looks forphrases relevant to the task at hand. For in-stance, if we can correctly identify the inflectedword form Malmille from the input string, wecan be quite certain of the user wishing to knowsomething about getting to Malmi.

The current speech input does not go throughany special morpho-syntactic analysis becauseof the strict context-free grammar used by thespeech recognizer. The dictionary used by therecognizer is tagged with the needed morpholog-ical information and the context-free rules aretagged with the needed syntactic information.

3.3 Language generation

The language generation function is located inthe system’s Presentation Manager module. Un-like language analysis, for which different ex-isting Finnish morphosyntactic analyzers canbe used, there are no readily available general-purpose Finnish language generators. We aretherefore developing specific generation compo-nents for this project. The flexible system ar-chitecture allows us to experiment with differentgenerators.

Unfortunately the existing Finnish syntacticanalyzers have been designed from the outset as“parsing grammars”, which are difficult or im-possible to use for generation. However, the two-level morphology model (Koskenniemi, 1983) isin principle bi-directional, and we are work-ing towards its use in morphological generation.Fortunately there is also an existing Finnishspeech synthesis project (Vainio, 2001), whichwe can use together with the language genera-

tors.Some of our language generation components

use the XML-based generation framework de-scribed by Wilcock (2001), which has the ad-vantage of integrating well with the XML-basedsystem architecture. The generator starts froman agenda which is created by the dialogue man-ager, and is available in the system’s Informa-tion Storage in XML format. The agenda con-tains a list of semantic concepts which the dia-logue manager has tagged as Topic or NewInfo.From the agenda the generator creates a re-sponse plan, which passes through the genera-tion pipeline stages for lexicalization, aggrega-tion, referring expressions, syntactic and mor-phological realization. At all stages the responsespecification is XML-based, including the finalspeech markup language which is passed to thespeech synthesizer.

The system architecture allows multiple gen-erators to be used. In addition to the XML-based pipeline components we have some pre-generated outputs, such as greetings at the startand end of the dialogue or meta-acts such aswait-requests and thanking. We are also ex-ploiting the agent-based architecture to increasethe system’s adaptivity in response generation,using the level of communicative confidence asdescribed by Jokinen and Wilcock (2001).

4 Recognition of Discussion Topic

One of the important aspects of the system’sadaptivity is that it can recognize the correcttopic that the user wants to talk about. By’topic’ we refer to the general subject matterthat a dialogue is about, such as ’bus timetables’and ’bus tickets’, realized by particular words inthe utterances. In this sense, individual doc-uments or short conversations may be seen tohave one or a small number of topics, one at atime.

4.1 Topically ordered semantic space

Collections of short documents, such as news-paper articles, scientific abstracts and the like,can be automatically organized onto documentmaps utilizing the Self-Organizing Map algo-rithm (Kohonen, 1995). The document map

methodology has been developed in the WEB-SOM project (Kohonen et al., 2000), where thelargest map organized consisted of nearly 7 mil-lion patent abstracts.

We have applied the method to dialogue topicrecognition by carring out experiments on 57Finnish dialogues, recorded from the customerservice phone line of Helsinki City Transportand transcribed manually into text. The dia-logues are first split into topically coherent seg-ments (utterances or longer segments), and thenorganized on a document map. On the orderedmap, each dialogue segment is found in a spe-cific map location, and topically similar dialoguesegments are found near it. The document mapthus forms a kind of topically ordered semanticspace. A new dialogue segment, either an utter-ance or a longer history, can likewise be auto-matically positioned on the map. The coordi-nates of the best-matching map unit may thenbe considered as a latent topical representationfor the dialogue segment.

Furthermore, the map units can be labeled us-ing named topic classes such as ’timetables’ and’tickets’. One can then estimate the probabilityof a named topic class for a new dialogue seg-ment by construing a probability model definedon top of the map. A detailed description of theexperiments as well as results can be found in(Lagus and Kuusisto, 2002).

4.2 Topic recognition module

The topical semantic representation, i.e. themap coordinates, can be used as input for thedialogue manager, as one of the values of thecurrent dialogue state. The system architecturethus integrates a special topic recognition mod-ule that outputs the utterance topic in the In-formation Storage. For a given text segment,say, the recognition result from the speech rec-ognizer, the module returns the coordinates ofthe best-matching dialogue map unit as well asthe most probable prior topic category (if priorcategorization was used in labeling the map).

5 Dialogue Management

The main task of the dialogue manager com-ponent is to decide on the appropriate way to

react to the user input. The reasoning includesrecognition of communicative intentions behindthe user’s utterances as well as planning of thesystem’s next action, whether this is informationretrieval from a database or a question to clarifyan insufficiently specified request. Natural inter-action with the user also means that the systemshould not produce relevant responses only interms of correct database facts but also in termsof rational and cooperative reactions. The sys-tem could learn suitable interaction strategiesfrom its interaction with the user, showing adap-tation to various user habits and situations.

5.1 Constructive Dialogue Model

A uniform basis for dialogue management canbe found in the communicative principles re-lated to human rational and coordinated inter-action (Allwood et al., 2000; Jokinen, 1996).The speakers are engaged in a particular activ-ity, they have a certain role in that activity, andtheir actions are constrained by communicativeobligations. They act by exchanging new in-formation and constructing a shared context inwhich to resolve the underlying task satisfacto-rily.

The model consists of a set of dialogue states,defined with the help of dialogue acts, obser-vations of the context, and reinforcement val-ues. Each action results in a new dialoguestate. The dialogue act, Dact, describes the actthat the speaker performs by a particular utter-ance, while the topic Top and new informationNewInfo denote the semantic content of the ut-terance and are related to the task domain. To-gether these three create a useful first approx-imation of the utterance meaning by abstract-ing over possible linguistic realisations. Unfilledtask goals TGoals keep track of the activity re-lated information still necessary to fulfil the un-derlying task (a kind of plan), and the speakerinformation is needed to link the state to pos-sible speaker characteristics. The expectations,Expect are related to communicative obligations,and used to constrain possible interpretations ofthe next act. Consequently, the system’s inter-nal states can be reduced to a combination ofthese categories, all of which form an indepen-

dent source of information for the system to de-cide on the next move.

5.2 Dialogue agents and evaluators

A dialogue state and all agents that contributeto a dialogue state are shown in Figure 2. TheDialogue Model is used to classify the currentutterance into one of the dialogue act categories(Jokinen et al., 2001), and to predict the nextdialogue acts (Expect). The Topic Model rec-ognizes the domain, or discussion topic, of theuser input as described above.

Figure 2: Dialogue states for user’s utter-ance and system action, together with dialogueagents involved in producing various informa-tion.

All domains out of the system’s capabili-ties are handled with the help of a specialOutOfDomain-agent which informs the user ofthe relevant tasks and possible topics directly.This allows the system to deal with error sit-uations, such as irrelevant user utterances, ef-ficiently and flexibly without invoking the Dia-logue Manager to evaluate appropriate dialoguestrategies. The information about error situ-ations and the selected system action is stillavailable for dialogue and task goal managementthrough the shared Information Storage.

The utterance Topic and New Information(Topic, NewInfo) of the relevant user utter-ances are given by the parsing unit, and sup-plemented with discourse knowledge by ellipsis

and anaphora resolution agents (which are In-put Agents). Task related goals are produced byTask Agents, located in a separate Task Man-ager module. They also access the backenddatabase, the public transportation timetablesof Helsinki.

The Dialogue Manager (DM) consists ofagents corresponding to possible system actions(Figure 3). There are also some agents for inter-nal system interaction, illustrated in the figurewith a stack of agents labeled with Agent1. Oneagent is selected at a time, and the architecturepermits us to experiment with various compet-ing agents for the same subtask: the evaluatorsare responsible for choosing the one that bestfits in the particular situation.

Figure 3: The Dialogue Manager component.

Two types of evaluators are responsible forchoosing the agent in DM, and thus implement-ing the dialogue strategy. The QEstimate eval-uator chooses the agent that has proven to bemost rewarding so far, according to a Q-learning(Watkins and Dayan, 1992) algorithm with on-line ε-greedy policy (Sutton and Barto, 1998).That agent is used in the normal case and thedecision is based on the dialogue state presentedin Figure 2. The underlying structure of theQEstimate evaluator is illustrated in Figure 4.

The evaluator is based on a table of real val-ues, indexed by dialogue states, and updated af-ter each dialogue. The agent with the highest

Figure 4: The QEstimate evaluator.

value for the current dialogue state gets selected.Adaptivity of the dialogue management comesfrom the reinforcement learning algorithm ofthis evaluator.

On the other hand, if one of the error evalu-ators (labeled with Error1..N) detects that anerror has occurred, the QEstimate evaluator isoverridden and a predetermined agent is selectedto handle the error situation (Figure 5). In thesecases, only the the correct agent is given a non-zero value, forcing the dialogue manager to se-lect that agent. Examples of such errors includesituations when the user utterance is not recog-nized by the speech recognizer, its topic is irrele-vant to the current domain, or its interpretationis inconsistent with the dialogue context.

Figure 5: An error evaluator.

Because all possible system actions arereusable agents, we can easily implement a dif-ferent dialogue management strategy by addingevaluators, or replacing the current QEstimateevaluator. We are developing another strategybased on recurrent self-organizing maps, thatlearns to map dialogue states to correct actionsby fuzzy clustering, minimizing the amount ofhuman labor in designing the dialogue strategy.

6 Demo System and Future Work

The project deals with both speech and textcommunication, interfacing with the user by

telephone and web-applications. An initial in-tegrated system with basic functionality and in-teraction capabilities concerning bus timetableinquiries has been built. Sample interactions aregiven in Figures 6 and 7. Various other appli-cation types such as intelligent question-answersystems, automatic call-centers, and other ser-vices that would benefit from flexible naturalinteraction can also be envisaged.

The current system is the first prototype andintended to provide useful basis for further ex-perimentation and system development. Theplanned activities include further investigationsof the limitations and advantages of various ap-proaches for language technology, as well as us-ability studies concerning real users and userswith special needs.

(1) UT: mmillas bussilla ma paasen malamille’with which bus do I get to Malami’UR: milla bussilla paasen malmille’with which bus do I get to Malmi’

SU: mista olittekaan lahdossa malmille’where do you want to leave for Malmi’

UT: no hakaniemesta’well from Hakaniemi’UR: hakaniemesta’from Hakaniemi’

SU: bussi 70 lahtee sielta malmille’bus 70 leaves from there to Malmi’

UT: selva kiitos hei’ok thank you bye’UR: bussi kiitos hei’bus thank you bye’

SU: kiitos hei’thank you bye’

Figure 6: Sample dialogue with low recognitionscore for the destination Malmille ’to Malmi’(UT = Utterance as actually made by the user,UR = Utterance as recognized by the system,SU = System utterance).

(2) UT: koskas mennee seuraava pussi arapi-asta malmille’when will the next bus go from Arapia toMalmi’UR: koska menee seuraava bussi arabiastamalmille’when will the next bus go from Arabia toMalmi’

SU: kello 14 30’at 14:30’

UT: minka numeroinen se on’which number is it’UR: mina nelonen selva’I four okay’

SU: olen pahoillani osaan vastata vainbussiaikatauluja koskeviin kysymyksiin’sorry I can reply only to questions con-cerning bus timetables’

UT: aha no joo jaa kiitos hei’I see ok thank you bye’UR: onko aijaa kiitos hei’is I see thank you bye’

SU: kiitos hei’thank you bye’

Figure 7: Sample dialogue with misrecognitionof user utterance minka numeroinen se on(UT = Utterance as actually made by the user,UR = Utterance as recognized by the system,SU = System utterance).

7 Acknowledgements

The research is carried out in the context of theUSIX Interact project, a collaboration projectbetween four Finnish universities, and fundedby the National Technology Agency, the leadingIT companies ICL Invia oyj, Sonera oyj, Ling-soft oy, and Gurusoft oy, as well as the FinnishAssociation for the Deaf and the Arla Institute.

References

J. Allwood, D. Traum, and K. Jokinen. 2000. Coop-eration, dialogue and ethics. International Jour-nal of Human-Computer Studies, 53:871–914.

L. Carlson. 2001. CPARSE manual. http://www.ling.helsinki.fi/ lcarlson/cparse09en.html.

T. Jauhiainen. 2001. Using existing written lan-guage analyzers in understanding natural spokenFinnish. In Proceedings of Nodalida ’01, Uppsala.

K. Jokinen and G. Wilcock. 2001. Confidence-basedadaptivity in response generation for a spoken di-alogue system. In Proceedings of the 2nd SIGdialWorkshop on Discourse and Dialogue, pages 80–89, Aarhus.

K. Jokinen, T. Hurtig, K. Hynna, K. Kanto,M. Kaipainen, and A. Kerminen. 2001. Self-organizing dialogue management. In Proceedingsof the 2nd Workshop on Natural Language Pro-cessing and Neural Networks, pages 77–84, Tokyo.

K. Jokinen. 1996. Goal formulation based on com-municative principles. In Proceedings of the 16thCOLING, pages 598–603.

F. Karlsson. 1983. Suomen kielen aanne- ja muoto-rakenne. WSOY, Juva.

T. Kohonen, S. Kaski, K. Lagus, J. Salojarvi,V. Paatero, and A. Saarela. 2000. Organizationof a massive document collection. IEEE Transac-tions on Neural Networks, Special Issue on NeuralNetworks for Data Mining and Knowledge Discov-ery, 11(3):574–585, May.

T. Kohonen. 1995. Self-Organizing Maps. Springer,Berlin.

K. Koskenniemi. 1983. Two-level morphology: ageneral computational model for word-form recog-nition and production. University of Helsinki,Helsinki.

K. Lagus and J. Kuusisto. 2002. Topic identifica-tion in natural language dialogues using neuralnetworks. In Proceedings of the 3rd SIGdial Work-shop on Discourse and Dialogue, Philadelphia.

S. Seneff, E. Hurley, R. Lau, C. Pao, P. Schmid, andV. Zue. 1998. Galaxy-II: A reference architecturefor conversational system development. In Pro-ceedings of ICSLP-98, Sydney.

R. Sutton and A. Barto. 1998. Reinforcement Learn-ing: An Introduction. MIT Press, Cambridge,Massachusetts.

M. Turunen and J. Hakulinen. 2000. Jaspis - aframework for multilingual adaptive speech appli-cations. In Proceedings of the 6th InternationalConference on Spoken Language Processing, Bei-jing.

M. Turunen and J. Hakulinen. 2001a. Agent-basedadaptive interaction and dialogue management ar-chitecture for speech applications. In Text, Speechand Dialogue. Proceedings of the Fourth Interna-tional Conference (TSD-2001), pages 357–364.

M. Turunen and J. Hakulinen. 2001b. Agent-basederror handling in spoken dialogue systems. In Pro-ceedings of Eurospeech 2001, pages 2189–2192.

M. Vainio. 2001. Artificial Neural Network BasedProsody Models for Finnish Text-to-Speech Syn-thesis. Ph.D. thesis, University of Helsinki.

C. Watkins and P. Dayan. 1992. Technical note:Q-learning. Machine Learning, 8:279–292.

G. Wilcock. 2001. Pipelines, templates and transfor-mations: XML for natural language generation. InProceedings of the 1st NLP and XML Workshop,pages 1–8, Tokyo.