Dialogue Structure of Coaching Sessions

Iwan de Kok1,2, Julian Hough1,3, Cornelia Frank1,4,David Schlangen3, and Stefan Kopp1,2

1CITEC, 2Social Cognitive Systems,3Dialogue Systems Group, 4Neurocognition and Action (Biomechanics)

Bielefeld Universityidekok@techfak.uni-bielefeld.de

Abstract

We report initial findings of the ICSPACE(‘Intelligent Coaching Space’) project onvirtual coaching. We describe the gather-ing of a corpus of dyadic squat coachinginteractions and initial high-level modelsof the structure of these sessions.

1 Introduction

While interactive tutoring systems which performfactual teaching have been established for sometime (Litman and Silliman, 2004; Graesser et al.,2005), dialogue systems capable of skill coachingare much rarer. We introduce preliminary workon the ICSPACE (‘Intelligent Coaching Space’)project, which aims to create a virtual intelligentcoaching agent in an interactive environment totrain users to perform complex motor actions.

Coaching physical movement skills requirescombining communication with real-time track-ing, assessing and correcting the motor action ofthe coachee. In particular, giving online feed-back while the coachee is carrying out an exercise(Sigrist et al., 2013) is an interesting challenge im-posing specific requirements on the system .

To identify these requirements more preciselywe analyse two recordings of a professional coachtraining individuals to perform a squat. We fo-cus on the overall dialogue structure and observewhich dialogue situations arise.

2 Recordings

We invited a professional coach to our lab torecord two coaching sessions. The coach wasasked to instruct coachees how to do a squat as hewould teach it in the gym. The coachees (one fe-male (A), one male (B)) were familiar with doingsquats, although they had not received instructionfrom a professional coach before. Each interactionlasted between 4 and 5 minutes.

Explain

Demonstrate

AskToDo

Assess Online Feedback

Feedback

coachee training attempt

Introduction

InitialAssessment

Closing

CoachingCycle

Squat Session

Figure 1: Overall structure of a squat coachingsession (left) and structure of a coaching cyclewithin the session (right).

3 Dialogue Structure

In analysing the two dialogues, several common-alities to the structure of the interaction were ob-served, as well as some interesting differences.From these dialogues a common overall coachingstructure can be inferred, represented in the left-hand shaded box in Figure 1 – temporal depen-dencies are represented by arrows.

In both dialogues, there was an Introductionphase where rapport with the coachee was estab-lished by the coach, consisting of questions aboutpersonal details and also establishing their previ-ous experience of the squat exercise.

In the second phase of Initial Assessment, whilein A’s session this was completed after lengthy ex-planation, for B the coach begins by asking thecoachee to do a squat before any explanation. Inboth cases the coach assesses the coachee’s abil-ity at performing the squat, identifying the sub-movements and aspects of their technique whichfall short of the coachee’s potential, and sub-sequently planning the following coaching be-haviour.

What follows are a series of coaching cycles,each explaining a particular area in which themovement is being executed below the potential

167

of the coachee, or which the coach wants to bringto attention to ensure the coachee will continue ex-ecuting that aspect correctly. As soon as the coachevaluates the action as being performed appropri-ately after several iterations, the session is closed.

Coaching cycles Each coaching cycle follows asimilar structure as depicted in the right-hand di-agram of Figure 1 – the optional components andtransitions have dashed lines. In both dialogues,the coach starts by explaining the particular (as-pect of the) movement this coaching cycle will fo-cus on (see visually, top-left in Figure 2). Oftenthe coach demonstrates the movement and high-lights the area of interest with gestures. The coachthen asks the coachee to perform the movementand they comply, else this is done without prompt-ing as the coachee takes initiative (as was the casein the top-right image of Figure 2). The latter caseshows the possibility of mixed initiative in coach-ing dialogues and is analogous to question ac-comodation in issue-based dialogue management(Larsson, 2002), however the “answer” to the ac-comodated question here is non-linguistic.

As the coachee attempts the squat, either in asingle effort or in repetition, the coach assessesthe movement being performed by the coachee(bottom-left in Figure 2) and may give online feed-back during execution to adjust the movement, ei-ther in the form of short utterances as verbal feed-back, gesture or even by performing the movementsynchronously with the coachee (bottom-right inFigure 2). Once the coach is satisfied with theresult or can not adjust the movement during ex-ecution, he will stop the coachee and give morelengthy final feedback on the movement. Duringthis stage he will often explain why this particu-lar (aspect of the) movement is important. If thecoach is satisfied he moves on to the next aspect(if there are any remaining to be corrected), other-wise the cycle will be partially repeated by eitheranother demonstration or request to try again.

Spatial positioning of the coach The coachingcycles were not only noticable in the dialoguestructure, but also in the coach’s movement. Theexplanation and feedback phases were always per-formed in front of the coachee – the instructionspace (see top two images in Figure 2). Dur-ing the assessment and online feedback phases thecoach usually moved to the side of the coacheeto get a good profile view– the observation space

Figure 2: Images from the recordings show-ing explaining (top-left), demonstrating withuser initiated following of (top-right), assessing(bottom-left) and finally giving online feedback on(bottom-right) the squat movement. The imagesalso highlight the instruction space (top) and ob-servation space (bottom).

(see bottom two screenshots in Figure 2). Ini-tial demonstrations are usually performed in theinstruction space. If the coach demonstrates themovement during the online feedback phase, thisis performed in the observation space. Thismulti-locational instruction behaviour is similar tothat exhibited by music teachers in instrumentallessons, who tend to move between the work zoneand the listening zone (Duffy and Healey, 2012).

4 Conclusion

We have analyzed the dialogue structure of twoexemplary squat coaching sessions to identify therequirements of a virtual intelligent coaching sys-tem. The initial recordings show the need formulti-modal turn-taking, use of different spacesfor different coaching phases and the ability togenerate fast incremental feedback as describedby (Kopp et al., 2013) during phases of onlinefeedback. As a next step we plan to collect morerecordings with different coaches with coachees ofdifferent skill levels. This should give evidencefor whether the dialogue structure we hypothe-size generalizes to the domain, and it will informthe design of an artificial coach capable of onlinefeedback and dialogue in coaching.

Acknowledgments

This research is supported by the DeutscheForschungsgemeinschaft (DFG) in the Center ofExcellence EXC 277 in Cognitive InteractionTechnology (CITEC).

168

ReferencesSam Duffy and Patrick GT Healey. 2012. Spatial co-

ordination in music tuition. In Proceedings of the34th Annual Conference of the Cognitive ScienceSociety.

Arthur C Graesser, Patrick Chipman, Brian C Haynes,and Andrew Olney. 2005. Autotutor: An intelligenttutoring system with mixed-initiative dialogue. Ed-ucation, IEEE Transactions on, 48(4):612–618.

Stefan Kopp, Herwin van Welbergen, RaminYaghoubzadeh, and Hendrik Buschmeier. 2013.An architecture for fluid real-time conversationalagents: integrating incremental output generationand input processing. Journal on Multimodal UserInterfaces, pages 1–12.

Staffan Larsson. 2002. Issue-based dialogue manage-ment. Department of Linguistics, Goteborg Univer-sity.

Diane J Litman and Scott Silliman. 2004. Itspoke:An intelligent tutoring spoken dialogue system. InDemonstration Papers at HLT-NAACL 2004, pages5–8. Association for Computational Linguistics.

Roland Sigrist, Georg Rauter, Robert Riener, and Pe-ter Wolf. 2013. Augmented visual, auditory, haptic,and multimodal feedback in motor learning: A re-view. Psychonomic bulletin & review, 20(1):21–53.

169