Supporting Autism Therapists:Co-designing Interventions

Carlos DuarteUniversity of LisbonFCUL, Campo Grande1749-016 Lisboa, Portugalcad@di.fc.ul.pt

Carla AlmeidaPIN – Progresso InfantilRua Pedro Alvares Cabral 472775-551 Carcavelos, Portugalcarla.almeida@pin.com.pt

Luis CarricoUniversity of LisbonFCUL, Campo Grande1749-016 Lisboa, Portugallmc@di.fc.ul.pt

Soraia NobrePIN – Progresso InfantilRua Pedro Alvares Cabral 472775-551 Carcavelos, Portugalsoraia.nobre@pin.com.pt

Tiago GuerreiroUniversity of LisbonFCUL, Campo Grande1749-016 Lisboa, Portugaltjvg@di.fc.ul.pt

Ana Margarida CamposUniversity of LisbonFCUL, Campo Grande1749-016 Lisboa, Portugalanamarcampos@gmail.com

Permission to make digital or hard copies of part or all of this work for personalor classroom use is granted without fee provided that copies are not made ordistributed for profit or commercial advantage and that copies bear this noticeand the full citation on the first page. Copyrights for third-party componentsof this work must be honored. For all other uses, contact the Owner/Author.Copyright is held by the owner/author(s).CHI 2014 , April 26–May 01 2014, Toronto, ON, Canada.ACM 978-1-4503-2474-8/14/04.http://dx.doi.org/10.1145/2559206.2581223

AbstractIn this paper we report on the design of a support systemfor autism therapists. The system was co-designed with ateam of therapists working with autistic children andsupports session management and data collection andanalysis. It is integrated with a storytelling interactiveenvironment in the context of improving the social skillsof children with Autistic Spectrum Disorder. Preliminaryresults show the potential advantages of such a system forthe therapists’ work.

Author KeywordsAustim therapy; Interactive stories; Participatory design

ACM Classification KeywordsK.4.2 [Computers and society]: Social issues.

General TermsDesign, Human Factors

IntroductionAutism therapy is usually a very comprehensiveundertaking, involving a child’s family and a team ofprofessionals. Each autistic child has different needs andstrengths. A range of different therapies have beenemployed targeting specific conditions, and, typically,autistic children receive treatment to more than one

condition. Each case being truly specific, translates todifferent treatments that need to be tailored by thetherapists involved.

The range of variables involved make this a very complexprocess for the therapists. Their activities comprisepreparing interventions, following the child’s evolution,collecting and analyzing data, communicating with theparents, coordinating with educators, discussing withfellow therapists, etc. Yet, most of these activities are stillmainly supported by pen and paper [3]. While technologyhas been used in the autism context for some years, it hasfocused almost exclusively in helping the autistic children,mainly through games [2]. The therapists themselvescould also benefit greatly from the support technology canoffer, in their information acquisition, searching, retrieval,archiving and sharing tasks.

While there are general tools (e.g. note takingapplications, communication tools, etc.) that can supportindividual needs, and some therapists do use them, thereis no support for the complete therapy workflow.Moreover, there is an absence of tools to support thetherapy sessions – their preparation, running, and analysis.

For the past year, we have been working with a team ofautism therapists towards understanding theirrequirements with the ultimate goal of designing a systemthat supports their work processes, and by supporting orfreeing them from some of their current tasks, allowingthem to devote more time to the children. The approachwe adopted consisted in jointly designing a system forsocial skill development, that would be used in thecontext of therapy sessions. In this paper we report on thedesign efforts and results from a preliminary evaluationwhere the system has been used in seven therapy sessions.

Related workTechnology is no stranger to autism therapy and totherapy sessions [5, 4]. Different technologies have beenused in different autism interventions: video, virtualreality, mobile devices, interactive tables, robots, avatars,social networks, etc. Some of these have seen more usagein the context of therapy sessions, while others have beenexplored outside of the sessions’ context. Video has beenthe technology most used [4] because of its ability toaddress a large range of interventions, and the relativeease with which therapists are able to work with it. Othertechnologies require development or configuration skillsthat therapists hardly possess.

However, technology use has focused almost exclusivelyon the children. Therapists have to manage large amountsof data relative to the children and the interventions theyhave administered [3], making them a target user groupthat could reap real benefits from supportive technology.Albeit systems exist that collect data frominterventions [6], therapists are still missing a system thatcould help them manage the complete workflow ofpreparing and managing sessions, and collecting,processing, analyzing and sharing data.

Supporting autism therapy sessionsTo design a system that addresses the needs of therapists,we opted for a participatory design approach. A total ofseven therapists collaborated with us, with different levelsof involvement. Two of the therapists had frequentmeetings with our team, and contributed heavily to thedesign. The remaining therapists participated lessfrequently, being involved in early focus group meetings,specific control points throughout the project’s life cycle,and the preliminary evaluation activities. Almost allmeetings took place at the therapists’ institution, which

allowed us to gauge the physical space and understand themobility requirements. The early meetings focused onrequirements collection. We then begun an iterativeprocess, with each iteration focusing on improving currentfeatures or incrementally adding new features. Therapistscontributed heavily to feature design and assessmentduring this process. In the following we describe theconcept behind the system and how it was implemented.

The conceptFollowing the therapists’ lead, we opted to focus onsupporting social skills development. From the range ofpossible activities to address (e.g. establishing eyecontact, choosing the correct stance, etc.) we initiallyfocused on teaching social interactions that requiregestures (e.g. waving goodbye, shaking hands, etc.) andlater added additional mechanisms for evaluatingknowledge about social interactions.

Figure 1: The session managerinterface.

One of the techniques used in therapy sessions isstorytelling [1]. We also based our system design on theconcept of interactive storytelling. An interactive storyconsists of one or more scenes. We have partitioned eachscene in 3 parts. The first is the introduction part wherethe social skill being taught is introduced. This will bebased on a video or an animation showing a story leadingto a point where the social skill is to be exercised.

Figure 2: Editing one scene froma story.

The scene follows with the activity part. In this part thechild is expected to conduct or reflect upon a socialactivity. The therapists’ aim in this part is to be able toassess the child while performing the activity. The systemneeds to collect performance data and the therapist’sassessment of the activity, to enable progress monitoringand other evaluation reports. During the activity, thesystem should continue to accompany the child eitherthrough instructions (e.g. a video of a gesture being

executed) or context information to enable the child toanswer questions about social situations.

The scene ends with the feedback part. The aim of thispart is for the therapist to provide feedback to the childbased on what has been done in the activity. Thetherapist should have the possibility to configure thefeedback given. The feedback should act as reinforcement,positively prompting the child to perform better. Forinstance, a child might not perform a gesture correctly,but it might still be an improvement over what the childwas able to do before. In that instance, the therapistmight give positive feedback to encourage the child, evenif the gesture was not performed correctly. Technically,the feedback can consist of another video or animation.

System designIn order to put the interactive story concept into practice,we needed to support two main activities: story creationand story presentation. This lead to a system with thefollowing modules: story and scene creation; storypresentation; therapy session manager; and a backendstoring all the information and connecting all modules.

The session manager (figure 1), besides allowing sessionscheduling, allows the therapist to assign a story to atherapy session. If the story does not exist yet, or thetherapist wants to change a story, the story creationmodule (figure 2) can be launched from the sessionmanager. To create stories we had to consider severalrequirements: every part of the scene might include videosor animations; the process of composing scenes andstories should be simple, because therapists do not haveknowledge of video processing or animation creation; eventhough every child will have interventions designedspecifically for him, scenes and stories should be easilyreusable and easily modified.

We have designed and explored several interfaces for storyand scene creation. All that involved any type of video oranimation manipulation proved to be too complex for thetherapists. Still, we learned that therapists knew clearlywhat they were looking for, and were often capable offinding the needed content in different sources (e.g.youtube for videos). Consequently, the scene editor loadsvideo or animation files that are already stored on thetherapists’ computer drive or makes use of the ones thathave been previously uploaded to the backend.

When a session is about to start, the therapist can launchthe session presentation module from the sessionmanager. This module supports two views, coordinated intwo different browsers (that might run on two differentdevices). One is the child view, where the multimediacontents are presented. The other is the controller view,where the therapist controls the flow of the story.

When designing the story presentation interface the mainchallenge was to provide the control needed by thetherapists in a way that does not disrupt the flow of thetherapy session. Given that most contents to display aremultimedia contents, we followed a multimedia controllerapproach, with standard play and pause commands. Whilethis provided some level of control, there were additionalrequirements. Because some children will be distractedduring the session, and others may not understand with asingle viewing what is requested from them, the controlpanel includes a replay command. Additionally, becausethe child’s performance during the session cannot beforeseen, and it might lead to blocking situations or delays,the control panel also includes a command to stop a storyat the end of any scene and not just at the final scene.

The two different activities supported required differentdesigns for the activity part of the scene. For the gesture

recognition activity we initially considered using anautomatic gesture recognition system. However, thetherapists expressed reservations regarding having anautomated system assessing the gestures performedduring the sessions. There were two main reasons. First,some scenarios present difficulties for automaticrecognition. The therapists had strong reservations withthe possibility of having both false positives and falsenegatives, as they could severely hinder the experiencechildren had with the system. Second, even gestures thatare not correctly performed can be judged to be positiveevolutions regarding what the child was able to previouslyperform. This kind of evolution is important knowledgefor the therapist, but difficult to collect with an automaticrecognition system. As a result, the presentation interfaceis capable of operating with and without the recognitionsystem. Even when present, the therapist always has thepower to override the recognition system decision.

The second type of activity involves assessing the child’ssocial skills in different situations. This is usually assessedthrough a series of questions and challenges. The storypresentation interface, for this type of activity, allows thetherapist to register the performance of the child in eachquestion. Furthermore, the therapists requested flexibilityto add new questions to the questionnaire during theintervention. This means that the functionality to addquestions is present not only during story creation but alsoduring story presentation.

The final part of the scene presentation is responsible forproviding feedback to the child, which offered us thepossibility to explore gamification concepts. The goal is toincrease the children motivation to correctly employsocially relevant skills, by encouraging them to improve ontheir prior performance. According to the therapists, this

concept can be valuable to motivate autistic children, butonly for older children. Younger children are not able tograsp the concepts associated with scores andperformances. We incorporated gamification conceptsoptionally, with the therapist deciding to use them forspecific children. Given the target audience weincorporated very simple mechanisms. The more basicvariation just presents a score between one and five starsfor each scene. More complex variations includepresenting a score for the whole story (adding eachscene’s score), and presenting scores across sessions.

Figure 3: An individual sessionwith the child’s view on themonitor and the therapistcontrolling the session with atablet.

Data collection and presentationThe system collects all session data. This includes: thenumber of times the therapist paused the session, andwhen each pause occurred; the number of times contentwas repeated, and when each request for repeatingoccurred; and the child’s performance as classified by thetherapist (and the gesture recognition system if beingused). All information is tagged with the time it wasgenerated. Additionally, the therapist can createannotations at any time during the session. Theseannotations can store whatever information the therapistdeems necessary and that might be important to interpretsome of the data collected (e.g. the reason why thetherapist needed to repeat some content).

Figure 4: Graphical visualizationof a child’s performance data.

The collected information is available for the therapist toconsult during and after the session. This allows thetherapist to assess the evolution of the child across severalsessions. Graphical visualizations of the child’sperformance allow the therapist to check on the evolutionof the child (figure 4), and to compare the evolution ofdifferent children, which might help assessing theefficiency of different interventions.

Preliminary evaluation resultsBefore starting a more extended trial we performed apreliminary evaluation of the system. During thisevaluation, the system was used in 7 sessions, with a totalof 7 children. Two of the sessions were group sessions,with three children. The remaining five sessions wereindividual (figure 3), two with the same child, and theother 3 sessions with different children. The children’s ageranged from 3 to 10 years old. They were six boys andone girl. The participating children were selected by thetherapists. Their parents were informed about theobjectives of the project, and signed a consent form. Twoof the sessions were based on the gesture recognitionactivity, while the remaining were based on the socialbehavior rules questionnaire. In six sessions the controllerdevice was an Android tablet. In the other, it was a PC.

We tried to assess the adequacy of the system for use inthe context of a therapy session and its impact on thelevels of interest of the children. One of our teammembers was authorized to attend two sessions, and couldcollect in-situ data from observation of the use of thesystem by the therapist, as well as the child’s reaction tothe interactive story presentation. After the sessions weconducted a post session walkthrough with the therapist,and a short interview with the child (conducted by thetherapist).

All therapists were able to use the available featureswithout any problems. The single exception was theannotations feature, used by only one therapist. Thejustification for this was that writing the annotations onthe tablet keyboard was considered too cumbersome, andit would break the flow of the session. Therapists didmention that if they were able to input the annotationswith their fingers or a stylus they might have made use of

the feature. The tablet’s mobility, allowing the therapistto move freely inside the session’s room, was alsoappointed as an added benefit.

Regarding the impact on the sessions flow, the therapistsexpressed a very positive opinion. They reported higherlevels of interest, motivation and attention from thechildren, because of the introduction of a newtechnological apparatus in their work practice. Childrenbecame more interactive with the technology, not in a waythat isolates them, but in a way that made them morecommunicative (with the therapist or other children in thegroup sessions). The children did not show any signs ofstress caused by the introduction of this system. They allreported to enjoy the new tool.

We found that therapists would now like the system tooffer an even greater level of flexibility, supportingdifferent endings to a story, customizable by the children.

Conclusions and future workAutism therapists manage large amounts of informationwhile running sessions and interventions. Mostly they dothis without support from any IT tool. In this paper, wereport on the period of one year, where we worked withautism therapists to design a system that could help themmanage therapy sessions. Currently we are preparing alongitudinal study, where the system will be used forseveral months in therapy sessions, which will allow us toevaluate its effects both on the therapists work and theintervention results on the children. The work so far hasopened additional intervention possibilities. One that isbeing considered is to employ automatic gesturerecognition to support children exercises in contextswithout the presence of a therapist (e.g. at home) basedon remote sessions configured by the therapist.

AcknowledgementsThis work was supported by FCT through funding ofSiTECOSE project, ref. PTDC/EIA-EIA/117058/2010,and LaSIGE Strategic Project, ref.PEst-OE/EEI/UI0408/2014

References[1] Dillon, G., and Underwood, J. Computer mediated

imaginative storytelling in children with autism. Int. J.Hum.-Comput. Stud. 70, 2 (2012), 169–178.

[2] Hourcade, J. P., Williams, S. R., Miller, E. A.,Huebner, K. E., and Liang, L. J. Evaluation of tabletapps to encourage social interaction in children withautism spectrum disorders. In Proc. CHI (2013),3197–3206.

[3] Marcu, G., Tassini, K., Carlson, Q., Goodwyn, J.,Rivkin, G., Schaefer, K. J., Dey, A. K., and Kiesler, S.Why do they still use paper?: Understanding datacollection and use in autism education. In Proc. CHI(2013), 3177–3186.

[4] Reed, F. D. D., Hyman, S. R., and Hirst, J. M.Applications of technology to teach social skills tochildren with autism. Research in Autism SpectrumDisorders 5, 3 (2011), 1003 – 1010.

[5] Tse, J., Strulovitch, J., Tagalakis, V., Meng, L., andFombonne, E. Social skills training for adolescentswith asperger syndrome and high-functioning autism.Journal of Autism and Developmental Disorders 37,10 (2007), 1960–1968.

[6] Westeyn, T. L., Abowd, G. D., Starner, T. E.,Johnson, J. M., Presti, P. W., and Weaver, K. A.Monitoring children’s developmental progress usingaugmented toys and activity recognition. PersonalUbiquitous Comput. 16, 2 (2012), 169–191.