www.elsevier.com/locate/intcom

Interacting with Computers 19 (2007) 587–596

The efficacy of narrative video for raising awareness in ICTdesigners about older users’ requirements

A. Carmichael *, A.F. Newell, M. Morgan

Division of Applied Computing, University of Dundee, Dundee DD1 4HN, Scotland, UK

Received 29 June 2006; received in revised form 23 March 2007; accepted 11 June 2007Available online 20 June 2007

Abstract

This paper discusses the efficacy of narrative video to communicate some of the fundamental differences between older users of ICTinterfaces and the interface designers who tend not to be familiar with the general perspectives and user requirements of this and other‘non-typical’ target groups. Preliminary results show the impact such videos can have on relevant audiences’ perspective on designingsystems for older adults. The findings suggest that they can influence the mind set of those with little or no experience of designingfor older users and that this influence can persist in the longer term. The findings also suggest that the extent of this influence can bean appropriate alternative to that of meeting and interacting with older users in a user centred design process, which although very valu-able can be a logistically (and otherwise) challenging element in the training of prospective software designers. The potential utility andlimits of this approach are also discussed.� 2007 Elsevier B.V. All rights reserved.

Keywords: Film/video; User requirements; Interface design; Older people; Inclusive design

1. Introduction

1.1. The importance of suitably informed empathy with users

Successful inclusive design requires that designersshould develop a degree of empathy with their potentialusers, and subsequently inform this with sufficient relevanthuman factors information in order to make appropriatedesign decisions. This requirement is emphasised whenthe user population includes older and/or disabled people,who are not likely to be well represented in the peer groupof the designer. This seems to be particularly markedamongst Information and Communication Technology(ICT) designers, as much current software, and the associ-ated user interfaces, seem to have been designed by young

0953-5438/$ - see front matter � 2007 Elsevier B.V. All rights reserved.

doi:10.1016/j.intcom.2007.06.001

* Corresponding author. Tel.: +44 0 1382 385054; fax: +44 0 1382285509.

E-mail addresses: acarmichael@computing.dundee.ac.uk (A. Carmichael),afn@computing.dundee.ac.uk (A.F. Newell), mmorgan@computing.dundee.ac.uk (M. Morgan).

male computer scientists with a limited, or even non-exis-tent, understanding of the relevant characteristics of thewider population, particularly older people (Newell andGregor, 1997). Danowski and Sacks (1980) comment that‘‘older people differ from ‘typical’ computer users. . .[and]. . .as a result, software designed for such ‘typical’users can present barriers to learning and use for older peo-ple’’ (p. 125). Further, the somewhat less than positive ‘atti-tudes’ toward ICT which many older people can have tendto be very alien to the younger software engineer who is soproficient with, and enamoured of, digital technology thatit is effectively inconceivable that others may have noknowledge or understanding whatsoever. Thus, beyondthe need to provide such designers with more specifichuman factors information about the needs, wants andcapabilities of older (and other ‘non-typical’) users, it isimportant to raise their awareness about the disparity(from themselves) and the diversity of their potential users.It is only with such awareness that the potential for empa-thy can be realised, such empathy will provide the motiva-tion to seek out and utilise suitable user information as the

588 A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596

basis of more appropriate designs for more, and morediverse, groups of users.

There are two distinct (but ultimately interrelated) waysthat older people differ from ‘typical’ computer users. Theyare less likely to have the degree of familiarity (and hence,confidence) with digital technology that younger people do(e.g. Docampo-Rama et al., 2001). This will impact upontheir potential ability to develop an understanding of theprinciples of operation, and upon their motivation to per-severe if difficulties are encountered. Secondly, the physi-cal, sensory and cognitive limitations brought about byageing can add significant barriers to the use of computer(and computer-like) interfaces as they tend currently tobe designed (Carmichael, 1999; Czaja and Lee, 2003).The combination of these types of factors can be particu-larly deleterious to a ‘non-typical’ user. Not accommodat-ing their basic capabilities into the interface designincreases the difficulties they will encounter, making itmore likely that their (relatively fragile) confidence willbe damaged, which in turn increases the likelihood thatthey will ‘give-up’ (and consequently be even more con-vinced that they are ‘too-old’ to use this modern technol-ogy). Thus older people’s entire relationship withtechnology can be negatively impacted simply because rel-atively minor effects of age have not been taken intoaccount in the design. It is, for example, completely inap-propriate that predominantly visual interfaces often donot address the diversity of the population’s eyesight, asthis will cause difficulties for many people, older ones inparticular.

Eyesight can be affected by a wide range of factors, someage-related and some not, which can each have very wideranging impacts on the use of visual interfaces (Fozard,1990). All older people have some minor visual impair-ments either age-related per se and/or added to by theeffects of injury and/or disease. Advancing age can affectmost optical parameters, each of which can have a dispa-rate impact on visual aspects of the interface. Althoughmany older people may not consider themselves ‘VisuallyImpaired’, they find the use of ‘typical’ screen layouts muchmore difficult than they did when they were younger.Awareness of (and accommodating for) such visual limita-tions will also be beneficial for ‘typical’ users in ‘non-typi-cal’ situations (e.g. sub-optimum lighting conditions).Many older people (and other ‘non-typical’ users) alsohave combinations of other minor impairments and theseimpairments may also interact with one another, creating,in effect, a more serious handicap (Carmichael, 1999; New-ell and Gregor, 1999; Gregor and Newell, 2001). Clearlydesigners should address such issues for the benefit of theseuser groups, but, by doing so, benefits will also accrue formany ‘typical’ users in relatively more challenging usagecontexts.

There are also significant challenges in achieving suc-cessful communication between designers and older people(another important foundation for empathy), again, partic-ularly in the context of designing ICT. One example of this

is jargon (and associated technological concepts, meta-phors, etc.) used almost unavoidably by technically profi-cient software designers, which many older (and other)users may be unfamiliar with or may misunderstand. Thismismatch in understanding and the basic human urge toavoid looking ‘ignorant’ may be related to the general find-ing that older users tend to be positive about prototypesthey are confronted with, even if they actually had difficultyusing them, rather than give what might otherwise be amore objective view (Newell et al., 2006b). Also many olderpeople who experience difficulty with ‘modern technology’,will tend to blame themselves rather than consider the pos-sibility of poor design (e.g. Mitchell et al., 2007; Hawthorn,2000; Rabbitt and Carmichael, 1994). Another important(and, again related) issue is that some older people’s confi-dence regarding their ability to use technology can be veryfragile (e.g. Zajicek, 2005), and it is vital from an ethicalperspective not to put people in a position where whateverconfidence they may have is put under threat (Gregor andNewell, 2001). These, and other issues demonstrate that avariety of aspects of the communication between older peo-ple and designers about the design of ICT is far fromstraightforward (Zajicek, 2004; Eisma et al., 2004).

The present paper focuses on the communication of rel-atively general insights about ‘non-typical’ users (in thisinstance older adults) to designers of ICT in a relativelyefficient but informative and engaging way. This is impor-tant because the information and data which are currentlyavailable are in predominantly static and text based for-mat, which limits their effective utility for many ICTdesigners. Following previous success in the use of videosequences to facilitate discussions between older usersand designers (McKenna et al., 2003, and see Newellet al., 2006a) we decided to examine the potential of thismedium to convey insights about older users of ICT todesigners. We thus utilised the skills of theatre profession-als to produce a series of videos, the UTOPIA Trilogy, toaddress this challenge. Although the UTOPIA Trilogy doescontain some ‘design guidance’ the emphasis in the currentstudy is less on information transmission and more on‘awareness raising’, in regard particularly to the notion thatpotential users of technology are not the same as its design-ers. Other aspects of the communication channels betweendesigners and ‘non-typical’ users, and the role theatricaltechniques can play in this (Boal, 1995; Dishman, 2003)and of which the UTOPIA Trilogy is but one element,are discussed further in (Newell et al., 2006a), wherein acomprehensive account of the motivation and processbehind the Trilogy is given.

1.2. ‘Guidelines’ about older users’ requirements

One approach to informing designers of the needs ofolder and disabled people within a design brief is the useof guidelines. Many such guidelines have been produced,including those based on accessibility standards for theWeb (World Wide Web Consortium, 1998, 1999; Office

A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596 589

of the e-Envoy, 2002a,b,) and more generalised guidancebased on human factors principles in relation to the char-acteristics of older people (e.g. Carmichael, 1999; Haw-thorn, 2000). There is also a body of HCI researchinformation in the academic literature on the particularproblems older people face using ICT systems (e.g. Coyneand Nielsen, 2002; Williamson et al., 1997; Hawthorn,2002). However, Robey and Markus (1998) report thatmany designers find these sorts of ‘academic’ sources ofuser information, ‘‘literally unreadable’’ (p. 8). Similarly,Blomberg et al. (2002) state that ‘‘re-presentation of obser-vational data or lists of disconnected findings is not partic-ularly useful as an aid to design’’ (p. 974). Further, manydesigners simply do not utilise such guidelines as do exist,and there is evidence suggesting that even when they areused, guidelines in isolation, have limited efficacy (Sloanet al., 2002; Kelly, 2002; Diaper and Worman, 2003) andcan often be inappropriately applied in the absence ofbroader knowledge about the context of the user popula-tion to which they relate (Thatcher, 2003). In a detailedstudy of the accessibility of web sites, Petrie and Hamilton(2004) have shown that many sites are not accessible, andmany of those which have complied with the W3C accessi-bility guidelines are still effectively unusable by disabledpeople. Another factor impacting the utility of guidelinesmay be that designers are typically visually oriented andmuch design education (and subsequent practice) is doneby example. In contrast to this, most user information ispresented in textual and numerical/tabulated form andrequires a marked degree of interpretation if they are tobe successfully incorporated into a design. This indicatesthat the mere availability of guidelines, or equivalent infor-mation, is not enough to ensure their utility, nor, it wouldseem, are the relevant legal imperatives (Gregor et al.,2005).

Another, distinct, approach to inform designers aboutusers (including the implications for design), focuses on this‘by example’ preference and is based on ‘claims’ or ‘pat-terns’ that relate ‘good’ interface design elements in a briefbut adequate context (e.g. Zajicek, 2005). This type ofapproach may have much potential, but is limited (cur-rently at least) by being static and predominantly textbased and is also likely to lag behind state-of-the-art designquestions by being ‘tried and tested’ in the context of ear-lier product generations.

A sizeable corpus of information thus exists about theabilities and requirements of ‘non-typical’ users such asolder adults, but much of this is currently unknown or con-sidered effectively inaccessible by many designers andtherefore is only of limited or non-existent utility. Thereis also a paucity of appropriate population based anthro-pometric data particularly in reference to older age groups(see for example; Charness and Bosman, 1990; Kelly andKroemer, 1990), and a need to gather data on anthropo-metric parameters (relevant to modern digital products)that include the full diversity of the population. While suchdata is potentially crucial, however, the question remains

of how to ensure that designers are aware of their needfor such information and how to make it accessible anduseful for designers.

1.3. User Centred Design

‘User Centred Design’ methodologies, and ‘usabilitytesting’ were developed to provide ways in which designerscan address the actual characteristics of their user popula-tion, and these have proved to be very useful in conven-tional software design (i.e. for ‘typical’ users), but, asNewell and Gregor (1997) have pointed out, such method-ologies do not cater well for the much wider spread of char-acteristics that are embodied by ‘the population’, whichincludes older and disabled people. This is particularlyimportant as increasing numbers of interfaces are beingdeveloped where the ‘target user group’ is the entire popu-lation (e.g. the world wide web and digital television).

The wide ranging characteristics and functionality ofolder and disabled people, make it difficult if not impossi-ble to identify and select meaningfully ‘representative’ usersamples and to produce ‘universal’ guidelines (Gregor andNewell, 2001). Newell and Gregor (2000) thus suggest thata new design paradigm should be developed which theydescribe as User Sensitive Inclusive Design. Based on theidea that ‘‘inclusive’’ is more achievable than ‘‘universal’’or ‘‘for all’’, and that ‘‘sensitive’’, rather than ‘‘centred’’,identifies the lack of a truly representative sample of a usergroup (especially when that group is ‘the general public’)and thus the need to be sensitive to the diverse needs of alarge group rather than be centred on the needs of a partic-ular (and not necessarily representative) sub-group or,indeed, of some notional ‘ideal’, ‘typical’ user.

Another approach commonly used in User CentredDesign to foster designers’ engagement with users is thatof personas and scenarios (see for example, Cooper,1999; Carroll, 1997; Benyon and Macaulay, 2002). Animportant motivation in the development of theseapproaches was to help designers to break away from amindset that (albeit implicitly) encourages them to essen-tially design for themselves (Keates et al., 2000). However,personas and scenarios are limited by similar presenta-tional factors as mentioned above for ‘user data’, by beingstatic and essentially text based. Despite the undoubtedbenefits of this approach, the diversity of older and dis-abled people limit the ability of personas and scenarios toproduce more ‘inclusive’ designs (Newell et al., 2006b),but the ‘person’ within a persona does promote engage-ment and some degree of ‘empathy’ between the designerand their potential users. However the efficacy of suchempathy can be offset by its focus on a (quasi) individualrather than on a relatively diverse group of individuals.

The development of effective methods for obtaining andpresenting ‘human factors’ and other ‘user requirements’information to designers in order for them to be optimallyused has become a pressing issue, particularly in the con-text of the design of ICT that will be used by older people,

590 A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596

and the UTOPIA Trilogy videos aim to impart an initialbasis for ‘sensitivity’ to the needs and wants of this usergroup, and to embody the combined strengths of personasand scenarios in a way that maintains a focus on a ‘usergroup’ rather than a ‘user’.

1.4. The UTOPIA project

The UTOPIA project (Usable Technology for OlderPeople: Inclusive and Appropriate; see http://www.com-puting.dundee.ac.uk/projects/UTOPIA) funded by theScottish Higher Education Funding Council, was led bythe University of Dundee and involved Abertay, Glasgowand Napier Universities. It is notable in this context, thatthis project shares a name (acronym notwithstanding) withthe seminal participatory design study in Sweden (see, forexample, Bodker et al., 2000). The Scottish UTOPIA pro-ject was established to examine a range of issues surround-ing older people’s use (or not) of modern technology. Oneaspect of this was the development of a volunteer panel ofolder people who act as surrogate users in a variety ofcapacities. A major objective of the project was to promote(to both academia and industry) the need for suitable typesof technology to be appropriately designed for this group(Dickinson et al., 2002; Eisma et al., 2003) an importantelement of which being the development of various toolsand methods to support designers in achieving this.

1.5. The UTOPIA Trilogy

An important strand of the work in UTOPIA was thedevelopment of video portrayals of some of the commonchallenges and difficulties older people can experience withmodern technologies. This was motivated in large part byan earlier academic/industry collaboration to develop aneasy to use e-mail system, during which one of the industrybased designers, who had been briefed about the needs ofolder people, having something of a ‘road to Damascus’experience after witnessing first-hand the complete bewil-derment of a group of elderly users when faced with hisprototype design. This particular issue and a fuller discus-sion of the collaborative project are given in Newell et al.(2006b). Based upon the marked influence of this experi-ence on these designers, the Foxtrot Theatre Companywas commissioned to produce a series of narrative basedvideos to illustrate the kinds of problems that many olderpeople face with ICT (see Newell et al., 2006a, for moreon this collaboration). The ‘problems’ depicted covered arange of the ‘lifestyle’, confidence, ‘jargon’ and ‘user abil-ity’ issues discussed above. The stories portrayed were anamalgamation and distillation of many older people’s realexperiences and the findings from various human factorsand usability research with older people drawn both fromthe UTOPIA project and the wider literature. These dataand experiences were distilled by the scriptwriter into nar-ratives which encapsulated a range of issues within anengaging, cohesive and ‘dramatic’ storyline. These were

then produced as short vignettes using professional actors,director, and crew. These vignettes formed the ‘‘UTOPIATrilogy’’, which consisted of dramatizations of:

• A new (older) user of a mobile telephone• A new (older) user of email both at home and in a ‘train-

ing’ setting• A new (older) user of a home PC dealing with a new

peripheral device (a webcam)

Brief synopses of the final stories included in the trilogyare given in Fig. 1 below.

A variety of audiences including academics, usabilitypractitioners (with experience with ‘non-typical’ users)and older people from our volunteer panel viewed the Uto-pia Trilogy. Various ‘focus group’ and other discussionswith these audiences suggested that the videos accuratelyportray the kinds of challenges which many older peopleexperience with much new technology, and that as such,had the potential to ‘enlighten’ many ICT designers. Amajor advantage of this methodology was that the scriptand the actors could emphasise aspects of the situationsin ways which would not normally occur in more tradi-tional usability studies. These exaggerations, and thehumour generated by them, were very powerful in trans-mitting the message. For example the actor who was strug-gling to use an email system swore on a number ofoccasions, which had not been witnessed directly, but theolder people who viewed the videos all said ‘‘that was whatwe really felt’’. It was clear from the audience reactions tothe swearing that they really understood the frustration ofthe user. Similarly, within a sequence illustrating a trainingevent, the older user was much more aggressive to the trai-ner than would usually occur in actual sessions, and thisprovided a much clearer depiction of the difficulties thatthe user had than would have been seen in the more com-mon and much more polite response of an insecure olderperson being taught to use a computer for the first time.

The laughter and lively discussion periods after viewingthe videos gave an indication of the success of this method,but this needed to be tested more formally.

2. Testing the impact of the video presentations

We thus wished to explore whether these videos wouldhave a measurable impact on the ‘mind set’ which seemscommon amongst ICT designers. Because undergraduatecomputing students are particularly likely to have such amind set, we incorporate a ‘user centred’ practical projectas part of their training. This includes the evaluation ofpaper prototypes with small groups of older people. Whilethis approach has had clear educational value, its organisa-tion raises the ethical challenges of the inexperience of theundergraduates in dealing appropriately with older adultsin this sort of context. Thus one of the motivations behindthe development of the Trilogy was to serve as ‘orientationtraining’ in preparation for the ‘user involvement’ course

Peter and Jane Buy a WebcamHaving been given her son's old computer, Jane has set about learningthe basics and now feels confident using it for email and word processing. Having come across an article in a paper, she has decided to take the plunge and buya webcam so that she can talk to her daughter and grandchildren in Australia.

Sandy's Mobile AdventureSandy has been given his daughter's old mobile phone. He never uses it but carries it around with him to keep her happy. Today he finally finds a use for it when he locks himself out of his house...

A few months later...

After his problems with the phone, Sandy's daughter gave him a quick lesson and a cheat sheet of simple instructions. Sandy still doesn't use the phone, but carries it with him in the car fully charged... just in case.

A year and a half later...

Since saving him a long walk home, Sandy has changed his attitude towards mobile phones and now uses his frequently. Over a glass of wine his daughter brings this change up...

After the debacle of trying email on his own, Peter has bitten the bullet and signed up to computer classes at the local university. He's feeling his petulant self as ever...

Frustrated with his experiences so far, Peter decides to give email one last try. The computer class is demonstrating a new cut down email application which has been designed for simplicity, clarity and ease of use...

Email ExperiencePeter is quietly jealous of his wife's confidence with using a computer, but is too proud to admit it. One day he finds that she's left it on when she's gone shopping. He decides that this is the chance to give email a go in privacy...

Fig. 1. Synopses of the three films in The Utopia Trilogy (running time 35 min).

A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596 591

activities and the ‘impact’ was assessed at the level of gen-eral awareness rather than more detailed levels of ‘under-standing’ or ‘knowledge’ of the content of the stories.This is based on the view that awareness (e.g. that olderusers are different from the designer) is a prerequisite forseeking out appropriate kinds of ‘user data’ in order toachieve a more suitable design.

A ‘‘before and after’’ questionnaire was developed tomeasure the impact on awareness. The eight items on thequestionnaire were developed to reflect some of the rela-tively stereotypical ‘attitudes’ that often seem to be held

by designers with regard to older people and ICT (seeFig. 2). The questions were designed so that, In relationto the Likert rating scale used, more ‘agreement’ with thesestatements suggested an affinity with the ‘mind set’ that thevideos are intended to challenge.

Initially, two opportunistic samples of potentially suit-able respondents presented themselves. One was a first yearclass of Applied Computing undergraduates, who had notyet undergone any ‘HCI’ courses. The other consisted ofHCI professionals and academics attending a workshopon older people at the HCI 2004 conference. Although

1.Most current interfaces are easy for most people to use.

2.Successful inclusive design is a matter of following the appropriate guidelines properly.

3.Older people are not interested in new technology.

4.A clear and comprehensive user manual is the key to usable technology.

5.Designing for older people is the same as designing for any niche market.

6.If an older person has difficulty with technology there will usually be a younger relative around who can help.

7.Older people have more in common than do younger people.

8.Specialist (rather than mainstream) companies should provide technology suitable for use by people with disabilities or impairments.

Response Categories

Strongly Agree Agree Neutral Disagree Strongly Disagree

Fig. 2. The eight ‘attitude’ statements and associated response categories.

592 A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596

involved in HCI (with many describing themselves as‘interface designers’ or similar), most of this group didnot have any particular experience with older people asusers, and the data from the few who did is not includedin the analysis. There remains a certain degree of self-selec-tion bias in this group, on the basis that they chose toattend a workshop on HCI and older adults, but this wouldtend to limit the potential for raising their awarenessregarding older adults as users.

All groups viewed the videos under essentially the samecircumstances. The UTOPIA Trilogy runs for about35 min overall, and was apparently sufficiently engagingto the audiences such that no interruptions occurred duringany screenings. A brief introduction was given, includingan overview of the content of the videos, and an indicationof their role as the basis for subsequent discussion. For the‘before and after’ questionnaires, respondents indicatedtheir dis/agreement with each of the eight statements ona five point Likert scale (Fig. 2). They completed one setof responses before they had viewed the trilogy and all ofthese forms were collected before the screening began.No explicit mention was made about the ‘after’ question-naires until they were distributed on completion of thescreening. Great care was taken while distributing and col-lecting the forms to ensure that ‘before’ and ‘after’ responseforms were correctly paired. Subsequently, further samplesof suitable student groups have viewed the Trilogy andcompleted the questionnaire under the same circumstances,both from Applied Computing at Dundee and elsewhere(we acknowledge the kind assistance of Dr. Nina Reeves,Department of Multimedia and Computing, University ofGloucestershire, in this regard) and the viewing sessionswere all equivalent to that described above.

These data reflect only the immediate impact of viewingthe Trilogy, and not any longer term impact on the charac-terized ‘mind set’. The initial sample of (Applied Comput-ing) undergraduates who viewed the Trilogy, was thusfollowed up in their subsequent academic year. This wasapproached in a similar way to the now common practiceof obtaining student feedback as part of course review pro-cedures. Thus the students were asked to rate how influen-

tial different elements of the module had been for them inconsidering the people who may use the applications theywere about to develop. The module involved lectures onHCI, the development of (paper) prototypes which weresubsequently utilized to obtain feedback in user evaluationsessions with older people from our volunteer panel, fol-lowed by the ‘completion’ and documentation of their ‘usercentred’ application. The module elements included in thisevaluation questionnaire were; the student’s prior knowl-edge (reading etc.), the lectures, the process of developingtheir prototype, the feedback sessions with the older peo-ple, and the Trilogy session which also included some dis-cussion of issues raised by the videos. Each element wasrated on a five point scale (i.e. ‘not influential’, ‘slightlyinfluential’, ‘moderately influential’, ‘fairly influential’,and ‘very influential’) and the students were asked to con-sider all the elements before giving a rating to each. The‘influence’ questionnaires were distributed in a similarmanner to other ‘course review’ material for anonymouscompletion and return (this occurred around eight monthsafter this undergraduate group had viewed the Trilogy andcompleted the ‘before’ and ‘after’ forms).

The data from this questionnaire is also presentedbelow. The requirements for anonymity disallowed‘within-subjects’ analysis of the relationship between indi-vidual students’ ‘immediate’ (i.e. ‘before’ and ‘after’) and‘delayed’ (i.e. ‘influence’) responses, and the disparate nat-ure (e.g. non-comparable rating scales) of the two question-naires disallowed a meaningful examination of theirconnection using ‘within-groups’ analysis. We are currentlyinvestigating how we can explore any relationship between(for example) the extent to which an individual’s awarenesswas raised and their subsequent perception of how influen-tial that experience had been within the ethical constraintsof course assessments.

3. Results

An initial analysis was carried out on the responses of thetwo samples. One group consisted of first year Applied Com-puting undergraduates (N = 40). The second group was

Before After

Agree

Neutral

Disagree

Ugrads HCI

Fig. 3. Averaged ratings given to all ‘before & after’ statements bycomputing undergraduates and HCI researchers.

A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596 593

made up of HCI professionals and academics (N = 51). Ananalysis of variance for the responses (averaged across allstatements, which gave acceptably normal distributionsand thus allowed examination of interaction effects) showeda significant ‘improvement’ between the ‘before’ and ‘after’responses across both groups (F(1,89) = 40.96, p < .0001),see Fig. 3. Fig. 3 also shows that the HCI group showed more‘disagreement’ overall than did the undergraduates(F(1, 89) = 27.51, p < .0001). The effects of these two factorsalso interacted significantly (F(1, 89) = 10.59, p = .0016)such that the Undergraduates’ responses showed a greaterextent of change than did those of the HCI group. Howeverit is worth noting that the simple effects of this analysis indi-

Undergraduates1 3 7

Agree

Neutral

Disagree

2 4 5 6 8

Fig. 4. Difference in ‘before & after’ ratings for all statem

cate that the extent of change is statistically significant inboth cases (F(1,39) = 43.19, p < .0001) and (F(1,51) =6.45, p = .014), respectively. It is also worth noting thatexamination of the distributions in these analyses indicatesthat none of the results are due to floor or ceiling effects.

A more detailed breakdown of these groups’ responses(see Fig. 4) shows that the overall pattern of effects is sim-ilar across the different statements. One exception is themarked difference in views about the utility of user manuals(Statement 4), which may be due to different levels of expe-rience between the undergraduates and the HCI profes-sionals regarding the; quality, usefulness, usability, andactual usage (by ‘end users’) of such documentation. A sim-ilar but smaller disparity can be seen for Statement 1;‘‘Most current interfaces are easy for most people touse’’, which also seems to be an issue that the undergradu-ates changed their view on the most, having seen thevideos.

These findings suggest that this approach can have animpact on the mind set of people involved (or soon tobe) in the design of ICT for older people.

Subsequent to this analysis, the undergraduate data hasbeen supplemented by further samples of (mostly ‘comput-ing’) students with varying degrees of exposure to HCIissues. This larger sample (N = 198) replicated the overallsignificant change in ‘before’ and ‘after’ responses reportedabove (F(1,196) = 138.88, p < .0001) and as can be seen inFig. 5, the overall pattern of responses across statements isconsistent with the initial smaller undergraduate samplealthough there is a slight tendency for more ‘disagreement’overall in the larger sample.

HCI researchers1 3 72 4 5 6 8

Before

After

ents, between undergraduates and HCI researchers.

Statements

Agree

Neutral

Disagree

1 2 3 4 5 6 7 8

BeforeAfter

Fig. 5. Difference across ‘before & after’ ratings by larger undergraduatesample (N = 198).

594 A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596

Finally, an analysis of the ‘influence’ questionnaires wascarried out. As mentioned above these were completed bystudents involved in the initial ‘before & after’ sampleand it would have been potentially informative to examinethe relationship between these sets of ratings. However,constraints of anonymity for the student feedback didnot allow such ‘within subjects’ analysis between the ‘influ-ence’ data to the earlier ‘before & after’ data. Fig. 6 showsthat no striking differences emerged between the perceivedinfluence of the different module elements, with the aver-aged ratings falling between ‘fairly’ and ‘moderately’ influ-ential (i.e. below ‘very’, and above ‘not’ and ‘slightly’).However, it can be seen that meeting (and getting feedbackon ‘paper’ prototypes from) the older volunteers, andwatching and discussing the UTOPIA Trilogy are consid-ered the most (and similarly) influential.

Comparisons between the ratings for the individualmodule elements using a Wilcoxon matched pairs test indi-cated that the only significant differences among the ‘influ-ence’ ratings for the different module elements werebetween ‘Reading’ and ‘Video’ (N = 43, Z = 2.52,p = .011) and between ‘Reading’ and ‘Meet Users’ (i.e.meeting with the older volunteers) (N = 43, Z = 2.44,

Reading Prototype

Moderately

Fairly

How

Influ

entia

l?

Fig. 6. Ratings for degree of influen

p = .014). Although not significantly different, it can beseen that watching the videos (and meeting older users)were perceived by the students as more influential than ashort series of hour long lectures which had a similar ‘mes-sage’ as a major theme.

4. Discussion

This project produced and evaluated a series of videoswhich were designed to focus on the common issues andto indicate some of the general motivations and difficultiesolder people often have with regard to using ICT. It wasexpected that this would increase the insights which design-ers had about the challenges ICT presented to this group ofusers.

If designers are to appreciate the need to investigate fur-ther the diversity of their users’ preferences and require-ments and gather relevant information so that theresulting design is more suitable for use by a wider rangeof people, acquiring this general insight about the diversityof the older population is an important step. It would bemost effective if this ‘message’ could be conveyed earlierduring the formative education of those aiming for careersinvolving ICT design, but there is also a need to direct thesesorts of message to the professional design community andthe results from questionnaires indicate that the UTOPIATrilogy is effective in both situations.

The importance of this insight regarding the diversity of‘user populations’, is addressed for our undergraduates bythe provision of modules incorporating direct interactionwith older people, an approach which currently seems tobe far from widespread. While these are undoubtedly suc-cessful course elements, there are significant ethical, logisti-cal, administrative, financial and educational challengesinvolved. Therefore, while not in any way a replacementfor ‘the real thing’, an encapsulation of some of the mainbackground issues on video has potential to be a usefulcomplement to the other ways of highlighting the role of

Lecture Meet Users Video

Module

ce for different module elements.

A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596 595

users in interface design. Our experience has also shownthey can act as a helpful orientation for students in prepa-ration for interaction with ‘real’ users.

Much was learned in developing the UTOPIA Trilogy(see Newell et al., 2006a). It was found to be extremelyimportant to ensure that the researchers, script-writer,director, producer and the actors were are all aware ofthe various, and potentially conflicting, agendas involvedin producing an effective video. The tension between theartistic requirements of a rich and rewarding story andthe scientific requirement of conveying particular issuesaccurately needs to be carefully managed to ensure thatthe videos encapsulate a satisfactory balance. There is alsotension between a narrative which primarily raises aware-ness of issues and one which aims to convey ‘‘information’’or other forms of prescriptive guidance and in the presentcase we opted for the former. We are also exploring thepotential for this approach to convey more prescriptiveguidance, which seems greater in regard to coarser ratherthan finer degrees of detail.

There were also constraints due to time/budget pres-sures. The planning of any video shoot must be based onthe script, and once the shoot is concluded (particularly iffurther shooting days are disallowed by the time/budget,as was the case here) there is only so much that can be donein the cutting room. The scripts which were produced couldhave benefited from an additional developmental sessionwith an invited audience where the actors acted out whathad been developed so far. Comments from the audience,researchers and ‘older participants’ in the project, couldhave been added at that stage and the script be sharpenedup in places. We are thus examining ways in which we canrefine the process to give ‘tighter’ and thus more powerfulscripts. There are also interesting challenges exploring howwell this approach can be used to convey relatively more‘detailed’ (or otherwise, more ‘instructional’ for thedesigner) information about users. The aim of the UTO-PIA Trilogy was to present a general, overall ‘message’regarding the idea that other people can have a very differ-ent ‘relationship’ with technology, and this was comple-mented with specific ‘messages’ about some of the typesof difficulty older people often encounter (e.g. the use of‘jargon’, the effects of fragile confidence, etc.). The decisionwas thus made to focus on well established technologies,with two of the vignettes relating to the PC, one involvingthe learning and use of software applications and the otherto the management of hardware peripherals, and oneinvolving coming to terms with owning and using a mobilephone provided by a concerned adult daughter. The com-bination of a general ‘message’ about older users in thecontext of established mainstream technologies makes theUTOPIA Trilogy a good facilitator for discussion on awide range of associated issues in this area and as a rela-tively efficient surrogate for interaction with actual users.

Theatrical techniques for augmenting the communica-tion links between designers and users have been shown inother areas to be very effective in stimulating discussion,

and we will continue investigating the use of this and othertechniques both in conjunction with videos and live perfor-mances for advancing and promoting the ‘‘user sensitiveinclusive design’’ message within the design community.We are also investigating ways in which we can more closelylink such theatrical presentations to scientific and demo-graphic data about older users, so that the messages in suchpresentations can be more grounded in scientific data,whilst still retaining the impact characterized by this genre.Any readers interested in viewing or otherwise utilising theUTOPIA Trilogy should contact the corresponding authoras the videos can be made available on CD-ROM.

References

Blomberg, J., Burrell, M., Guest, G., 2002. An ethnographic approach todesign. In: The Human–Computer Interaction Handbook, vol. 2, pp.964–986.

Boal, A., 1995. The Rainbow of Desire. Routledge, London.Bodker, S., Ehn, P., Sjogren, D., Sundblad, Y., 2000. Co-operative design

– perspectives on 20 years with ‘the Scandinavian IT model. Report forCID (CID-104) Royal Institute of Technology, Stockhom, Sweden,ISSN 1403-073X.

Benyon, D., Macaulay, C., 2002. Scenarios and the HCI-SE designproblem. Interacting With Computers 14, 397–405.

Carmichael, A., 1999. Style Guide for the Design of Interactive TelevisionServices for Elderly Viewers, Independent Television Commission,Winchester. <http://www.computing.dundee.ac.uk/projects/UTOPIA/publications/Carmichael%20-%20DesignStyleGuideFinal.pdf/>.

Carroll, J., 1997. Scenario-based design. In: Helander, M. et al. (Eds.),Handbook of Computer Interaction. Elsevier Science B.V.

Charness, N., Bosman, E., 1990. Human factors and design for olderadults. In: Birren, J., Schaie, K. (Eds.), Handbook of the Psychologyof Aging, 2. Academic Press, Inc., San Diego, CA, pp. 446–463.

Cooper, A., 1999. The Inmates are Running the Asylum. Macmillan, USA.Coyne, K., Nielsen, J., 2002. Web Usability for Senior Citizens. Nielsen/

Norman Group.Czaja, S., Lee, C., 2003. Designing computer systems for older adults. In:

Jacko, J., Sears, A. (Eds.), The Human–Computer Handbook:Fundamentals Evolving, Technologies and Emerging Applications.Lawrence Earlbaum Associates, Mahwah, NJ, pp. 413–427.

Danowski, J., Sacks, W., 1980. Computer communication and the elderly.Experimental Aging Research 6 (2), 125–135.

Diaper, D., Worman, L., 2003. Two falls out of three in the automatedaccessibility assessment of World Wide Web Sites: A-Prompt v.Bobby. In: Johnson, P., Palanque, P. (Eds.), People and ComputersXVII. Springer-Verlag.

Dickinson, A., Eisma, R., Syme, A., Gregor, P., 2002. UTOPIA: Usabletechnology for older people: Inclusive and appropriate. In: Brewster,S., Zajicek, M. (Eds.), A New Research Agenda for Older Adults,Proc. BCS HCI, London, pp. 38–39.

Dishman, E., 2003. Designing for the New World. In: Laurel, B. (Ed.),Design Research. MIT Press, pp. 41–48.

Docampo-Rama, M., de Ridder, H., Bouma, H., 2001. Technology generationand age in using layered user interfaces. Gerontechnology 1 (1), 25–40.

Eisma, R., Dickinson, A., Goodman, J., Mival, O., Syme, A., Tiwari, L.,2003. Mutual inspiration in the development of new technology forolder people, Include 2003, London, pp. 7:252–7:259.

Eisma, R., Dickinson, A., Goodman, J., Syme, A., Tiwari, L., Newell, A.,2004. Early user involvement in the development of InformationTechnology-related products for older people. Universal Access in theInformation Society 3/2, 131–140.

Fozard, J., 1990. Vision and hearing in aging. In: Birren, J.E., Schaie,K.W. (Eds.), Handbook of the Psychology of Aging. Academic Press,London, pp. 150–170.

596 A. Carmichael et al. / Interacting with Computers 19 (2007) 587–596

Gregor, P., Newell, A., 2001. Designing for dynamic diversity – makingaccessible interfaces for older people. In: Jorge, J., Heller, R., Guedj,R. (Eds.), Proceedings of the 2001 EC/NSF Workshop on UniversalAccessibility of Ubiquitous Computing: Providing for the Elderly,A.C.M, pp. 90–92.

Gregor, P., Sloan, D., Newell, A., 2005. Disability and technology:building barriers or creating opportunities. In: Zelkowitz, M.(Ed.), Advances in Computers. Elsevier, Academic Press, pp.284–346.

Hawthorn, D., 2000. Possible implications of aging for interface designers.Interacting With Computers 12 (5), 507–528.

Hawthorn, D., 2002. Issues related to application complexity whendesigning for older users, SIGCHI-NZ Symposium, PalmerstonNorth.

Keates, S., Lebbon, C., Clarkson, P., 2000. Investigating industry attitudesto universal design. In: RESNA 2000. Orlando, USA, pp. 276–278.

Kelly, B., 2002. WebWatch: An Accessibility Analysis of UK UniversityEntry Points, Ariadne, Issue 33. <http://www.ariadne.ac.uk/issue33/web-watch/>.

Kelly, P.L., Kroemer, K.H.E., 1990. Anthropometry of the elderly: statusand recommendations. Human Factors 32 (5), 571–595.

McKenna, S., Marquis-Faulkes, F., Newell, A., Gregor, P., 2003.Scenario-based drama as a tool for investigating user requirementswith application to home monitoring for elderly people. In: Proceed-ings of the 10th International Conference on Human–ComputerInteraction, Crete.

Mitchell, V., Nicolle, C., Maguire, M., Boyle, H., 2007. Web-basedinteractive TV services for older users. Gerontechnology 6 (1),20–32.

Newell, A., Carmichael, A., Morgan, M., Dickinson, A., 2006a. The use oftheatre in requirements gathering and usability studies. Interactingwith Computers 18 (5), 996–1011.

Newell, A., Dickinson, A., Smith, M., Gregor, P., 2006b. Designing aportal for older users: A case study of an Industrial/AcademicCollaboration. Transactions on Computer–Human Interaction 13(3), 347–375.

Newell, A., Gregor, P., 1997. Human computer interfaces for people withdisabilities. In: Helander, M. (Ed.), Handbook of Human–ComputerInteraction. Elsevier Science, pp. 813–824.

Newell, A., Gregor, P., 1999. Extra-ordinary human-machine interaction:what can be learned from people with disabilities? CognitionTechnology and Work 1 (2), 78–85.

Newell, A,. Gregor, P., 2000. User sensitive inclusive design. In: Scholtz,J., Thomas, J. (Eds.), CUU 2000 First ACM Conference on UniversalUsability 39–44.

Office of the e-Envoy, 2002a. Guidelines for UK Government web-sites: Illustrated handbook for web management teams (May 2002),see; <http://www.e-envoy.gov.uk/oee/oee.nsf/sections/webguidelines-handbook-top/file/handbookindex.htm/>.

Office of the e-Envoy, 2002b. Guidelines for UK government websites:Framework for local government (August 2002), see; <http://www.eenvoy.gov.uk/oee/oee.nsf/sections/webguidelines-handbook-top/file/handbookindex.htm/>.

Office of the e-Envoy, 2002c. Government Metadata Standard (April2002), see; <http://www.e-envoy.gov.uk/oee/oee.nsf/sections/guide-lines-metadata/$file/index.htm/>.

Petrie, H., Hamilton, F., 2004. The Disability Rights Commission FormalInvestigation into Web Site Accessibility. In: Dearden, A., Watts, L. (Eds.),Proceedings HCI 2004: Design For Life (Leeds, UK), pp. 157–158.

Rabbitt, P., Carmichael, A., 1994. Designing communications andinformation-handling systems for disabled and elderly users. In: Snel,J., Cremer, R. (Eds.), Work and Aging: A European Perspective.Taylor and Francis Ltd., London, pp. 173–195.

Robey, D., Markus, M., 1998. Beyond rigor and relevance: producingconsumable research about information systems. InformationResources Management Journal 11, 7–15.

Sloan, D., Gregor, P., Booth, P., Gibson, L., 2002. Auditing Accessibilityof UK Higher Education Web sites. In: Novick, D., Scholtz, J. (Eds),Interacting with Computers, vol. 14 (4), Elsevier Science, pp. 313–325.

Thatcher, J., 2003, Web Accessibility – What not to do. <http://www.jimthatcher.com/whatnotp.htm/>.

Williamson, K., Bow, A., Wale, K., 1997. Older People and the Internet,Link-Up, March, pp. 9–12. <http://infotech.monash.edu/itnr/reports/olderp2.html/>.

World Wide Web Consortium, 1999. Web Content Accessibility Guide-lines 1.0, (May 1999). <http://www.w3.org/TR/WCAG10/>.

World Wide Web Consortium, 1998,.Cascading Style Sheets, level 2, (May1998). <http://www.w3.org/TR/REC-CSS2/>.

Zajicek, M., 2004. Successful and available: Interface design exemplars forolder users. Interacting with Computers, vol. 16, Elsevier Science, pp.411–430.

Zajicek, M., 2005. Older adults: Key factors in design. In: Perhonen, A.et al. (Eds.), Future Interaction Design. Springer Verlag, London, pp.151–176.