capturing teachers’ experience of learning design through case studies

This article was downloaded by: [Tufts University]On: 08 October 2014, At: 09:14Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Distance EducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cdie20

Capturing teachers’ experience oflearning design through case studiesElizabeth Masterman a , Jill Jameson b & Simon Walker ba Oxford University Computing Services , 13 Banbury Road,Oxford, OX2 6NN, UKb School of Education and Training , University of Greenwich ,Mansion Site, Bexley Road, Eltham, London, SE9 2PQ, UKPublished online: 22 Jul 2009.

To cite this article: Elizabeth Masterman , Jill Jameson & Simon Walker (2009) Capturing teachers’experience of learning design through case studies, Distance Education, 30:2, 223-238, DOI:10.1080/01587910903023207

To link to this article: http://dx.doi.org/10.1080/01587910903023207

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/cdie20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/01587910903023207

http://dx.doi.org/10.1080/01587910903023207

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Distance EducationVol. 30, No. 2, August 2009, 223–238

ISSN 0158-7919 print/ISSN 1475-0198 online© 2009 Open and Distance Learning Association of Australia, Inc.DOI: 10.1080/01587910903023207http://www.informaworld.com

Capturing teachers’ experience of learning design through case studies

Elizabeth Mastermana, Jill Jamesonb* and Simon Walkerb

aOxford University Computing Services, 13 Banbury Road, Oxford OX2 6NN, UK; bSchool of Education and Training, University of Greenwich, Mansion Site, Bexley Road, Eltham, London SE9 2PQ, UKTaylor and FrancisCDIE_A_402493.sgm(Received 3 November 2008; final version received 5 February 2009)10.1080/01587910903023207Distance Education0158-7919 (print)/1475-0198 (online)Original Article2009Open and Distance Learning Association of Australia Inc.302000000August [email protected]

This article distinguishes three dimensions to learning design: a technologicalinfrastructure, a conceptual framework for practice that focuses on the creation ofstructured sequences of learning activities, and a way to represent and sharepractice through the use of mediating artefacts. Focusing initially on the second ofthese dimensions, the article reports the key findings from an exploratory study,eLIDA CAMEL. This project examined a hitherto under-researched aspect oflearning design: what teachers who are new to the domain perceive to be its valueas a framework for practice in the design of both flexible and classroom-basedlearning. Data collection comprised 13 case studies constructed from participants’self-reports. These suggest that providing students with a structured sequence oflearning activities was the major value to teachers. The article additionallydiscusses the potential of such case studies to function as mediating artefacts forpractitioners who are considering experimenting with learning design.

Keywords: learning design; case studies; pedagogic practice; mediating artefacts;LAMS; Moodle

Introduction

Sharing has been a cornerstone of the concept and practice of learning design since itsearliest days (e.g., Britain, 2004). The purpose of making materials available to others(and, conversely, consulting materials created by others) is not just to provide exem-plar solutions to specific problems that crop up in everyday practice, but also – andarguably more importantly – to point newcomers in the domain towards good or effec-tive practice in designing for online learning in a range of settings, including flexibleand classroom-based. However, it is not enough to present such practitioners with afinished lesson plan to copy or emulate; they also need guidance on how to create onefor themselves. For, as the Chinese proverb reminds us, ‘Give a man a fish and youfeed him for a day. Teach a man to fish and you feed him for a lifetime.’ This, though,raises further questions: should that guidance on process be prescriptive: that is, tellteachers in a didactic manner, by giving them a set of steps to follow with the impli-cation that this will lead to success? Or, might it be descriptive: that is, offer teachersaccounts of what has been done by others, from which they might discover lessons forthemselves (in both senses of the phrase)? Moreover, should a descriptive account

*Corresponding author. Email: [email protected]

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014

224 E. Masterman et al.

report expert practice, as much of the present learning design literature does, and focuson success as measured in terms of the performance improvements achieved by thestudents? Or, might there also be value in making available accounts of teachers’initial experience of using learning design: the uncertainty as well as the excitement,the failings as well as the positive outcomes, the trade-offs as well as the new possi-bilities afforded by learning design tools?

This article takes the first steps towards evaluating descriptive accounts of emer-gent (i.e., as distinct from expert) practice in learning design. It examines case studiesdeveloped from school and college teachers’ first-hand accounts of their early encoun-ters with learning design as a framework for practice and with its supporting technol-ogies (learning design tools). The distinctive feature of these case studies was anemphasis on teachers’ learning outcomes in terms of what they gained from the expe-rience, rather than on those of their students. The case studies were developed in anexploratory study, the eLIDA CAMEL project, funded by the UK’s Joint InformationSystems Committee (JISC) from May 2006 to December 2007. Their purpose was toanswer a specific research question: ‘What do teachers who are new to learning designperceive to be its value?’ Building on the findings of that project, the article sets itselfan additional objective: to explore the extent to which case studies of this kind mightfunction as ‘mediating artefacts’ that capture and represent practice and can subse-quently be deployed in scaffolding the learning design practice of other teachers(cf. Conole, 2008).

The article opens by outlining the theoretical position that it adopts vis-à-vis learn-ing design as a domain, and case studies as mediating artefacts within that domain,before giving an overview of the eLIDA CAMEL project and the case studiesproduced. A cross-case analysis of teachers’ experiences of learning design follows,and a reflective discussion of the findings in relation to the objectives indicates possi-ble directions for future research.

Theoretical position

Dimensions of learning design

Broadly speaking, we perceive three distinct, but overlapping, dimensions to therapidly evolving domain of learning design: learning design as a technological infra-structure, learning design as a framework for practice, and learning design as a way tomodel and share practice through appropriate representations.

Work within the technological dimension of learning design has largely centred onthe development of an infrastructure of authoring applications (editors) and runtimeplayers that implement a computational model of teaching and learning, notably theIMS LD specification. Koper (2006) and Griffiths and Liber (2008) have providedoverviews of these tools, which include Reload, CopperCore, and COLLAGE. Thescope of such tools extends to those that are either partially compliant with IMS LD,such as the Learning Activity Management System (LAMS), or not yet compliant. Anexample of the latter is Moodle, which lends itself to an activity-oriented approach todesign and is en route to becoming IMS LD-compliant (Griffiths & Liber, 2008).

Learning design as a conceptual framework for practice – that is, as a set of prin-ciples for thinking about, and approaching, what one does – has been the subject of abody of research carried out in the UK under the aegis of the JISC. It upholds threebasic concepts: (1) a focus on learning through activity rather than through theabsorption of content, (2) the structuring of those activities into sequences, and (3)

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014

Distance Education 225

the sharing of such sequences for the dissemination of good practice (cf. Britain,2004). In addition, it pays attention to the systematic nature of design as a plannedprocess:

designing, planning and orchestrating learning activities as part of a learning session orprogramme [with] a new emphasis on those aspects of the process that can be ‘designed’in advance, and articulated and shared across different contexts of practice. (JISC, 2006)

This dimension embraces any technology to support learning, not just IMS LD-compliant tools, and led JISC to rebrand practice in 2006 as ‘design for learning’ toreinforce the distinction from the technological dimension. However, even thoughthe work described here was conducted under the rubric ‘design for learning,’ forconsistency with other articles in this special issue we revert to the term learningdesign.

Both the technological and the practice dimensions of learning design are, of course,interdependent in that designing learning experiences is a human activity mediated, andideally enhanced, through the deployment of purpose-designed tools. They convergein the third, representational dimension: learning design as a way to model and sharepractice in the creative use of technology (Agostinho, 2008) through a range of repre-sentations, or ‘mediating artefacts’ (Conole, 2008). The challenge that researchers inthis dimension set themselves is to formulate a notation to document teaching and learn-ing practice ‘so that it can serve as a description, model, or template that can be adaptableor reused by a teacher to suit his/her context’ (Agostinho, 2008, p. 3). Such notationsinclude IMS LD and other structured languages, patterns (McAndrew & Goodyear,2007), tabular lesson plans, concept maps, and most pertinent to this article, narrativetext in the form of case studies (Falconer & Littlejohn, 2008).

Case studies as mediating artefacts

Mediating artefacts have a twofold role. First, they capture and represent learning activ-ities – ‘tasks that students undertake to achieve a set of intended outcomes’ (Conole,2008, p. 190) – in such a way that other practitioners can understand them. Second,through functioning as either ‘runnable’ or ‘inspirational’ (Falconer & Littlejohn,2008) representations, they can help to inform the decisions made by practitioners whowish to design learning activities for their own students. For this purpose, they maybe aggregated into metamediating artefacts: for example, repositories containing casestudies (amongst other artefacts), and pedagogy planner tools that incorporate casestudies into the advice and guidance offered to users.

The strength of case studies is that they ‘provide rich contextually located mediat-ing artefacts’ (Conole, 2008, p. 193) that describe a design for a specific interventionand its attendant outcomes. However, this specificity is also their great shortcoming:‘because they are so contextually located, they may be difficult to adapt or repurpose’(Conole, 2008, p. 193). Moreover, the narrative format of case studies makes it harderfor others to discern the plan underlying the learning experience being described: thus,they are inspirational (i.e., inspire teachers to change their practice) rather than runna-ble (i.e., can be implemented as a sequence of learning activities for students to workthrough).

The question of interest to this article is to what extent case studies of novice learn-ing design practice can be considered as inspirational mediating artefacts, particularly

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


given their emphasis on the experiences and learning outcomes of the teachers. Cansuch case studies really serve to mediate decisions about the design process, or mightthey have an altogether different mediational role: namely, in motivating those teach-ers who are fearful in the face of technology to overcome their affective barriers toengagement with learning design?

Evidence lends support to this proposal. For example, Falconer and Littlejohn(2008) have found that teachers seem to find contextualised examples more conduciveto changing their practice than abstract models, while Masterman (2008) has reportedevidence that they may relate more easily to the experience of peers than to guidancefrom outsiders such as learning technologists. Therefore, in addition to examining thevalue to teachers of the practice dimension of learning design manifested in theeLIDA CAMEL case studies, this article will infer, from an analysis of the casestudies, their potential for this alternative mediational role in the representationaldimension.

Context: the eLIDA CAMEL project

A multi-partner project, eLIDA CAMEL was led by the University of GreenwichSchool of Education and Training and formed part of the JISC Design for LearningProgramme. It aimed to build a community of practice for the critical evaluation of,and feedback on, practitioners’ use of learning design tools and pedagogic practiceamong the partner institutions. The project comprised two complementary strands:pedagogic and social.

The pedagogic strand (eLIDA: e-Learning Independent Learning Design Activities),which forms the subject of this article, was an exploratory study of the development,implementation, and evaluation of learning design activities in post-compulsory educa-tion (i.e., education over the age of 16) in the UK. It expanded on the work of a prede-cessor project, JISC eLISA (Walker & Masterman, 2006), which had investigated thedevelopment, in LAMS and Moodle, of learning activity sequences to develop studyskills.

The social strand focused on community activities – both online and face-to-face– to support teachers’ engagement with learning design in the pedagogic strand. Theseactivities were structured according to the CAMEL (Collaborative Approaches to theManagement of e-Learning) model of a peripatetic supportive community (Ferrell &Kelly, 2006; Jameson, Ferrell, Kelly, Walker, & Ryan, 2006) and are described inmore detail by Jameson (2008). The strands were connected through a mentor–menteerelationship, in that individual practitioners working in the eLIDA (pedagogic) strandwere supported within their institutions by mentors who were themselves members ofthe cross-institutional CAMEL community.

The principal outputs from the pedagogic strand were a set of learning designscreated in either LAMS or Moodle, together with case studies detailing how individ-ual practitioners created those designs and ran them with students in classroom-onlyor combined classroom + distance settings. These case studies were intended toanswer the question: ‘What do teachers who are new to learning design perceive to beits value?’ The distinctive feature of these case studies was their focus on the learningexperiences (and learning outcomes) of the teachers. As well as finding out whatworked well for practitioners, what was effective and what was achieved, the researchteam wanted to know the converse: what did not work, was difficult, or remainedunachieved.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


Method

Participants

The participants in eLIDA CAMEL who contributed case studies were a mixture ofmentors and mentees drawn from five of the institutions involved in the project part-nership: four further education colleges and a secondary school. All participants wereexperienced teachers and familiar to varying extents with technology-enhanced learn-ing. Mentors were selected according to the aptitude for community-focused workingthat they had demonstrated through involvement either in the JISC eLISA project orin previous CAMEL activities. They were required to:

(1) select mentees in their institution;(2) introduce mentees to the basic principles of learning design as a framework for

practice as outlined in the ‘Dimensions of Learning Design’ section above;(3) train mentees in the tool to be used;(4) help mentees develop a sequence of learning activities, including advising on

appropriate activities, locating appropriate resources to support the learningactivities so that mentees could focus on planning the activities themselves,and critiquing mentees’ draft designs;

(5) assist in running the sequence with students.

For their part, mentees were to produce at least one learning activity sequence,evaluate the sequence(s), and contribute a case study. Some mentors also contributedactivity sequences and case studies. Although they had experience creating andrunning sequences in LAMS or Moodle, this had only been in the context of the JISCeLISA project. Thus, they were only a few steps ahead of their mentees and could stillbe considered as relative novices in learning design.

Data collection

Data were collected both from teachers, using structured templates that were latercollated into narrative case studies, and from students, using an online questionnaire.

Teacher case studies

Data for the case studies were collected remotely through teachers’ self-reports. Wecreated two structured templates for this purpose, as this form of artefact can ‘ensurethat essential information about each case is captured, and allow for easier reading andanalysis’ (JISC, 2006). Questions were derived from a number of existing templates,including a ‘template for describing an example of (e)learning’ developed by JISC(n.d.) and the ‘Effective practice planner’ (JISC, 2004). Template A addressed theexperience of creating a learning design and was completed when teachers had finishedcreating their sequence of learning activities. Template B covered the experience ofrunning a learning design and was completed after the actual learning session. Thetemplates were finalised in collaboration with the mentors to ensure their acceptability.

As well as providing factual information that would enable readers of the casestudies to understand the context, the questions laid emphasis on the affective andreflective dimensions, in order to capture the experience of using learning design andits impact on practice. Aspects of learning design practice covered by the questionsincluded:

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


● Context: Topic; age of students; whether the lesson was a new one or was rede-signed from an existing plan; setting (classroom, distance, or both); individualdifferences among students: for example, abilities, learning styles, and linguisticskills.

● Initial expectations: What teachers hoped to achieve by using their chosen learn-ing design tool: for example, motivate students, structure learning, support flex-ible learning.

● Design process and tools: The starting point of the design process (learningoutcomes, assessment criteria, content, specific activities); tools used (paper-and-pencil, digital); difference from their usual approach; what was gained (orlost) by using LAMS or Moodle; guidance received from mentor.

● Learning outcomes: Intended outcomes, assessment criteria.● Learning activities: Rationale for the activities chosen and the overall format of

the lesson.● Running the sequence: Number of students; additional staff present; extent to

which the learning outcomes were achieved; any differences in performancefrom previous modes of delivery; any unexpected outcomes; impact of LAMSor Moodle on lesson delivery and on students’ experience (as perceived by theteacher).

● Reflections: What worked well (and the converse); what benefits and problemshad been envisaged and whether they had been realised; teachers’ level ofenjoyment; impact of the experience on teachers’ approach to design and theiruse of technology.

Most of the questions captured qualitative data; however, we included a few multi-ple-choice questions that could be analysed quantitatively: for example, a questioneliciting teachers’ initial expectations which offered a list of choices used in an earlierevaluation of LAMS (Masterman & Lee, 2005).

Case studies were compiled from the teachers’ templates only in those instanceswhere teachers had completed both templates and the activity sequence itself wasavailable to the project team. They were written by the first author, and each took theform of a short (one- to two-page) third-person narrative interspersed with directquotations from the teacher and, where available, students. The third person wasadopted in preference to the first person as the narratives were not first-hand testimo-nies, but had been constructed from structured templates by a near-stranger.

Each case study was divided into six sections: (1) contextual information, (2) anoutline of the design challenge facing the teacher, (3) the process of designing thesequence, (4) the experience of running the sequence with students, (5) impact ofusing the chosen learning design tool, and (6) the learners’ experience (where theirfeedback had been obtained).

For confidentiality, teachers were given pseudonyms and were asked to approvetheir own case study before it was made publicly available.

Student questionnaire

To triangulate teachers’ accounts, we designed an online questionnaire to elicitstudents’ comparisons of their experience of learning in LAMS or Moodle with theirusual mode of learning. Teachers administered this questionnaire themselves at theend of the learning activity sequence. For reasons of space, the data are not reported

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


in detail here, although some quotations have been included to supplement the teach-ers’ testimonies.

Data analysis

Teachers’ completed templates were analysed in order to elicit cross-cutting themesthat would help answer the question: ‘What do teachers new to learning designperceive to be its value?’ Content analysis (Krippendorff, 2004) was applied in an iter-ative process of reading, examining, identifying, coding, and comparing themeswithin and across the different case studies. Given the small number of templates, andtheir structured format, it was feasible to code them using a word processor, collatingrelated contributions in tables. A number of themes had been pre-identified in thetypes of question included in the template, on the basis that teachers’ responses wouldyield data pertinent to the research question: for example, their expectations and theextent to which these were met, and differences from their usual practice. However,in our analysis we remained alert to unexpected sub-themes that might emerge fromthe data. These would be established from the frequency of their occurrence and theprominence accorded to them by individual participants.

Quantitative data, both from the teachers’ templates and from the students’ ques-tionnaires, were transferred to spreadsheets for analysis. As already noted, students’free-text questionnaire responses were used to triangulate their individual teachers’reports, and so were not subjected to detailed analysis across respondents andinstitutions.

Findings

Overview of case studies by institution

In all, 13 narrative case studies were compiled from contributions by 11 teachers, assummarised in Table 1. Student feedback from the online questionnaire was suppliedfor seven lessons. The case studies can be found at http://dfl.cetis.ac.uk/wiki/index.php/ELIDA_CAMEL.

College W constitutes an example of a mentor (Rachel) with experience of LAMShelping her two colleagues – both novice users of the system – to create and run theirown learning designs for classroom use with students of English as a Second or OtherLanguage (ESOL), who were immigrants from a wide range of countries. Thesequences fulfilled the dual function of developing students’ language and informa-tion technology (IT) skills simultaneously. Rachel also tried a mentee’s sequence withher own students, providing the project with its only example of sharing and reuse.

Mentor–mentee relationships were also strong in College E. Here, the mentees hadthe support of two mentors: one a technical expert in Moodle (Edward); the other asubject expert who had also learned to use the tool. This was also the only institutionin which the tool was already well embedded, although the two mentees who contributedcase studies had not used it before. The project yielded designs for three blended learningexperiences extending over several weeks. The ‘Child Care’ and ‘First Aid’ courseswere intended to support students who fit their studies around other commitments.

College M was the only institution without prior experience of a learning designtool. However, two business studies teachers contributed five short sequences tosupport students’ revision and teach a generic skill, although technical problemsimpeded plans to support flexible off-campus learning.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


College N’s participation in eLIDA CAMEL was limited by other commitments.Nevertheless, a senior English teacher, Rhona, was able to create a simple set ofMoodle activities to support the study of poetry.

The teacher at School K, Christine, was the project’s only lone practitioner.Technologically speaking, she was also its most adventurous, trying out not onlyMoodle and two versions of LAMS, but also running a LAMS sequence directly fromMoodle. Her three case studies demonstrate, inter alia, the use of technology for flex-ible learning by older school students.

The value of learning design to practitioners

In this section we present the data from the case studies that help to answer theresearch question: ‘What do teachers who are new to learning design perceive to beits value?’ For this purpose, we have grouped the data into four categories:

(1) the value of learning design in terms of the learning experiences that teacherscould offer their students;

(2) the value of learning design to teachers’ practice;(3) the less successful aspects of teachers’ experience that detracted from its value;(4) teachers’ messages to other practitioners regarding the value of learning design.

Table 1. The eLIDA CAMEL case studies.

Name Case studyAge group Tool Setting

Student feedback

College WRachel ESOL – Verbs Adult LAMS v.1 Classroom YRachel ESOL – Learning Styles Adult LAMS v.1 Classroom NWilliam ESOL – Adjectives Adult LAMS v.1 Classroom YAndrew,

RachelESOL – Punctuation Adult LAMS v.1 Classroom Y

College EEdward E-learning – VLE Adult Moodle Class+Dist NLinda First Aid Adult Moodle Class+Dist YAlison Child Care 16–19 Moodle Class+Dist Y

College MOlivia Business (four sequences) 17–20 LAMS v.2 Classroom;

Class+DistN

Nicola Referencing Adult LAMS v.2 Classroom N

College NRhona Literature – Poetry Adult Moodle Classroom N

School KChristine Literature – Novel 16–17 Moodle Class+Dist YChristine Literature – Critical Reading 16–17 LAMS v.1 in Moodle Classroom YChristine Literature – Travel Writing 15–16 LAMS v.2 in Moodle Classroom N

Note: In the Setting column, Classroom denotes that teacher and students were present in the same location, with the teacher facilitating the session; Class+Dist denotes a combination of classroom activities and flexible learning.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


1. Value in terms of learning experiences offered to students

We consider the value of learning design firstly in the ways that teachers’ key expec-tations were met regarding the learning experiences they could offer to their students.These expectations, which were elicited when they completed the case studytemplates, were motivating students, providing more structured learning experiences,supporting different abilities, and supporting flexible learning.

All of the teachers shared the expectation that implementing learning design wouldmotivate students, if only to make a dry topic more interesting (as in Nicola’s case).A number reported that their students did indeed seem more engaged than usual bythis experience of technology. Indeed, one of Linda’s students noted: ‘It helps you toremember what you did in class. It is more interesting than going through a book orhandout.’ However, as Rhona observed, the experience might not motivate studentswith low IT skills, particularly if the technology appears to be bolted on, rather thanintegral to the learning session.

Another expectation was that adopting learning design would result in more struc-tured learning. In her ‘ESOL – Verbs’ design, Rachel spoke of breaking down thelearning into manageable chunks. Linda’s ‘First Aid’ course was explicitly structuredto combine theory with practice (which she felt would be more interesting for thestudents), and classroom with home-based learning.

For Christine, the ability to structure students’ online discussions, in particular,was a key difference from designing for a face-to-face setting. In the ‘Literature –Critical Reading’ case study, she wrote that LAMS added both breadth and structureto the design, allowing her to structure and vary the discussions more than she mightotherwise have done. She also structured the activities in accordance with the typesof skills students were to apply. Students had already indicated that they found higher-order skills such as critical analysis of the language difficult, and so these werepractised in the classroom. Lower-order skills centred on understanding the narrative,so Christine allocated these activities to independent study, providing questions tosupport students’ reading.

Olivia saw the structured learning experience as a stepping-stone to independence:‘LAMS […] allows students to become independent learners and take responsibilityfor their own learning within a framework of guidance from their teachers.’

Providing meaningful learning experiences for a range of abilities (differentia-tion) was another key expectation. Both Rachel and Andrew built extension activitiesfor early finishers into their LAMS sequences. Having students working indepen-dently through a LAMS sequence that automatically moved them on to the next activ-ity meant that Rachel could give personal support to any student in need.

Linda used collaborative learning activities, as well as flexible access to her ‘FirstAid’ course, to accommodate differentiation:

When using the auditory and video sequences students were asked to take notes, thensplit into groups to share the information they had written, then feedback to whole class.This was done to enable all students to contribute and ensure the weaker students had acomplete set of notes to aid their development.

In this way, she believed, weaker students would not feel out of their depth as theywould be able to go through the topic at home and at their own pace.

Providing supportive flexible learning opportunities to part-time students withfamily responsibilities was important to Linda and Alison, and their learning designs

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


specifically exploited the affordances of online learning in this respect. Linda’s expec-tations were met by a spectacular 20% improvement in the number of students passingthe exam first time:

I believe it was because students had time to go over the sessions at home […] This gavethem confidence, and when asked they attributed this to being able to go revisit the areasthey were unsure about, in their own time and a pace to suit them.

Their students fully appreciated the benefits of flexible access to the course materials,as these two comments from the student questionnaire show:

The fact that the resource is accessible from anywhere was comparable to having a tutoron call 24 hours a day. It is also easy to review any bits you didn’t understand during theclass.

We knew that the resources were the ones that were important for our course, so we didnot need to try to find additional material, which save[s] valuable [time]. This is impor-tant if you have a family.

Although Andrew created his ‘ESOL – Punctuation’ for classroom use, he wasinterested in fostering students’ independent learning by giving them different typesof learning resources outside class and allowing them to work at their own pace. Afterrunning the sequence he reported that students had indeed expressed interest in flexi-ble access, from which he concluded that LAMS had stimulated in them a will to learnindependently.

2. Value of learning design to teachers’ practice

In this section, we present evidence of the extent to which learning design had influ-enced, in a positive way, how teachers thought about, and approached, their practice(i.e., the extent to which they had modified their conceptual framework). Themesincluded extending teachers’ thinking in relation to learning activities, a greaterawareness of learners’ point of view, a more dynamic teacher–learner relationship,and an intention to embed the tools in their practice.

Teachers were already taking an activity-focused approach to designing theirstudents’ learning before their participation in eLIDA CAMEL, but found that usinglearning design tools helped to extend their thinking. William wrote: ‘LAMS encour-ages learning through a range of activities. I suppose that I was encouraged to thinkhow different activities could be adapted to achieve the learning objectives.’ Christinefound her attention being re-drawn to students’ engagement, noting the importance ofproviding a variety of activities in order to sustain their interest. However, she recog-nised also the need to stage the activities on offer according to students’ existing skills,and to use some activities as an introduction to the technology as much as to the topicof study (‘Literature – Novel’).

Some teachers described how the experience of learning design had stimulatedthem to consider more deeply how things might look from the students’ point of view.For Nicola, reinterpreting the ‘Referencing’ design in a blended environment includedrethinking how to motivate her students. Previously when teaching the Harvard refer-encing system, she would either let them work through it in their own time or highlightthe main points and provide multiple-choice questions as a small-group activity. The

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


latter technique was fairly easy to incorporate into LAMS; however, choosing activi-ties for the students ‘made me look at the content from a learner’s perspective, so thatI could ensure that the elements would be engaging and easy to understand, as well asaccomplishing the learning that I want the learner to achieve.’ The payback was anovel opportunity to stimulate the thinking and responses of individual students:something that had not been possible in the ways she had previously covered the topic.

There is also some evidence of a developing awareness of a more dynamic rela-tionship between teachers and learners. Christine attributed her changing awarenessspecifically to technology:

It has taken me out of the centre – I think technology will change the role of the teacher… [The students] have gained a deep knowledge of the text with perhaps more original-ity in their comments because I have taken a step back.

This led her to:

think about the place of the teacher and the role of the teacher. I would like to experi-ment in some way with students as designers. We often do presentations on books, andI would like to do a group work activity where instead […] students put together a seriesof on-line sessions designed to allow [other] students to familiarise themselves with thetopic.

However, a teacher-led approach may retain its place: for example, Andrew feltthat in tasks that are aimed at conceptual change, there is a need for the teacherconstantly to check students’ emergent understanding. Moreover, students themselvesstill want a human teacher on the scene, whether they are at a dependent stage in theirlearning (like the ESOL students at College N) or have reached the level of indepen-dent sophisticated analysis (like Christine’s class).

Many of the teachers were not only confident that they would use their learningdesign tool again, but were also contemplating redesigning all their lessons to incor-porate it. For example, Rhona planned ‘to take my whole scheme of work and recreateit in LAMS.’ However, they also recognised a continued need for mentoring, some toreinforce their confidence in the basics of designing in LAMS or Moodle, and othersto extend their reach. In this respect Olivia noted: ‘I would need further support todevelop more complex sequences in order to produce a wider variety of activities.’

Having participated in both the JISC eLISA and eLIDA CAMEL projects, Racheland Christine had reached the point where learning design tools formed a natural partof their practice:

LAMS fits seamlessly into the way I usually plan lessons. (Rachel)

Thinking about [Moodle and LAMS] has become an integral part of the planning processto the point where I now find it difficult not to naturally load things into Moodle and thenplay with the structure. […] I don’t always plan on paper or in Word now. For me thatis a real change that has happened in the last year. I think that Moodle and LAMS makeplanning much easier. (Christine)

Indeed, Christine now planned her lessons directly in Moodle, explaining:

It was useful to have put all resources into the Moodle site first. This enabled me to getan overview – it was like an effective filing system. I think also the fact that all the toolsare listed enabled me to think about what to use and when.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


3. Less successful aspects of the experience of learning design

Although teachers’ experience of learning design was overwhelmingly positive, somereported aspects of their experience that were less successful, and which detractedfrom its overall value.

Some teachers, who lacked learning support assistants to work with individualstudents, found their attention was taken up helping particular students who had diffi-culties using IT. They were therefore unable to check on the rest of the class. Intheory, the LAMS Monitor should make it easy to keep an eye on the whole class but,as Nicola found, it can help only if the teacher has time to look at it.

Improvement in learning outcomes has become one of the principal yardsticks bywhich digital technologies stand or fall. However, despite some success stories (asreported by, for example, Linda, Rhona, and Rachel in ‘ESOL – Verbs’), the intendedlearning outcomes were not achieved in a number of the eLIDA CAMEL case studies.

The technology itself was one factor: not just because it broke down (as in William’ssequence), but also because it was simply inappropriate for particular tasks. After exper-imenting with LAMS for teaching modal verbs, Rachel concluded: ‘LAMS is good forreading and writing but not speaking or listening, although you could add a listeningexercise from the Web. I use more listening and speaking exercises when I don’t useLAMS.’

For Andrew, another factor was the need for closer teacher involvement in blendedtasks: ‘to achieve a real improvement, students need more controlled tasks beforedoing the final assessment, with a chance for the teacher to check their work closelyand provide feedback along the way.’ Interestingly, a similar view was voiced bysome of Christine’s students.

Other teachers felt that their design was to blame, rather than the technology. Forexample, Olivia ascribed the lack of improvement in some students’ essay writing tothe writing frame she had created for them.

The reuse of an existing learning design proved unsatisfactory for the only personwho tried it: namely, Rachel. One reason appeared to be Andrew’s personalisation ofhis original ‘ESOL – Adjectives’ design to his students – specifically, their knownmisconceptions – and this made Rachel unsure whether it would be suitable for herown class.

4. Teachers’ message to others about the value of learning design

Overall, teachers had a positive message to communicate to others regarding the valueof learning design, not least because of its potential to benefit both their own profes-sional development and their students’ learning. Linda wrote: ‘It is worth working onand developing your skills to benefit your teaching and your learners’ experience,’while her colleague Alison commented: ‘You develop additional skills, reflect uponyour teaching methods and how to adapt and change them.’

Some wanted to share the excitement of their triumph over initial uncertainty: ‘IfI can do it, anyone can!’ (Linda); ‘Do it! The results surprised me. I didn’t expect thelearners to enjoy it quite so much’ (Rhona); ‘it’s not as scary as you think it might be’(Nicola). However, they wanted others to be aware that time, effort, and support wereneeded: ‘teachers require a limited amount of training to get started, but on-goingsupport is needed if they are going to progress in using [Moodle] on a regular basis,making use of its many tools’ (Edward). More specifically, Andrew commented thatRachel’s extensive support and guidance made him ‘very clear about the process of

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


designing a sequence.’ Linda’s mentor located the resources for her, leaving Lindafree to concentrate on the design itself. Linda also felt that collaboration with a mentor‘helped me to think about the changes I could make and the best way to use theresources.’

Finally, Rachel (‘ESOL – Verbs’) offered practical advice on the value of carefulplanning and experimentation when designing a sequence:

Work through a couple of existing sequences first to see how they work. Plan your activ-ities very carefully, as it can be hard to change them once the session is under way. Aska colleague to look at your sequence before you use it. If possible, try out the sequencewith a student.

Discussion

The value of learning design to teachers’ practice

In relation to the elements of learning design as a framework for practice – focus onactivities, the structuring of learning, sharing and reuse, and attention to elements thatcan be planned in advance – the data from the eLIDA CAMEL case studies suggestthat the ability to offer students a structured learning experience with a wider varietyof activities was perhaps its major value to participants. Although teachers werealready comfortable with activity-focused (i.e., as distinct from content-driven)design, the range of activities available in both LAMS and Moodle prompted them toconsider the students’ perspective more.

The benefit of a systematic approach is well illustrated in Rachel’s step-by-stepprescription for the design process, reflecting as it does the value of careful advanceplanning and piloting with another teacher or an individual learner. Christine’s claimthat LAMS and Moodle actually make planning easier offers a positive message aboutthe way in which the tools can become part of one’s everyday working. However,Rachel’s unsatisfactory experience is insufficient for us to draw conclusions regardingthe value of reusing another person’s sequence.

Teachers’ expectations regarding the learning experiences they could offerstudents appeared largely to be met, including motivating students to take more inter-est in their learning, and scaffolding their growing independence. None of the teachersfully articulated what they meant by the concept independent learning, but it seemsthat they envisaged students working through activities without the immediate supportof the teacher, rather than students determining their own learning goals and designingtheir own learning activities (self-directed learning). However, there are hints of amore adventurous pedagogic approach in Christine’s embryonic thoughts on studentsas learning designers.

The discussion of independent learning leads us to the consideration of a keytheme of this special issue: designing for flexible learning. Although the eLIDACAMEL project did not specifically set out to explore flexible learning, five partici-pants developed activity sequences that gave students anytime, anywhere access tolearning materials and activities. Furthermore, data from a number of teachers haveimplications relevant to the design of flexible learning: for example, ensuring that thetopic is amenable to technology-enhanced learning; ensuring that the concepts andskills being addressed are those that can be acquired or practised in the absence of amore able human partner; and ensuring that the students themselves have the requisiteIT skills and are ready for this kind of learning.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


The eLIDA CAMEL case studies as mediating artefacts

The eLIDA CAMEL case studies provide the learning design researcher withevidence of what it is like to engage in the practice of learning design for the first time,albeit largely with the assistance of a more able peer (the mentor). An additionalobjective of this article was to explore the extent to which these case studies mightserve as mediating artefacts: that is, as representations for capturing and sharing effec-tive practice in learning design. In the ‘Case Studies as Mediating Artefacts’ sectionearlier in the article, we suggested that the usefulness of case studies may lie in theirinspirational function: here, motivating practitioners who may not have embarked onlearning design yet and are hesitant (even fearful) about using technology in theirteaching. We also speculated that such case studies may have value in offering thepersonal testimonies of teachers (rather than the prescriptions of learning designprofessionals) and in describing specific examples of practice. For the present thisremains speculation only; further research is needed to ascertain the actual value ofsuch case studies to their intended audience.

Questions that might guide such research include the extent to which teachers findthe content of the case studies inspirational in the manner intended. Do the case stud-ies, in the methodological parlance, lend themselves to ‘naturalistic generalisation’ onthe part of readers through ‘recognizing essential similarities’ (Stake, 1978/2000,pp. 22, 23) to their own situations? Or does the strongly contextualised nature of thecase studies detract from their meaningfulness to readers who lack knowledge ofLAMS, Moodle, or the CAMEL model? It would therefore be advisable to collectadditional case studies of novice practice under different conditions: for example,where teachers are using other tools or do not have ready access to a mentor.

We also need to ask whether the case studies fulfil a valid developmental functionwithin the domain of learning design. That is, to what extent do the experiencesdescribed in them embody what we might term the values of learning design (in so faras these have been codified) and demonstrate the seeds of expert practice? There aresigns that the eLIDA CAMEL case studies might meet both these requirements, butindividual pieces of evidence are scattered across individual case studies, and there isno guarantee that, without guidance, the audience would discern the learning designvalues embedded in them.

This concern leads us to consider the aggregation of the case studies into metame-diating artefacts. The speculation in the previous paragraph suggests that they shouldnot stand alone, but should be interpreted by an expert and presented in a supportivetool such as a pedagogy planner. The current project to develop a ‘learning designsupport environment’ for teachers and lecturers (Laurillard & Masterman, in press)could provide a test bed for aggregation in this manner.

Conclusion

This article reported the key findings of the eLIDA CAMEL project, which investi-gated an aspect of learning design that has been hitherto overlooked: namely, whatteachers who are new to the domain perceive to be its value as a framework for prac-tice in the design of both flexible and classroom-based learning. For this purpose, thearticle adopted a perspective on learning design as (a) a conceptual framework for thecreation of structured sequences of learning activities in both flexible and classroomsettings, and (b) a way to share practice and scaffold the design process throughmediating, and metamediating, artefacts.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


The 13 narrative case studies compiled by the project suggest that learning designas a practice is indeed valuable as teachers engage with technology-enhanced learn-ing. However, learning design can be challenging, and face-to-face mentoring mayplay an important role during the early days. Success in this period may lie, not instudents’ improved performance, but in increased confidence and a new pedagogicperspective on the teacher’s part.

The article additionally considered the value of the case studies as inspirationalmediating artefacts. However, given the exploratory nature of the eLIDA CAMELproject, no generalisable conclusions can yet be drawn. Recommended future work tothis end includes an empirical evaluation of their usefulness to teachers, throughaggregation into a supportive tool.

AcknowledgementsThe authors thank JISC for funding the eLIDA CAMEL project, and the eLIDA CAMEL, JISCeLISA, and JISC infoNet CAMEL partners, institutions, and agencies for contributing theirwork towards the content of this article. We are also grateful for the advice of the two anony-mous reviewers of the initial version.

Notes on contributorsElizabeth Masterman is senior researcher in the Learning Technologies Group, University ofOxford. She has been involved in research into learning design for the past four years, special-ising more recently in pedagogy planner tools. Elizabeth was responsible for evaluating theJISC eLISA project and acted as consultant to the eLIDA CAMEL project.

Jill Jameson is director of research and enterprise in education and training at the University ofGreenwich and was co-chair of the ALT-C Conference in 2008. Her research interests includelifelong learning, post-compulsory and continuing education, e-learning, and leadership. Jillwas director of both the JISC eLISA and eLIDA CAMEL projects.

Simon Walker is the head of the Educational Development Team at the University of Greenwichand is a National Teaching Fellow. His research interests lie in the development and evaluationof learning design and the management of change. Simon was extensively involved in the JISCeLISA project and was deputy director of the eLIDA CAMEL project.

ReferencesAgostinho, S. (2008). Learning design representations to document, model, and share teaching

practice. In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook ofresearch on learning design and learning objects: Issues, applications and technologies(Vol. 1, pp. 1–19). Hershey, PA: Information Science Reference.

Britain, S. (2004). A review of learning design: Concept, specifications and tools. RetrievedJanuary 9, 2009, from http://www.jisc.ac.uk/uploaded_documents/ACF83C.doc

Conole, G. (2008). The role of mediating artefacts in learning design. In L. Lockyer, S.Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of research on learning design andlearning objects: Issues, applications and technologies (pp. 108–208). Hershey, PA:Information Science Reference.

Falconer, I., & Littlejohn, A. (2008). Representing models of practice. In L. Lockyer, S.Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of research on learning design andlearning objects: Issues, applications and technologies (pp. 20–40). Hershey, PA: Infor-mation Science Reference.

Ferrell, G., & Kelly, J. (2006). Collaborative approaches to the management of e-learning(CAMEL). In T. Lillemaa (Ed.), Is information technology shaping the future of highereducation? Proceedings of the 12th International Conference of European UniversityInformation Systems (pp. 333–337). Estonia: University of Tartu.

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014


Griffiths, D., & Liber, O. (2008). Opportunities, achievements, and prospects for use ofIMS LD. In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook ofresearch on learning design and learning objects: Issues, applications and technologies(pp. 87–112). Hershey, PA: Information Science Reference.

Jameson, J. (2008). The eLIDA CAMEL model of collaborative partnership: A community ofpractice in design for learning. In D. Remenyi (Ed.), Proceedings of ICEL 2008, the 3rdInternational Conference on e-Learning (pp. 225–232). South Africa: University of CapeTown.

Jameson, J., Ferrell, G., Kelly, J., Walker, S., & Ryan, M. (2006). Building trust and sharedknowledge in communities of e-learning practice: Collaborative leadership in the JISCeLISA and CAMEL lifelong learning projects. British Journal of Educational Technol-ogy, 37(6), 949–968.

Joint Information Systems Committee (JISC). (2004). Effective practice with e-learning: Agood practice guide in designing for learning. Bristol: JISC. Retrieved January 9, 2009,from http://www.jisc.ac.uk/uploaded_documents/jisc%20effective%20practice3.pdf

Joint Information Systems Committee (JISC). (2006). Background to the JISC Circular 1/06: Design for learning programme (JISC Briefing Document). Bristol: JISC.Retrieved April 3, 2009, from http://www.jisc.ac.uk/media/documents/funding/2006/02/backgrounddesignforlearningv3%20final.pdf

Joint Information Systems Committee (JISC). (n.d.). Template for describing an exampleof (e)learning. Bristol: JISC. Retrieved January 9, 2009, from http://www.jisc.ac.uk/whatwedo/programmes/elearningpedagogy/templates

Koper, R. (2006). Current research in learning design. Educational Technology & Society,9(1), 13–22.

Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.).Thousand Oaks, CA: Sage.

Laurillard, D., & Masterman, E. (in press). Online collaborative TPD for learning design. InJ.O. Lindberg & A.D. Olofsson (Eds.), Online learning communities and teacher profes-sional development: Methods for improved education delivery. Hershey, PA: InformationScience Reference.

Masterman, E. (2008). Activity theory and the design of pedagogic planning tools. In L. Lockyer,S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of research on learning designand learning objects: Issues, applications and technologies (pp. 209–227). Hershey, PA:Information Science Reference.

Masterman, L., & Lee, S. (2005). Evaluation of the practitioner trial of LAMS: Final report.Retrieved January 9, 2009, from http://www.jisc.ac.uk/uploaded_documents/LAMS%20Final%20Report.pdf

McAndrew, P., & Goodyear, P. (2007). Representing practitioner experiences through learn-ing design and patterns. In H. Beetham & R. Sharpe (Eds.), Rethinking pedagogy for thedigital age (pp. 92–102). London: Routledge.

Stake, R.E. (2000). The case study method in social enquiry. In R. Gomm, M. Hammersley, &P. Foster (Eds.), Case study method: Key issues, key texts (pp. 19–26). London: Sage.(Original work published 1978.)

Walker, S., & Masterman, L. (2006). Learning designs and the development of study skills:Reuse and community perspectives. In R. Philip, A. Voerman, & J. Dalziel (Eds.),Proceedings of the First International LAMS Conference 2006: Designing the future oflearning (pp. 89–98). Sydney: LAMS Foundation. Retrieved January 9, 2009, from http://lamsfoundation.org/lams2006/refereedpapers.htm

Dow

nloa

ded

by [

Tuf

ts U

nive

rsity

] at

09:

14 0

8 O

ctob

er 2

014

capturing teachers’ experience of learning design through case studies

Documents