Project Number: 531262-LLP-2012-ES-KA3-KA3MP

METIS - Meeting teachers co-design needs by means of

Integrated Learning Environments

D3.2: Draft of pilot workshop

WP3: Workshop Design WP Leader: OU

Author(s)/Editor(s): Andrew Brasher, Chris Walsh, Patrick McAndew, Yishay Mor (OU)

Project Number: 531262-LLP-2012-ES-KA3-KA3MP

Project information

Project acronym: METIS

Project title: Meeting teachers' co-design needs by means of

Integrated Learning Environments

Project number: 531262-LLP-1-2012-1-ES-KA3-KA3MP

Sub-programme or KA: KA3 Multilateral projects

Project website: http://www.metis-project.org

Reporting period: From 31/1/13

To 31/5/13

Report version: 0.1

Date of preparation: 10/5/13

Beneficiary organisation: University of Valladolid (UVa), Spain

Project coordinator: Prof. Yannis Dimitriadis

Project coordinator organisation: University of Valladolid (UVa), Spain

Project coordinator telephone number: +34 983 423696

Project coordinator email address: info@metis-project.org

WP Leader: Yishay Mor (OU)

WP Leader email address: Yishay.Mor@open.ac.uk

Document history

Date Version Author(s) Description 10/5/13 01 Andrew Brasher, Chris

Walsh, Patrick McAndrew, Yishay Mor

Final draft for formal internal review

16/5/13 0.1.1 Gráinne Conole Review 20/5/13 0.1.2 Michael Derntl Review 29/5/13 02 Andrew Brasher Edited in response to reviewers’

comments. 6/6/13 03 Andrew Brasher, Yishay

Mor Final version

This work is licensed under a Creative Commons Attribution-NonCommercial-

ShareAlike 3.0 Unported License.

This project has been funded with support from the European Commission. This

publication reflects the views only of the author(s), and the Commission cannot be

held responsible for any use which may be made of the information contained

therein.

Project Number: 531262-LLP-2012-ES-KA3-KA3MP

Executive Summary Several decades of research in Technology-Enhanced Learning (TEL) have clearly

demonstrated the potential of digital technology to transform education. Yet the

impact of TEL research on daily teaching-learning practices is still far from fulfilling

this potential (Mellar, Oliver, & Hadjithoma-Garstka, 2009). Teachers lack the

necessary digital literacy skills (Jenkins, 2009) to harness the potential of new

technologies. Arguably, this is a gap in the capacity for learning design1: educators

need the tools and competencies which would allow them to identify educational

challenges, describe the context in which they arise, identify the opportunities

afforded by technology, project the insights derived from research, and devise new

learning experiences. To address this gap, educators need tools and practices. Tools

that would support them through the cycle of learning design – from conception to

deployment and evaluation of techno-educational innovations. Professional

practices that use such tools to ensure the robustness and effectiveness of their

innovations and make learning design a daily habit and part of their professional

identity. The METIS project (http://metis-project.org/) aims to contribute to this aim,

by providing educators with an Integrated Learning Design Environment (ILDE) and a

workshop package for training educators in using the ILDE to support effective

learning design.

Work Package 3, led by the OU (UK), is concerned with the design and development

of the workshop package.

Deliverable D3.2 represents the analysis of best practices and workshop design from

the first cycle of the METIS project methodology. Alongside this report a prototype is

provided to allow access to the package of resources representing a workshop

structure developed from the preliminary analysis of best practices in teacher

training reported in Deliverable D3.1. (This prototype is included in appendix 1).

Section 2 provides an account of the review of best practices – the methodology

used for this review, its current status and outcomes, and plans for the future.

Section 3 provides an account of the design of the METIS workshop – the process,

current status and outcomes, and plans for the future. It also lists risks and

challenges and implications to and from WP 2 and 4.

The METIS partners responsible for designing the workshops have extensive

experience of running and evaluating learning design workshops. This includes the

1 . Typically “Learning Design” (with capital letters) is used to refer to specific tools or projects e.g. ‘

IMS Learning Design’, and the ‘Integrated Learning Design Environment’. The phrase “learning design” (all lower case) is used to refer to general practices and instantiations e.g. ‘enactment of innovative learning designs’ .

Project Number: 531262-LLP-2012-ES-KA3-KA3MP

work undertaken by the OULDI project at the OU, the Carpe Diem workshops run by

the University of Leicester, the 7Cs of Learning Design framework (developed by

University of Leicester), the recent OLDS MOOC (led by the OU) and the JISC-funded

SPEED project.

The workshop structure presented here is a “meta-design”, which needs to be

customized and specified for each user group. The structure provides a flexible basis

for developing the ready-to-run workshops, and is the input for subsequent tasks in

WP3/WP4 in the second cycle. The partner methodologies (provided in Appendices)

show some of the activities envisioned that can be placed within the structure to

develop the different workshops.

This report consists of an account of the workshop design process and critical

decisions, an overview of the design principles and outline of the workshop

structure, and a review of the pedagogical framework and best practices informing

the design. The current version of the design prototype, as well as some of the best

practices data, are includes as appendices.

METIS PROJECT

D3.2: Draft of pilot workshop Page 5 of 64

Contents 1 Introduction ............................................................................................................................. 7

2 Review of best practices .......................................................................................................... 9

2.1 Method .............................................................................................................................. 9

2.2 Results ............................................................................................................................. 11

3 Outline of draft design of the METIS Workshops .................................................................. 16

4 Next steps and implications for other work packages ........................................................... 17

4.1 Risks ................................................................................................................................. 17

4.2 Implications for and from WP 2 ...................................................................................... 18

5 Conclusions ............................................................................................................................ 19

6 Appendices ............................................................................................................................. 20

7 References .............................................................................................................................. 20

Appendix 1. Workshop design .................................................................................................. 22

Appendix 2. Methodology descriptions ................................................................................... 34

a. Learning Design Workshop Methodologies from the ICOPER Best Practice Network .......... 34

b. 7Cs of learning design framework ......................................................................................... 35

c. Design-Practice ....................................................................................................................... 37

d. Learning Design Studio ........................................................................................................... 39

e. Collage / LdShake workshops................................................................................................. 41

f. Participatory Pattern Methodology ....................................................................................... 42

g. Design Challenge .................................................................................................................... 44

h. OULDI Learning Design Training Module ............................................................................... 45

Appendix 3. Methodology design narratives and design patterns .......................................... 48

a. “Design Narratives” task presented to partners .................................................................... 48

b. Design narrative template ..................................................................................................... 49

Appendix 4. Sample of Methodology Design Narratives.......................................................... 51

a. Meta-Pyramid design ............................................................................................................. 51

b. Role-playing on problematic situations and technology ....................................................... 56

c. Mini-focus-groups .................................................................................................................. 59

Appendix 5. Personas template ................................................................................................ 63

Appendix 6. Survey of user groups ........................................................................................... 64

METIS PROJECT

D3.2: Draft of pilot workshop Page 6 of 64

METIS PROJECT

D3.2: Draft of pilot workshop Page 7 of 64

1 Introduction

The last few decades have seen tremendous progress in the use of ICT for education and

training across Europe. However, effective integration of ICT should go beyond replacing,

streamlining or accelerating current practices. It must also support “pedagogical and

organisational innovation” (EACEA, 2009). The current gap between research and practice in

Technology-Enhanced Learning (TEL) highlights the need for a shift in attention from the

development of specific tools and resources to the support for their integration, e.g. in teacher

practice for the design of ICT-based learning activities. This integration into practice needs

support for the whole design and implementation life-cycle, from (co-)design to enactment

(Kelly, Lesh, & Baek, 2008; Laurillard, 2008).

The METIS project (http://metis-project.org/) aims to provide this kind of support, synthesising

the achievements of the design paradigm and making them available to a broad circle of

practitioners across multiple educational sectors such as adult education, vocational training

and higher education. This is to be achieved mainly through a practitioner-centred approach,

which combines a) technological support for the whole learning design life-cycle (in the Learning

Design Environment, or ILDE); b) professional development support in the form of ready-to-use

workshop packages; and c) the dissemination of these project outcomes to a wide community

of practitioners.

Work Package 3 of the METIS project is concerned with the design of the professional

development workshops. This work package is led by the OU (UK), in collaboration with other

partners. We will work closely with the projects’ user groups to ensure that the workshop

design addresses their constraints and concerns, and with the evaluation partners to ensure

that the design is robust and effective. The workshop design will be evaluated by the user group

partners (task T3.4), and eventually offered as an open educational resource on the project

website (as part of deliverable D3.4, task T3.9).

Deliverable D3.2 represents the analysis of best practices and workshop design from the first

cycle of the METIS project methodology. Alongside this report a prototype is provided to allow

access to the package of resources representing a workshop structure developed from the

preliminary analysis of best practices in teacher training reported in Deliverable D3.1.

The workshop structure presented here is a “meta-design”, which needs to be customized and

specified for each user group. The structure provides a flexible basis for developing the ready-

to-run workshops, and is the input for subsequent tasks in WP3/WP4 in the second cycle. The

partner methodologies (provided in Appendices) show some of the activities envisioned that

can be placed within the structure to develop the different workshops.

This report consists of an account of the best practices review process and its outcomes (section

2), an overview of the design principles (section 2.2.1), an outline of the workshop structure and

its rationale (section 3), and a discussion of the next steps in the process (section 4) and the

perceived risks (section 4.1) and implications (section 4.2). The current version of the design

METIS PROJECT

D3.2: Draft of pilot workshop Page 8 of 64

prototype, as well as some of the best practices data, are includes as appendices (indexed in

section 6).

METIS PROJECT

D3.2: Draft of pilot workshop Page 9 of 64

2 Review of best practices

2.1 Method

D3.1 (Brasher & Mor, 2013) identified a set of workshop aims for participants, a selection of

successful approaches and a set of design principles. These were derived from a survey of

learning design experts, a series of 4 semi-structured interviews of representatives of user

groups and a workshop held on 9th January 2013 in which representatives of user groups and

other project partners were guided in describing their context and characterising their

prospective workshop participants. These aims, approaches and principles have guided the

development of the draft design of METIS workshops below. In parallel to working on the

design, we have continued the process of reviewing past and current practices, which informs

our design work and will continue to feed into it as we proceed into greater detail.

Our aim is to capture these best practices in the form of transferable design knowledge, which

can then be applied to the core generic workshop design, used in the process of customising this

design to the needs of the user groups, and could also be shared with anyone who wishes to

design their own learning design workshops.

To achieve this aim, we choose to represent this design knowledge using a combination of

overview descriptions, design narratives, design principles and design patterns. This approach is

based on the SNaP! Methodology (Mor, 2013)2 and the PPW methodology (Mor, Warburton and

Winters, 2012)3.

A design narrative is an “account of critical events in a design experiment from a personal,

phenomenographic perspective”4. A design pattern “describes a recurring problem, or design

challenge, the characteristics of the context in which it occurs, and a possible method of

solution”5. A design principle is “…an intermediate step between scientific findings, which must

be generalized and replicable, and local experiences or examples that come up in practice.” (Bell

et al, 2004, p. 83, in Kali, 2009)6. Together with the overview descriptions, these cover a full arc

from the pedagogical framework, through the high-level design, and down to specific activities.

The design principles will be declared upfront in our workshop design documentation and serve

as a pedagogical contract between the workshop designers, facilitators and participants:

defining a set of mutual expectations regarding the roles and interactions between them. We

2 Mor, Yishay (2013). SNaP! Re-using, sharing and communicating designs and design knowledge using scenarios, narratives and

patterns. In: Luckin, Rosemary; Puntambekar, Sadhana; Goodyear, Peter; Grabowski, Barbara L.; Underwood,

Joshua and Winters, Niall eds. Handbook of Design in Educational Technology. London, UK: Routledge, (In press). 3 Mor, Yishay; Warburton, Steven and Winters, Niall (2012). Participatory pattern workshops: a methodology for open

learning design inquiry. Research in Learning Technology, 20 4 http://www.ld-grid.org/resources/representations-and-languages/design-narratives 5 http://www.ld-grid.org/resources/representations-and-languages/design-patterns 6 In: http://www.ld-grid.org/resources/representations-and-languages/design-principles

METIS PROJECT

D3.2: Draft of pilot workshop Page 10 of 64

will review the general flow of the workshop as well as the individual activities to ensure that

they are compliant with these principles.

The design patterns will be used to mold specific activities, or even elements of activities. They

will also be offered as a resource for workshop facilitators as an aid in customizing the workshop

design, and as a resource for participants as an illustration of an exemplar learning design

process.

The end product of this process with be deliverable D3.4 “Final workshops packages: workshops

for different educational levels and education contexts”. This will contain instructions for the

trainers as to how to run the workshop; a sequence of activities for the trainer and the trainees;

a description of the rationale and pedagogical methodology on which the workshop is based;

and attached learning resources to be used in the workshop.

The review process consists of the following phases:

Phase 1. Collating overviews of the workshop methodologies, with example resources and

links and references to detailed descriptions (completed, see Appendix 2)

Phase 2. Collation of design narrative of successful workshop practices (completed, see

Appendix 3)

Phase 3. Identifying design principles in the design principles database and in the overviews

(completed, see section Results’).

Phase 4. Extraction of initial design patterns from the design narratives, to be achieved by

analysis and cross-comparison of narratives.

Phase 5. Refinement, elaboration and substantiation of the design patterns and design

principles

Phase 6. Identifying the implications of the design patterns and principles for WP 2 (ILDE),

WP 4 (enactment) and WP 5 (evaluation).

Phase 7. Applying the patterns and principles to the workshop design.

Originally, it was planned to complete this process before elaborating the workshop design.

However, our discussions with WP2 and WP5 suggested that they need an early draft of the

design in order to align their work with it. Furthermore, it has become evident that the user

groups need to be engaged early on with the general framework of the workshop design to

verify that they can commit to its structure. Consequently, we are proceeding with the

methodology review and the workshop design in parallel rather than sequentially.

Initial results from phases 1 to 4 of the review process are presented in section Results’. The

next steps required to support this process through phases 5-7 are:

Step 1. WP3: Provide guidance to partners to embark on a more robust and critical of the

new design patterns using the additional guidance provided (completed June 4,

2013);

METIS PROJECT

D3.2: Draft of pilot workshop Page 11 of 64

Step 2. ALL: Elicit/evoke additional and refined design narratives;

Step 3. All: Elicit/evoke additional and more refined design patterns into the ILDE by

refining/building on the design patterns that have already been generated. We will

do this by providing high quality and relevant examples from workshops conducted

by the OU to work as a catalyst to evoke the additional examples;

Step 4. All: Migrate the rest of the original design narratives into the ILDE;

Step 5. WP3: Author a design principles document drawing on the work already completed

and the expected work to be completed in 1-5 above;

Step 6. WP3 and User groups: Use the design principles, patterns and narratives to inform

future versions of the generic as well as the user-group specific workshop designs;

Step 7. WP3 and WP2: review the patterns and principles, and consider their implications

for the ILDE design and implementation;

Step 8. OU and WP4: review the patterns and principles, and consider their implications for

the workshop enactment (WP4);

Step 9. OU and WP5: review the patterns and principles, and consider their implications for

the Metis evaluation strategy;

2.2 Results

Appendix 2 lists the methodology descriptions. Appendix 3 presents the procedure for collating

the design narratives (including our design narrative template). Appendix 4 offers a sample of

design narratives of successful workshop activities. In section 2.2.1 we describe the design

principles which have been extracted so far.

We initiated the process using the project internal collaborative work space

(http://internal.metis-project.org/workpackages/wp3-workshop-design/methodology-cross-

review/). We collected 12 design narratives there (

Figure 1).

METIS PROJECT

D3.2: Draft of pilot workshop Page 12 of 64

Figure 1: screen shot of design narrative repository on the project internal work space.

In parallel, ILDE was launched for internal beta, and we decided to shift our activity there. We

began migrating the design narratives to ILDE, and in parallel began extracting patterns in ILDE.

Currently, we have 10 design narratives in ILDE (Figure 2).

Figure 2: Design narratives in ILDE

So far, we have identified 6 design patterns (Figure 3)

METIS PROJECT

D3.2: Draft of pilot workshop Page 13 of 64

Figure 3: Design patterns in ILDE

Figure 4 shows an example of a design pattern in ILDE. This pattern is not yet available for public

viewing. Another pattern, which has been published and is available for public viewing, can be

found at: http://ilde.upf.edu/v/bjd

METIS PROJECT

D3.2: Draft of pilot workshop Page 14 of 64

Figure 4: example design pattern in ILDE

2.2.1 Design principles

Table shows a first set of design principles extracted during phase 3. This table shows the source

of each principle, the principle itself, and some remarks on how it has or could be applied in the

workshop design presented in Appendix 1.

Source Principle Application Principles from the Educational

principles database that UVA have found useful in their workshops and can be applied to the METIS workshops

1. Build on student ideas (http://www.edu-design-principles.org/dp/viewPrincipleDetail.php?prKey=166)

2. Reuse student artifacts as resource for learning (http://www.edu-design-principles.org/dp/viewPrincipleDetail.php?prKey=371)

3. Integrate online with offline activities (http://www.edu-design-principles.org/dp/viewPrincipleDeta

METIS workshops will be on a specific theme relevant to participants’ context, and will focus on participants own design problems.

During the workshop participants will be encourage to share resources produced during a workshop both through the ILDE and face-to-face. This principle is embodied in activity 9

METIS PROJECT

D3.2: Draft of pilot workshop Page 15 of 64

il.php?prKey=330) 4. Connect to personally relevant

contexts (http://www.edu-design-principles.org/dp/viewPrincipleDetail.php?prKey=171)

‘Evaluate’ in which participants will evaluate and learn from others’ designs.

The workshop design includes activities which use both online and offline tools.

METIS workshops will be on a specific theme relevant to participants’ context, and will focus on participants own design problems.

Additional principles from the Educational principles database that UPF have found useful in their workshops and can be applied to the METIS workshops

5. Encourage learners to learn from others (http://www.edu-design-principles.org/dp/viewPrincipleDetail.php?prKey=224)

6. Employ multiple social activity structures (http://www.edu-design-principles.org/dp/viewPrincipleDetail.php?prKey=238)

7.

This principle is used explicitly in activities 9, 10, and underlies all the other activities as they are collaborative in nature.

The workshop design includes multiple social activity structures. The emphasis is on working in small groups, with individual activity and whole class activity occurring occasionally (e.g. activity 2, activity 4).

Principles from reviewing the LDS methodology:

8. Continued work on a challenge/design project: "the main activity of a course is the students' continued work on design challenges in a defined domain of practice" (akin to project-based learning)

9. Public review of group artifacts: "classroom sessions are mostly dedicated to group work and public review of design artefacts"

10. Iterate!: implicit in "continued work on design challenges in a defined domain of practice"

METIS workshops will be on a specific theme relevant to participants’ context, and will focus on participants own design problems.

This occurs in activities 4 and 9. Iteration is implicit in the

workshop design (see figure 2), but time limitations may limit the iteration that is possible during the workshop itself.

Some principles from CARDET's Design-Practice methodology:

11. Include an early "how to ruin X" activity to get people started thinking about the topic X

12. Break down a (part of a) learning design to a full detail level -- this is an important activity to get teachers started thinking in detail about design decisions and implications of those decisions (see also this paper)

13. Pitching results -- Important to let people pitch their results and have peers discuss those. In the CARDET

Included as activity 2. Creating a fully detailed design

occurs in activity 8 ‘Prototype’.

This occurs in activity 9, ‘Evaluate’.

METIS PROJECT

D3.2: Draft of pilot workshop Page 16 of 64

Table 1: design principles, their source, and comments on their application to the workshop design

3 Outline of draft design of the METIS Workshops

Based on the aims, approaches and principles outlined in D3.1 (section 3, Brasher & Mor, 2013,

pp. 8-11), we have proceeded to iteratively develop the design for the METIS workshops. Early

sketches of this design have been presented to partners and user groups for their review and

feedback. The current state of the draft design is presented in Appendix 1.

This is a generic, or “template” design, in the sense that it needs to be customized for each of

the user groups to ensure it meets local needs and contexts. The process of customization will

involve further discussions and negotiations with each user group so as implement context

specific versions of the activities described in the template during WP3’s work towards D3.3 (the

three ready to run workshop structures). The design decisions taken in this process will be

recorded by WP3 and used to produce guidance for customisations for application of the

template to other contexts. However, it sets a common framework which we see as necessary

for an effective workshop:

A METIS workshop consists of a sequence of collaborative activities, spanning 6-8 hours. These

can be conducted as one full day event or two half day events. The workshop requires a space

which is set up for group work. It is optimized for 4-5 groups of 3-6 participants each, overall 15-

25 participants. Ideally, it would need a facilitator per every 5 participants. To be effective, a

workshop will require participants to invest 1-2 hours pre-event and 1-2 hours post event.

methodology it sounds asynchronous, but also very relevant in a synchronous/f2f setting.

Some principles from ICOPER methodologies:

14. Solving a proposed task (e.g., providing a narrative example to solve) -- helpful when training specific skills associated to improving proficiency around the elements of a design modelling language / tool.

15. Improving participants' previous designs -- participants reconceptualise their actual designs (courses) using theoretical input provided gradually.

Examples are provided in activity 5 ‘Evidence and examples of ‘X’’, and also through patterns in activity 7. However, decisions need to be taken about how these patterns and examples are instantiated and present to the different user groups.

Iteration is implicit in the workshop design (see figure 2), but time limitations may limit the iteration that is possible during the workshop itself. Theoretical input is provided gradually (e.g. activities 5 and 7).

METIS PROJECT

D3.2: Draft of pilot workshop Page 17 of 64

The current draft of the workshop design is presented as a word document (Appendix 1) and a

CompendiumLD7 archive. This format is easy to share, manipulate and edit.

4 Next steps and implications for other work packages

The next phases of the WP 3 design and development process will be:

1. Discussing this draft with partners and user groups, and reviewing their feedback

(This work is part of part of Task T3.4, face to face discussion to occur at the METIS

meeting in Barcelona, July 2013; preliminary online discussion already underway).

2. Work with tool providers for CADMOS, OpenGLM, and WebCollage on integrating

detailed and meaningful activities using these tools into the current workshop structure.

3. Specifically, consider ways of incorporating the collaborative learning flow patterns

(Hernández-Leo, Asensio-Pérez, Dimitriadis, & Villasclaras, 2010) both into the workshop

design and as meaningful resources for workshop participants.

4. Incorporate the outcomes of the “best practices” cross-review process, described in

section 2.

5. Based on the feedback and the “best practices” review, deploying a revised design to a

shared Virtual Learning Environment (VLE). (This work is part of Task T3.5, to occur in

project months 8 – 10, June –August 2013).

6. Run an open survey of potential participants to assess their interest in the topics and

structure which emerged so far (an initial survey of the 3 user partners representatives

within METS highlighted the topics of formative assessment, collaborative learning and

project based learning). (This work is part of Task T3.4, the survey will be launched in

June 2013, project month 8).

7. Work with user groups to customize this design per their specific needs, concern and

constraints (coordinated by WP4). The workshop design draft may be modified in

response to the requirements for customisations, so that it remains a generic template

for the customisations that are produced. The customised instances of this meta design

will be delivered as METIS Deliverable D3.3 (month 11). (This work is part of Task T3.5, to

occur in project months 8 – 10, June –August 2013).

8. Work with WP5 on evaluation procedures for the design. On-going communication with

and reporting to WP5 about the evolution of the workshop design.

4.1 Risks

In the course of the work on the workshop design we have identified several risks which need to

be mitigated. We are collaborating with WP 4 (‘Report on the pilot workshops and LD

enactment’) on addressing these:

7 CompendiumLD is a a software tool for designing learning activities using a flexible visual interface that can be

downloaded from http://compendiumld.open.ac.uk/. This site also provides a ‘getting started’ guide for new users.

METIS PROJECT

D3.2: Draft of pilot workshop Page 18 of 64

1. User groups need to commit to one full day / two half day workshops. Initial

discussions with user groups suggest this may be difficult for some of them. However,

we are convinced that it would be ineffective to run shorter workshops.

2. User groups need to recruit sufficient participants and facilitators for each workshop.

In order to properly evaluate and revise the design between the two cycles, we will need

3 workshops (1 per user group) of 10-20 participants in the first and second rounds.

These workshops will require 2-3 facilitators each. In addition, we note that after the

workshops least one participant from each workshop must use the ILDE in their own

teaching practice, applying techniques they have learnt through the workshops.

3. Input from user groups required to customize and add content to their instance of the

workshop design. Some of the activities are common, some are specified in a generic

manner, and need to be elaborated per the specific topics, context and concerns of

every workshop. Furthermore, user groups may want to translate the workshops

materials to their local language.

4. User groups need to accommodate diverse technical skills. The workshop design

assumes a basic level of computer and internet proficiency. The workshops are

supported by a VLE, the ILDE and a variety of computational tools. Some of the

participants may not have the required skills, and the user groups will need to

accommodate them by prior training or by grouping them with other participants who

can assist them.

4.2 Implications for and from WP 2

Table 10 in D2.1 (Hernández-Leo, Asensio, Chacón, & Prieto, 2013) describes the WP2 revised

use cases and requirements stressed by the user groups. These revised use cases are listed

below, along with a comment indicating how the workshop structure reported herein relates to

the use case. We also make a note of any specific requirements for WP2 to bear in mind. Some

of the activities require specific resources to be provided in the ILDE. We will coordinate these

with WP2 after the next round of reviews and edits.

(1) Choose a learning design tool between several available. The skill to select an

appropriate design tool will be developed through several of the activities that occur

within the workshop structure. For example in Activity 7 participants chose one or more

design patterns appropriate to the requirements of their design vision. In this activity

they are also introduced to the ILDE tools with the appropriate affordances to produce

runnable designs from these patterns.

(2) Produce a learning design by both adapting reused designs and starting from scratch.

In the workshop structure we have described, we envisage that workshop participants

will produce many design artefacts during the design process, prior to and during the

creation of the runnable design. These artefacts will be sketches, notes, evaluation

checklists and so on.

METIS PROJECT

D3.2: Draft of pilot workshop Page 19 of 64

(3) Co-produce a learning design. The workshop structure is built around team work and

coproduction.

(4) Share a learning design. The ILDE tools facilities for sharing will be used in Activity 9.

Designs produced during the workshop will be shared amongst all the teams

participating in the workshop and participants will be would encouraged to share them

more widely afterwards.

(5) Instantiate a learning design. Instantiating a learning design occurs during Activity 8

‘Prototype’.

(6) Deploy an instantiated learning design into chosen VLEs. Deploying a learning design

occurs during Activity 8 ‘Prototype’.

(7) Provide feedback and reflections. In Activity 9 participants evaluate each other’s designs

using the checklists developed during earlier activities.

(8) Explore designs, instantiations and feedback. This is the focus of Activities 9 and 10 in

the workshop structure.

5 Conclusions

We have completed the first cycle of the workshop design and development. This deliverable is

the end product of that cycle, and consists of package of resources representing the workshop

structure developed from the best practices identified in Deliverable D3.1 (Brasher & Mor,

2013). This package includes: a sequence of activities for the trainer and the trainees; a

description of the rationale and pedagogical methodology on which the workshop is based; and

attached learning resources to be used in the workshop.

This package provides a meta-design for the Metis workshops. Over the next few months we

will use this as the basis for developing the detailed designs of the actual workshops to be run

by and for the user groups.

In the course of our work towards this deliverable, we identified a need to deepen and broaden

our review of best practices. Due to the high level of interdependencies between the workshop

design and other work packages, we decided to prioritise progress on the workshop design, and

continue the review of best practices in parallel.

Also, during the work on the design and review of best practices, the ILDE had moved into

private beta phase. Once we verified that it is stable and the basic functionality is in place, we

decided to shift our work from the project’s internal workspace to the ILDE. This entailed a

transition cost, but will allow us to develop our expertise in using ILDE, and will provide WP2

with “real world” testing and evaluation.

Our next goals would be to synchronise the workshop design with the ILDE development, and

use the generic design here to develop the structures for the three pilot workshops (D3.3). We

will continue our review of best practices, and these will feed into the design process

continuously.

METIS PROJECT

D3.2: Draft of pilot workshop Page 20 of 64

6 Appendices

Appendix 1: Workshop

designAppendix 1 The sequence of activities, rationale and pedagogical framework, and associated resources. This will form the basis for the structure of the three pilot workshops.

Appendix 2: Methodology descriptions

High-level descriptions of the workshop methodologies considered in the review of best practices.

Appendix 3: Methodology design narratives and design patterns

Process and template used for collecting the methodology design narratives.

Appendix 4: Sample of Methodology Design Narratives

Example of design narratives collected in the review of best practices.

Appendix 5: Personas template An example resource used in a workshop activity. Also used by us as an aid in designing the workshops.

Appendix 6: Survey of user groups

Summary of data collected to ensure the workshops address user concerns and constraints.

7 References

Brasher, A., & Mor, Y. (2013). METIS deliverble D3.1: Report 2 on meetings with user groups:

Early feedback on candidate best practices for teacher training on learning design.

Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). OULDI-JISC Project Evaluation Report

Retrieved 3/8/2012, from http://oro.open.ac.uk/34140/1/OULDI_Evaluation_Report_Final.pdf

EACEA. (2009). Lifelong Learning Programme Key Action 3: Information and Communication

Technologies (ICT) Retrieved 22/5/2013, from

http://eacea.ec.europa.eu/llp/ka3/information_communication_technologies_en.php

Galley, R. (2010). Activity: 30 mins: Learning Outcomes view Retrieved 9/5/2013, from

http://cloudworks.ac.uk/cloud/view/4036

Hernández-Leo, D.; Asensio-Pérez, J.I.; Dimitriadis, Y.; & Villasclaras, E.D. Generating CSCL

Scripts: From a Conceptual Model of PAttern Languages to the Design of Real Scripts. In:

Goodyear P.; Retalis, S. (eds.). Technology-Enhanced Learning, Design patterns and pattern

languages, Sense Publishers, Series Technology-Enhanced Learning; 2010. p. 49-64. Appendix

available at http://ulises.tel.uva.es/%7Edherleo/dpbook/appendix-chapter.pdf

Hernández-Leo, D., Asensio, J. I., Chacón, J., & Prieto, L. P. (2013). METIS deliverble D2.1: Report

1 on meeting with stakeholders: early feedback on ILDE requirements.

Jenkins, H. (2009). Confronting the Challenges of Participatory Culture: Media Education for the

21st Century The John D. and Catherine T. MacArthur Foundation Reports on Digital Media and

Learning (pp. 146). Retrieved from

http://mitpress.mit.edu/sites/default/files/titles/free_download/9780262513623_Confronting_

the_Challenges.pdf

METIS PROJECT

D3.2: Draft of pilot workshop Page 21 of 64

Kelly, A. E., Lesh, R. A., & Baek, J. Y. (2008). Handbook of Design Research Methods in Education.

New York: Routledge.

Laurillard, D. (2008). The teacher as action researcher: using technology to capture pedagogic

form. Studies in Higher Education, 33(2), 139-154. doi:

http://dx.doi.org/10.1080/03075070801915908

Mellar, H., Oliver, M., & Hadjithoma-Garstka, C. (2009). The role of research in institutional

transformation'Transforming Higher Education through Technology-Enhanced Learning. York,

UK: Higher Education Academy. Retrieved from

http://www.heacademy.ac.uk/assets/York/documents/ourwork/learningandtech/Transforming.

pdf.

METIS PROJECT

D3.2: Draft of pilot workshop Page 22 of 64

Appendix 1. Workshop design

METIS Workshop Design (version 0.7) Summary

Despite the abundance of high-quality educational technology, and the wealth of research

demonstrating its value, educators are still struggling to make effective use of these

technologies and the associated pedagogies in their daily practices. Arguably, this is a gap in the

capacity for learning design: educators need the tools and competencies which would allow

them to identify educational challenges, describe the context in which they arise, identify the

opportunities afforded by technology, project the insights derived from research, and devise

new learning experiences. To address this gap, educators need tools and practices. Tools that

would support them through the cycle of learning design – from conception to deployment and

evaluation of techno-educational innovations. Professional practices that use such tools to

ensure the robustness and effectiveness of their innovations and make learning design a daily

habit and part of their professional identity. The METIS project (http://metis-project.org/) aims

to contribute to this aim, by providing educators with an Integrated Learning Design

Environment (ILDE) and a workshop package for training educators in using the ILDE to support

effective learning design.

This document describes the aims, organisational requirements and activity structure for METIS

learning design workshops. The aims are specified in terms of outcomes for participants and the

METIS project. The organisational requirements describe the human and other resources

required to run a workshop. The activity structure describes a reusable structure of activities to

enable participants to reach the learning outcomes specified. It includes a description of the

tools and the information resources for both participants and facilitators for each activity.

Workshop aims and organisation

1 Learning and other outcomes

Our research of user concerns suggests that practitioners have limited interest in training on

learning design in general, but are have much higher interest in learning design for specific

themes, such as collaborative learning, formative assessment or project-based learning (Brasher

& Mor, 2013). Hence, the workshop design presented here is a meta-design that is flexible and

can be customised to a specific theme. In this document, this theme is noted as ‘X’.

The intended learning and other outcomes for a METIS workshop on learning design for ‘X’ are

shown in figure 1. The learning design is intended to be applicable across a range of topics.

METIS PROJECT

D3.2: Draft of pilot workshop Page 23 of 64

Figure 5: Learning and other outcomes of the workshop

2 Workshop organisation

2.1 Duration

The total duration of the activities in the workshop(s) is 7.5 hours (Table 2). The activities need

to run in the order they are presented in the ‘Workshop activity structure’ section, but they can

be split into two sessions (e.g.one 4 hour workshop including activities 1 to 6 followed by one

3.5 hour workshop consisting of activities 7 to 11).

Activity Hours

1 Introduction 0.3

2 How to ruin a course / pedagogical features 0.33

3 Personas 0.5

4 Barriers and challenges 0.66

5 Evidence and examples of X 0.5

6 Initiate, Ideate, Investigate: produce your Vision 1.5

7 Connect: gather tools and resources 0.75

8 Prototype 1.5

METIS PROJECT

D3.2: Draft of pilot workshop Page 24 of 64

9 Evaluate 0.75

10 Reflect 0.5

11 Wrap up 0.25

Total 7.74

Activities 1 to 6 (half day workshop 1: Context and vision,

plus wrap up) 3.79

Activities 7 to 11 (half day workshop 2: Prototype and

evaluate, plus intro and ice breaker) 3.25

Table 2: activity durations

2.2 People: Participants, facilitators and others

Participants work in teams composed of 3 to 6 members. This team size encourages and

facilitates each team member to be fully involved in design discussions throughout the

workshop. A team size of 3 to 6 members is optimal because it allows for a diverse range of

views to be debated. Teams of more than 6 often split up into smaller teams and lose focus; this

should be avoided if possible.

It is highly advisable to allocate a ‘critical friend’ to each team to challenge design thinking and

stimulate focused and informed discussion (Cross, Galley, Brasher, & Weller, 2012, p. 28). This

‘critical friend’ role could be played by the workshop facilitators circulating amongst the teams,

but if there are many more teams than facilitators it is advisable to recruit additional ‘critical

friends’ to ensure that each team can benefit from their input and challenges. A learning design

workshop facilitator must have detailed knowledge of the both the workshop topic and the

Integrated Learning Design Environment (ILDE) so he/she can answer any question that might

arise during the workshop. A distinction needs to be made between a critical friend and a

facilitator. A critical friend is required to have knowledge of the topic in order to prompt and

encourage a focused and robust discussion. Obviously, some awareness of the ILDE will be

useful for critical friends. A learning design workshop facilitator is an expert on both the topic

and the ILDE.

2.3 Resources

Each team needs access to one or more laptop or desktop computers with internet access to

interact with the ILDE and other online resources. Prior to the workshop starting, an empty

learning design for each team will be created in the ILDE, and all team members will be given

editing rights for that learning design. If other resources such as printed material and specific

online resources are required for a particular activity, they are described within the relevant

activity in the activity structure that follows.

METIS PROJECT

D3.2: Draft of pilot workshop Page 25 of 64

Workshop activity structure A diagram of the structure of the participants’ activities during the workshop is shown in Figure

2. This illustrates the relationships between the activities, the tools to be used, the resources to

be produced.

METIS PROJECT

D3.2: Draft of pilot workshop Page 26 of 64

Figure 6: Participants activities, tools used, and resources produced

1 Introduction (0.30 hour)

Facilitators introduce themselves, describe the structure of workshop, the organisation in terms

of the available human resources, tools and information and the intended learning outcomes.

They also introduce the critical friends and explain their role in workshop.

Facilitators’ resource: slideshow with notes, ILDE.

2 How to ruin a learning experience (0.33 hour)

(Note: this activity reuses the OULDI activity http://cloudworks.ac.uk/cloud/view/2597).

This icebreaker activity will focus on the key issues and strategies that impact on the success (or

otherwise) of learning and teaching within their context. The output of this activity will be a

design checklist which can be used as one of the design evaluation tools in a mid-way design

review and at the end of the workshop.

Resources (all to be available online and in printed form at the workshop)

Participants’ resources:

Instructions

1. List the 10 best ways to ensure that the learning experience you are designing will

fail! (or: 10 ways technology can ruin the learning experience you are designing)

2. Share these with the others by uploading your lists to the ILDE.

3. What are the key themes?

Output

A first version of a design checklist which can be used as one of the design evaluation tools in a

mid-way design review and at the end of the workshop. The checklist will be uploaded to the

ILDE.

3 Personas (0.5 hour)

“Personas are a tool for sharing our understanding of our expected users, as a starting point for

design” (Mor, 2013). In this activity the participants will create descriptions of 2 or 3 personas

relevant to the context in which they teach. This activity is a first step towards a detailed

specification of the context in which the “X” learning activity will occur.

Resources

METIS PROJECT

D3.2: Draft of pilot workshop Page 27 of 64

Printed and online versions of the Personas template (appendix x) will be provided for

participants. (The online version will be in the ILDE).

Output

The personas created will be uploaded or linked to the ILDE.

4 Barriers and challenges (0.66 hour)

20 minutes team work, 20 minutes plenary (assuming 3 or 4 teams)

In this activity the participants are asked to describe what they see as the barriers and

challenges with respect to designing and running ‘X’ learning activities in their context.

Facilitators refer to the activity “How to ruin a course”, where they will have already created a

list that is likely to be applicable to any type of course or activity. This activity prompts

participants to relate their current understanding of the workshop topic (X) to the context in

which they teach. It also allows the facilitators and critical friends to gain an understanding of

each team’s context. The participants will use the personas created in the “Personas” activity to

discuss and answer the following questions for a learning activity on ‘X’ (e.g. collaborative

learning)

a) What are the barriers and challenges from a learners’ perspective of ‘X’?

b) What are the barriers and challenges in implementing ‘X’ in your course or topic

area?

This will be a collaborative mapping exercise:

Each participant writes down 3-4 barriers and 3-4 motivations, each one on a separate

post-it (5 minutes).

Participants place their post-its on an A1 paper and arrange them in some order or map

(15 minutes).

Each group presents its map to the whole workshop (10 minutes)

Each participant thinks about their own view, discusses them with one other member

and works collaboratively towards a team list of at least 5 barriers/challenges for

learners and 5 for barriers/challenges for implementation. (20 minutes)

Each team shares their list with the wider group. These will be shared with the other teams by

displaying them in the workshop and each team will describe one from the learners’ perspective

and on from the implementation perspective in the plenary (one only to limit the time spent on

sharing). Facilitators foster focused discussion of remaining barriers/challenges between

different teams to occur during breaks.

Resources (all to be available online and in printed form at the workshop)

METIS PROJECT

D3.2: Draft of pilot workshop Page 28 of 64

Facilitators’ resources: slideshow of questions (as above).

Participants’ resources: slideshow with questions as above with notes indicating themes

that they might discuss (e.g. technical, motivational, temporal challenges). One A1 sheet

and several post-it packs per team(As an alternative to post-its, participants can use a

concept mapping tool of their choice.)

Instructions

1. As individuals, write down 3-4 barriers and 3-4 motivations, each one on a separate

post-it. Use green post it notes for barriers/challenges from learners’ perspective,

yellow post-it notes for barriers/challenges for implementation (5 minutes).

2. Share these with the others in your team by placing your post-its on an A1 paper and

collaborate to arrange them in some order or map (15 minutes)

3. Each team should present its map to the whole workshop; focus on describing up to

5 barriers/challenges for learners, and up to 5 barriers/challenges to implementation

from the map (20 minutes)

Critical friends’ resources: document describing suggestions for questions to ask.

Output

Each team should produce a list of up to 5 barriers/challenges for learners, and a list of up to 5

barriers/challenges to implementation. These will be added to the output produced in the first

activity ‘How to ruin a course’ to produce a new version of the team’s evaluation checklist

5 Evidence and examples of ‘X’ (0.5 hour)

The facilitators present a few examples of X that are chosen so as to be relevant to the

participants. (four or five examples should be sufficient). Evidence demonstrating that each

example is effective is also presented.

Each team selects one or two for review, and notes design features that may be transferred to

their context.

Facilitators and critical friends will support the discussion by pointing out particular aspects of

the example designs critical to the success of the example.

Resources

Slides showing the example designs, descriptions of their use, and evidence of their

success.

Copies of academic papers will also be provided. The focus of the presentation will vary

depending on the participants, and will vary from workshop to workshop.

METIS PROJECT

D3.2: Draft of pilot workshop Page 29 of 64

Output

Notes on potentially useful design features added to the ILDE.

6 Initiate, Ideate, Investigate: produce your Vision (1.5 hours)

In this activity each team will describe their vision for a ‘X’ activity. This is a first draft, and it may

be modified during the workshop. This vision should focus on describing the effects the activity

is intended to have on the learners (not on how these effects will be achieved which is the focus

of activity 8 ‘Prototype’).

Resources

Printed templates for the Course Map (see appendix B.5 in Hernández-Leo, Asensio,

Chacón, & Prieto, 2013) need to be provided to participants and they will be able to

complete this within the ILDE.

For the Learning Outcomes View (Galley, 2010) A3 sheets of paper and CompendiumLD

icon post-it notes need to be provided, along with a guide showing participants how to

create a Learning Outcomes View in CompendiumLD should participants wish to

produce a digital version..

Output

The vision should be described in terms of

Learning outcomes

Other outcomes (e.g. affective outcomes such as individual motivation, confidence, team

building)

Leaners’ outputs

An initial description of the evidence required to indicate that the learners have reached

the learning outcomes. Examples include a written piece of work, and an observable

behaviour.

Participants will produce representations such as Course Map and Learning Outcomes View.

The Learning Outcomes View is probably most appropriate as that view is very simple to

produce, and gives a specification that a single activity can be evaluated heuristically against. In

contrast the Course Map is intended to describe complete courses as a whole such as a course

of 100 or more hours of study time. The Course Map could still be applied to show the context

for the ‘X’ learning activity that the participants will design. Other representations of context

could also be introduced (e.g. factors and concerns table) to help focus participants’ attention

on consideration of the context where the design will be run.

METIS PROJECT

D3.2: Draft of pilot workshop Page 30 of 64

7 Connect: gather tools and resources (0.75 hour)

By this stage of the workshop participants will have an idea of the kind of behaviours they want

their learners to demonstrate, as well as the kind of products/competencies they desire learners

to produce/become proficient in during the learning activity they are collaboratively designing.

These need to be described in the learning outcomes view.

In this activity the learners are shown a set of ‘X’ learning patterns. The examples used in the “5

Evidence and examples of ‘X’” activity will provide concrete examples of some or all of the

patterns that are now made available to the participants.

Each team will select one or more patterns which best suit their articulated Vision as defined by

the Course Map, Learning Outcomes view, Personas and the barriers and challenges that they

developed during the preceding activities. If more than one pattern is chosen it could be

because there are alternatives, or because the team thinks that a sequence of ‘X’ activities is

necessary.

They now begin the detailed design. Questions that should be used as prompts for this session

include:

Which parts of the activity should be synchronous, and which should be asynchronous?

Which tools have the right affordances for your activity?

This could be carried out further using a Think-Pair-Share activity.

Resources

A set of design patterns for X. These should be provided both online and in printed

form.

Output

The selected pattern(s), a set of resources that are to be used within the activity and an

annotated diagram of the pattern(s) showing:

asynchronous and synchronous stages within the design

where particular tools and resources are used.

8 Prototype (1.5 hour)

Participants work together to produce a prototype of an ‘X’ activity. The prototypes illustrate

tasks the facilitator and learners may choose to undertake. Prototyping would use a suitable

ILDE tool or be carried out using the technology of the team’s choice (including paper).

Resources

METIS PROJECT

D3.2: Draft of pilot workshop Page 31 of 64

Participants’ resources: a guide to the affordances of the ILDE instantiation and deployment

tools which explains which tools may be best suited to instantiate and run patterns of type X.

9 Evaluate (0.75 hour)

An appropriate summary view of each team’s design is selected by the team to be shared and

evaluated. The summary view to be used will depend on ‘X’ (e.g. if ‘X’ is Collaborative Learning

the appropriate summary view will be generated by Web Collage).

Designs are swapped amongst each group and evaluated heuristically (http://www.ld-grid.org/resources/methods-and-methodologies/heuristic-evaluation).

Facilitators and critical friends advise on how to apply the teams evaluation checklist to the

design in question.

The final step is to share the result of the evaluation back to group as a whole.

Resources

Participants’ resources: the design of another team, along with the same team’s evaluation

checklist (i.e. the output from Activity 4).

10 Reflect (0.5 hour)

In the last activity (‘Evaluate’), each group’s design was evaluated by another group, and the

results of the evaluation shared. Now everyone should have an understanding of all the designs.

Participant groups should now reflect on if and how their activity design could be connected to

other groups’ designs to produce a course.

11 Wrap up (0.25 hour)

The facilitators conclude by describing how participants can find out more about learning design

in general, and the ILDE in particular.

Resources

Facilitators’ resource: slideshow with notes, ILDE.

References Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). OULDI-JISC Project Evaluation Report.

METIS PROJECT

D3.2: Draft of pilot workshop Page 32 of 64

Retrieved 3/8/2012, from http://oro.open.ac.uk/34140/1/OULDI_Evaluation_Report_Final.pdf

Mor, Y. (2013). Personas - The Learning Design Grid, Retrieved 9/5/2013, from

http://www.ld-grid.org/resources/representations-and-languages/personas

METIS PROJECT

D3.2: Draft of pilot workshop Page 33 of 64

METIS PROJECT

D3.2: Draft of pilot workshop Page 34 of 64

Appendix 2. Methodology descriptions

This appendix lists the high-level descriptions of methodologies which formed the

basis for the review of best practices. For each methodology, we compiled an

overview and an index of key references.

a. Learning Design Workshop Methodologies from the ICOPER Best

Practice Network

Summary

Two methodologies were developed and then delivered several times during the

ICOPER eContentplus project8 by the people in the Instructional Design work

package. ICOPER was about interoperability and the alignment of all phases in the

instructional analysis, design and deployment lifecycle based on intended and

achieved learning outcomes.

(1) The first series of workshops was targeted at propelling the learning design

proficiency (in particular IMS LD) at partner institutions. We gave them a brief

narrative description of a concrete unit of learning. We presented the idea of

learning design and the set of elements that are used in LD to abstractly describe

a teaching/learning process [1]. We then asked them to describe the unit of

learning using the IMS LD level A + B concepts (i.e., role, activity, activity

structure, environment, role-part, property, condition). For results see [2].

(2) The second series of workshops was to give teachers at partner institutions a

hands-on feeling of how to think of their teaching / courses in terms of concepts

that are demanded by sound instructional design (alignment of learning

outcomes, teaching methods, and assessment) as well as specified in the

European Qualification Framework for lifelong learning (EQF). The latter was

used in particular to foster understanding of the concept of learning outcomes

and how to formulate those. The procedure: we gave them some background on

learning outcomes (from EQF perspective), then some theoretical input on how

to align intended outcomes, methods and assessment [3]. Then we had them

"re-conceptualize" one of their actual courses using the provided input.

Both workshops were heavily based on hands-on work based on concrete problems

from the participants' context. While we had workshop procedure (1) in both

software-based and paper-based settings, we conducted workshop procedure (2)

mostly paper based using simple, uniform templates.

8 http://icoper.org/

METIS PROJECT

D3.2: Draft of pilot workshop Page 35 of 64

Key references

[1] M. Derntl (2010). ICOPER IMS LD Learning Design Hands-on Workshop.

Presentation recording: http://distance.ktu.lt/vips/flash/play.php?&rid=5846 and

slides: http://www.slideshare.net/mikederntl/introduction-to-ims-learning-design

[2] M. Derntl, S. Neumann, D. Griffiths, P. Oberhuemer (2012). The Conceptual

Structure of IMS Learning Design Does Not Impede Its Use for Authoring.

Transactions on Learning Technologies, 5(1): 74-86

[3] M. Derntl, S. Kabicher, P. Oberhuemer (2010). Workshop on outcome based

education. Workshop material. Available at METIS internal

site:https://docs.google.com/viewer?a=v&pid=sites&srcid=bWV0aXMtcHJvamVjdC5

vcmd8bWV0aXMtaW50ZXJuYWx8Z3g6NDIxZTY0MDUzYzk1Yjc2Ng

b. 7Cs of learning design framework

Summary

The 7Cs of learning design framework (Figure 1) illustrates the key stages involved in

the design process, from initial conceptualisation of a learning intervention through

to trialling and evaluating it in a real learning context.

METIS PROJECT

D3.2: Draft of pilot workshop Page 36 of 64

Figure 1: The 7Cs of Learning Design Framework

The framework consists of the following stages:

1. Conceptualise: Vision for the course, including:

a. Why, who and what you want to design

b. The key principles and pedagogical approaches

c. The nature of the learners.

2. Capture: Finding and creating interactive materials, including:

a. Undertaking a resource audit of existing OER

b. Planning for creation of additional multimedia such as interactive

materials, podcasts and videos

c. Mechanism for enabling learners to create their own content

3. Communicate: Designing activities that foster communication, such as:

a. Looking at the affordances of the use of different tools to promote

communication

b. Designing for effective online moderating

4. Collaborate: Designing activities that foster collaboration, such as:

a. Looking at the affordances of the use of different tools to promote

collaboration

b. Using CSCL (collaborative) Pedagogical Patterns such as JIGSAW, Pyramid,

etc.

5. Consider: Including three elements:

a. Designing activities that foster reflection

b. Mapping Learning Outcomes (LOs) to assessment

c. Designing assessment activities, including diagnostic, formative,

summative assessment and peer assessment

6. Combine: Combining the learning activities into the following:

a. Course map, providing a holistic overview of the nature of the course

b. Activity profile, showing the amount of time learners are spending on

different types of activities

c. Storyboard, creating a temporal sequence of activities mapped to

resources and tools

d. Learning pathway, providing a temporal sequence of the learning designs

7. Consolidate: Putting the completed design into practice in the following ways:

a. Implementation in the classroom, through a VLE or using a specialised

Learning Design tool

b. Evaluation of the effectiveness of the design

c. Refinement based on the evaluation findings

METIS PROJECT

D3.2: Draft of pilot workshop Page 37 of 64

d. Sharing with peers through social media and specialised sites like

Cloudworks

For each of the seven stages a series of conceptual design tools have been

developed, including the Course Features card, Course Map, Activity Profile,

Storyboard, Resource Audit tool, and E-tivity template.

The framework and its tools and resources have been trailed in the HE context,

including four UK universities and overseas institutions such as SAIDE (South African

Institute of Distance Education). In addition, we have run a series of 7Cs workshops

at a number of international conferences, with participants from a variety of

different educational sectors. In METIS, we think that the 7Cs methodology offers

the potential to be applied to other educational sectors, such as adult learning and

vocational training. The conceptual design tools developed from the 7Cs enable

teachers at all level of education to rethink their design practice and to create more

engaging learning experience for their learners.

Key references

Conole, G. (forthcoming). Innovative approaches to learning design – harnessing new

technologies for learning. In T.D. Bilham (eds.) For the Love of Learning: innovations

from outstanding university teachers. Palgrave MacMillan.

Conole, G. (2013). Current thinking on the 7Cs of learning design (blog post).

http://e4innovation.com/?p=628.

SPEED project. 7Cs toolkit: http://www2.le.ac.uk/departments/beyond-distance-

research-alliance/projects/speed, and SPEED blog:

http://speedprojectblog.wordpress.com/

c. Design-Practice

Summary

Design-Practice project (http:// www.design-practice.org) was aimed at preparing

them in integrating ICT in their teaching and advancing their lifelong learning skills by

building a community of teachers for sharing, discussing, debating, and improving

instructional activities and learning designs. During the project an online portal with

community of teachers and face-to-face and online teacher training modules were

developed.

The workshops were held in Cyprus, Greece and United Kingdom and were based on

the work of the Open University UK (OU) on Learning Design, which was first

METIS PROJECT

D3.2: Draft of pilot workshop Page 38 of 64

developed in the context of the OULDI (Open University Learning Design Initiative)

project. A detailed description of the workshops is available at:

http://www.open.ac.uk/blogs/OULDI/?page_id=987

The workshops included the following 7 activities:

Activity 1 - How to ruin a course: The main goal of this activity was to pinpoint key

issues and strategies that can affect the success (or otherwise) of learning and

teaching within an educator’s context.

Activity 2 - Comparing four web 2.0 tools: This activity was aimed at introducing

participants to four web 2.0 tools and to motivate them to think of various ideas on

how to use these tools in the classroom.

Activity 3 - Affordances: Tools in use: The aim was to explore the ‘affordances’ of one

or more technological tools. The goal was to encourage participants to develop

critical thinking and to make judgements as to which tools are the right ones to use

in their teaching based on their affordances.

Activity 4 - Course Map: The goal of the activity was to propose a way to represent

lessons which can help the design and implementation of a lesson and it can also

facilitate the sharing of designs amongst educators.

Activity 5 - Mapping a design using learning design notation: The goal was to

introduce a method for designing learning activities to the participants.

Activity 6 - Breaking-down a design into the micro-level: This activity was aimed at

further analyze the lesson representation designed in Activity 5.

Activity 7 - Sharing and discussing designs: The goal was to develop an environment

of cooperation and sharing, which can be facilitated with the use of social

networking tools such as Cloudworks.

Key references

Vrasidas, C., Conole, G., Retalis, S. (2010). Usable representations of Learning Design

for Educators & Instructional Designers. Workshop at Online EDUCA Berlin,

December, 2010. (http://www.slideshare.net/pambos/usable-representations-of-

learning-design)

Vrasidas, C., Theodoridou, K., Theodoulou, F., Aravi, C., Pattis, I. (2010). Design

Practice: A Framework for Preparing Teachers to Teach with ICT. International Visual

Literacy Association Conference.

METIS PROJECT

D3.2: Draft of pilot workshop Page 39 of 64

d. Learning Design Studio

Summary

The learning design studio is a collaborative, blended, project based framework for

training educators in effective and evidence-based use of educational technology.

This approach is modelled after the tradition of studio-instruction in arts and design

disciplines (such as architecture). In this model, the main activity of a course is the

students' continued work on design challenges in a defined domain of practice.

Students typically work in groups. They identify an educational challenge, research it,

and devise innovative means of addressing it. The course instructor guides the

students through the process, and classroom sessions are mostly dedicated to group

work and public review of design artefacts.

The Learning Design Studio manifests a model of teaching a Design Inquiry of

Learning. Design, in this context, is the informed creative practice of devising

“courses of action aimed at changing existing situations into desired ones” [0Simon,

1996, p 129]. Inquiry-based learning attempts to shape educational experiences in

the model of scientific investigation. Similarly, an inquiry approach to the training of

educational practitioners should mimic the form of design research in education.

Thus, the learning design studio mimics the structure of a design experiment (Mor &

Winters, 2007), with the exception that students do not have the resources or the

time to conduct several iterations, scaling up from a conceptual prototype to an

extensive deployment.

METIS PROJECT

D3.2: Draft of pilot workshop Page 40 of 64

Key references

Mor, Yishay and Mogilevsky, Orit (2012). A Learning Design Studio in Mobile

Learning. In: The 11th World Conference on Mobile and Contextual Learning (mLearn

2012), 16-18 October, Helsinki.

Mor, Y, & Mogilevsky, O. (submitted to EC TEL) The Learning Design Studio:

Educational Practice as Design Inquiry of Learning

https://docs.google.com/file/d/0B5iNZunquTigUVN1bTdwYi1UVTA/edit

Other references

Anastopoulou, S.; Sharples, M.; Ainsworth, S.; Crook, C.; O'Malley, C. & Wright, M.

(2012), 'Creating Personal Meaning through Technology-Supported Science Inquiry

Learning across Formal and Informal Settings', International Journal of Science

Education 34 (2) , 251-273

Cox, C.; Harrison, S. & Hoadley, C. (2008), 'Applying the "studio model" to learning

technology design',Educating learning technology designers: guiding and inspiring

creators of innovative educational tools, 145

Kali, Y, & Ronen-Fuhrmann, T. (2011). Teaching to design educational technologies.

International Journal of Learning Technology, 6(1), 4–23.

METIS PROJECT

D3.2: Draft of pilot workshop Page 41 of 64

Mor, Yishay and Winters, Niall (2007). Design approaches in technology enhanced

learning.Interactive Learning Environments, 15(1) pp. 61–75.

Mor, Yishay (2013). SNaP! Re-using, sharing and communicating designs and design

knowledge using scenarios, narratives and patterns. In: Luckin,

Rosemary; Puntambekar, Sadhana; Goodyear, Peter; Grabowski, Barbara

L.; Underwood, Joshua and Winters, Niall eds. Handbook of Design in Educational

Technology. London, UK: Routledge, (In press).

Ronen-Fuhrmann, T, & Kali, Y. (2010). The role of concretization in acquiring design

knowledge. Proceedings of the 9th International Conference of the Learning Sciences-

Volume 1 (pp. 468–475).

Ronen-Fuhrmann, T., Kali, Y., & Hoadley, C. (2008). Helping Education Students

Understand Learning Through Designing. Educational Technology, 48, 26–33.

Simon, H. A. (1996), The Sciences of the Artificial - 3rd Edition , The MIT Press ,

Cambridge, MA

e. Collage / LdShake workshops

Summary

At the UVA team (and later, also at UPF as Davinia moved from Valladolid to

Barcelona), we have hosted several teacher workshops related with our research

efforts in the field of Computer-Supported Collaborative Learning and Learning

Design. These workshops were mainly directed at university teachers (although the

approach can be applied to practitioners at other levels – as long as the

technological tools proposed make sense in the actual context). The workshops

aimed to promote participants’ awareness of collaborative learning techniques (e.g.

in the form of collaborative patterns such as the Jigsaw). The main focus was on

providing practical guidance about how to design such learning experiences (both

conceptually by using and combining patterns at different levels, and using

technological support such as the Web Collage authoring tool). In the last editions

there has also been an emphasis on how to implement those collaborative designs

using VLEs (such as Moodle) and external ICT tools (e.g. wikis, Google apps, etc.),

again conceptually and through the use of the GLUE!-PS technological system.

An example of workshop can be found in Hernández-Leo et al. (2011), in which

LdShake (the precursor to METIS’s ILDE) is shown and used by teachers. Only very

METIS PROJECT

D3.2: Draft of pilot workshop Page 42 of 64

recently we have made efforts in reflecting on the workshop approach that has

emerged through these years of research and professional development

interventions. Prieto et al. (accepted) describe one instance of the last round of

workshops that have been held (focusing on design, implementation and

enactment), and tries to extract some of the main guiding principles and patterns of

the approach:

Focusing on multiple aspects of the activities’ “orchestration” (beyond pure

design, also management, adaptation, pragmatic restrictions, etc.)

Focusing on multiple phases of the activities’ lifecycle (beyond design,

towards implementation, enactment)

Use of pedagogical patterns (e.g. design patterns like Jigsaw, but also

deployment/enactment patterns like “on-the-fly monitoring of activities”)

Providing hands-on practical experience not only in the concepts of design,

but also the use of technologies to design and implement the activities

Modeling: the workshops themselves are designed and modelled using the

tools and strategies being taught at the workshop

Authentic/meaningful problems: Participants work on design/enactment

problems that are meaningful for their own concrete courses

Key references

Description of one workshop instance and main characteristics of the approach:

Luis P. Prieto, Yannis Dimitriadis, Juan I. Asensio-Pérez, Sara Villagrá-Sobrino, Iván M.

Jorrín-Abellán (accepted). Fostering CSCL adoption: an approach to professional

development focused on orchestration. Accepted at the International Conference on

Computer-Supported Collaborative Learning (CSCL 2013).

Example of LdShake workshop with teachers: Davinia Hernández-Leo, Lauren

Romeo, Miguel A. Carralero, Jonathan Chacón, Mar Carrió, Pau Moreno, Josep Blat

(2011). LdShake: Learning design solutions sharing and co-edition, Computers &

Education, 57(4), 2011, p. 2249-2260. (see sp. Table 1 and section 3.1)

f. Participatory Pattern Methodology

Summary

The Participatory Methodology for Practical Design Patterns is a process by which

communities of practitioners can collaboratively reflect on the challenges they face

and the methods for addressing them. The outcome of the process is a set of

METIS PROJECT

D3.2: Draft of pilot workshop Page 43 of 64

Design narratives, design patterns and design scenarios situated in a particular

domain of practice. This pattern is an “envelope” for the rest of the patterns in this

paper, and the context described here is the baseline for all the others.

At the heart of the methodology is the PARTICIPATORY PATTERN WORKSHOPS

pattern, which describes the interrelation between three COLLABORATIVE

REFLECTION WORKSHOPS: a DESIGN NARRATIVES WORKSHOP, a DESIGN PATTERNS

WORKSHOP and a DESIGN SCENARIOS WORKSHOP.

Apart from these, the methodology includes a “toolkit” of support patterns, which

address critical points in the process or specific recurring needs.

The methodology is based on two fundamental assumptions: we are all experts, and

we are all designers. This methodology utilises narrative epistemology: practitioners

are prompted to recount their experiences as design narratives, and discuss these

with their peers. The construction and discussion of these narratives are scaffolded

by a set of tools and activities to extract transferable and verifiable elements of

design knowledge in the form of design patterns.

Key references

Mor, Yishay; Warburton, Steven and Winters, Niall (2012). Participatory pattern

workshops: a methodology for open learning design inquiry. Research in Learning

Technology, 20.

METIS PROJECT

D3.2: Draft of pilot workshop Page 44 of 64

Mor, Yishay (2010). Embedding design patterns in a methodology for a design

science of e-Learning.In: Kohls, Christian and Wedekind, Joachim eds. Problems

Investigations of E-Learning Patterns: Context Factors, Problems and

Solutions. Hershey, PA, USA: IGI, pp. 107–134.

http://projects.lkl.ac.uk/ppw

Video - Part 1

Video - Part 2

g. Design Challenge

Summary

This is one example of a series of related workshop designs conceived, delivered and

evaluated by the Open University Learning Design Initiative (OULDI). This one-day

workshop was developed to introduce course teams to using a learning design

approach at the curriculum level as opposed to lower level detailed design of

specific activities. The example below (originally developed in collaboration with the

Open University’s Faculty of Education and Language Studies (FELS) in summer 2009)

uses the OULDI learning tools and activities but tools and activities from other

projects (especially the Viewpoints project) have been used successfully and the

format holds well. For a list of other activities and tools you could use try the

learning design toolbox.

Designed to be fun and engaging, the event gives an awareness of the latest in

thinking innovatively about curriculum design and is designed to be proactive rather

than being composed around an uncontextualised set of one-to-many presentations.

The time-limited challenge enables participants to make a preliminary assessment of

which of the ideas, tools and resources are useful and gives them some feel for what

might be possible in a longer, term real course production process. The challenge is

intended to open avenues for participants to pursue this further.

The OULDI team delivered almost twenty workshops during the project pilots

including fourteen directly associated with the pilots. Post-workshop questionnaires

and later impact surveys reveal a wide range of reaction from participants - even

those present at the same workshop. Overall, feedback has been overwhelmingly

positive (Cross, et al., 2012).

With respect to application to METIS, the following issues should be considered.

METIS PROJECT

D3.2: Draft of pilot workshop Page 45 of 64

(1) The OULDI workshops were designed for Higher Education practitioners. All

of the activities and tools are transferable to other sectors such as those

represented by KEK and Agora, but customisation will be necessary.

(2) In general, the OULDI tools and workshops focus on curriculum level design

whereas METIS is focused on activity design. This means that a selection of

the tools and activities can be usefully applied within METIS (for example, to

specify the educational context of a particular learning activity), they must be

complemented by workshop activities and tools targeted at activity level

design.

Key references

A schedule for the workshop including a description of the activities and tools:

The Open University Learning Design Initiative. (2012). Workshop template: Design

Challenge Retrieved 9/5/2013, from

http://www.open.ac.uk/blogs/OULDI/?page_id=985

Evaluation of the workshops is contained in:

Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). OULDI-JISC Project Evaluation

Report Retrieved 3/8/2012, from

http://oro.open.ac.uk/34140/1/OULDI_Evaluation_Report_Final.pdf

h. OULDI Learning Design Training Module

Summary

As part of the EU Design-Practice project the OU Learning Design Initiative delivered

d a learning design training module to around 40 OU Associate Lecturer participants

in the UK using Cloudworks. This module has a higher theoretical content than the

“Design Practice” workshops (see appendix 2.c; these “Design Practice” workshops

were modelled on the OULDI’s “LD Lite” template which uses more of a practice

based approach).

The learning outcomes of the training module are to improve participants’

knowledge and understanding of:

The design process for learning activities

Choices that practitioners make about the ways of applying technologies for

a variety of learners

and to improve participants’ ability to :

METIS PROJECT

D3.2: Draft of pilot workshop Page 46 of 64

Use and evaluate particular technologies and tools for individual and

collaborative learning

Share and collaborate learning designs.

The training had three components:

(1) A five hour (approximately) self-paced introduction to learning design

module which was held online. This provided the background to (2) below.

(2) A five hour face-to-face ‘Using Technologies in Teaching’ workshop.

(3) A five hour online self-paced activity, which draws on the work undertaken in

the workshop, and is based on the collaboration and sharing of learning

designs.

The training module provides an overview of the methodology for learning design

developed by the Open University's Institute of Educational Technology. The module

is aimed at participants with an interest in using technology in their teaching practice

and/or are involved in the design of learning activities at all scales.

With respect to application to METIS, the following issues should be considered.

(1) The OULDI workshops were designed for Higher Education practitioners (e.g.

it requires the participant to read academic papers).

(2) The duration of the complete module is 15 hours. However, activities from

within the face-to-face and second online components could be customised

to suit METIS participants.

(3) In general, the OULDI tools and workshops focus on curriculum level design

whereas METIS is focused on activity design. However, activities from within

the face-to-face and second online components of this module are targeted

at activity design and sharing and so could be customised to suit METIS

participants.

Key references

A schedule for the workshop including a description of the activities and tools:

http://cloudworks.ac.uk/cloudscape/view/2294.

METIS PROJECT

D3.2: Draft of pilot workshop Page 47 of 64

METIS PROJECT

D3.2: Draft of pilot workshop Page 48 of 64

Appendix 3. Methodology design narratives and design patterns

a. “Design Narratives” task presented to partners

The following instructions where provided to project partners asking them to

provide design narratives of successful workshop practices:

METIS Methodology Design Narratives: telling a ‘good’ story

Task: Each METIS partner authors 2-3 Methodology Design Narratives

(see http://www.ld-grid.org/resources/representations-and-languages/design-

narratives), based on an activity, from your own methodology (the methodology

descriptions you authored are located here[1]).

The idea behind the methodology design narrative is simple: narrate a ‘good’ story

about a workshop activity (e.g. icebreaker, task, wrap-up, etc.) that was successful.

This can be from a workshop you either led or attended.

The narrative should describe what participants were asked to do “X” (e.g.

participate in an activity and/or complete a task, etc.) and what was the intended

outcome “Y”.

In writing the narrative please describe the “how” in as much detail as possible (e.g.

steps A, B, which led them to the intended outcome Y).

You can do this by providing a “thick description” of the activity and sequence of

events participants (protagonists) were given, the steps (challenge), their choices

and the results (positive ones). If you have a sample of what materials were used in

the activity or photos this is also useful.

Rationale: We require these Methodology Design Narratives for WP3. Once we have

them, we will guide you in extracting patterns from them. Our goal is to extract

applicable patterns from existing Methodology Design Narratives, with a view to

applying some (or all) of the applicable patterns to the METIS workshop design.

LD Grid: Please have a look at The Learning Design Grid’s[2] "Healthy Eating[3]"

as a Design Narrative example. This example is far more extensive than what

we're asking for here, but if anyone wants a broader view it may be useful.

Process: Please download the file below 'metis-methodology design narratives-

template_template.docx' (Sample Methodology Design Narrative Template) and

metis-methodology design narratives_Sample.docx (Sample Methodology Design

Narrative). Then using the template (and sample) author 1 to 3 Methodology

METIS PROJECT

D3.2: Draft of pilot workshop Page 49 of 64

Design Narratives and upload your narratives (each one in a separate file) on this

page by May 15, 2013.

To assist you in this task and for consistency across the project, we are providing the

following template and example in the word document below. The example is of

one activity, out of eight, from a workshop that lasted 2 hours.

[1] See https://sites.google.com/a/metis-project.org/metis-internal/workpackages/wp3-workshop-

design/methodology-cross-review/methodology-descriptions

[2] http://www.ld-grid.org/resources/representations-and-languages/design-narratives

[3] https://sites.google.com/a/ld-grid.org/www/resources/learning-designs/pi-project-healthy-eating-activity/healthy-eating-design-narrative

b. Design narrative template

Situation

A Tagline for your project or work Describe the user group and the work context Describe your technological setup

Task

Describe what you are trying to achieve

Actions (How did you try to address the issue?)

Activity .

Activity goal(s) or intended outcome (“X”)

Activity title & description Title:

Activity materials

METIS PROJECT

D3.2: Draft of pilot workshop Page 50 of 64

The “how” (steps A, B, C, that lead to Y or the activity outcomes)

.

.

.

Activity outcomes (“Y”) (positive)

The activity was successful because it achieved the intended outcomes as well as:

Results (What were the results of the actions you took?)

Activity outcomes (“Y”) (positive)

The activity was successful because it achieved the intended outcomes as well as:

Reflection

Observation and what you have learned.

METIS PROJECT

D3.2: Draft of pilot workshop Page 51 of 64

Appendix 4. Sample of Methodology Design Narratives

a. Meta-Pyramid design

1. Situation

The GSIC-EMIC research group at the UVA has conducted several workshops in the last two years, for professional development of academic teaching staff of the university. The main aim of these workshops was to promote the adoption of computer-supported collaborative learning in university teachers’ everyday practice, including both the conceptual side of designing and running collaborative activities, and the technological one of using commonly available ICTs (Moodle, web apps), but also specific ones (e.g. the Web Collage authoring tool).

This narrative is extracted from a workshop held in February 2012, which made more emphasis on the conceptual side of designing collaborative activities, and deciding about ICT tools to use to support such learning. The workshop included 2 face-to-face sessions and a few online activities, for a total of about 12 hours of workshop. There were 25 university teachers from a variety of backgrounds and levels of teaching experience in the workshop. The technological setup was based around a Moodle course with embedded Google Docs documents for collaborative work; also, extensive use of pen and paper was also part of the workshop philosophy.

2. Task

The main aim of the workshop was to provide teachers with a few best practices for collaborative learning and its implementation using ICTs, at different levels, including high-level patterns such as the Jigsaw9, and lower level design and implementation “tricks” (which we called “routines” or “atomic patterns”10) extracted from successful collaborative learning practice of other teachers. The ultimate goal of the workshop was that, after the workshop, each teacher had created a collaborative scenario for his/her own practice, as a necessary step for applying such techniques in practice. It is also important to note that the workshop activities themselves were designed and structured around these best practices, so that teachers could experience what collaborative learning “felt like” as an student (which not everyone had done in their own previous education).

3. Actions (How did you try to address the issue?)

9 A very common collaborative strategy for addressing complex problems by dividing them in parts.

See http://pandora.tel.uva.es/wic2/patterns/en/jigsaw/ for a more complete description. 10

See Prieto et al.’s “Recurrent routines: Analyzing and supporting orchestration in technology-enhanced primary classrooms” at http://www.sciencedirect.com/science/article/pii/S0360131511000091

METIS PROJECT

D3.2: Draft of pilot workshop Page 52 of 64

Activity Designing a collaborative learning scenario.

Activity goal(s) or intended outcome (“X”)

Engage teachers in their first collaborative learning design

Encourage collaboration among participants Make a number of best practices available to teachers (patterns,

routines)

Show the value of iteration in learning design

Activity title & description

Title: Design a Pyramid activity using a Pyramid This was a lengthy activity that encompassed three main phases, of 30-45 minutes each, where they had to address a fictitious but realistic scenario (the need for a zero-course on bibliographic search and collaborative report writing for undergraduates) that required them to design (and later, enact) a set of collaborative learning activities. These three phases introduced progressively collaborative best practices of different kinds (overall strategies or patterns, such as the Pyramid11; more concrete design “routines”; and even more concrete deployment/implementation “routines”), which were explained briefly by facilitators and provided to participants in the form of cards (see below, translated from the original Spanish).

Also, this activity led participants to review their designs and those of their peers, and iterate over them, going from the initial, rather abstract idea, up until they had a clearer idea of how they would use ICTs to implement it. Later in the workshop, this design was taken further, and parts of it were enacted through a role-playing, to discuss yet another set of enactment “routines” that might be useful when running such collaborative activities.

Activity materials

Large white paper surfaces

Pens, highlighters, pencils A digital camera Scenario description (see an example in Spanish here).

11 See http://pandora.tel.uva.es/wic2/patterns/en/pyramid/, Hernández-Leo, D.; Asensio-Pérez, J.I.;

Dimitriadis, Y.; & Villasclaras, E.D. Generating CSCL Scripts: From a Conceptual Model of PAttern Languages to the Design of Real Scripts. In: Goodyear P.; Retalis, S. (eds.). Technology-Enhanced Learning, Design patterns and pattern languages, Sense Publishers, Series Technology-Enhanced Learning; 2010. p. 49-64. Appendix available at: http://ulises.tel.uva.es/%7Edherleo/dpbook/appendix-chapter.pdf

METIS PROJECT

D3.2: Draft of pilot workshop Page 53 of 64

Pattern, design routine and deployment routine cards (examples in Spanish are available here, here, and here, respectively), and an English translation of the titles of such cards can be found here (see appendix B)

The “how” (steps A, B, C, that lead to Y or the activity outcomes)

This activity took place after a short introduction by the facilitators to the workshop methodology and goals, and about collaborative learning in general.

A. Explanation of the scenario: Although participant teachers had had to read the scenario previously, the facilitators remind participants about the scenario (zero course in bibliographic search and joint report writing) and the scope of the design (10 hours of blended student work).

B. First design iteration (with patterns): facilitators present very briefly the idea of collaborative design patterns, and describe two of them (Pyramid and Jigsaw). Then, participants are divided in heterogeneous groups of three people (they had already been seated in such groups at the beginning of the workshop to minimize hassle) and are provided with a paper description of several such design patterns. Then, for 30 min, they are asked to design the learning activities, applying one of the patterns (the Pyramid) to the provided scenario. The design is to be done in paper, in whatever format they desire (textual, graphical, diagrammatic, tabular...). Below we can see the result from one of the groups:

C. Second design iteration (with design routines): Facilitators then briefly

present other set of best practices, of a lower granularity (design routines), as elements of blended CSCL practice that have been observed in successful practice by other practitioners, that can be used to further flesh out the overall structure denoted by the design pattern. Facilitators give participants a set of these routines (each on a yellow card) and, in the same groups as in phase B, participants are asked to re-iterate their design, using such cards to make the activities and strategies to use more concrete. Again, 30 min are allocated to this

METIS PROJECT

D3.2: Draft of pilot workshop Page 54 of 64

task. Although it was not explicitly said, many teachers at this point glued the yellow cards over different parts of the previous design, to mark when a certain design routine would be used. At the end of this phase, a short break is scheduled, which facilitators use to make photos of the designs so far, uploading them to the workshop’s Moodle.

D. Cross-review of designs (with implementation routines): Now, facilitators provide yet another set of routines (“deployment routines”, represented by green cards), which are also lower-level, and which deal with implementation and deployment details of a CSCL activity using the workshop’s concerned ICTs (Moodle, Google Docs, etc.). Then, for 15 min, participants are asked to look at the design of another group (and vice-versa), and provide three aspects they liked, three problems they saw in the design, and suggestions of deployment routines that might be useful in the context of that design. These suggestions are to be written in Google Docs that have been set up beforehand by facilitators (one per group), and are accessible through the workshop’s Moodle.

E. Third design iteration: Participants now are asked to work in 6-person groups by joining the two groups that have cross-reviewed each other (thus, following the Pyramid pattern that is being used in the designs). For another 30 min, participants are asked to produce a complete design using all the elements (by re-iterating either of the groups’ original designs, or doing an entirely new one that takes advantage of the strategies of both). Below, one of the final products is reproduced:

METIS PROJECT

D3.2: Draft of pilot workshop Page 55 of 64

Afterwards, other workshop activities were meant for teachers to reflect on the experience they had just undergone, and to individually apply their recently-discovered skill (designing collaborative activities) to their own real courses.

Activity outcomes (“Y”) (positive)

4. Results (What were the results of the actions you took?)

Activity outcomes (“Y”) (positive)

The activity was successful because it achieved the intended outcomes:

It exposed participants to a (fairly large and heterogeneous) number of best practices

It engaged participants in collaborating with each other, intensely and for a lengthy period of time

Teachers had a quite concrete, finished design they had done themselves

It provided them with an actual learning experience of the kind they were designing

It exposed them to the challenges and benefits of enacting such collaborative activities using ICT (by seeing facilitators struggle to follow the workshop’s collaborative design)

5. Reflection

We use this kind of activity a lot lately in PD actions, especially in those workshops that try to convey the concepts of collaborative learning to people that may have little or no

Despite the fact that teachers tend to like this kind of activity a lot (unless they are a-priori against collaborative learning), it poses several important challenges (which are nevertheless intrinsic to CSCL):

o It is quite long, but also very active (participants have to produce something tangible regularly), and some teachers find it a bit “stressful”

o It requires a good amount of preparation (writing, printing and distributing the cards, making the participant groups beforehand, etc.). The technological side also has to be prepared (creation of the Moodle course, GoogleDocs, etc.), but lately that has been relieved a lot by the usage of WebCollage + GLUE!-PS (now part of the ILDE)

o Although it seems to be a very long activity, strict timekeeping is essential, since the creative effort can be easily derailed by discussions, musings, people just getting to understand each other (take into account that a group may have people from wildly different disciplines).... Facilitators should monitor the activities, solve any doubts early, and “push” groups to produce

METIS PROJECT

D3.2: Draft of pilot workshop Page 56 of 64

something (even if only a partial product) on time o Some teachers often find it difficult to start the design process unless

concrete tips are given about what kind of product is expected – i.e. what elements should a design have: is it groups and group sizes, timeframes, materials to use... (but you do not want to make it too restrictive either to the point that it becomes uncreative). The routine cards seem to help a bit to make the level of detail needed explicit

There is a tension between making the scenario (and the participants’ designs) specific and relevant to only a certain domain (e.g. Engineering), which might make people from other disciplines uninterested. On the other hand, making the scenario too generic can be bland and get all participants uninterested. For now, we are trying to make the scenario to be about a transversal problem (bibliographic search and report writing), and let each group navigate the problem of how domain-specific they want their design to be.

b. Role-playing on problematic situations and technology

1. Situation

The GSIC-EMIC research group at the UVA has conducted several workshops in the last two years, for professional development of academic teaching staff of the university. The main aim of these workshops was to promote the adoption of computer-supported collaborative learning in university teachers’ everyday practice, including both the conceptual side of designing and running collaborative activities, and the technological one of using commonly available ICTs (Moodle, web apps), but also specific ones (e.g. the Web Collage authoring tool and the GLUE!-PS system, now part of the ILDE).

This narrative is extracted from a workshop held in April 2012, which made more emphasis on the technological side of designing collaborative activities, implementing such designs with concrete ICT tools available to the teachers (Moodle, Google Docs and other web apps). The workshop included 2 face-to-face sessions and a few online activities, for a total of about 12 hours of workshop. There were 22 university teachers from a variety of backgrounds and levels of teaching experience in the workshop. The technological setup was based around a Moodle course with links to the authoring and deployment groups (Web Collage, GLUE!-PS), and embedded Google Docs documents for collaborative work.

2. Task

The main aim of the workshop was to provide teachers with the necessary technological skills to implement a collaborative learning and its implementation using ICTs, starting from a fairly concrete conceptualization of the design (in descriptive form). The ultimate goal of the workshop was that, after the workshop, each teacher had implemented a collaborative scenario for his/her own practice, ready to be enacted by the students. The workshop also emphasized the fact that problematic situations may arise when ICT technologies are used for collaborative learning, and aimed to show teachers how to use the technology to address

METIS PROJECT

D3.2: Draft of pilot workshop Page 57 of 64

those problematic situations in practice. It is also important to note that the workshop activities themselves were implemented and structured around the set of best practices and technologies that participants were learning, so that teachers could experience what ICT-supported collaborative learning “felt like” as an student (something that almost none of the participants had done before).

3. Actions (How did you try to address the issue?)

Activity Problem-based technology practice

Activity goal(s) or intended outcome (“X”)

To be conscious of problems of using ICT for CSCL, especially when using learning design approaches (e.g., due to a-priori instantiation)

Review some of the “enactment routines” (run-time best practices in CSCL) from previous workshops

Show teachers practically how to address very common run-time problems using the technology being taught at the workshop (GLUE!-PS)

Activity title & description

Title: Solving run-time problems with GLUE!-PS This was a 1-hour activity that took place once teachers had learned how to do a learning design with Web Collage and to deploy it using GLUE!-PS into a Moodle or MediaWiki platform (first by following a walkthrough of the tools, then by using them to do a design for their own practice and deploying it into a VLE of their choice). In this activity teachers are presented with three common situations that pose problems when ICTs (sp. those addressed by the workshop: Web Collage, GLUE!-PS, Moodle) are used to support CSCL. These situations include the formation of groups at the beginning of an activity, and their re-formation later on, in the face of unexpected occurrences; also, the need for changing the supporting ICT tools on-the-fly during enactment (e.g. due to a failure in the service provider – think, what happens if Google Docs goes down?). Participants are prompted to solve one of the three situations by freely exploring the GLUE!-PS interface in dyads, and then share the solution to all situations in 6-people groups. This activity occurred towards the end of the workshop, but still there was time set for participant reflection on the challenges and advantages of using this kind of ICTs to implement CSCL in everyday teaching practice (see the mini-focus-group design narrative).

Activity materials

Documents describing the problematic situations and prompting participants to action (see example in Spanish here).

Documents describing step-by-step the solution to each problematic situation (to be distributed after the workshop for reference purposes) – see example here.

Laptop computers (one per two participants), with Firefox/Chrome browser

A GLUE!-PS server and Moodle/MediaWiki servers, where each

METIS PROJECT

D3.2: Draft of pilot workshop Page 58 of 64

participant/dyad has an account and an example design/deployment/course to experiment with

The “how” (steps A, B, C, that lead to Y or the activity outcomes)

F. General intro: Facilitators introduced briefly and in a general manner the fact that ICTs can be useful in supporting CSCL and in designing CSCL... but they may also fail and introduce problems, especially during the enactment (see slide 12 here, in Spanish).

G. Situation description: Facilitators briefly describe each of the three problematic situations, which are also given as short documents (see example in Spanish here).

H. Participants, in dyads, are asked to enter the GLUE!-PS system (linked from the workshop’s Moodle course) and try to solve one of the problems using GLUE!-PS.

I. The dyads are joined in groups of 6 people (each dyad had solved a different situation) and are asked to explain to their partners the situation they solved and how they did it (by performing the change live in the system).

J. Once the activity has ended, participants are given documents detailing, step by step, the solution to each of the three problematic situations (so that they can use them later in their practice, if needed). These walkthroughs include both text and images (see below):

4. Results (What were the results of the actions you took?)

Activity outcomes (“Y”) (positive)

The activity was successful because it achieved the intended outcomes as well as:

Participants collaborated and helped each other to find a solution

Participants internalized the solutions by explaining them to fellow participants

Participants explored freely the GLUE!-PS user interface and found intuitively solutions to the presented problems

All this was achieved in an unexpectedly short amount of time

METIS PROJECT

D3.2: Draft of pilot workshop Page 59 of 64

5. Reflection

One of the main values of this part of the workshop (something that participants also acknowledged later in questionnaires) was that it reckoned the possibility of failures when using ICTs, and showed ways of tacking them, something that is seldom portrayed in PD actions (sp. shorter ones). Other reflections:

This activity’s progress was surprisingly smooth, and was performed by all groups in an even shorter time than allocated. This may indicate that teachers understood rapidly the internal logic of the GLUE!-PS system, or that we mis-assessed the effort necessary by two people to tackle it.

Related to this, the work in dyads at the beginning of the activity (dyads that were pre-defined by facilitators, on the basis of being heterogeneous in tech-savviness) seemed to help a lot in making progress along the different activities (sp. in the beginning stages of a workshop, where participants still do not know the technology). Of course, this runs the risk of one member of the dyad “doing all the techie work” – but we tried to offset this by encouraging participants to change roles in each new activity.

Although in the end we could not do it because of lack of time, this activity could be pre ceded by a short warm-up role-playing exercise, in which e.g. a 6-people group enacts the classroom situation, posing as teacher and students.

c. Mini-focus-groups

1. Situation

The GSIC-EMIC research group at the UVA has conducted several workshops in the last two years, for professional development of academic teaching staff of the university. The main aim of these workshops was to promote the adoption of computer-supported collaborative learning in university teachers’ everyday practice, including both the conceptual side of designing and running collaborative activities, and the technological one of using commonly available ICTs (Moodle, web apps), but also specific ones (e.g. the Web Collage authoring tool and the GLUE!-PS system, now part of the ILDE).

This narrative is extracted from a workshop held in April 2012, which made more emphasis on the technological side of designing collaborative activities, implementing such designs with concrete ICT tools available to the teachers (Moodle, Google Docs and other web apps). The workshop included 2 face-to-face sessions and a few online activities, for a total of about 12 hours of workshop. There were 22 university teachers from a variety of backgrounds and levels of teaching experience in the workshop. The technological setup was based around a Moodle course with links to the authoring and deployment groups (Web Collage, GLUE!-PS), and embedded Google Docs documents for collaborative work.

METIS PROJECT

D3.2: Draft of pilot workshop Page 60 of 64

2. Task

The main aim of the workshop was to provide teachers with the necessary technological skills to implement a collaborative learning and its implementation using ICTs, starting from a fairly concrete conceptualization of the design (in descriptive form). The ultimate goal of the workshop was that, after the workshop, each teacher had implemented a collaborative scenario for his/her own practice, ready to be enacted by the students. The workshop also emphasized the fact that problematic situations may arise when ICT technologies are used for collaborative learning, and aimed to show teachers how to use the technology to address those problematic situations in practice. It is also important to note that the workshop activities themselves were implemented and structured around the set of best practices and technologies that participants were learning, so that teachers could experience what ICT-supported collaborative learning “felt like” as an student (something that almost none of the participants had done before).

Of course, there was a flip side to these pedagogical objectives, which was the fact that we were doing research on the support that the workshop and the tools employed in it gave teachers to apply CSCL in their everyday practice. This secondary goal of the workshop enters into play prominently in this activity (since it may be shared by the METIS workshops also).

3. Actions (How did you try to address the issue?)

Activity Small-group discussion

Activity goal(s) or intended outcome (“X”)

For participants to reflect upon the applicability of the practices and tools presented in the workshop, to their everyday practice.

For researchers to gather participants’ opinions on the usefulness of the proposed tools and practices (later to be triangulated with other data sources)

Activity title & description

Title: Problems and issues in real use of ICTs in (teaching) practice This activity was to happen towards the end of the workshop, after participants had designed, deployed, and solved run-time problems in CSCL scenarios using ICT tools such as Web Collage and GLUE!-PS. The idea is to have all participants reflect Participants were divided in 6-people groups and three main “reflection prompts” were provided (regarding the applicability of collaborative learning to their practice, the applicability of the ICT tools presented, and about the run-time problems presented and elicitation of further problematic situations). Each of these small groups had one workshop facilitator to moderate the debate, audio record it and keep participants on track, and other roles are distributed among the members of the group: timekeeper, scribe, spokesman. After 30 minutes of discussion, the spokesmen highlight the one most important idea that emerged from the discussion of each of the three prompts. Once this activity was finished, the facilitators did a short wrap-up with parting reflections and recommendations, before the workshop finishes.

METIS PROJECT

D3.2: Draft of pilot workshop Page 61 of 64

The Google Doc documents with the rest of the debate’s ideas for each group are shared among all the participants through the workshop’s Moodle.

Activity materials Slides presenting the debate dynamics and the reflection prompts for the group discussion (see slide 19 here, in Spanish)

Large paper surfaces, post-its, pens and markers A Google Doc document for each small group

One audio recorder for each small group (a laptop with an audio recording program will also do)

The “how” (steps A, B, C, that lead to Y or the activity outcomes)

K. Facilitators introduce briefly the dynamic of the debate (see slide 18 here, in Spanish), the issues to be discussed, and the roles to be taken: moderator/recorder (one of the facilitators), scribe (writing the discussion’s ideas in a Google Docs), timekeeper (ensuring that each issue is dedicated 10 min at most), spokesman (voicing the main group conclusion on each issue after the debate)

L. Participants are divided into 6-person groups, and given 30 minutes to discuss about the three issues. They are encouraged to draw and write their ideas in paper (so that they are not forgotten), while one of the members tries to write them down in a Google Docs that was previously created by facilitators). The facilitator in each group keeps a background role, just to spark up discussion if it seems to dwindle.

M. Once the time is up, facilitators remind about each of the issues, and ask the spokespeople to provide a short, 1-2 minute conclusion about that issue by their small group. The Google Docs documents are left in the workshop’s Moodle for everyone in the participant community to access.

4. Results (What were the results of the actions you took?)

Activity outcomes (“Y”) (positive)

The activity was successful because it achieved the intended outcomes as well as:

Allowed some (limited) time for teachers to collectively reflect on the usefulness (or not) of the presented tools/practices

Gives researchers first insights into the ecological validity of the proposed tools/practices

Captures all participants’ opinions in multiple forms (audio, written), even if there is not time to share all of them with the whole group

Serves to elicit new problems, issues, use cases for the tools, which researchers might not have thought of

5. Reflection

METIS PROJECT

D3.2: Draft of pilot workshop Page 62 of 64

One of the main values of this activity is as an additional data source for evaluation of the workshops, but also as the seed to begin building a community of practitioners interested (in this case) in CSCL, which previously were quite isolated (since they often report being the “rara avis” in their respective departments). Also, it gives the adequate feeling of “closure” needed at the end of the workshop.

METIS PROJECT

D3.2: Draft of pilot workshop Page 63 of 64

Appendix 5. Personas template

This work by the OU H817 module ““Openness and innovation in elearning”

(http://www3.open.ac.uk/study/postgraduate/course/h817.htm) is licensed under a

Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales

License.

See http://www.ld-grid.org/resources/representations-and-languages/personas for

a short introduction to personas. To use this template, open it at

http://goo.gl/m1Fp6, make a copy and edit.

Name: Gender: Age: Lives in ... with ... Likes ...

Education and experience

Role and responsibilities

Technical skills

Subject domain skills and knowledge

Motivation and desires

Goals and expectations

Obstacles to their success

Unique assets

METIS PROJECT

D3.2: Draft of pilot workshop Page 64 of 64

Appendix 6. Survey of user groups

In March 2013 a survey of representatives of the three METIS user groups was

carried out. The full results of the survey are available to project partners at

https://sites.google.com/a/metis-project.org/metis-internal/workpackages/wp3-

workshop-design/task-3-3-development-of-a-draft-structure-for-pilot-workshops.

Findings from the survey include

That 2 of the 3 user groups expect that their target audience will be prepared

to do work related to the workshop before the workshop occurs, but the

other group will not.

Representatives of 2 of the 3 user groups think that their target audience

would prefer no more than 2 half-day workshops in the first round

(September 2013 – February 2014). The other thinks that 1 half-day

workshop is the maximum that their target audience will be able to attend.

All 3 groups can be expected to do some follow up work afterwards, provided

it is focused or linked to certification.

All 3 representatives believe a blend of face-to-face workshops and online

activities before or after the workshops will best suit their user group.

One of the representatives stated that certification would be very important

for their target audience; it was less important for the other two groups.

The representatives thought that a variety of tangible outputs would be

useful for their groups. The range includes electronic and printed handbooks,

draft of learning design as a Word or image file and in a printed form.