Orchestrating collaborative technology-enhanced

ecosystems: How to support teachers with conceptual and technological toolsProf. Yannis Dimitriadis

GSIC/EMIC research groupUniversity of Valladolid

May 30, 2012Based on work by L.P. Prieto – S.

Villagrá

University of SydneyCOCO-CHAI-LATTE Joint Seminar

TEL/CSCL ecosystem

2

Outline

3

The two hats of this seminar Hat 1: Conceptual tools

A framework on orchestration Multi-level patterns as mediating tools

Hat 2: Technological tools for Learning design Flexible deployment

The two hats of this seminar

4

Personal Engineering background + Interest in

education Audience (and associated drinks …)

CHAI-LATTE and CoCo TEL research

sound pedagogy + efficient engineering Quest for sustainable innovation

good research for understanding, abstraction and evidence

effective support to actors – teachers, learners or even service and technology providers

The two hats of GSIC/EMIC

5

“Dark and bright side” in a group since 1994

Focus on CSCL: complex and innovative Support real-world actors and own action-

research Inquiry cycle (design-enactment-evaluation)

for teaching/learning and technological

support

Hat 1: Conceptual tools

6

Analyze and understand the field of orchestration in complex TEL ecosystems The “5+3 aspects” framework for

orchestration

Collect and use knowledge in terms of good practices during the CSCL cycle Multi-level patterns

Orchestration: Moving target

7

Increasing presence in literature, but what is it about? Is there a new focus?

Efforts to define the field/concept/metaphor: Stellar NoE deliverable, CSCL 2011 and

ICLS 2012 workshops, collective paper New proposal for “5+3” orchestration

framework and validation through expert panel

Focus of orchestration on …

8

Acknowledge complex ecosystems for innovative TEL/CSCL environments

Support teachers as orchestrators (and think of how to minimize-distribute the orchestration load on teachers, learners and systems)

Build on pragmatism and minimalism as keys for sustainability

Explicit emphasis on how to flexibly manage (learning time) enactment

Design for adequate orchestrating and orchestrable technologies (and conceptual tools)

The “5+3” frameworkA diagramatic view

9http://prezi.com/aa2vighak7hh/orchestration-in-tel-cscl-as-easy-as-53/

The “5+3” frameworkA process view

10

Evaluating the frameworkThe setup

11

Research questions on Completeness of common understanding Usefulness for researchers

Two panels (March – May 2012) Pilot phase: 22 Spanish researchers of low-

medium expertise (2.58) and more technological profile (16T-7E-1M)

International panel: 24 researchers of medium-high expertise (4,46), high research experience in TEL (15,8 years), more balanced profile (6T-11E-7M)

Completeness of framework

12

Quantitative and qualitative data Non-normative and inclusive Need to clarify role of actors, technology,

theory More than 4,5 (Lickert scale of 1-6) for all

items Logical: 4,78 –Comprehensive: 4,52 –

Relevant: 5,13

Usefulness of framework

13

To be referenced in papers, checklist and didactic tool for young researchers

But need for normative advice and real-world examples

Will be used in future (4.43)

Provided new insights (4.17)

Multi-level patterns

14

But the “5+3” Orchestration framework is a conceptual tool mainly for researchers

Tools (conceptual or technological) for orchestration should also support “teachers” Exploiting articulated (design) knowledge

(Pedagogical) patterns as mediating tools regarding various orchestration aspects Design and Management of learning flow,

Assessment, Adaptation … Related fields of

Learning Design, Educational Modeling, Scripting

Collaborative Learning Flow and assessment patterns

15

Learning Flow

Assessment

Aligning both sets of patterns

16

Usefulness of patterns

17

Are these patterns useful for practitioners? Evidence for learning flow patterns for OER

repurposing to collaborative activities Appropriation in University (Higher

Education – HE) teachers (Professional Development - PD) workshops

But not always If not included in appropriate technological

tools Too complex (abstract) for (Primary

Education – PE) practitioners Do not take into account enactment issues

The need for atomic patterns

18

Two independent studies by GSIC and SRI using GroupScribbles in Primary and Secondary Education schools (2008-2011)

Studying enactment and disciplined improvisation

Showed that other small-scale, informal, contextual, actionable patterns emerged

A learning design in actionin a Primary Education classroom

19

Using atomic patterns in a HE PD workshop

20

Atomic patterns as mediating conceptual tools

21

And we need to “fill the gap”, “flesh the bones” with appropriate mediating artifacts (conceptual tools)

Evaluation of atomic patternsin PD workshops

22

Four (2 in PE, 2 in HE) PD workshops (2010-2011)

WS4 data quantitative data (scale 1 to 7): Enactment patterns useful (5,31) Feasibility (5,22) and nearness (6,06) 19 out of 22 teachers enriched design

Useful but not revolutionary Especially important for non-experienced or

pre-service teachers Need to condense and classify long list of AP What about inclusion in technological tools?

Approach for multi-level patterns

23

And the way to elicit and use multi-level patterns

Hat 1: Wrap-up

24

Knowledge regarding orchestration for researchers: “3+5 Aspects” framework Approach for multi-level patterns Atomic pattern catalogue and LD

representation Support teachers too with conceptual tools:

Classroom norms for participation, discourse moves for discussion, decision rules for contingent teaching (SRI)

Multi-level patterns Use in PD workshops Role playing and

learning design enrichment using cards (GSIC)

Hat 2: Technological tools

25

But TEL is not only about Conceptual tools (patterns, moves, …)

It is also about enacting learning in TEL complex

ecosystems especially in HE distributed learning

environments where VLE/PLE are combined with third-party tools

And also support teachers (and learners) with technological tools when orchestrating TEL/CSCL ecosystems (less

orchestration load for teachers – more for tools)

Learning Design tools

26

Articulate/exploit the knowledge of patterns and support the learning design process

Create a model or representation of a learning design (unit of learning, script, scenario, etc.) That allows understanding and reflecting

on the pedagogy, making the learning design explicit, sharing it with others

Through the Web Collage tool Or other Learning tools by the Stellar

Learning Design Grid Theme Team http://www.ld-grid.org/home

Web Collage: Learning & Assessment flows

27

Teacher

Select and work individually in a

subproblem

Support individual work

Students

Work in expert groups of students

with the same subproblem

Form Jigsaw groups and explain each other the different

subproblems

Support expert group work

Support jigsaw group work

Write activity report

Review report

Work in expert groups of students

with the same subproblem

input for…

provide feedback…

http://pandora.tel.uva.es/wic2

A learning design(HE teacher in a PD workshop)

28

Web Collage:Evolution and evaluation

29

Evolution of the Collage tool since 2005 Towards a more stable and user-friendly

Web-based tool that includes learning and assessment patterns, as well as support for instantiation

Extensive use and evaluation using mixed methods Multi-case studies in authentic settings and

PD workshops with HE teachers Wide acceptance of the pattern-based

design process

The deployment gap

30

But teachers wanted to complete the cycle deploy their design in their TEL

environment, especially their institutional VLE (e.g. Moodle)

Alternatives for deployment

31

Have the learning design “in mind” and then go through the VLE/PLE interface and struggle to “translate” the design to a course/lesson plan

Use a single system (LAMS) to do both design and deployment in the same environment

Use an LD Tool (LDSE) to create the interpretable script and then deploy it to one environment

But there are many LD tools and VLEs: Provide intermediary to connect them:

Glue!-PS

Glue!-PSAn architectural view

32

Glue!-PS:A functional view

33

Glue!-PS Validation

34

Proof of concept validations using the Planet Game (ICALT 2006) scenario Two LD languages: LDL + IMS-LD Two Learning Environments: Moodle +

MediaWiki Design and deployment

by GSIC researchers of 36 learning designs created using pen and paper at two HE PD workshops (Sep-Nov 2011)

by 24 teachers in a follow-up PD workshop (Feb 2012) Immediate use in real life: 6,19 (scale 1

to 8)!

Glue!-PS and atomic patterns

35

Atomic patterns are especially important for flexible orchestration

University teachers suggested embedding atomic patterns in technological tools

Initial evidence of use in Glue!-PS Implementation of a subset of atomic

patterns without architectural changes as e.g. AP: Spontaneous use of additional IT Actions: Add and synchronize in real-

time resources in an activity Feasibility evidence in 2 cases (Mar-Apr

2012)

Hat 2: Wrap-up

36

Technological tools to support orchestration Patterns (that reflect good educational

practices) guide a flexible design and deployment process

Glue!-PS allows to bridge the deployment gap in distributed learning environments time-efficient and less error-prone allows sustainable migration to new VLE

and connection to various LD tools More naturalistic evaluation is necessary Important role of PD workshops for innovation

and feedback to research

Hat 2 (cont.): More tools to …

37

Regulate through monitoring and the detection of critical aspects of the learning designs (GLUE!-CAS)

Find and select appropriate tools and include them in learning designs (SEEK)

Integrate third-party tools in learning designs directly through the VLE interface (GLUE!)

Design evaluation of CSCL case studies and generate multimedia reports (CSCL-EREM)

….

Hats 1 and 2: wrap-up

38

Support bits and pieces but also the complete cycle of orchestration flexibly and sustainably

Design for orchestration to show added value of TEL innovative pedagogies using old and new

technologies Integrate conceptual and technological tools

for researchers, teachers and students

And many open questions

39

Do researchers need an orchestration theory and framework and should it be normative?

Is the proposed approach for pattern elicitation sustainable? (crowd-sourcing ….)

How can we support co-orchestration with students and integrate efficiently PLE?

How various (Web, 3D worlds, augmented physical) spaces can be orchestrated?

Is there a way to solve the ethical and security problems that are involved?

…

Some links

40

Publications http://www.gsic.uva.es/public.php?lang=en

&list_public=all GSIC/EMIC channel at YouTube:

http://www.youtube.com/user/gsicemic?feature=watch

WS on orchestration at ICLS2012 (July 3) https://sites.google.com/site/iclsorchestrati

on2012/