+

QUALITY ASSURANCE FOR E-LEARNING

AN ESSENTIAL EXTENSION FOR LIFELONG LEARNING

Oulu 19-20.5.2009

Keith Williams

Leo Wagemans

Jo Boon

Pekka Kess

European Association of

Distance Teaching Universities

EADTU

EADTU is Europe’s leading representative associationfor Lifelong Open and Flexible (LOF) learning in distance HE.

• Members are:• 12 Open and distance teaching Universities• 15 Consortia and Associations of conventional

universities operating with e-learning and DE• 3 Associate members

• Representing over 200 Universities and • 2 million students.

Added value of e-learning

• QA in HE is receiving much attention at institutional, national and European level.

• Still missing: QA specific to e-learning-criteria based on ease of access-new forms of interaction (students and staff)-flexibility, personalisation and other pedagogicalaspects that are more relevant to e-learning

These express the added value of e-learning to HE.

E-xcellence: Benchmarking Quality in e-learning

Main Contributors:EADTU OULU-University (Finland)OUNL (Netherlands)OUUK (United Kingdom)

with support from:CNED (France)UNED (Spain) UOC (Spain) EITSA (Estonia) NETTUNO (Italy) APERTUS (Hungary)NVAO (Netherlands/Belgium)European University Association (EUA)

APPROACH•Analysis of existing quality processes of partners and/or requirements of national quality organisations

•Making inventory of missing criteria (by 12 European e-learning experts)

•Stakeholders consultation procedure

•Translation of these parameters into a quality assessment tool based on benchmarks

•Testing in 3 pilots the benchmarks, the manual and approach:-the institutions benefited from the process and interactive discussions. -they received useful information in further developing the course/ programme. The exercise was relevant for the course but also for the university as a whole.-the process was however time-consuming and there is a need for improved instructions beforehand.

The E-xcellence manual• Mindmap QUALITY manual as backbone of web-tool

Products to work with

manual– reference tool for the design and assessment of e-

learning programmes – benchmarks, quality criteria and notes for guidance

against which e-learning programmes and their support systems may be judged

assessors notes– provide a more detailed account of the issues and the

approaches. Good practices for various situations. web-based instrument Quick scan Full assessment

+

http://www.eadtu.nl/e-xcellenceQS/

E-xcellence Products

Quickscan

• Simplified version of the full e-xcellence assessment tool• Quick self-assessment of your institution’s e-learning

performance• Identifies areas in your e-learning programme/course that need

more attention • Score yourself on the most relevant aspects

Results will show after completing quick-scan. These help you: • To identify elements to be improved• To guide the internal discussion • To learn if a full quality assessment procedure is useful

http://www.eadtu.nl/e-xcellenceQS/

Full e-learning assessment

Evaluate the institution/program/course using the E-xcellencemethod and tool.

Manual; Benchmark statements, in relation to criteria and indicators.

Questionnaire: comments on the specific issue: in the box you can refer to documents or other references which can be used as evidence on that specific aspect of e-learning.

The on-line questionnaire needs to be filled out by different disciplines in your organisation coming from management, course designers, tutors and students

On-site visit of e-learning expert

Full assessment

REVIEW TOOLAGAINST INSTITUTIONALSTRUCTURE+ PROCESS

PREPARE SELF ASSESSMENT USING THE TOOL

SITE VISIT - SELF ASSESSMENT ANALYSIS- INTERVIEWS- CONCLUSIONS

ACTION PLANNING

Support: manual, proposed agenda, assessors’ notes, glossary.

E-xcellence +

Goal: From project to mainstream implementation of the E-xcellence instrument European wide at the local level.

• E-XCELLENCE+ brings together the expertise and experience of universities in lifelong learning from 13 countries as well as the expertise of quality assurance and accreditation processes from several QA agencies as a framework for educational improvement and innovation.

• E-XCELLENCE+ promotes the use of E-xcellence European wide and envisages increased performance and innovation in e-learning by integration of the instrument in the institutional and national policy frameworks.

How to reach out and influence current bussiness models?

+

Local introduction

Steps of local introduction

1. Sensibilisation on including QA for e-learning

2. Information on the E-xcellence instrument

3. Organising cooperation universities and QA-agencies

4. Finetuning instrument and existing systems

5. Integration

6. Implementation

What to achieve in the local seminars

• Mapping experiences• Determining the local impact (dialogue,

shift of attention, roadmaps to improvement, etc)

• Finetuning• Establishing a sustainable use of the

instrument (internal + external QA)• Receive feedback on the instrument

Local introduction

13 Local seminars

OUNL (Heerlen)

Oulu University (Oulu)

OUUK (Milton Keynes)

EITF (Tartu)

Hungarian Virtual University Network (Budapest)

Uninettuno (Rome)

Lund University (Lund)

Czech Association of the Distance Teaching Universities + University of Hradec Králové

(Brno/ Hradec Králové)

UNED (Madrid)

KU Leuven (Leuven)

Slovak university of technology (Bratislava)

MESI (Moscow)

FernUni Schweiz (Bern)

+

2 full Assessments:

-Open Universiteit Nederland

-Lund university

Universities

External QA agencies

E-learning experts

• Yearly update of the instrument, available as open sourceImproving accessibility, interactiveness and flexibility for the LLL-student.

• E-xcellence Associates LabelPromote continuous educational improvement cycles for LLL.

• 13 and more countries working with the E-xcellence instrument (institutional/QA agency) Mainstreaming by integration.

Reaching out by networking

• ENQA-EADTU

• UNESCO

E-xcellence contributing to LLL

+

2009

Experience Rome

UninettunoA student questionnaire is developed based on E-xcellence framework to analyse the perception of students about e-learning with regard to:-integration and interaction of multimedia technologies-flexibility and adaptability-instruments of collaborative learning

Experience Finland

‘The evaluation with the E-xcellence instrument createda lively discussion inside the university.

In all, the experience was very useful to most peopleparticipating internally in the evaluation:- new ideas were created to the course design- new foundations were found to justify decisions- experience exchange between the evaluatorsand staff was extremely valuable- new working methods were discovered’(Pekka Kess: OULU University)

Peer review with other universities

Remarks about the Manual, Tool and Audit process, Finland

• Using the tool: individually vs. team (from management to students)

• Good exercise for the institution and the members to go through the stages of the process of benchmarking; people become aware of choices and implementations.

• It brings the institution really to a formulation of the policy of e-learning.

E-xcellence Project

Remarks about the Manual, Tool and Audit process, Italia

• Two kind of benchmarks: benchmarks which describe products and benchmarks which describe processes

E-xcellence Project

Remarks about the Manual, Tool and Audit process, Estonia

• The tool is too general and too much focused on management level

• There must be an instruction for the use of the tool: a manual.

E-xcellence Project

Remarks about the Manual, Tool and Audit process, Leuven

• Function of the QS: too much as an assessment tool; better to use it as an improvement tool

• Possible scenario’s to use QS

• Too much from the perspective of DL, less focus on blended learning.

E-xcellence Project