Download - Quality In Computer Supported Collaborative eLearning by Lambropoulos Romero

Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learning

Jueves, 25 de febrero de 2010

Quality Assurance in Computer Supported Collaborative Learning

Environments

Euro-CAT Ergonomic Evaluation

Dr. Niki Lambropoulos * Dr. Margarida Romero

Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010

• What is Quality?

• Utility, usability, acceptability– Case study: Toll staff training (learning situation + ergonomic

criteria)

• CSCL ergonimic evaluation

– Case study: Euro-CAT Ergonomic Evaluation

• 5 quality criteria

Quality Assurance in Computer Supported Collaborative Learning Environments

Agenda


What is quality?


• Fit of purposeBergman and Klefsjö (1994):

“The quality of a product (article or service) is its ability to

satisfy

the needs and expectations of the customers”

• European Foundation for Quality in eLearning (EFQUEL)


1.

Utility, usability, acceptabilityCase study: Toll staff training (learning situation + ergonomic criteria)

Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010Tooll staff training needs

• USA Toll Tariffs:No training is necessary, even

children can became toll staff ! • European Toll Tariffs:• Intensive training needs to achieve 2 main

learning objectives1. Vehicle category identification2. Angry management


Where and when can a toll worker could follow an elearning course?

During the work hours?

During the coffee break?

At home?

Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria Is the

course useful?

Is the course easy to

use?

…mmmh, if the course is not useful, compatible to my workplace and out of the work life, reliable… and my colleagues don’t like the course, will I accept

the course?


• As if the learning situation was not complex enough.... we will consider the ergonomics criteria

– UTILITY– USABILITY – ACCESSIBILITY

Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria

UTILITY

Relevance and efficacy (Tricot, 2007) for the performance in academic settings or the workplace

Is the course

useful?


Is the course easy to

use?

USABILITY

Computer Learning Environments easy to learn and

easy to use (Preece et al, 1994)


ACCEPTABILITY Not only consider the usefulness and usability,

but also robustness, cost and reliability of ICT applications

…mmmh, if the course is not useful, compatible to my workplace and out of the work life, reliable… and my colleagues don’t like the course, will I accept

the course?


SOCIAL ACCEPTABILITY Not only consider the usefulness and usability,

but also robustness, cost and reliability of ICT applications

Joel says the course is nuts; for me, is quite

boring, and you?

No, Joel is nuts, the course is just

not enough challenging and

interactive

[Joel] The course don’t work in my computer… I needed to use my son computer to

follow the course…


Nielsen’s Taxonomy of System Acceptability


Toll staff training – Case study Conclusions

1. In addition to pedagogical criteria we need to consider the utility, usability and (social) acceptability criteria

2. All these criteria need to be considered for a successful elearning course


2.

CSCL ergonomic evaluationCase study: Euro-CAT Ergonomic Evaluation


HCI-Ed in 7 Steps

1.1. Context & Learning Values - HypothesisContext & Learning Values - Hypothesis2.2. (Iterative) Design – Requirements (Iterative) Design – Requirements 3.3. Evaluation with user groups/expertsEvaluation with user groups/experts4. Development5. Evaluation with user groups6. Re-Design & Development7. Study & Research - Tool Release

1CH 2DR 3E 4D 5E 6D 7RTR


Research Problem

Distributed learning teams in team-based learning situations aim to achieve organisational convergence.

However, there are collaboration problems due to mediated distance & time flexibility.



1. Key CSCeL concept: Collaboration Awareness Students’ difficulties Group Awareness Time convergence

2. Learners’ Intentions (learning goals): Task Convergence3. Users’ operations (actions/tasks): Pedagogical Usability

Step1a: Context & Learning Values



Being HumanCollaboration Awareness

Students’ difficulties: In addition to f2f Collaborative Learning difficulties, CSCeL situation adds Computer Mediated Communication (CMC) and ILE difficulties (Kirschner,

2009)

Group Awareness: understanding of the activities of others, which provides context for your own activity (Dourish &

Bellotti, 1992)

Group time awareness: Distributed learning students need to develop an awareness of their team-mates’ temporal patterns in order to regulate the Time-on-Task efficiently in the collective level (Romero, 2010)



3C Model: Communication, Coordination & knowledgeConvergence (Ellis et al, 1991)

How?



If group awareness is a prerequisite for convergence then increase of group awareness will result in increase of

organisational convergence and thus, collaboration awareness.

Step1b: Hypothesis



Euro-CAT…• Collaboration Awareness Tool • Task-independent• Supports 3C via

– Mirroring capabilities– Awareness cues

• Does not replace other 3C tools

Step2: (Iterative) Design - Requirements



Step2: (Iterative) Design - Requirements

The design can be defined as an individual and collective activity, finalized by a project to develop a physical and symbolic artefact. The artefact is rarely well defined in this step.



Key Concepts for Design

Group ProgressUsersUsers

Scheduling

Trail tracking

NowNow

Awareness Cues



Euro-CAT Screenshot



Step3: User/Expert Evaluation @ this workshop

Euro-CAT Ergonomic Evaluation – Pedagogical Utility/Usability/Accessibility Questionnaire &

System Usability Scale (SUS)– Experts/users evaluation– Think aloud protocol



Results 1a/3: Questionnaire


Results 1b/3 Questionnaire

EuroCAT Ergonomics Mean Std. Deviation

Walk up Usability 4.4000 .54772

Ease of Use 4.0000 .70711

Functionality Integration 3.6000 .54772

Learnability 4.2000 .44721

Interface Attractiveness 2.4000 1.14018

Originality 3.6000 1.51658

Collaboration 3.0000 .70711

NavigationNavigation 4.8000 .44721

Task Regulation 3.8000 1.48324

Message Legibility 4.0000 1.22474

Observe Availabilities 4.2000 .83666

Facilitate Group Work 3.2000 .83666

Overall Satisfaction 3.8000 .83666

Overall Enjoyability 3.6000 1.14018

Overall Usability 3.0000 .70711

Overall Score 3.706/5.00


Results 3a/3


Results 3b/3


Results 3c/3


Solutions


• Ask the e-tutors/e-learners for solutions!• Direct fit between educational task and the method chosen

to pursue it• Iterative Design

– Engaging different user/expert groups• Next step: Evaluation in context

Conclusions & Insights


1. Utility

2. Usability

3. Acceptability

4. Learner/user-centred design (HCI-Ed)

5. Adaptive eLearning:

- Diverse Teaching/Learning Styles

- Learning Alone/Learning Together

- Learning anywhere/anytime

5 criterios de calidad en E-learning

Ergonomics


Thank you!

Questions?

Dr. Niki Lambropoulos

[email protected]

Dr. Margarida Romero

[email protected]

Download - Quality In Computer Supported Collaborative eLearning by Lambropoulos Romero

Top Related