a proposal for skill evaluation via complex tasks in virtual learning environments

A proposal for skill evaluation via complex task in

virtual learning environments



Jaione Cubero IbáñezMaría Soledad Ibarra Sáiz

Gregorio Rodríguez Gómez Universidad de Cádiz (Spain)

Jaione Cubero IbáñezMaría Soledad Ibarra Sáiz

Gregorio Rodríguez Gómez Universidad de Cádiz (Spain)

EE ALforALfor EVALfor Research Group

Presenting author: Gregorio Rodríguez Gómez






1. Introduction2. Context3. EDECOM program4. Objective5. Methodology6. Results7. Conclusions

1. Introduction2. Context3. EDECOM program4. Objective5. Methodology6. Results7. Conclusions

A proposal for skill evaluation via complex task in virtual learning environments

1. CONTEXTUALIZACIÓN


1. INTRODUCTION

The growing relevance of Information and Communication Technologies (ICT) in all aspects of society means that university education must adapt to new learning environments which are conditioned by virtualisation and digitalisation.

The use of e- assessment in higher education is a relatively new education practice that has been more frequently studied in recent years (Stödberg, 2012). It is, however, already recognised as a fundamental part of e-learning.(Whitelock, 2009).




1. INTRODUCTION

Skills, competences"a latent attribute, knowledge, attitude, ability, proficiency or faculty for the practice of a profession, a job or an academic task, carrying out the required labour or academic tasks and activities properly and correctly” (Rodríguez, G. e Ibarra, M.S. 2011: 70)

e- Assessment“electronic evaluation process in which ICTs are used for the presentation and implementation of activities and assessment tasks and record the answers”. JISC (citado en Rodríguez, G. e Ibarra, M.S. 2011: 29)

e-Assessment of skills New strategy for the skills e-assessment in virtual learning

environments.




2. CONTEXT: DevalSimWeb PROJECT

Skills development through participatory assessment Skills development through participatory assessment and simulation using web tools and simulation using web tools

ALFA III (2011)-10 ALFA III (2011)-10




3. EDECOM training program

“Evaluation and Development of Professional Skills in Higher Education”

- Blended-learning Program- Virtual learning environment- Destined to final year university students. - Aims to develop professional skills (which are transversal to evaluation) via their participation in e-assessment processes.

- analytical and critical thought- problem solving- sense of ethics- decision making - teamwork




3. EDECOM training program

The students will actively participate in e-assessment processes via the use of EvalCOMIX web service and electronic tools, such as forums, wikis and simulation games

It is based on the idea that training students in assessment skills enhances the self-regulation of their learning and therefore their academic success and employability.




4. AIM

Develop a methodological design for the e-assessment of skill level



WHICH ARE THE PROFESSIONAL SKILLS TRANSVERSAL TO EVALUATION?

A catalogue of work-related and professional skills was created.

To write this catalogue a study was developed in order to analyse the importance given to different soft skills by stakeholders (companies, teachers and students).

The results showed a need to promote five competences which are considered transversal to evaluation :

- analytical and critical thought- problem solving- sense of ethics- decision making - teamwork

5. METHODOLOGY



Skills Definition

Analytical and critical thoughtActs systematically, establishes hierarchies and uses tools for analysis; questions the

situations which arise and discusses the pertinence of the judgements in order to make decisions within his/her role.

Problem solvingIdentifies, analyses, builds criteria and applies knowledge when facing specific

situations, makes decisions and evaluates possible risks.

Sense of ethicsActs responsibly according to universal axiological principles and respects diverse

cultural contexts.

Decision making

Acts opportunely, efficiently and efficiently, agilely and proactively; uses support from relevant information which facilitates the identification, understanding, evaluation

and acceptance of risks in situations of uncertainty; chooses the best alternative via consultation with the most appropriate sources, comparing and contrasting them and

integrating them.

Teamwork

Makes responsible commitments and collaborates with other people in the achievement of common objectives, shares resources and knowledge, harmonises

various interests, evaluates the opportunities and therefore contributes to a working environment which facilitates the implementation and development of tasks.

Figure 2. Definition of the skills, according to the DevalsimWeb Catalogue of Skills.(Ibarra, Rodríguez y García, 2013)



Level Description

Everyday

Shows an intuitive knowledge and basic grasp of assessment, allowing the development of elementary

and complex activities concerning the assessment actions to be carried out.

Thoughtful (technical)

Establishes analysis and connections between the actions, using procedures, strategies and resources

which allow sensible explanations based on the obtained results.

Comprehensive (inclusive and

creative)

There is integration, application, creativity and critical thought in the decisions made concerning the

actions and results of the evaluation; they are discussed and information is provided which is objective, reliable,

useful and oriented towards improvement.

Figure 3. Levels of skill development (Ibarra, Rodríguez y García, 2013)



HOW TO ASSESS STUDENT SKILLS LEVEL? COMPLEX TASK

5. METHODOLOGY

An assessment system “based on the active participation of the students, allowing them to use their knowledge creatively to resolve real problems” (Villardón, 2006). Complex activities or problematic situations (Fernández, 2011) must involve:

•The assessment requires the observation of dimensions and situations

•Application to more than one situation to facilitate transference

•Being close to reality (authentic)

•Mobilisation of various resources, both internal (knowing, knowing how and knowing how to be) and external (requested autonomously by the student)

•Development indicators (levels of command)

•Involvement of various agents in the assessment process (concordance)

A complex activity was designed for each of the skills and the student must address and respond to the established questions.



This is a space through which the students will be able to give a response to different situations/problems (complex tasks) which are related to the selected skills.

6. RESULTS

The “EDECOM-DevalSimWeb Skill Level” virtual campus was designed.

Their responses will be evaluated by external evaluators using e-rubrics so as to determine their initial skill level.

The proposal to evaluate the initial skill level of students participating in the EDECOM-DevalSimWeb Training Program

The proposal to evaluate the initial skill level of students participating in the EDECOM-DevalSimWeb Training Program



6. RESULTS

All participants in the Training Program must provide a response to the five complex tasks as an obligatory requirement to be able to access the EDECOM-DevalSimWeb Training Program.

Each complex task presents three unavoidable elements to assist the students’ comprehension:

•A short introduction

•The material which must be watched or read (videos, articles, etc.)

•The submission area, which details the specifics of how to respond to the activity.

example



6. RESULTS

Skill Name of complex task Resource

Decision making It’s time to decide Video

Sense of ethics What would you do? Moral dilemma

Analysis and critical thought

Robot invasion! Opinion article

TeamworkWhat if we set up a

restaurant?Video

Problem solving What’s the problem? Written problem

Figure 4. Complex task and resource used for each skill.



Figure 5. e-rubric to evaluate the initial skill level of the Sense of ethics skill.

Authentic assessment and student participation: is it all too much?


7. CONCLUSIONS

• This proposal was shared in order to provide new strategies to address skill assessment via virtual learning environments, which are appearing ever more frequently in a university setting.

• The evaluation questions and tasks used in e-assessment must evolve into sophisticated tasks (Boyle & Hutchison, 2009), which are understood to be: media-rich stimulus materials which are interactive and complex - aspects which feature in the tasks designed in our proposal.

• The final goal is to be able to discover and compare the students’ final skill development level once they have completed their training in the EDECOM-DevalSimWeb Training Program.

http://devalsimweb.eu


THANK YOU FOR YOUR ATTENTIONTHANK YOU FOR YOUR ATTENTION

EE ALforALfor

EVALfor Research Group

PARTICIPANTS

Jaione Cubero Ibáñez. Universidad de Cádiz (Spain)[email protected]

María Soledad Ibarra Sáiz. Universidad de Cádiz (Spain)[email protected]

Gregorio Rodríguez Gómez. Universidad de Cádiz(Spain) [email protected]

e-mail: [email protected]ágina Web: http://www.uca.es/evalfor