e assessment presentation ver2 june 2008

26
Jo Richler Assessment has always had an e in it Jo Richler June 2008 Assessment has always had an ‘e’ in it or A quick look at (e-)assessment

Upload: jo-richler

Post on 27-May-2015

773 views

Category:

Technology


2 download

DESCRIPTION

This is a presentation to start the process of exploration and deveopment of e-assessment.

TRANSCRIPT

Page 1: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Assessment has always had an ‘e’ in it

or

A quick look at (e-)assessment

Page 2: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

In the beginning we had…..

www.adelaide.edu.au

Page 3: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

http://www.outsider.co-uk.com/

Page 4: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

• Interactive resources for musiciansThis is a Palatine funded project, with the support of City College Manchester, Vortex WebHosting and RealStrings.com

• Due to a server upgrade (to php5) many of the old interactive tests on this site will no longer mark your answers correctly.

• Pop music theoryHelp files and online tests, supporting the LCM Popular Music Theory syllabus

• Comment on and discuss these resources - click here. • KEYBOARDNew project in development - music language exercises on a

keyboard, with auto-marking and data collection.• Study Songs - a Flash based elearning project, to develop aural skills and

musical analysis through a song. • Pop Music Theory exam - History section questions - your suggestions for

essay content

• inside the music inside the music inside the music inside the music inside the music inside the music

Page 5: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Then we added self assessment…Diagnostic to identify student needs and to determine prior or existing knowledge as wellas strengths and weaknesses. Often diagnostic assessments occur prior toanother learning experience.

Self-Assessment to assist individual students to review their own performance, to assess theirown learning and to obtain feedback which can support future learning andreflection.

Formative Assessment to assist the student's learning by providing feedback during the learningprocess.

Summative Assessment Assessments which are designed to evaluate student’s overall knowledge andunderstanding of the material presented in the unit. They give a quantitativegrading and may be given as assessed coursework or as part of an end of unitexamination.

Page 6: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008 http://ims.ode.state.oh.us/ODE/IMS/images/classroom_assess

ment_types.gif

Page 7: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Summary of decision-making issues for assessment formats (Crisp 2005)

Low stakes assessment

Medium stakes assessment

High stakes assessment

Purpose of assessment

Improve learning, identify teaching gaps

Improve learning, progression to new concepts

Credentials, gate keeping, progression, certification

Consequences if problems arise

Few with low impact Some with modest impact

Significant with high impact

Resources required

Often minimal, can use low threshold software

Modest investment in large scale system

Significant investment in enterprise system

Consequences of cheating

Few Some Significant

Authentication of learner

Not important Maybe important Very important

Invigilation required

Not usual Sometimes Always

Development effort

Minor Medium Major

Evaluation of reliability and validity

Not usual, anecdotal feedback from colleagues and learners sought

Subject matter expert provide feedback

Requires professional psychometric analysis

Page 8: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

• Guidelines for E-Assessment• These Guidelines apply to summative forms of e-assessment, although a department• may also wish to consider the principles below as good practice for formative forms of • e-assessment.• Operational Guidance• Students should have had access to/experience of the exam format and the technology prior to the

summative exam. Good practice would be to give students prior exposure to a formative exam with feedback.

• The examination should be rehearsed in the same technical environment (although not necessarily with the same group of students) as the actual examination, to ensure that it is robust.

• Invigilation will be appropriate to the type of e-assessment being used and will require technical invigilation where the server which delivers the assessment is located. This may require prior liaison and agreement with Information Services staff.. Invigilators should be fully briefed prior to the assessment.

• The summative exam should only be accessible by secure password and the performance recorded by university-approved secure management tools suited for the purpose.

• Computers used for summative exams should wherever possible have both internet and communication tools disabled, except as needed for the purpose of the assessment.

• Reasonable and appropriate adjustments should be made for students whose disability would put them at a disadvantage due to the format of the exam. Students who have made a case for special arrangements must be offered an alternative to the computer based exam e.g. a paper based assessment.

• The use of a large pool of exam questions from which a randomised sub-sample of questions are generated to produce individual student exams is acceptable so long as the pool covers all aspects of the examinable material and the sub-sample is representative. The appropriateness of utilising a pool of questions depends on the subject-specific content of the questions and how they were designed.

•www.ltss.bris.ac.uk

Page 9: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Title: Feasibility study of implementing e-assessment: lessons to be learned.Authors: John Andresen (School of Chemical & Environmental Engineering), Dragos Axinte (School of Mechanical Materials & Manufacturing Engineering and Management), David Hann (School of Mechanical Materials & Manufacturing Engineering and Management), Mark Haw (School of Chemical & Environmental Engineering), Karen Steel (School of Chemical & Environmental Engineering), Katy Voisey (School of Mechanical Materials & Manufacturing Engineering and Management).Keyword: Assessment.The project evaluates the suitability of existing e-teaching environments to fulfil students’ learning needs, academics’ requirement to handle versatile “teaching/ assessment tools” and “sine qua non” constraints in teaching engineering subjects.Dealing with “precise quantities” the engineering subjects are, theoretically, suitable for employing e-assessment/learning methods. E-assessment automates marking, hence reducing the marking load on academics. Moreover, e-assessment enables monitoring of large numbers of students while providing academics with necessary feedback to evaluate students’ progress and their needs for attaining better module understanding.

Page 10: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

• What are the advantages of e-assessment?• Richer assessment experience – questions can be made clearer and more detailed

through the use of text, sound and video which can aid motivation. For example, e-portfolios allow the use of digital video, animations, presentations etc to be submitted electronically for assessment – impossible in a 'paper world!'

• Increased flexibility – assessment can be provided at a greater range of locations. This means assessment on-demand can become completely achievable. This allows learners to formally demonstrate their understanding at a time and place that is convenient for them. For example, e-tests can be taken (under test conditions) in a range of locations as diverse as the community hall through to a formal e-testing centre.

• Instant feedback – results are often available within minutes of taking an e-test, as well as diagnostic information on a learner's performance, highlighting areas that can be improved upon.

• Reduce the administration burden – less paper forms to complete, no posting of test papers, no printing and posting candidates portfolios and no storage space needed!

• It's what candidates want!

Page 11: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Embedding e-Assessment and

e- Portfolios to support Learning and Teaching

Amanda BlackStuart Jones

Page 12: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

e-Portfolio Components

e-Portfolio Management ToolOrganise own learning and

Collaborate with others

ShowcasePresent evidence to others

Personal Learning SpaceRepository / Storage

Virtual Learning EnvironmentLearning platform providing

content and support

Transcript FileROA/ LAR

Page 13: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Why e-assessment for learning?• Why ‘e’? Assessment embedded into resources, immediate

feedback, the ability to record progress and ‘distance travelled’ in a format accessible to more than just the DfES, LEA or teacher. Peer group assessment. ICT enables learners to engage in, and assess progress on authentic tasks.

• The importance of the e-portfolio process. A clear progression of work supporting reflection, analysis and learning transactions. The portability of elements of work and progress across a lifetime and not just a school year.

Page 14: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

http://eassessmentinwales.jiscinvolve.org

/about/• The University of Wales Institute, Cardiff (UWIC) has long held

strong partnerships with colleges in the South East Wales region. Like many courses nationwide across the FE and HE sectors, retention has been identified as a key issue, as has the requirements of effective and timely student feedback. Work has begun to identify possible critical moments when some sort of intervention might make a large positive difference on the student experience. One sort of intervention might well be e-assessment in the form of formative feedback, for instance.

•  The HE Academy has recognised the degree to which e-assessment could help support universities and colleges in addressing these issues; recognising also the need for effective curriculum design if assessment of any sort is to remain a beneficial part of the learning and teaching process.

Page 15: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 16: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 17: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 18: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 19: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 20: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 21: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 22: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 23: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 24: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

Page 25: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

And finally….

Page 26: E Assessment Presentation Ver2 June 2008

Jo Richler Assessment has always had an e in itJo Richler June 2008

• Computers now marking free text responses better than humans

• March 6, 2008 at 4:34 pm · Filed under OU VLE, eAssessment

• Sally Jordan gave a workshop today on how to use the Intelligent Assessment Technologies system we’ve got plugged into our VLE to develop short text response questions. One example she gave (I’ve reworded it slightly):

• A raindrop falls vertically with constant speed. What can you tell from this about the forces acting on the raindrop?

• The answer, which I could just about recall from my Higher Physics, is that they’re equal and opposite. You can enter this in all sorts of ways, with misspelling, synonyms and a variety of grammatical structures. Merely enter ‘equal’ and �you’ll be given another chance with appropriate feedback saying you’ve only got it partially correct. Sally’s trials show that students are marked accurately by the computer 97% of the time.

• In a study to ascertain the effectiveness of this technology responses from students on our introductory science course (S103) to seven questions were marked by the Intelligent Assessment system and six tutors. There were variations in the marking of four of these questions among the tutors, some of whom disagreed with the question author as to the correct response.

• There was no surprise that the computer could mark more consistently than the tutors overall and more in line with the question author. What was surprising was the number of misunderstandings, slips and inconsistencies which occurred with the human markers.