e assessment presentation ver2 june 2008
DESCRIPTION
This is a presentation to start the process of exploration and deveopment of e-assessment.TRANSCRIPT
Jo Richler Assessment has always had an e in itJo Richler June 2008
Assessment has always had an ‘e’ in it
or
A quick look at (e-)assessment
Jo Richler Assessment has always had an e in itJo Richler June 2008
In the beginning we had…..
www.adelaide.edu.au
Jo Richler Assessment has always had an e in itJo Richler June 2008
http://www.outsider.co-uk.com/
Jo Richler Assessment has always had an e in itJo Richler June 2008
• Interactive resources for musiciansThis is a Palatine funded project, with the support of City College Manchester, Vortex WebHosting and RealStrings.com
• Due to a server upgrade (to php5) many of the old interactive tests on this site will no longer mark your answers correctly.
• Pop music theoryHelp files and online tests, supporting the LCM Popular Music Theory syllabus
• Comment on and discuss these resources - click here. • KEYBOARDNew project in development - music language exercises on a
keyboard, with auto-marking and data collection.• Study Songs - a Flash based elearning project, to develop aural skills and
musical analysis through a song. • Pop Music Theory exam - History section questions - your suggestions for
essay content
• inside the music inside the music inside the music inside the music inside the music inside the music
Jo Richler Assessment has always had an e in itJo Richler June 2008
Then we added self assessment…Diagnostic to identify student needs and to determine prior or existing knowledge as wellas strengths and weaknesses. Often diagnostic assessments occur prior toanother learning experience.
Self-Assessment to assist individual students to review their own performance, to assess theirown learning and to obtain feedback which can support future learning andreflection.
Formative Assessment to assist the student's learning by providing feedback during the learningprocess.
Summative Assessment Assessments which are designed to evaluate student’s overall knowledge andunderstanding of the material presented in the unit. They give a quantitativegrading and may be given as assessed coursework or as part of an end of unitexamination.
Jo Richler Assessment has always had an e in itJo Richler June 2008 http://ims.ode.state.oh.us/ODE/IMS/images/classroom_assess
ment_types.gif
Jo Richler Assessment has always had an e in itJo Richler June 2008
Summary of decision-making issues for assessment formats (Crisp 2005)
Low stakes assessment
Medium stakes assessment
High stakes assessment
Purpose of assessment
Improve learning, identify teaching gaps
Improve learning, progression to new concepts
Credentials, gate keeping, progression, certification
Consequences if problems arise
Few with low impact Some with modest impact
Significant with high impact
Resources required
Often minimal, can use low threshold software
Modest investment in large scale system
Significant investment in enterprise system
Consequences of cheating
Few Some Significant
Authentication of learner
Not important Maybe important Very important
Invigilation required
Not usual Sometimes Always
Development effort
Minor Medium Major
Evaluation of reliability and validity
Not usual, anecdotal feedback from colleagues and learners sought
Subject matter expert provide feedback
Requires professional psychometric analysis
Jo Richler Assessment has always had an e in itJo Richler June 2008
• Guidelines for E-Assessment• These Guidelines apply to summative forms of e-assessment, although a department• may also wish to consider the principles below as good practice for formative forms of • e-assessment.• Operational Guidance• Students should have had access to/experience of the exam format and the technology prior to the
summative exam. Good practice would be to give students prior exposure to a formative exam with feedback.
• The examination should be rehearsed in the same technical environment (although not necessarily with the same group of students) as the actual examination, to ensure that it is robust.
• Invigilation will be appropriate to the type of e-assessment being used and will require technical invigilation where the server which delivers the assessment is located. This may require prior liaison and agreement with Information Services staff.. Invigilators should be fully briefed prior to the assessment.
• The summative exam should only be accessible by secure password and the performance recorded by university-approved secure management tools suited for the purpose.
• Computers used for summative exams should wherever possible have both internet and communication tools disabled, except as needed for the purpose of the assessment.
• Reasonable and appropriate adjustments should be made for students whose disability would put them at a disadvantage due to the format of the exam. Students who have made a case for special arrangements must be offered an alternative to the computer based exam e.g. a paper based assessment.
• The use of a large pool of exam questions from which a randomised sub-sample of questions are generated to produce individual student exams is acceptable so long as the pool covers all aspects of the examinable material and the sub-sample is representative. The appropriateness of utilising a pool of questions depends on the subject-specific content of the questions and how they were designed.
•www.ltss.bris.ac.uk
Jo Richler Assessment has always had an e in itJo Richler June 2008
Title: Feasibility study of implementing e-assessment: lessons to be learned.Authors: John Andresen (School of Chemical & Environmental Engineering), Dragos Axinte (School of Mechanical Materials & Manufacturing Engineering and Management), David Hann (School of Mechanical Materials & Manufacturing Engineering and Management), Mark Haw (School of Chemical & Environmental Engineering), Karen Steel (School of Chemical & Environmental Engineering), Katy Voisey (School of Mechanical Materials & Manufacturing Engineering and Management).Keyword: Assessment.The project evaluates the suitability of existing e-teaching environments to fulfil students’ learning needs, academics’ requirement to handle versatile “teaching/ assessment tools” and “sine qua non” constraints in teaching engineering subjects.Dealing with “precise quantities” the engineering subjects are, theoretically, suitable for employing e-assessment/learning methods. E-assessment automates marking, hence reducing the marking load on academics. Moreover, e-assessment enables monitoring of large numbers of students while providing academics with necessary feedback to evaluate students’ progress and their needs for attaining better module understanding.
Jo Richler Assessment has always had an e in itJo Richler June 2008
• What are the advantages of e-assessment?• Richer assessment experience – questions can be made clearer and more detailed
through the use of text, sound and video which can aid motivation. For example, e-portfolios allow the use of digital video, animations, presentations etc to be submitted electronically for assessment – impossible in a 'paper world!'
• Increased flexibility – assessment can be provided at a greater range of locations. This means assessment on-demand can become completely achievable. This allows learners to formally demonstrate their understanding at a time and place that is convenient for them. For example, e-tests can be taken (under test conditions) in a range of locations as diverse as the community hall through to a formal e-testing centre.
• Instant feedback – results are often available within minutes of taking an e-test, as well as diagnostic information on a learner's performance, highlighting areas that can be improved upon.
• Reduce the administration burden – less paper forms to complete, no posting of test papers, no printing and posting candidates portfolios and no storage space needed!
• It's what candidates want!
Jo Richler Assessment has always had an e in itJo Richler June 2008
Embedding e-Assessment and
e- Portfolios to support Learning and Teaching
Amanda BlackStuart Jones
Jo Richler Assessment has always had an e in itJo Richler June 2008
e-Portfolio Components
e-Portfolio Management ToolOrganise own learning and
Collaborate with others
ShowcasePresent evidence to others
Personal Learning SpaceRepository / Storage
Virtual Learning EnvironmentLearning platform providing
content and support
Transcript FileROA/ LAR
Jo Richler Assessment has always had an e in itJo Richler June 2008
Why e-assessment for learning?• Why ‘e’? Assessment embedded into resources, immediate
feedback, the ability to record progress and ‘distance travelled’ in a format accessible to more than just the DfES, LEA or teacher. Peer group assessment. ICT enables learners to engage in, and assess progress on authentic tasks.
• The importance of the e-portfolio process. A clear progression of work supporting reflection, analysis and learning transactions. The portability of elements of work and progress across a lifetime and not just a school year.
Jo Richler Assessment has always had an e in itJo Richler June 2008
http://eassessmentinwales.jiscinvolve.org
/about/• The University of Wales Institute, Cardiff (UWIC) has long held
strong partnerships with colleges in the South East Wales region. Like many courses nationwide across the FE and HE sectors, retention has been identified as a key issue, as has the requirements of effective and timely student feedback. Work has begun to identify possible critical moments when some sort of intervention might make a large positive difference on the student experience. One sort of intervention might well be e-assessment in the form of formative feedback, for instance.
• The HE Academy has recognised the degree to which e-assessment could help support universities and colleges in addressing these issues; recognising also the need for effective curriculum design if assessment of any sort is to remain a beneficial part of the learning and teaching process.
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
Jo Richler Assessment has always had an e in itJo Richler June 2008
And finally….
Jo Richler Assessment has always had an e in itJo Richler June 2008
• Computers now marking free text responses better than humans
• March 6, 2008 at 4:34 pm · Filed under OU VLE, eAssessment
• Sally Jordan gave a workshop today on how to use the Intelligent Assessment Technologies system we’ve got plugged into our VLE to develop short text response questions. One example she gave (I’ve reworded it slightly):
• A raindrop falls vertically with constant speed. What can you tell from this about the forces acting on the raindrop?
• The answer, which I could just about recall from my Higher Physics, is that they’re equal and opposite. You can enter this in all sorts of ways, with misspelling, synonyms and a variety of grammatical structures. Merely enter ‘equal’ and �you’ll be given another chance with appropriate feedback saying you’ve only got it partially correct. Sally’s trials show that students are marked accurately by the computer 97% of the time.
• In a study to ascertain the effectiveness of this technology responses from students on our introductory science course (S103) to seven questions were marked by the Intelligent Assessment system and six tutors. There were variations in the marking of four of these questions among the tutors, some of whom disagreed with the question author as to the correct response.
• There was no surprise that the computer could mark more consistently than the tutors overall and more in line with the question author. What was surprising was the number of misunderstandings, slips and inconsistencies which occurred with the human markers.