icmas for computing courses centre for open learning of mathematics, science, computing and...
TRANSCRIPT
![Page 1: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/1.jpg)
iCMAs for Computing Courses
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).
Michael Isherwood
Associate Teaching Fellow, COLMSCT
![Page 2: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/2.jpg)
Presentation Summary
• Project Aims• Background• The Quizzes (including demonstration)• Initial Analyses• Student Views• Developing Questions and Quizzes• A Way Forward• A Vision
![Page 3: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/3.jpg)
Project Aims - summary
• Support students in their understanding of new and difficult parts of M150 (in effect, Block 2)
• Confirm understanding prior to tma feedback• Investigate extension to other computing courses• Through better understanding, aid retention
![Page 4: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/4.jpg)
Background
• Students find getting to grips with programming difficult• This leads to frustration and dropping out of M150 (and
therefore further computing courses)
![Page 5: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/5.jpg)
Background – 2006J TMAs
![Page 6: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/6.jpg)
Background - TMA02/03 Marks
![Page 7: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/7.jpg)
On-Line Quizzes• 6 Quizzes on 2008J and 2009B presentations
– Structured English– Conditions, truth and trace tables– Basic JavaScript– Selection - the if statement– Repetition - the while statement– Repetition - the for statement
• Question Types– Selecting tick boxes, radio buttons
– Free text and expression entry– Entering values– Drag and drop
![Page 8: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/8.jpg)
if constructs – Q1
• 67% correct at 1st try • further 21% at 2nd attempt• 3% correct at 3rd attempt• 10% remained wrong
Most errors were
• < > instead of !=
• = rather than ==
feedback for this Q is to indicate wrong answers. As always, correct answer is shown at the end
![Page 9: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/9.jpg)
if constructs – Q2• 83% correct at 1st attempt• Only 4% wrong after 3rd attempt
Note Feedback after 1st attempt
After 2nd attempt the correct answers remain, so student concentrates on incorrect answer(s)
![Page 10: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/10.jpg)
if construct – Q2 cont
![Page 11: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/11.jpg)
if constructs – Q3•75% of responses correct at 1st attempt• 3% wrong after 3rd attempt.
After a 2nd attempt, the correct answers were retained, as shown, for the final attempt.
The majority of the errors were at the boundary, as here, though several students reversed
![Page 12: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/12.jpg)
if constructs – Q6
On 1st attempt the number of wrong answers is given along with a reminder of the OR (||) operator's effect. The 2nd attempt has feedback as above and the 3rd attempt starts with all the correct responses retained.
![Page 13: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/13.jpg)
Conclusions on Questions• Most students complete quizzes in 10 – 15 minutes• Most/many students get questions right first time and
few fail to be correct after third attempt – there are exceptions
• Most students find the quizzes valuable and the feedback helpful
• Only one question “didn’t work” on 1st pilot - discarded• Variants of a question generally seemed equivalent
![Page 14: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/14.jpg)
iCMAs on Courses • Philosophy is generally to use formatively• However Students need “encouragement”
– hence must form part of assessment • M150 has 10-20% attempting as not part of assessment• S104 has similar number as attempt tma (effectively 100 in
my TG)
• Students take seriously if part of assessment• Courses using iCMAs (some in pilot form) include
MU120, M150, MST121, M256, S104, S342
![Page 15: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/15.jpg)
Building a Quiz for Assessment
• Scope of Quiz• Each Question needs 5 variants – must be equivalent• 3 attempts allowed for each question• Each Question requires feedback after each attempt
– feedback targeted to answer given• therefore must anticipate wrong answers• free text (or symbol) entry much harder to anticipate
• Correct answers may not be apparent, particularly with free text entry
![Page 16: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/16.jpg)
Validating Questions
• Initial testing by Setter• Further testing by Others• Reviewing Student Responses
– for variant equivalence– for errors – then zero rated
![Page 17: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/17.jpg)
Learning from Questions• What is understood• What is misunderstood• How course material might be amended
![Page 18: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/18.jpg)
Moodle and/or OpenMark• OpenMark questions very versatile • It is intended to run OpenMark questions through
Moodle – work is proceding• OpenMark feedback highly tailored• Openmark requires specialist programming
• Moodle has question types• Moodle feedback is available, though fairly rigid• Moodle questions can be set up by anyone who is
reasonably logical – but there is a learning curve
• Within Moodle a mixture of OpenMark and Moodle questions should soon be possible
![Page 19: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/19.jpg)
A Way Forward (i)• Identify Question Setter(s) for each iCMA• Appoint Coordinator / iCMA expert• Workshop(s) on Question and Feedback setting• Set Questions with Feedback• Review Questions• Program Qs (OpenMark and Moodle Qs separately)• Test Quiz } iterative• Make Corrections }• Review Quiz } iterative• Further changes }
![Page 20: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/20.jpg)
A Way Forward (ii)• Coordinator could run workshops• Reviewers may be others associated with Block• Coordinator reviewing could facilitate uniformity of
approach• Perhaps Coordinator could interface with developers• OpenMark Developer would be needed• Moodle Developer probably desirable (could be
Coordinator)
![Page 21: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56649eac5503460f94bb2edd/html5/thumbnails/21.jpg)
A Vision for Student
Studies
Section
Does
Quiz
Next
Section
“Pod” +
Ppt (IC)
Tutor Advice Special
Session