scholar: eassessment design for science and engineering - helen ashton

eAssessment Design

for Science & Engineering

Dr Helen Ashton

Head of eAssessment, SCHOLAR

Heriot-Watt University

A programme of Heriot-Watt University Prògram Oilthigh Heriot-WattSCHOLAR

Overview Why bother? What is important? Different approaches Assessing the right learning outcome Supporting the learner


Why bother?

Support students Independent learning Reduce marking Immediate response Improve feedback

To students To staff


What is important?

Repeated practice Constructed answers Feedback Appropriate assessments Progress monitoring

How am I doing? How are they doing? (But not ALL the time!)


Simplify3 (7 - x + 6x2 + 4x3 - 8x4) - 4 (2 - 2x + 9x3)

-1 + 7x + 6x2 - 32x3 - 8x4

29 - 11x + 18x2 + 48x3 – 24x4

23 - 5x + 18x2 + 21x3 – 24x4

13 + 5x + 18x2 - 24x3 – 24x4

The 4 options:

• Not using multiplier of 1st bracket

• Not using sign for 2nd bracket

• Not using multiplier of 2nd bracket

• Correct


Simplify3 (7 - x + 6x2 + 4x3 - 8x4) - 4 (2 - 2x + 9x3)

-1 + 7x + 6x2 - 32x3 - 8x4

29 - 11x + 18x2 + 48x3 – 24x4

23 - 5x + 18x2 + 21x3 – 24x4

13 + 5x + 18x2 - 24x3 – 24x4

The 4 options:

• Not using multiplier of 1st bracket

• Not using sign for 2nd bracket

• Not using multiplier of 2nd bracket

• Correct

Correct answer can be found by multiplying out the numbers only.

Need to be aware the question is different to a constructed version.

Often when MC versions are created, questions are changed dramatically.


y = m x + c

m: 2, ½, -2, -½c: 4, -2

Assume student knows:• the form of a straight line equation,• that m is the gradient & relates to how much we move up and along,• that c is the intercept & relates to where the line crosses an axis.

So… choices are expression of the form

with possible values of

What is the equation of the line?


y = 2 x + 4 y = 2 x - 2 y = ½ x + 4 y = ½ x – 2 y = -2 x + 4 y = -2 x – 2 y = -½ x + 4 y = -½ x - 2

Even with all these assumptions, we still have 8 choices for the answer.

How many choices are needed if we don’t make all of these assumptions?

How easy is it to find your answer in all of these choices?

What is the equation of the line?


Designing Assessments

Start with the learning objectives

Think about what you would LIKE to do

Technical Educational

Develop question

Review against learning objectives


Question Design

Often we are trying to create a question online which used to exist on paper.

We need to assess the same objectives.

What are our options?

How does this effect the appropriateness of the question?


Constructed Answers: ExampleDiagrams 1 and 2 on the worksheet show part of

the graph y = f (x).(a) On Diagram 1, draw the graph of y = - f (x).(b) On Diagram 2, draw the graph of y = f (x + 2).


Option 1 – Multiple Choice


Option 1 – Multiple Choice

Advantages Easy to create and mark Easy for student to answer Some randomisation possibleDisadvantages Recognition and Elimination NOT

construction High screen real estate required Limited possible answers


Option 2 - Hotspot


Option 2 - HotspotAdvantages Easy to mark Lower screen real estate Construction of an answer - many possibilities Partial credit possible

Disadvantages Answer is not a graph of –f(x) Could be fiddly for student to answer Time consuming to create Low reusability - no randomisation


Option 3 - Flash


Option 3 - FlashAdvantages Lower screen real estate Easy to mark Construction of an answer – many possibilities Partial credit possible Graph shows in context High reuse - randomisation possible

Disadvantages Creation requires multimedia authoring skills Some accessibility issues


Appropriate Learning Outcome

(a) algebraically (b) graphically

Solving simultaneous linear equations…Partly answered questionShowing feedback.

Graphically, student must demonstrate they can draw the appropriate graph.


Appropriate marking

Follow through: allowing students to gain credit even if they make an initial mistake.


Appropriate markingPartly answered question showing follow through and feedback.

Here the best fit line is incorrect, but the appropriate equation of this line has been entered.

Feedback and marks are given as appropriate.


Be adventurous!


What do teachers think?

“Constructing questions in this format that challenged the students was considerably harder than I'd expected but probably took no longer than constructing a traditional exam. And it was nice to have no marking!”

Computer Science Lecturer, Heriot-Watt University


What do teachers think?

“It moves the work from marking the exam to setting the exam, and allows one to concentrate on what is a suitable question.”

Computer Science Lecturer, Heriot-Watt University


Any Questions?

[email protected]

Online Demonstrationshttp://www.calm.hw.ac.uk/pass-it.htmlA variety of questions can be found in the demos.

SCHOLAR Examples: http://scholar.hw.ac.uk

(Choose Country> Try SCHOLAR)

scholar: eassessment design for science and engineering - helen ashton

Education

scholarheriotwatt university

learnera programme of

hotspota programme of

flasha programme of

graph of y

form y

ofthe graph y

appropriate graph