“Assessing group wiki projects using peer assessment”

Teaching and Learning Conference, University of Liverpool29th June 2011. James Gaynor, Department of Chemistry

Overview• What is a wiki?– Wiki statistics

• Peer and self assessment– Various modes– Why use it?

• Case study– Introducing CHEM331– Analysis/Results– Thoughts of students– Limitations/Improvements

What is a wiki?Online collaborative tool – group work.

What is a wiki?

Wiki statisticsSome major limitations:• Working in word and then transferring material over.• People being verbally helpful rather than actually logging in

and helping.• Students working together from one computer.

Wiki statistics

Peer and self assessment• Assessing group work can obviously be tricky. One mark for all

not necessarily fair.• PA/SA help develop a variety of skills:

– Reflection.– “Correcting a classmate’s work can be as much as a part of the

assignment as taking the test itself.” Sadler and Good.• Methods of PA/SA (detailed guidelines are important).

– Distribute marks, effort marks, signature block and bonus marks.• Web PA.

Sadler, Philip M. and Good, Eddie (2006) 'The Impact of Self- and Peer-Grading on Student Learning', Educational Assessment, 11(1), 1 — 31

Chemistry soft skills

Hanson, S.; Overton, T. Skills required by new chemistry graduates and their development in degree programmes, Higher Education Academy, Physical Sciences Centre, 2010

“With respect to your career since completing your undergraduate degree, whether working, training or undertaking other activities, please indicate the value of the areas of knowledge or skills listed.” % of those given as useful/very useful.

Case study: CHEM331• Final year BSc students – core organic chemistry (15 credits).• Biological chemistry portion contains three main areas.

Students need to cover two of them in detail (self-learn).• Cohort (32) split into 3 groups of 10/11.

– Part 1: Prepare individual essays on an assigned topic.– Part 2: As a group, prepare a website on a different topic.

• PA/SA new for this year.Activity % Assessment Notes

Lectures (22) 60 Examination Two hours

Tutorials (3) 3 Continual

Individual essay 18.5 Subjective 1500 words

Group website via a wiki

18.5 Numerous (next slide)

Individual and group marks awarded, both of which contained PA/SA portions

Case study: CHEM331Code Assessment

Procedure% How were these assessed?

Individual MarkA Individual PA

25All PA marks for an individual were averaged and adjusted to be out of 25.

B Contribution

25*

Same process as students used to complete PA, but using data from wiki stats, contribution at meetings (from minutes), effort on their page (not content), etc.

C Individual assignment

Each student had an individual task/webpage (determined by the group). The scientific content on each page marked by same assessors as essays.

Group MarkD Group PA 10 All PA marks averaged and adjusted to be out of 10.E Academic

Judgement35

Five academics. Average mark given. More emphasis on presentation here since scientific content is marked above.

F Meeting minutes

5Academic.

Case study: CHEM331Comparison of two sets of marking criteria:1. Individual contribution:

– From PA (25%): students distributed 100 marks between their own group (including themselves).

– Contribution mark (from staff, 25%*): 1 staff member distributed 100 marks between a group.

2. Judgement of wiki quality:– From PA (10%): students distributed 30 marks between all 3

groups.– Contribution mark (from staff, 35%): 6 staff members awarded

a mark out of 20.– Both marks were adjusted to be out of 35 for comparison.

Only 26/32 students submitted their peer assessment forms.

Case study: CHEM3311. a. Individual contribution

• Under marked: The mark given by academic judgement was lower than PA.

• Over marked: The mark given by academic judgement was higher than PA. • Academic judgement marks were awarded prior to knowing PA results.• Average difference is 1/25 (4% error).

Case study: CHEM3311. a. Individual contribution

Case study: CHEM3311. b. Individual contribution

• Weaker students gave themselves higher marks than they received from peers. Similar observations seen elsewhere (Boud, Sadler & Good).

Boud, D. (1989). The role of self-assessment in student grading. Assessment and Evaluation in Higher Education, 14, 20–30

Case study: CHEM3312. Judgement of wiki quality

• Staff marking not done via a PA method and hence there is a bigger deviations since marking criteria for students and staff different.

• Average for both staff and PA marking was 60%.

Case study: CHEM331Results/Conclusions• Determination of individual contribution marks by PA and academic

judgement are quite consistent (1 mark difference).• Determination of group mark by academic judgement vs PA was

less consistent – differing marking criteria.• For individual PA vs SA, the students who received a higher PA mark

from their group often awarded themselves a lower SA mark. The converse is also true.

• For group PA vs SA, all groups awarded their wikis higher SA marks than their peers.

Thoughts of students• Only 5 completed the evaluation. Mixed feedback.

Case study: CHEM331Limitations/Improvements• “Distribute marks” approach means there is a set average.• Not all students submitted forms – make it compulsory so that they

don’t only lose their own marks, they lose all PA marks.• Transparency issues with PA assessment procedure. Students were

told they would have to complete PA from the beginning, but were not told how they this would work until end.

• New fees/increased internationalization. • Perhaps increase PA portion of marks for next year. Shall we give

students their average PA marks back as feedback? Some careful thought required.

• Need to repeat with the 2011-2012 cohort.• Would be useful for our MChem students to do something similar.

Or introduce more group work in general.

Acknowledgments

• Organising committee.• Colleagues in chemistry: Nick, Richard, Rick, Andrew and Neil

• You for your attention.

ANY QUESTIONS?