personalised learning, support and feedback in a large...

1 Personalised learning, support and feedback in a large first year chemistry class, referred paper

Personalised learning, support and feedback in a large first year chemistry class

Adam J Bridgeman and Adrian V George School of Chemistry, University of Sydney, NSW 2006

Abstract

A coherent and integrated renewal of the teaching style of a large first year chemistry unit taken by students with no prior knowledge of chemistry has been undertaken to promote active learning and student engagement and improve student success. Blended and individualised in-class and online learning has been introduced to make best use of student and staff time. Instant informal in-class and online feedback and individualised formal feedback have been used to deal quickly with misconconceptions and provide hands-on practice with each topic in turn. Personalised student support has been incorporated through individualised communication leading to improvements in retention despite growing enrolments. As well as being extremely popular with students, this unified approach led to a significant decrease in the failure rate and improvements in learning.

Introduction

Chemistry is often considered to be the ‘central science’ as an understanding of the atomic and molecular world underpins the modern understanding of the physical, natural and life sciences. As a result, students take first year chemistry units of study as part of a wide range of degrees, including those in applied sciences and technology, such as health, medicine and engineering, as well as science. Alongside possessing a wide range of preparedness for studying chemistry, the motivation and attitude of these students towards the subject varies hugely. A growing number have not studied chemistry at high school and it is amongst this group, in particular, that attitudes and pre-conceptions can be most polarised. Whilst some approach the subject openly, many have actively avoided chemistry at high school and have negative images of its difficulty and dryness.

The ‘Fundamentals of Chemistry 1A’ CHEM1001 (semester 1) and ‘Fundamentals of Chemistry 1B’ CHEM1002 (semester 2) units at The University of Sydney are designed for students with no prior knowledge of chemistry. Students from every faculty in the university take these units, sometimes as an elective but commonly as compulsory units or as pre-requisites for their chosen field. Over 2000 students take our chemistry units in semester 1 each year and the fraction taking the Fundamentals of Chemistry route has grown steadily, to greater than a quarter of the total number of first year chemistry students in 2013. The learning outcomes from the Fundamentals of Chemistry programs are quite similar to those for our mainstream units. Students with no background in chemistry at the start of the year need to get to grips with much of the same theoretical ideas and practical skills as those who enter with two years of knowledge from high school.

The profusion of new language and symbolism and the intensity of the courses have led, unfortunately, to ongoing issues with attrition and failure rates, often signalled by lack of engagement and attendance. In this paper, we outline a renewal in the design of the teaching and learning in CHEM1001 in semester 1 of 2013. As discussed below, the goal of this re-


design was to involve students actively in their learning through a homogeneous blend of in-class and online learning activities. Alongside a focus on guided inquiry and ownership of learning, the opportunities for peer-to-peer and peer-to-instructor interactions and the provision of individualised support and feedback were to be maximised. The curriculum itself is determined by internal considerations in the School of Chemistry and by the external needs of the many partners for whom the course is a pre-requisite. As a result, the review was almost completely of the approach to student learning rather than of the content. As a result, the same material previously taught through highly time-efficient traditional, didactic lectures needed to be covered with no change in the same amount of face-to-face time.

Through the ‘Active Learning in University Science’ (ALIUS) group, a new direction in learning and teaching of chemistry in Australia has been developed (Bedgood et al, 2008; Bedgood et al, 2010a, 2010b; 2010c). Active learning techniques for large classes using worksheets have been introduced in our classes (Bridgeman, 2012a, 2012b, Badiola et al., 2013). These are purposefully designed to encourage students to actively interact with chemical structures and problems using a “pen and paper” approach rather than passively listening to lectures. The methodology guides them towards building their own understanding of concepts and ideas. It has evolved from the ‘Process Orientated Guided Inquiry Learning’ (POGIL) approach widely used in North America (Moog & Spencer, 2008) but has been adapted to suit the Australian context and to be sustainable and effective within our budgets and infrastructure. Although we now utilise the worksheets approach in many of our courses, their design and use in the Fundamentals of Chemistry course described here has been taken to a further level. In particular, we sought to deliberately combine inquiry (Abraham, 2005) with continuous in-class practice and feedback and to confront common misconceptions. Despite the large sizes of the classes and the restrictions in time and types of spaces available, we sought at all stages to embed active and collaborative learning and to maximise opportunities for peer-to-peer and student-teacher interaction, following Kift’s Transition Pedagogy (Kift, 2009).

A second important component of the renewal of the course was the delivery of content online through short video and web-based tutorials and mastery quizzes. These were devised to allow partial flipping of the course content, with material considered to be more effectively delivered online removed from the face-to-face classes. The online work was also designed to provide students with scaffolded practice and feedback and to blend seamlessly with their in-class learning. As described below, small weekly mastery tasks also provided data to enable individualised support to be provided.

Overview of the Fundamentals of Chemistry 1A (CHEM1001) Unit

CHEM1001 is a first semester course, and is designed* for students who have not taken high school chemistry, have a very weak background in chemistry or are returning to study. Most who enrol are either required to do so as part of their degree or choose it as an elective because it is a pre-requisite for second year units. A handful each year chooses to take it for interest. Whilst it is possible to major in chemistry via this unit, it is not recommended for students who plan to do this. Figure 1(a) shows the make up of the cohort in 2013 and Figure 1(b) shows the growth in total enrolment. The breakdown by faculty and the male to female ratio is quite typical of this unit in recent years. Failure rates in this unit of study have been between 15-20%, considerably higher than in our other CHEM1 units, despite concerted * Unit outline: http://firstyear.chem.usyd.edu.au/chem1001/summary.shtml Learning outcomes: http://firstyear.chem.usyd.edu.au/chem1001/learningoutcomes.shtml


efforts. It is noticeable that the increase in student numbers between 2009 and 2010 coincided with an increase in the number who unenrolled during semester.

(a)

(b)

Figure 1. Enrolment for CHEM1001 showing (a) breakdown for semester 1 of 2013 and (b) comparison of enrolment by year with number who unenrolled during semester

shown in blue.

Teaching involves three 1-hour lectures and a 1-hour tutorial each week and nine 3-hour laboratory sessions over the duration of a 13 week semester. The redesign described below did not involve the laboratory work. The cohort was divided by the central timetabling unit into 2 streams for lectures, taken in 300 seat, tiered theatres and into tutorial classes of 20-25, taken in flat seminar rooms. The assessment breakdown was 60% final examination, 15% for laboratory, 15% for tutorial quizzes and 10% online quizzes.

Redesign of learning and teaching activities

Active in-class learning

The 39 lectures in the semester followed the fairly consistent lesson plan illustrated in Figure 2. Each lecture included an instructor-led mini-review of the previous class with some review questions for students followed by a series of segments involving 10-12 minute mini-lectures,

0

200

400

600

800

2008 2009 2010 2011 2012 2013

numbe

r of stude

nts

year


chemical demonstrations and 4-5 minute worksheet-based group tasks and feedback. The approximate timing of each segment and the use of a regular routine is built from our own experience and the variation and pacing seems to be optimal for keeping students engaged in class (Bunce, Flens & Neiles, 2010, Bligh, 2000, McKeachie, 2002). Rotation of the delivery and activity throughout a 50-minute period leads to students’ attention being maximised. It allows variation for learners and teacher whilst ensuring that both have a structure to work within. It is also flexible enough that the teacher can go back over material or introduce additional practice examples based on feedback received at each of the class discussion points of the lecture. The mixture of the lecturer providing a coherent storey coupled with the student experiencing how to apply the concepts in surroundings in which guidance was available was used to support student learning.

Figure 2. Lesson plan for CHEM1001 lectures showing student (unshaded) and teacher (shaded) activities.

One 2-sided worksheet was used in each lecture and students collected a hard copy of it at the start of class. Soft copies and brief answers were made available online at the end of the lecture. As shown in Figure 2, the lecturer typically briefly covers some of the fundamental ideas for a topic† in 1 or 2 slides and then the students complete the sections of the worksheet in short intervals through discussions in small, self selected groups with some help from the instructor, as possible in the tiered space. The class results were then shared: the two-way feedback being used to direct the nature of the next activity. An example of a typical worksheet is given in Figure 3. The worksheets guide the students through each activity, building their understanding of the concept in the best possible way: by actively working, practicing and responding to feedback rather than just by trying to absorb the ideas from the lecture slides. Within their groups, each student learns and helps others at their own speed. The variation of time needed by students of different abilities was addressed by encouraging

† This is a modification of the POGIL approach (Moog & Spencer, 2008), where no lecturing takes place, partly reflecting the nature of our lecture theatres and the need to reduce costs and partly to reflect the effectiveness of short bursts of traditional lectures.


peer group work and by providing worksheet activities in several short bursts. Each student leaves the class with their own work and the correct answers which provides them with feedback on their understanding. The in-class experience is used to maximise peer-to-peer and student-teacher interaction even in the large class.

Alongside these activities, each lecture also incorporated at least one chemical demonstration (Badiola et al., 2013). These are extremely popular with students and provide an excellent way to illustrate chemical concepts and phenomena and for entertainment. As shown in the lecture 5 worksheet in Figure 3, the demonstrations themselves can form part of a worksheet activity with students learning how to observe and record experimental results and link these with the theoretical concepts of the lecture.

Figure 3. Example worksheet from CHEM1001 lectures.

Active online learning

In running student-centred lecture activities such as those described above, it is very difficult to cover the same amount of material as would be taught didactically: typically the number of lecture slides was halved and it was not possible to place the rest in the worksheets. Many topics in chemistry are, however, well or even better suited to online delivery as students can then absorb content in small packages, master ideas by practicing problems and use formative feedback to confront misconceptions (Lawrie et al. 2013a, 2013b).

Some topics in first year chemistry are, frankly, tedious and unsuitable for a face-to-face class. Examples include naming of chemical compounds and calculation of concentrations. Both of these topics are logical and are better learnt through working through the methods

CHEM1001: Worksheet – Lecture 5

Model 1: Cations and Anions

Atoms form ions by losing electrons, to form cations, or by gaining electrons, to form anions:

• Metals tend to form cations by losing enough electrons that they achieve a Noble gas configurations

• Non-metals tend to form anions by gaining enough electrons that they achieve a Noble gas configuration.

Critical thinking questions

1. Predict the ions likely to be formed by the atoms below: (a) K (b) Al (c) Ca

(d) Br (e) N (f) S

2. The electronic shell structures of aluminium and chlorine are shown below. Draw the shell

structures of the ions that are likely to be formed by these two elements.

3. The chemical equation for the formation of Na+ from Na is shown below.

Na ! Na+ + e-

Write similar equations for the formation of the ions you predicted in Q1.

(a) K (b) Al (c) Ca (d) Br (e) N (f) S

Al Al

Cl Cl

Model 2: Reactions of Metals and Non-Metals In the first demonstration, sodium metal reacts with chlorine gas to produce sodium chloride. Water is initially added as a mediator, reacting with the sodium to generate enough heat to melt the sodium. The hot sodium subsequently reacts with the chlorine gas. Critical thinking questions 1. Write down your observations of the reaction.

2. What ions are present in sodium chloride?

3. Use the charges of these ions to write down the formula of sodium chloride with the correct number of each ion so that there is no overall positive or negative charge.

In the second demonstration, aluminium foil reacts with liquid bromine to produce aluminium bromide. 4. Write down your observations of the reaction.

5. What ions are present in aluminium bromide? 6. Use the charges of these ions to write down the formula of aluminium bromide with the correct

number of each ion so that there is no overall positive or negative charge.

7. Write down the formulae of the compounds formed from each pair of metal and non-metal below. (a) K and Br (b) K and N (c) K and S (d) Al and N (e) Al and S (f) Ca and Br (g) Ca and N (h) Ca and S

Br2(l)

Al(s)

Cl2 (g)

Na (s)

Sand


required repeatedly. Such topics provide the vocabulary to conduct chemical conversations (Bridgeman, 2012c) so that their mastery enables students to gain much more from discussions in lectures and in the laboratory. For these topics, a flipped approach was therefore introduced. Students were asked to complete a “Pre-lecture tutorial and quiz” (Figure 2). These contributed to the overall unit mark and consisted of a short video and/or a web tutorial with examples and a mastery quiz which could be taken multiple times. For most of the quiz topics, a database containing many hundreds of questions was semi-automatically generated. For others, involving numerical problems, questions were generated on the fly. In both cases, students were served a selection of questions and were very unlikely to see any repetition whether they were attempting the quizzes for marks or later for formative feedback. A number of quiz styles, including multiple choice, short answer and drag and drop, were utilised. Examples are shown in Figure 4. Whilst completion of the pre-lecture quiz was given a mark incentive and was designed to improve understanding of the lectures, it was also important not to make non-completion too problematic and to discourage students. A fully flipped model of delivery would probably not be suitable for this cohort.

“Post-lecture practice questions” (Figure 2) were also introduced. These dealt with concepts that were covered in the worksheets and were designed to give more practice, feedback and to cement the ideas directly after class.

Figure 4. Example online quizzes from CHEM1001. Further examples and associated tutorials are available at http://firstyear.chem.usyd.edu.au/iChem/.

Personalised feedback

Opportunities for students and instructors to receive informal feedback on progress were built into the activities described. More formal in-semester feedback was also given through the tutorial quizzes, run in the tutorial classes in weeks 5, 9 and 12. In these assessments, students answered 10 multiple-choice questions and their answers were marked by a card reader. As described elsewhere (Bridgeman & Rutledge, 2010), prompt, detailed and, crucially, individualised feedback is delivered to each student within 24 hours of the assessment. Each student’s feedback contains statistical information on their performance relative to the rest of the class, individualised explanations for each incorrect answer built using the correct answer and the particular distractor chosen, and detailed suggestions on useful resources based on topics the student has performed poorly on.

The tutorial quizzes are taken in around 25 different classes during the assessment weeks, involving a teaching team of 8-10 tutors. At the same time as the student feedback is generated, class-by-class feedback is constructed and made available to the teachers. By


obtaining such immediate feedback on their class’s performance, each teacher is able to act in the next class on weaknesses and misconceptions that arise.

Personalised support

Each student in the class received a series of personalised emails (Bridgeman and Rutledge, 2010), from enrolment through to unit results. These are addressed by first name to each student and are individualised as much as possible according to available data (such as degree and previous results). More targeted advice was sent with each quiz feedback email, including how to obtain remedial support, such as access to a chemistry tutor at lunch times, for those with poor or declining performance.

Pre-lecture online quizzes give a rich and regular source of engagement data. At the end of week 1, a student who has not logged on to the quiz page receives a reminder email. A more urgent one is sent at the end of week 2. This pattern continues during semester for students who have not completed two consecutive quizzes.

Retention and course results

Figure 1(b) shows the ca. 50% growth in enrolment in CHEM1001 over the last few years. In 2013, 94% of those enrolled on day 1 were still enrolled at the end of semester; a significant improvement over the previous 3 years and comparable to the percentage retention before the growth spike. More qualitatively, attendance at lectures was higher than in recent years (despite the availability of recorded lectures and an industrial dispute). Figure 5(a) shows the merit grade distribution over the 2008-2013 period. The failure rate for 2013 has been significantly reduced over previous years and is ca. 5% lower than that for 2012. It is noteworthy that this reduction occurred despite the improvement in retention and larger enrolment. Direct comparison of learning outcomes is more difficult. The final examination consists of around 30% multiple choice and 70% short answer questions. The latter are changed each year but the multiple choice section was very similar in 2012 and 2013, though two fewer questions were asked in 2013. Figure 5(b) shows a comparison of the marks in this section, with a positive shift in the distribution.

Evaluation

A short survey was handed out in week 7 to evaluate students’ attitudes to the style of the lectures and use of worksheets. Figure 6 shows the results for the 5-point Likert questions:

Q1. The worksheets enhanced my learning experience. Q2. The link between the worksheets and the lecture context was clear. Q3. The worksheets aided my understanding of the chemical concepts covered. Q4. I generally enjoy worksheets in chemistry lectures. Q5. Worksheets enhance my motivation to attend lectures.

As can be seen, the students were extremely positive about the learning gains from using worksheets. The qualitative comments are consistent with this and suggest that students appreciate being able to work through material straightaway in a supportive environment:

‘…able to apply what we had just covered on the slides’; ‘immediate practice with the concept or calculations embedded it in my head’; ‘consolidating understanding’; ‘can get help straightaway. They are great’; ‘immediately forced you to transfer your (supposed)


understanding of a concept’; ‘immediate feedback in class from lecturer’; ‘helped grasp the new ideas and topics of chemistry that are usually too hard to understand’; ‘being able to do examples of the content straightaway made it easier to understand and retain’; ‘able to see how the content is put into questions therefore can gain a better understanding’

It is also clear that the style of learning also made the students more likely to enjoy studying chemistry and attend lectures:

‘the worksheets help involve the class’; ‘better use of my time rather than just sitting, listening and annotating lecture notes’; ‘promotes active learning rather than passive learning’; ‘you get to participate in the lecture!’

(a)

(b)

Figure 5. Results for CHEM1001 showing (a) merit grade distribution over 2008-2013 and (b) mark distribution in the multiple choice section of the 2012 (top) and 2013

(bottom) final exam (30 and 28 questions respectively).

F F F F F F

P P P P P P

CR CR CR CR CR CR

D D D D D D HD HD HD HD HD HD

2008 2009 2010 2011 2012 2013

Frac4o

n of stud

ents

Year


Figure 6. Evaluation of survey on feedback, showing percentage agreement: A = agree, SA = strongly agree, N = neutral, D = disagree and SD = strongly disagree (N = 205).

Summary

The teaching style of a large first year chemistry unit has been redesigned to promote active and individualised learning in-class and online and to blend these two elements together to make best use of each environment. Consistent informal and rapid formal but individual feedback has been used to support students and to quickly deal with misconceptions in this course for students with weak or no previous knowledge of chemistry. Individualised student support has been integrated through personalised communication leading to improvements in retention despite growing enrolments. The active learning strategies were extremely popular with students and, as shown by a significant decrease in the failure rate, effective in improving learning.

Acknowledgements

The approach described has been adopted in a number of units in chemistry 1. We would like to acknowledge the willingness of colleagues to implement this approach as well as the additional work they have expended to make it a success. Colleagues include: Brendan Kennedy, Elizabeth New, Siegbert Schmid, Tim Schmidt and Greg Warr.

References

Abraham, M. R. (2005). Inquiry and the learning cycle approach, in N. J. Pienta, M. M. Cooper & T. J. Greenbowe (Eds.), Chemists' guide to effective teaching (pp. 41-52). Upper Saddle River, NJ: Pearson Prentice Hall.

Badiola, K. A., Bartimote-Aufflick, K., Bridgeman, A. J., George, A. V., Hudson, T. S., Neto, C. & Schmid, S. A. (2013). Using interactive lecture demonstrations to invigorate chemistry lectures. In Proceedings of The Australian Conference on Science and Mathematics Education, Canberra.

Bedgood Jr, D. R., Yates, B., Buntine, M., Pyke, S., Lim, K. & Mocerino, M. (2008). Why are we still teaching the way we were taught in the 1980s?, Chemistry in Australia, 11, 22-23.

A+SA A+SA A+SA A+SA A+SA

N N

N

D+SD D+SD D+SD D+SD D+SD

0%

20%

40%

60%

80%

100%

Q1 Q2 Q3 Q4 Q5


Bedgood Jr, D.R., Yates, B., Buntine, M.A., Pyke, S., Lim, K., Mocerino, M., Zadnik, M. G., Southam, D. C., Bridgeman, A. Gardiner, M. & Morris, G. (2010a), The development of teaching skills to support active learning in university science, Journal of Learning Design, 3(3), 10-19.

Bedgood Jr, D. R., Yates, B., Buntine, M. A., Pyke, S., Lim, K., Mocerino, M., Zadnik, M. G., Southam, D. C., Bridgeman, A. J. Gardiner, M. & Morris, G. (2010b). Leading change in Australian science teaching, Chemistry in Australia, 77(5), 18-19.

Bedgood Jr, D. R., Mocerino, M., Buntine, M. A., Southam, D. C., Zadnik, M. G., Pyke, S., Lim, K., Morris, G., Yates, B., Gardiner, M. & Bridgeman, A. J. (2010c). ALIUS: Active Learning In University Science - Leading change in Australian science teaching. In M. Sharma (Ed). In Proceedings of the 16th UniServe Science Annual Conference, (p117). Sydney, NSW: UniServe Science.

Bligh, D. A. (2000). What's the Use of Lectures?, Jossey-Bass, San Francisco, CA. Bridgeman, A.J. (2012a). Language Support Strategies in First Year Chemistry in

Zhang, F., Lidbury, B. A., Richardson, A. M., Yates, B. F., Gardiner, M. G., Bridgeman, A. J., Schulte, S., Rodger, J. C. & Mate, K. E. Sustainable Language Support Practices In Science Education: Technologies And Solution, IGI Global, Hershey, PA.

Bridgeman, A.J. (2012b). Using Very Short Writing Tasks To Promote Understanding In Chemistry, In Proceedings of The Australian Conference on Science and Mathematics Education, Sydney.

Bridgeman, A.J. (2012c). Language Support Strategies in First Year Science Students. In Sustainable Language Support Practices in Science Education: Technologies and Solutions, Zhang, F., Lidbury, B. A. & Richardson, A. M. (Eds.), Information Science Reference.

Bunce, D. M., Flens, E. A. & Neiles, K. Y. (2010). How long can students pay attention in class? A study of student attention decline using clickers, Journal of Chemical Education, 87, 1438-1443.

Bridgeman, A. J. & Rutledge, P. J. (2010). Getting personal: feedback for the masses. Synergy, 30, 67–76.

Kift, S. M. (2009) Articulating a Transition Pedagogy to Scaffold and to Enhance the First Year Student Learning Experience in Australian higher education, Final Report for ALTC Senior Fellowship Program. ALTC Resources, http://www.altc.edu.au/resource-first-year-learning-experience-kift-2009. Accessed 8th January 2014

Lawrie, G. A., Tasker, R., Wright, A., O'Brien, G., Bedford, S., Bailey, C., Thompson, C., Dickson, H., Dargaville, T., Schultz, M. & William, M. (2013a). Feedback or feedforward: Supporting students with alternate or missing conceptions in chemistry as they transition into tertiary chemistry. In Proceedings of The Australian Conference on Science and Mathematics Education, Canberra.

Lawrie, G., Wright, A., Schultz, M., Dargaville, T., O'Brien, G., Bedford, S., Williams, M., Tasker, R., Dickson, H. & Thompson, C. (2013b). Using formative feedback to identify and support first year chemistry students with missing or misconceptions. A Practice Report. International Journal of the First Year in Higher Education, 4(2).

McKeachie, W. J. (2002). McKeachie's Teaching Tips, 11 ed., Houghton Mifflin, New York.

Moog, R. S. & Spencer, J. N. (2008). POGIL: Process Orientated Guided Inquiry Learning, Oxford University Press, New York.

personalised learning, support and feedback in a large...

Documents