Effective Pedagogical Use ofStudent Response Systems

Midwestern UniversityCollege of Health Sciences

Faculty Retreat – August 23, 2010

Dr. Angel HoekstraUniversity of Colorado

Outline for today’s workshopEffective Pedagogical Use of Student Response

SystemsQuestion and Answer SessionBreakSmall group work: Writing Effective Clicker

QuestionsUsing Student Response Systems to Collect Data

for Scholarship of Teaching and Learning Research

LunchSmall group work: Writing SoTL Clicker QuestionsWrap-up Discussion & Conclusion

Socrates proposed that teaching be done

by questioning, not telling…

Introduction to Clicker Use

- SRS use is not designed to replace lecture, but to act as a supplement

- Enhancing student learning through peer teaching, interaction

- Peer talk makes conceptual application more engaging

- Giving students the ability to ask for help without raising a hand

Presentation Outline

IntroductionPedagogical guidelines for

clicker useOverview of research programDiscussion of data: Effects of

clicker use under an active learning pedagogical approach

More guidelines for clicker use

Conclusion

Additional Information

My backgroundTheoretical & methodological orientationTeaching experience, courses using clickersRecent PhD graduate, currently on the job market

Effective pedagogical use of clickersUse should match the instructor’s learning goalsUse varies depending on nature of the course

material, size of course, composition of the learning

community

On first glance, a simple process…

Lecture: 5 – 15 minutesPosting a multiple choice clicker questionInstructor pedagogical decision: Encourage peer

discussion? If so, before or after they click in?Students click in their answersSRS software gathers, tabulates, displays

responsesClass viewing of clicker responsesInstructor explanationMove on to next part of lecture

… is actually not quite so simple.

Clickers are a tool, not a pedagogical strategyClickers can be used in a variety of waysHow one “frames” clicker use (instructor talk)

strongly influences student behavior in classE.g. Lifting the “ban on talking,” versus

unintentionally reinforcing it, in the Principles of Journalism course

Literature theme: Importance of instructor meta-narrative about why clickers are being used

General Purposes for Clicker Use

In-class conceptual applicationFostering critical thinking about course

conceptsEncouraging peer talk and social solidarityHelping students identify common mistakes

when applying, calculating or solvingDriving large group discussionEngaging students more during class

Simulating the Effects of Clicker UseTake a moment to write in

answer to the following:

What aspect of student response system use most interests you?

Put another way, what are you most hoping clicker use may accomplish in your course/s?

Please turn to a neighbor, introduce yourselves, and then discuss your answers.

Approaches toGrading Clicker Responses

No creditGreat for first time SRS usersLess motivation for students to click in/come to class

Assigning clicker points “extra credit status”Motivates students to attend, without requiring them toLess motivation for students to click in on individual

questions

Incorporating clicker points into the course gradeStrongest motivator for student participation in class Recommended weight: 5-10% of overall course grade

Beware of allowing the SRS to become an attendance monitoring device!

Consider the social status of most undergraduate students:

- Seeking affirmation as adults

- Recently “on their own,” learning to pay bills, etc.

- Especially sensitive to instructor behaviors that seem patronizing or seem to reduce the instructor’s workload

Problematic instructor behaviors:

Using clickers too infrequently

Asking clicker questions that are too simplistic, too often

Not fully explaining the answer and/or each of the potential answer choices for a question

Next… A look at the social effects of clicker use for student behavior

Considering what motivates students to want to participate in

the learning process…

The Research ProgramMixed Methods Approach, Ethnographic design (2004-

2009)

Participant Observation6 courses observed in four disciplines, 87 classes observed

Survey Design (N = 3045) Students surveyed in all 6 courses, plus my own course

Anonymous Free Writes (N = 675)One-page free writes address types of clicker questions used,

potential problems/concerns with clicker use under different pedagogical formats

Semi-structured Interviews (N = 38)20 interviews with General Chemistry students (spring 2005) 8 interviews with General Chemistry students (fall 2005)10 interviews with Sociology students (fall 2007 & spring

2008)

Disciplines Studied

Chemistry

Astronomy

Journalism

Sociology

Effects of Single-vote Pedagogy & Active Learning Approaches

General Chemistry & Stars and PlanetsClicker questions interspersed during lecturePeer discussion verbally encouraged in small groupsData generated by SRS technology structurally

prompts students to increase their level of attention to their peers

Social Statuses chosen by studentsRegular clicker group membersFloatersLonersSelf-Testers

Regular Clicker Group ParticipationPosition preferred by the majority of students: to establish a

regular clicker group early in the course and maintain it over time

Vanessa, a diligent reader and regular attendee in General Chemistry, explains her experience in a regular clicker group,

I sit by a guy and a girl, and we always work together on clicker questions. I met them in the class… we just sat by each other and now we sit by each other every day and work together. I met them through the clickers, really… a question came up and we just started talking about it, and I definitely am the kind of person where, I will keep to myself before I talk to somebody else, so I was glad that the girl talked to me first… With clickers, I mean, they present the material and then they ask the question, so if you didn’t get it right away, then you are kind of lost. So it helps [to work in a regular clicker group], because then someone explains it to you and you get to think about it again, or in a different way, as you are discussing the question.

Benefits of Regular Clicker Group Participation

Teams engage in shared activity around common goalsHelping one another understandHolding one another accountable

Survey data: General Chemistry (N= 814)

75-80% of General Chemistry students said clicker-prompted discussions were at least “somewhat helpful” for learning

45% said peer talk during clicker questions was “quite” or “extremely” beneficial

Student Experiences in Regular Clicker Groups

A male student in Stars and Planets,

At first I hated the clicker, I thought it was just a stupid piece of equipment we were being forced to spend our money on, [but] right from the first day, Dr. D said how clickers would be used, and [he] has stuck to his policy. He asks us clicker questions about the reading, or to apply concepts, or [to] express our opinions. I would never really [have] talked to the people next to me if not prompted to through clickers. In other classes I’ve had in this room, I’ve felt completely separate from the class and the material, but thanks to clickers, I feel involved.

Liz, explains her experience in a regular clicker group in General Chemistry,

I like the people I am working with a lot… [using clickers] gives us some time to chat in the class and see how each other are. And, there was a week… I had a really hard week, just personally I wasn’t doing so well, and I wasn’t caught up in the class and they noticed it, and they were like, “Hey, is everything going okay? You seem like you are kind of behind.” I was like, “Yeah, I am behind… okay, so by this Friday, I will have read up to this chapter… so you guys ask me on Friday.” And they did [ask her]… so it’s definitely been beneficial, just in accountability.

FloatersA term derived from the interview participants:

“those willing to work with others but not attached to a regular clicker group”

A small but significant percentage (15-20%) of sampleFloaters tended to sit in the middle/back of the

classroom

The extent to which individual students participated in discussions of clicker questions with others depended on:

1) the individual’s preference for social interaction2) whether s/he consistently did the reading before class

• Distinguishing between “floating” and “free-riding”

Loners and Self-testers

Both groups contain students who say they prefer to work alone on clicker questions Loners do, while self-testers work with others

Self-testers comprised a small but significant percentage (<20%) of each sample, while students who consistently worked alone were rare

Two primary reasons for working entirely on one’s own:1) The student is unprepared and so feels uncomfortable

discussing the material with others, and 2) shyness/introversion.

Loners & self-testers tended to sit in the back or on the side of the classroom, and were much more likely to be maleStereotypes about “who” does science; “who” works well with

technology; “who should be able to do things independently”

Additional GuidelinesExplain regularly to students how clickers

are designed to benefit their learning (they will forget)

Encourage peer talk during clicker questionsYou’re challenging years of educational

socialization!

Keep in mind, most research suggestsclicker use is beneficial to learning

in a variety of ways!

Two minute writing exercise inpreparation for the Q & A…

Please reflect on what you have just learned by writing for two

minutes.

What do you think about all this?What questions do you have?

Question & Answer Session

Followed by a break… please return at 10:30.

Writing Didactic Clicker Questions

Individual & Small Group Work10:30 – 11am

Guidelines for Clicker Question WritingMost questions should range from medium-difficultEasy clicker questions can be used but not too oftenResponse categories should offer answers that address

common mistakes or misconceptions about the conceptWhenever you explain a clicker question, you should

address both the correct and the incorrect answersPossible response categories with your system: 10

Students can choose a single answer from 0 – 9 on TurningPoint, or they can rank a set of multiple answers

Guidelines for Clicker Question UseQuestions should be used to “break up lecture” into

sections of conceptual material… helps keep young adults engaged

Consider not using questions in the last five minutes of class

Consider giving students a few seconds to read the clicker question before peer discussion begins

Explain to your students how the clicker questions they see in class may mirror, or be similar to, your exam questions

Writing exercise: Goals for clicker use in your classroom

Please list three to five goals that you hope to achieve through clicker use in your course/s. As you write, please take into account the particular nature of the material in your discipline. In addition, please write a little about your anticipated pedagogical approach to clicker use.

Small Group Work

Please work with your colleagues in your small group to write a few clicker questions together at this time. Introduce yourselves, choose a topic to write a few questions on, and then work through writing them together.

Using Student Response Systems to Collect Data for Scholarship of

Teaching and Learning Research

Midwestern UniversityCollege of Health Sciences

Faculty Retreat – August 23, 2010

Dr. Angel HoekstraUniversity of Colorado

What is SoTL Research?

“Published work on teaching and learning by college faculty” *

SoTL research focuses on post-secondary teaching & learning, whereas educational research refers to all levels

Professional associations that sponsor this kind of research:

Carnegie Academy for the Scholarship of Teaching and Learning (CASTL)American Association of Higher Education (AAHE)International Society of Scholarship of Teaching and Learning (ISSOTL)

_______________________________________________________________* A great resource: Maryellen Weimer, 2006. Enhancing Scholarly Work on Teaching and Learning: Professional Literature that Makes a Difference. Jossey-Bass.

The Challenge of Pedagogical ScholarshipHas not traditionally been valued in promotion &

tenureThis status is changing as more attention is being paid

in academic culture to:Using active learning to keep students engagedHolding students accountable for their own learning Increased focus on confirming learning outcomes through

research

“Faculty’s continuing fixation with teaching prevents them from seeing that teaching has no purpose unless it can be directly and explicitly connected with learning outcomes… If practice is to improve and college teaching is to develop respect… there must be a viable literature associated with it.” (Weimer 2006, 7)

Two basic approaches to SoTL Research

1) Wisdom of Practice Scholarship: experience-based work, sometimes called the “how-to literature of teaching”

2) Research Scholarship: addresses pragmatic and applied empirical questions about teaching and learning

SoTL scholarship makes some academics anxious because it challenges the traditional model of “discovering new knowledge by empirical means,” but SoTL scholars point out that there are other ways of knowing… ways that come from practice, from experience. These different ways of knowing are still legitimate, especially when paired with good data.

Wisdom of Practice Scholarship

Personal accounts of changeRecommended practices reportsRecommended content reportsPersonal narratives

Advice giving based on “personal experience or research evidence or both” (Weimer, 40)

Involves reflective and critical analysis of one’s teaching

Content sometimes has an emotional tone, sometimes does not

Research Scholarship

Strives to examine teaching practices and learning outcomes by means of rigorous objectivity & rational analysis

This work can be quantitative, qualitative, descriptive, or some combination of all three… but is defined by data collection, analysis

Many “hybrid” approaches are used (e.g. Dr. Hoekstra’s doctoral thesis work)

Handout: List of SoTL Periodicals

What characteristics of SoTL research make it credible, rigorous, valued?

Well-designed research program w/appropriate methods

Viable and timely research question/sAdequate to exceptional literature reviewPlacement of the phenomenon under study within an

appropriate theoretical and/or epistemological context

A well-written and insightful report of the resultsReliability and validity of the data can be establishedAuthor/s explain potential implications of the work

Benefits of using clickersto gather data for SoTL research

Clicker questions mimic Likert-style survey question formatData gathering is efficient, no more hand coding responsesData can be stored in clicker software by student ID #s

Optional nature of participation is obvious to studentsA few survey questions can be asked per class day, rather

than distributing an entire set of questions all at onceHandout: our study this fall in three science courses uses a

statement encouraging honesty in clicker survey responses

Individual work: Reviewing examples

Please choose two of the following four exemplary SoTL pieces to read before

lunch, and we will discuss them in small groups when we return to the workshop.

As you read these, please note in the margins:

What are the strengths and weaknesses of each piece?

Choose one from each category:“Walking on Eggs: Mastering

the Dreaded Diversity Discussion.” Frederick (1995). Personal narrative, some data Great practical advice for how

to manage conflict in discussion

“Helping Students Understand Grades.” Svinicki (1998). Nice placement of issue in

context Strong practical advice,

examples

“Using Clickers in Nonmajors- and Majors-Level Biology Courses.” Crossgrove & Curran (2008). Strong literature review Comprehensive approach to

research design, data analysis

“Active Learning and Cooperative Learning in the Organic Chemistry Lecture Class.” Paulson (1999). Combines personal change

account with strong quantitative data

Great practical advice, examples

Small & Large Group Discussions

What are the strengths and weaknesses of each of these exemplary SoTL

articles?

Which did you like better, and why?

What other questions do you have?

Concluding Comments…

Thank you for your participation!

Angel.Hoekstra@colorado.edu

Many thanks to Dr. Derek Bruff,Dr. Stefanie Mollborn & Dr. Douglas Duncan.

Some data in this presentation taken from the following:

Mollborn, Stefanie and Angel Hoekstra. 2010. “A Meeting of Minds: Using Clickers for

Critical Thinking and Discussion in Large Sociology Courses.” Teaching Sociology, 38(1): 18-27.

Hoekstra, Angel. 2008. “Vibrant Student Voices: Exploring Effects of the Use of Clickers

in Large College Classrooms.” Learning, Media, and Technology, 33(4): 329-341.