Encouraging College Student Active Engagementin Learning: The Influence of Response Methods

Michele L. Barr

# Springer Science+Business Media New York 2013

Abstract The purpose of this study was to examine the use of two student response methodswithin selected college lecture halls. Kinesiology majors from three universities were asked torespond to questions during two consecutive lectures, one using “clickers” and the other usinghand-raising. Participation and comprehension rates were statistically significantly differentfollowing the use of the different response methods. Participant survey responses revealedinsight into student participation, question presentation, perceived cognitive engagement, andoverall learning in traditional lecture settings using the response methods.

Keywords Active Engagement . Cognitive Engagement . Active Learning . ElectronicResponse Systems (ERS) . Clickers

Developing the ability to think critically is vital for success in the world today as the pace ofchange continues to accelerate and complexity and interdependence continue to intensify.Higher education teaching methods have been challenged to foster the deep understanding thatleads to academic success and real-world performance (Michael, 2006). According to Draper,Cargill, and Cutts (2001), a major problem with large lecture classes is the traditional approachinvolving content dissemination and lack of interaction resulting in extreme passivity on thepart of the students. Students dutifully take lecture notes, usually without thinking about whatthey are writing and without the opportunity to interact with the lecturer or with other students(Lass, Morzuch, & Rogers, 2007).

The challenge to cultivate student understanding has created a surge of interest in newer,alternative methods of teaching with a focus on “deep learning” rather than the content-centeredemphasis of many large lecture classes (Ramsden, 2003). Deep learning is learning withunderstanding in contrast to surface learning or rote learning. According to Ramsden, deeplearning occurs when the student is dynamically engaged with the subject matter. The student

Innov High EducDOI 10.1007/s10755-013-9276-x

Michele L. Barr is Lecturer in Kinesiology at California State University, Fullerton. She earned her Ed.D. inInstructional Leadership in Higher Education from Argosy University. Her research interests include cognitiveengagement strategies for college students and assessment as a teaching tool. Her special interests include theprofessional development of new teachers.

M. L. Barr (*)Department of Kinesiology, California State University, Fullerton, 800 N. State College Blvd, Fullerton,CA 92831, USAe-mail: mbarr@fullerton.edu

must focus on the content of a task such as problem solving, preparation of an essay, or practicalwork and the relationship of that task to other parts of the course or previous knowledge. Activelearning or active engagement is an alternative to traditional lecture methods and includes anyinstructional strategy that requires students to engage in meaningful learning activities and thinkabout what they are doing (Lasher, 2004). If then, as claimed by Ramsden, deep learning isdependent upon the student’s level of engagement, active learning strategies should result indeep learning.

Learning and Clicker Technology

New technologies are emerging that align with the goals of active engagement and deeplearning. One such technology is Electronic Response Systems (ERS), or “clickers”. Indevelopment since the 1960s (Judson & Sawada 2002), this technology enables the instruc-tor to pose a question to a class using electronic presentation software, such as PowerPoint.Questions are generally multiple-choice or true/false formats. Student responses are renderedvia small, hand-held remote keypads. Student responses are instantaneously recorded andcan be displayed, providing opportunity for immediate feedback and discussion. It isimportant to note that, while clicker technology is relatively new, the use of interactive,instructive questions to teach students has been used by instructors since the time of Socrates(Caldwell, 2007).

Recent years have witnessed a surge of interest in understanding the pedagogical implica-tions of clickers in the higher education classroom. Trees and Jackson (2007) concluded thatclickers could function as a potentially valuable classroom tool, stating, however, that “theireffects in the classroom are complex” (p. 1). Perceptions of students and instructors regardingthe use of clickers during a lecture are generally positive (Caldwell, 2007; Durbin & Durbin,2006; Fies &Marshall, 2006; Judson & Sawada, 2002; Simpson & Oliver, 2007). Hake (1998)reported that the use of clickers improved students’ problem-solving skills. Others havereported that clickers stimulated the interest of students for the subject matter (Beekes, 2006),improved the depth of student learning (Cue, 1998), and provided a useful tool for checkingcomprehension (Elliot, 2003). Judson and Sawada (2002) stated that clickers have demonstrat-ed a positive effect on student learning gains when used to enhance active-learning strategies.Dangel and Wang (2008) discussed the notion that clickers increase student engagement andprovide prompt feedback, both of which are known to promote learning. These authors furtherdiscussed that research supports the use of clickers when coupled with pedagogical techniquesto promote such deep learning. Davis (2003) summarized her findings using clickers stating thatthe technology has the potential to improve student participation overall, to reduce academicanxiety, to allow for increased equity of input, and to facilitate the teacher’s knowledge ofstudent understanding.

Evidence indicates that clickers result in improved learning outcomes. Caldwell (2007)reported that clickers have a positive effect on student exam grades and overall course grades. Itshould be noted, however, that in this study clickers were used over the entire semester andattendance was compulsory. It is impossible to ascertain whether improvement in exam gradesand overall course grades was the result of clicker use or of the increase in routine attendancealso reported.

Yourstone, Kraye, and Albaum (2008) reported that students in lecture courses utilizingclickers versus those who did not demonstrated significantly higher midterm and final exam-ination scores. The authors proposed that the immediate feedback provided to both the studentsand the instructor via clicker technology may change the dynamic of the classroom. They

Innov High Educ

suggested such dynamic alteration to the class process can ensure that the material beingpresented is understood. Lasry (2008) supported this notion, stating that the real contributionof clickers, due to the immediate and precise student feedback they offer, lies more on theteaching rather than the learning side of education. Lasry examined the effectiveness ofMazur’sPeer Instruction (PI) using both clickers and flashcards as student response methods. PI is astudent-centered teaching approach that requires every student to apply and explain coreconcepts to their classmates (Mazur, 1997). Crouch and Mazur (2001) reported thatimplementing PI in their introductory physics courses increased student engagement in discus-sion, understanding, mastery, quantitative problem solving, and conceptual reasoning. Lasryreported that clickers had a motivating effect over the course of the semester. However, Lasryconcluded there were no significant differences in conceptual learning gains or in examinationscores when comparing clickers and flashcards

Draper and Brown (2004) stated that getting students to be interactive is often a teachingchallenge. Graham, Tripp, Seawright, and Joeckel (2007) discussed advantages of using clickersincluding the ability to poll an entire class and to provide immediate and public feedback, whichallows the instructor to gain a sense of how well students comprehend the material. Poulis,Massen, Robens, and Gilbert (1998) noted that the difficulty inherent in gauging studentcomprehension during a lecture may result in a “house-of-cards effect,” which occurs whenthe lecturer is explaining a subject to students lacking the understanding of its precursor. Withoutthe precursory knowledge the student is, in effect, building a structure on a shaky foundation.Clicker technology allows the instructor tomake sure that the majority of students understand thematerial just presented before moving on, thereby decreasing the house-of-cards effect.

A common rationale for using clicker technology has been to engage more students in atraditional lecture setting. Participation in the classroom has been shown to have a positiveimpact on cognitive, meta-cognitive, and affective learning outcomes. Beyond favorablelearning outcomes, Gupta, Fadil and Kale (2009) argued that participation in class not onlyallows students to interact with each other and the instructor, it also allows them to gain socialexpertise as well as to access and process information they would not have had opportunity todo on their own. One drawback of a traditional lecture is the limited opportunity for students torespond. Often student responses to questions posed by the instructor are limited to the mostknowledgeable in the class who are also willing to answer in front of their peers (Graham et al.2007). The very nature of clickers is that they increase participation in a lecture class byallowing all students to respond to questions posed by the instructor (Caldwell, 2007).

Despite the importance associated with some form of participation in the classroom,research indicates that many students simply do not participate. Crombie, Pyke, Silverthorn,Jones and Paccinin (2003) reported that 64% of the 541 undergraduate students in their studynever, rarely, or only occasionally asked or responded to a question posed in the classroom.Caspi, Chajut, Saporta, and Beyth-Morom (2006) supported the findings of Crombie et al.(2003), stating that nearly 55% of the undergraduates they studied never or rarely partici-pated in class. Gorsky, Caspi, and Trumper (2004) estimated that only about 7-8% of thestudents attending a large lecture course posed questions during the lecture.

Bruff (2009) argued that clickers not only increase the percentage of students responding butthat instructors, because they can monitor the number of responses, can keep collecting responsesuntil the vast majority have at least had a chance to respond. Bruff further stated that using clickersis an effective means of increasing the participation of those students who normally do notrespond because they do not have enough time to formulate an answer. Bruff went on to say thatusing clickers can more fully engage students by providing the opportunity for independentthinking as they formulate a response.Many students will wait to hear the responses of their peersbefore thinking about the question on their own. Bruff explained that students typically remain

Innov High Educ

more engaged when they are asked to produce something that demonstrates their learning.Clickers allow instructors to request small demonstrations of such learning several times duringa lecture. Knowing that a request may bemade at any timemay help students remain attentive andengaged during the lecture.

Draper et al. (2001) discussed active participation through clickers to support studentlearning. They suggested that students must engage in some type of cognitive processing orreprocessing of the lecture content during the formulation of an answer to a clicker question.The increased cognitive processing required should be of benefit and ultimately enhance thestudents’ understanding of the material presented. While it can be argued that this is the casewith any question posed, the likelihood is that more students will participate and thus engagein such cognitive activity if clickers are the response method.

Active learning requires that students play an integral and contributory role in the learningprocess, and implementing active learning in the classroom can help students change frompassive to participatory learners (Yazedjian & Kolkhorst, 2007). True interactivity in a lectureclass requires that the teacher’s behavior be contingent on what students do in terms ofadequately communicating their existing attunement. Based on student feedback, the teachercan make adjustments if necessary (Draper et al., 2001). Draper and Brown (2004) termed thisapproach “contingent teaching” (p. 91) whereby the teacher makes the lecture dependent uponthe actions of the students rather than following a predetermined fixed sequence. The authorslater described this activity as “adaptive teaching: where what is done depends on the learner’scurrent state of understanding” (p. 92). Draper and Brown also discussed the real pedagogicalvalue of clickers technology is that it makes promoting interactivity in learning or contingentteaching, or both, “easier to achieve more often, in more contexts, and with much less effort andattention” (p. 93) than when using conventional response methods. Thus, the benefit of clickers,or any other technology, is not dependent on the technology itself but rather on how well it isutilized to foster thought and reflection in learners. Kirkwood and Price (2005) stated that themost important factor is not the technology itself but rather that technology is used creatively andconstructively aligned.

The Study and Methods

The purpose of the study reported here was to examine the use of two different student responsemethods in selected college lecture halls: clickers, which permit anonymous, electronic responseto instructor questions, and traditional, public hand-raising to indicate willingness to be called onby the teacher to respond. Specifically, this study examined response method effects on studentparticipation and, unlike previous research, the more immediate effects on comprehension ofcontent presented. Further, this study explored in-depth student perceptions of the role responsemethod plays in enhancing their overall learning experience in a traditional lecture setting in thehopes of providing insight as to how best to incorporate different response methods into effectiveteaching design.

Participants

Purposive sampling of courses and instructors was used for this research. Student participantsincluded 380 undergraduates majoring in kinesiology from three different California State Uni-versities – Fullerton, San Marcos, and San Marcos at Temecula. The 11 course sections usedconsisted of both upper and lower division courses required of all kinesiology majors at eachinstitution. Courses used included “Introduction to Kinesiology”, “Human Anatomy and

Innov High Educ

Physiology” and “Introduction to Biomechanics.” Class sizes ranged from 28-49 students, with anaverage of 39.4. To achieve a random sample within the purposive sample, all instructors teachingcourses meeting the course selection criteria were asked if they were willing to participate in thisstudy. Instructor participants were selected from the volunteer pool. Instructors already havingexperience using clicker technology in their classrooms were selected initially. Previous studiesinvolving clickers noted that technological problems, often due to the instructors’ lack of experi-ence with the technology, may have resulted in poor perceptions by students (Hall, Collier,Thomas, and Hilgers, 2005; Zhu, 2007). The remaining instructor participants selected were thosewilling to be trained by the investigator using Turning Point software and clickers in the classroom.

After receiving IRB approval from the three institutions involved in data collection,signed informed consents were collected from students and instructors. Participants wereinformed of the purpose of the research, and their participation was voluntary. Participantswere informed that their identities would remain confidential and that participation ornonparticipation would not affect their grade in the course.

Procedures

The instructors were asked to choose two consecutive lectures, and alternate student responsemethods were used during each lecture - clickers or hand-raising to indicate a student’swillingness to be called by the instructor to provide an answer. Instructors were asked to createa total of six response questions for each response method - four content and two comprehensionquestions, each with four multiple choice responses. The four content questions focused purelyon lecture content; the two higher order thinking comprehension questions had an emphasis onapplication, synthesis, or analysis of lecture content to assess themore immediate comprehensionof the lecture. The content questions were answered using clickers during one lecture and hand-raising for one lecture. All comprehension questions were answered using clickers. Instructorsattempted to make the content questions and the comprehension questions for each responsemethod of equal rigor. Instructors embedded all questions in their PowerPoint presentationsusing Turning Point clicker software, which seamlessly interfaces with PowerPoint.

To ensure the technology ran smoothly, the investigator was present in all lectures duringdata collection. Prior to each lecture, the investigator set up the clicker system and testedeach individual clicker to ensure it was working. Participants were provided with the clickerupon entering the classroom.

Prior to beginning the first lecture, participating students were instructed on the responsemethod to be used as well as on how to use clickers to respond by the investigator. Using clickers,participants responded to questions regarding their personal demographic information includinggender, year in school, chosen focus area within the kinesiology major, and race/ethnicity. Thisinformation was tracked using the individual clicker identification number on the back of eachclicker. When a student answered a question using a clicker, the system automatically stored theresponse and identification number of every clicker used. To ensure that each student received thesame clicker over the series of lectures, participants wrote their first name and the first initial oftheir last name on a piece of tape adhered to the back of their clicker. When the clickers werereturned, participants were assured that the researcher would not open the case of clickers until theclass met again.

When using clickers as the response method, the instructor displayed the question andmultiple choice responses on the screen. Once all participants had submitted their responses,the instructor closed the polling option and graphically displayed the frequency of answersselected by all students. The instructor then provided feedback to the class regarding thecorrect answer or incorrect answers.

Innov High Educ

When using hand-raising as the response method, the instructor also displayed eachquestion and multiple choice responses on the screen. However, without the technology forelectronic student response, monitoring the number of students who had responded andimmediate feedback of answer distributions was not possible. Participants indicated theirwillingness to state the answer they selected by raising their hand. The instructor called onone of the willing participants to give his or her answer and gave immediate feedback to thestudent regarding the answer. Hand-raising participation was monitored by using a preprintedindex card. Each participant had a response card with a box representing each question. If theparticipant raised his or her hand, he or she would indicate so by placing a check mark in thebox next to the appropriate question number. Participant tracking for the hand-raising portionwas done using the same clicker identification number as was used for responding todemographic and lecture comprehension questions. This identification number was alsoplaced on the participant response card. In addition, the investigator counted the number ofhands raised for each question and compared that to the number of students checkingindividual question boxes.

Following the use of each response method, the instructor presented two multiple-choicequestions to assess comprehension of the material presented using higher order thinkingquestions. The comprehension questions were determined by the instructors based on theirdetermination of the most important applications of the lecture content. Participants respondedto the comprehension questions using the clickers. The clicker data were stored and analyzedlater for comparison of comprehension based on the response method used during that lecture.

Immediately following the second lecture participants completed a survey consisting of sixLikert-scale questions, five multiple-choice questions, and three open-ended questions. Thesurvey was designed to determine student perceptions toward their experience with bothresponse systems in enhancing active engagement in undergraduate lecture classes.

Data Collection

The sources for data collection for this study were as follows:

& Student responses to Likert-scale questions,& Student responses to multiple-choice questions,& Student responses to open-ended questions,& Student responses to lecture content questions,& Student responses to lecture comprehension questions.

All quantitative data were analyzed using Statistical Package for the Social Sciences (SPSS20). Participant responses to the open-ended survey questions were initially transcribed into aspreadsheet. Comments were then summarized and clustered together based on similarity. Basedon the initial transcription, summarization, and clustering of text, dimensions and higher orderthemes were inductively identified. Where appropriate, higher order themes were broken downinto middle and sometimes lower order themes. Those comments were further analyzed andcoded using words and phrases.

Results

Observed participation rates using each response method were examined by determiningparticipant response frequency to the four lecture content-related questions placed throughoutthe lecture (coded as 0-4). The overall mean for observed clicker participation was 3.97 (n =

Innov High Educ

335). Of the 335 students responding using clickers, 97.9% (n = 328) responded to all fourquestions. The overall mean for observed hand-raising participation was 1.67 (n = 350). Of the350 students responding using hand-raising, the smallest group consisted of those raising theirhands for all four questions. Only 9.1% of participants (n = 32) raised their hands for all fourquestions; 22.3% (n = 78) raised their hands for three questions, 20% (n = 76) raised their handsfor two questions, and 20.3% (n = 71) raised their hands for only one question. The largestgroup consisted of those that did not participate at all; 26.6% (n = 93) did not raise their hands toanswer any questions. Dependent t- test results indicated a significant difference in participationrates between the two response methods (t = 29.911, df = 306, p = .000).

Student comprehension was based on correct responses to two higher order thinkingquestions following each response method (coded as 0-2). Pearson chi-square test resultsindicated a significant difference in comprehension when comparing response methods (X2 =19,153, df = 1, p < .001). Examination of comprehension based on the two response methodsrequired the collapsing of data as some of the groups did not meet the minimum numberrequired per cell for the Pearson chi-square test. Those answering neither or only onequestion correctly were collapsed into a single group. Of the 301 participants used in thecross-tabulation (Table 1), the same 35.5% (n = 107) answered either zero or only onecomprehension question correctly following each of the response methods. The majority ofthose answered one correctly; fewer than five participants answered neither comprehensionquestion correctly, resulting in that group being collapsed into those who answered onequestion correctly. Of the same participants, 26.9% (n = 81) answered both comprehensionquestions following each response method.

Based on the same cross-tabulation (Table 1), 50.2% (n = 151) of participants answeredneither or only one comprehension question correctly and 49.8% (n = 150) answered bothquestions correctly following the lecture using clickers as the response method. When hand-raising was used as the response method, 58.5% (n = 176) of those same participantsanswered neither or only one comprehension question correctly while only 41.5% (n =125) answered both comprehension questions correctly.

Participants were asked, “In general, how likely are you to actively think about questionsposed by the instructor when no opportunity to respond is given?” Survey responses werebased on a 6-point Likert scale (1 = not at all; 6 = very). The mean response was 3.70 (n =335). Of the 335 responses, only 7.5% (n = 25) responded they would not at all activelythink about questions if not given the opportunity to respond. More than half the participants(55.2%; n = 185) indicated they were still likely to actively think about questions even if notgiven an opportunity to respond with 14.3% (n = 48) selecting very likely as their response.

Survey responses provided insight into the influence of response methods on the confi-dence to answer questions, the perception of being more cognitively engaged during the

Table 1 Cross-Tabulation for Comprehension Post Response Method Collapsed

Comprehension post hand-raising collapsed * comprehension p clickers collapsed cross-tabulation

Comprehension post hand-raisingcollapsed

Comprehension p clickers collapsed Total

Answered 0 or 1Correctly

Answered bothcorrectly

Answered 0 or 1 correctly 35.5% (107) 22.9% (69) 58.5% (176)

Answered both correctly 14.6% (44) 26.9% (81) 41.5% (125)

Total 50.2% (151) 49.8% (150) 100% (301)

Innov High Educ

lecture, and perception of the method most enhancing to learning. Table 2 compares theparticipant response frequencies and percentages for those questions. Overall, of the 335respondents, 74.3% (n = 249) indicated their preference was for clickers whereas only 5.7%(n = 19) indicated hand-raising as their preference; 13.4% (n = 46) indicated they had nopreference, and only 2.1% (n = 7) indicated none of the above. Of the 335, 5% (n = 273)indicated that using clickers increased their confidence to respond to questions while 8.1%(n = 27) indicated hand-raising. Related to increased understanding of the lecture content,65.2% (n = 217) favored clickers while only 5.7% (n = 19) favored hand-raising. Increasedconcentration during the lecture was reported to be greatest using clickers with 63.3% (n =213) selecting clickers and 13.1% (n = 44) selecting hand-raising. Using clickers providedparticipants with the greatest perception of increased cognitive engagement with 72.4% (n =239) indicating clickers, and only 11.5% (n = 38) selecting hand-raising. Lastly, 63.9% (n =211) indicated that using clickers as the response method gave them the greatest perceptionof enhancement of learning while 8.8% (n = 29) indicated hand-raising.

It should be noted that the large disparity indicated in terms of response methodpreference was influenced by those selecting no preference, that is, both response methodshad a positive influence; 23.9% (n = 79) indicated no preference related to the responsemethod most enhancing to learning. Further, 20.4% (n = 68) indicated no preference relatedto understanding of the lecture. Finally, 16.4% (n = 55) indicated no preference related toincreasing concentration during the lecture.

Confidence to Answer Questions

Results indicated a preference for participating using clickers as it gave participants theconfidence to answer questions in class. When asked, “Which response method most increasesyour confidence in answering a question posed by the instructor?” 81.5% (n = 273) ofparticipants selected clickers, 8.1% (n = 27) selected hand-raising, 7.8% (n = 26) selected nopreference, and only 2.7% (n = 9) said that neither response method (none of the above)increased their confidence when answering questions in class.

There was not an open-ended question on the survey for comments regarding the impactof each response method on the confidence to respond. However, examination of thequalitative results from all three open-ended questions provided insight into such influence.A recurring theme throughout the responses to the three questions indicated that fear,anxiety, and embarrassment are associated with answering questions during a lecture.Compiling responses for all three open-ended questions, 373 comments indicated thatanonymity decreased these types of emotions when answering questions in class. Of those,7.5% (n = 28) directly indicated that being anonymous by using clickers gave them theconfidence to respond to questions in class.

Table 2 Participant Survey Responses

Which most increases your: Clickers Hand-raising No preference None of the above

Confidence to answer questions 81.5% (273) 8.1% (27) 7.8% (26) 2.7% (9)

Understanding of the lecture 65.2% (217) 5.7% (19) 20.4% (68) 8.7% (29)

Concentration during the lecture 63.6% (213) 13.1% (44) 16.4% (55) 6.9% (23)

Cognitive engagement 72.4% (239) 11.5% (38) 13.0% (43) 3.0% (10)

Learning enhancement 63.9% (211) 8.8% (29) 23.9% (79) 3.3% (11)

Overall preference 74.3% (249) 5.7% (19) 13.4% (46) 2.1% (7)

Innov High Educ

An interesting counter to this increase in confidence is that 4.8% (n = 18) commented thatthey only raised their hand if they were “100% confident” of having the correct answer first.Forty-six comments related to hand-raising indicated that participants perceived this re-sponse method as “intimidating” and required “confidence” and “courage” to participate.

Participants indicated that they liked the immediate feedback regarding the entire classresponse, the ability to make normative comparisons with classmates, and the increase inconfidence gained when they had answered correctly. They remarked that they preferredbeing asked questions in a multiple-choice format as opposed to an open-ended format;choosing an answer was cognitively less difficult than thinking of one independently.

Perceived Understanding of the Lecture

When asked, “Which response method most increases your understanding of the lecture?”65.2% (n = 217) selected clickers while only 5.7% (n = 19) responded with hand-raising;20.4% (n = 68) responded no preference, and 8.7% (n = 29) responded with neither clickersnor hand-raising (none of the above). It is clear that the majority of participants believed theuse of clickers increased their understanding of the lecture.

Concentration during the Lecture

When asked “Which response method most increases your concentration during the lec-ture?” 63.6% (n = 213) responded clickers, 13.1% (n = 44) responded hand-raising, 16.4%(n = 55) had no preference, and 6.9% (n = 23) responded that neither method (none of theabove) increased their concentration. Clearly the majority of participants perceived the useof clickers as increasing their concentration during the lecture.

Comments about the open-ended statement, “Explain how each particular responsemethod used by the instructor influences your likelihood of thinking about questions andresponding to them during a lecture,” provided further information. Participants were able toconcentrate and stay more “focused” because there were questions in the lecture regardlessof the response method. Eight participants stated that clickers helped them to remain focusedbecause they knew they were going to answer and desired to be correct even thoughanonymous. However, an equal number stated that the posing of questions in itself aidedin keeping them focused “on the main points” and “during the lecture,” regardless of theresponse method and whether or not they answered.

Cognitive Engagement

When asked, “In which response method do you perceive yourself as being more engagedduring the lecture?” 72.4% (n = 239) selected clickers, 11.5% (n = 38) selected hand-raising,13% (n = 43) selected no preference, and 3% (n = 10) said that neither method (none of theabove) increased their perception of being more engaged.

Results were supported by participant comments to the statement, “Explain how eachparticular response method used by the instructor influenced your likelihood of thinkingabout questions and responding to them during a lecture.” One theme regarding the use ofclickers related to the amount of cognitive engagement (n = 84). This theme was divided intotwo middle order themes. First, 49 comments related to the feeling of being cognitivelyengaged during the lecture. Comments often indicated participants were “thinking more”about the questions when responding anonymously using clickers. There were severalreasons for thinking more. Remarks indicated that participants thought more about questions

Innov High Educ

knowing they were going to answer them. One student remarked, related to the use ofclickers, “Knowing that I had the chance to answer questions based on the lecture made mecomprehend the lecture more so I would have a better chance of answering the questioncorrectly.” Further, participants indicated that they were thinking more because, even thoughanswering anonymously, they wanted to answer correctly.

Remarks indicated that responding anonymously using clickers helped students to stay morefocused during the lecture. This was summarized in one participant’s statement, “Since I knowthere will be questions I might need to answer, I tried to pay more attention andwrite everythingthe teacher has to say. My brain is more active, I’m more focused.” Finally, 25 remarks relatedto responding with clickers involved aspects of the clicker software. The presentation of thequestion on the screen (versus audibly) and the polling of student answers remaining open untilthe instructor closed it provided more time for students to read the question, think about it, andfinally respond. Some remarked that, when instructors ask questions during lecture, theytypically call on a student to answer before other students have had enough time to processwhat the question is really asking as well as thinking about possible answers.

The second theme to emerge related to responding with clickers and the influence oncognitive engagement was that participants were cognitively disengaged. Many stated thatthey “think less” about the question because they are responding anonymously. Thoughthere were only 10 comments indicating this perception, it is worth addressing as a possiblenegative aspect of anonymity when responding. The two lower order themes associated withthinking less are that participants did not mind getting the answer wrong because they wereanonymous, and they simply did not have to think as much.

Twenty-eight comments related to responding using hand-raising and the influence on theamount of cognitive engagement. Eight participants indicated they were more cognitivelyengaged when responding to questions by raising their hand. Few qualifying reasons wereprovided in these comments; however, one stated, “My likelihood of thinking aboutquestions is the same, if not greater because I do have to worry about being wrong if I’mcalled on.” An equal number (n = 8) indicated that they were cognitively disengaged aboutquestions when hand-raising was the response method. Several participants commented thatthey “thought less,” some not at all, about the question because they knew they would notraise their hands. One stated, “Hand-raising always causes my heart to beat faster, makes menervous so I tend to not answer, and in turn not always pay too much attention because I feelsomeone else will answer anyway.” Four respondents indicated they would not raise theirhand due to the risk of giving an incorrect answer in front of the class.

The remaining 12 comments indicating participants were cognitively disengaged whenusing hand-raising to respond related to cognitive processing of the question asked. Theseparticipants noted that someone has already responded to the question or the instructor hascalled on someone before they had had enough time to think about the question.

There were 50 comments related to the use of both response methods and the influence oncognitive engagement. The single major theme to emergewas that these participants think aboutthe question regardless of the response method used. From these, three lower order themesemerged. First, participants were cognitively engaged about the questions regardless of themethod but would only respond using clickers (n = 24). One participant stated, “Both clickersand hand-raising caused me to think about the questions; however, when it comes toresponding, I am more confident in using the clicker than raising my hand.” Second, partici-pants were cognitively engaged simply because there were indeed questions (n = 12). Finally,seven comments indicated that students participated by thinking about the questions.

Some participants indicated that they were cognitively engaged during both lectures due tothe presence of questions throughout the lecture. In addition, many of these same participants

Innov High Educ

identified their form of participation as purely cognitive versus the overt act of participating byhand-raising. This was further supported by participant survey results. More than half therespondents (55.2%) indicated that they were still likely to think about questions asked by theinstructor even if not given an opportunity to respond at all, with 14.3% indicating they werevery likely to do so.

Perceived Learning Enhancement

When asked, “Which response method provides the greatest enhancement to your overalllearning?” 63.9% (n = 211) selected clickers, 8.8% (n = 29) selected hand-raising, 23.9% (n =79) selected no preference, and 3.3% (n = 11) participants said that neither method (none of theabove) increased their perception of most enhancing to learning.

Unexpected Finding

There was one unexpected finding of this research, i.e., the magnitude and depth of thenegative feelings such as shame, embarrassment, and fear associated with answering aquestion aloud in class. Qualitative results yielded a total of 1535 comments (n = 393)related to the influence of anonymity on participation. It is clear from these many commentsthat using clickers to respond anonymously eliminates much of the negative affect associatedwith answering incorrectly in front of one’s classmates. The quantity of data received relatedto this was so extensive that it merits further exploration.

Discussion and Conclusions

Based on the results of this study, several conclusions can be drawn:

& Statistical results demonstrated a significance difference both in participation and incomprehension with the different response methods.

& Increased participation by students responding anonymously using clickers was likely dueto the decrease in fear, pressure, anxiety, and embarrassment associated with answeringquestions in class.

& Response methods influence cognitive engagement. A far greater number of participantsreported being cognitively engaged when using clickers to respond to questions asopposed to hand-raising. Further, the majority (74.3%) indicated their preference forclickers as the response method that most increased their understanding of the lecture,concentration during the lecture, confidence when answering questions, and perceivedengagement during the lecture. It also provided the greatest enhancement to their overalllearning.

& The lack of overt public participation in class does not necessarily equate to cognitivedisengagement. Participants indicated they were cognitively engaged due to the presenceof questions in the lecture. Further, though participation drastically declined when usinghand-raising, 41.5% (n = 125) still answered both comprehension questions correctlyfollowing that lecture. Following the use of clickers for participation, 49.8% (n = 150)answered both comprehension questions correctly. While the results of the chi-squaretest indicate comprehension was significantly better following the use of clickers as theresponse method, the raw data indicate that using hand-raising as the response methodhad a positive influence on comprehension.

Innov High Educ

& Of importance were student remarks regarding the manner of presenting the question. Itis likely that questions are most frequently posed verbally. However, this study showedthat many students prefer a visual representation.

& Further related to cognitive processing is the time factor for answering questions. As aninstructor, waiting for students to respond verbally can seem like an eternity; but many ofthem need more time to read, think, and respond to questions. The majority of them willlikely never raise their hand to answer. The lack of overt participation is often mistakenfor cognitive disengagement. Results of qualitative data indicate this may not be the casewith many students. They participate by engaging cognitively, not physically.

Based on this research it can be concluded that classroom response method influencesstudent participation, comprehension, and perception of cognitive engagement. However,further research is needed to determine if the response method itself or the systematicpresentation of questions throughout the lecture resulted in improved cognitive engagementand comprehension. Further, future research should systematically explore the underlyingcharacteristics of the response method. Rather than focus on the technologies themselves, thefocus of inquiry regarding technology and its role in the learning process requires a shift tothe attributes underlying the technology. Of particular interest to this researcher is theattribute of anonymity. Future research should examine the effects of varying degrees ofanonymity on participation and comprehension.

References

Beekes, W. (2006). The “millionaire” method for encouraging participation. Active Learning in HigherEducation, 7, 23–36. doi:10.1177/1469787406061143

Bruff, E. (2009). Teaching with classroom response systems: Creating active learning environments. SanFrancisco, CA: Jossey-Bass.

Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best practice. CBE – Life SciencesEducation, 6, 9–20. doi:10.1187/cbe.06-12-0205

Caspi, A., Chajut, E., Saporta, K., & Beyth-Morom, R. (2006). Participation in class and in online discussions:Gender differences. Computers and Education, 50, 718–724. doi:10.1016/j.compedu.2006.08.003

Crombie, G., Pyke, S. W., Silverthorn, N., Jones, A., & Piccinin, S. (2003). Students’ perceptions of theirclassroom participation and instructor as a function of gender and context. The Journal of Higher Education,74, 51–76. Retrieved from http://muse.jhu.edu/journals/journal_of_higher_education/v074/74.1crombie.html

Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. American Journal ofPhysics, 69, 970–977. doi:10.1119/1.1374249

Cue, N. (1998, December). A universal learning tool for classrooms. Paper presented at the First Quality inTeaching and Learning Conference, Hong Kong International Trade and Exhibition Centre, Hong Kong.Retrieved from: http://celt.ust.hk/files/public/nelsoncue.pdf

Dangel, H. L., & Wang, C. X. (2008). Student response systems in higher education: Moving beyond linearteaching and surface learning. Journal of Educational Technology Development and Exchange, 1, 93–104. Retrieved from http://www.sicet.org/journals/jetde/jetde08/paper08.pdf

Davis, S. (2003). Observations in classrooms using a network of handheld devices. Journal of ComputerAssisted Learning, 19, 298–307. doi:10.1046/j.0266-4909.2003.00031.x

Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an electronic voting system.Journal of Computer Assisted Learning, 20, 81–94. doi:10.1111/j.1365-2729.2004.00074.x

Draper, S. C., Cargill, J., & Cutts, Q. (2001). Electronically enhanced classroom interaction. AustralasianJournal of Education, 18, 13–23. Retrieved from http://www.ascilite.org.au/ajet/ajet18/draper.html

Durbin, S. M., & Durbin, K. A. (2006). Anonymous polling in an engineering tutorial environment: A casestudy. In D. A. Banks (Ed.), Audience response systems in higher education (pp. 116–126). Hershey, PA:Information Science Publishing.

Elliot, C. (2003). Using a personal response system in economics teaching. International Review of Econom-ics Education, 1(1), 80–86. Retrieved from http://www.economicsnetwork.ac.uk/iree/i1/elliott.htm

Innov High Educ

Fies, C., & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of ScienceEducation and Technology, 15, 101–109. doi:10.1007/s10956-006-0360-1

Gorsky, P., Caspi, A., & Trumper, R. (2004). Dialogue in a distance education physics course. Open Learning,19, 256–277. Retrieved from http://www.oranim.ac.il/sites/heb/SiteCollectionImages/personal/ricardo/Paul-OpenLearningJournal.pdf

Graham, C. R., Tripp, T. R., Seawright, L., & Joeckel, G. (2007). Empowering or compelling reluctantparticipants using audience response systems. Active Learning in Higher Education, 8, 233–258.doi:10.1177/1469787407081885

Gupta, S., Fadil, P., & Kale, R. (2009). The influence of culture and demography on classroom participation:Integrating technology into the pedagogical paradigm. International Business: Research, Teaching andPractice, 3(1), 71–89.

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey ofmechanics text data for introductory physics courses. American Journal of Physics, 66, 64–74.

Hall, R. H., Collier, H. L., Thomas, M. L., & Hilgers, M. G. (2005). A student response system for increasingengagement, motivation, and learning in high enrollment lectures. Proceedings of the Eleventh AmericasConference on Information Systems. Omaha, NE. Retrieved from http://ctel.mst.edu/documents/hall_et_al_srs_amcis_proceedings.pdf

Judson, E., & Sawada, D. (2002). Learning from past and present: Electronic response systems in collegelecture halls. Journal of Computers in Mathematics and Science Teaching, 21, 167–181. Retrieved fromhttp://www.has.vcu.edu/prism/ResearchReviewCPS.pdf

Kirkwood, A., & Price, L. (2005). Learners and learning in the twenty-first century: What do we know aboutstudents' attitudes towards and experiences of information and communication technologies that will helpus design courses? Studies in Higher Education, 30, 257–274. doi:10.1080/03075070500095689

Lasher, W. C. (2004). Using active learning to improve technical text comprehension and increase studentparticipation. Ethos, 9, pp.

Lasry, N. (2008). Clickers or flashcards: Is there really a difference? The Physics Teacher, 46, 242–244.doi:10.1119/1.2895678

Lass, D. A., Morzuch, B. J., & Rogers, R. T. (2007). Teaching with technology to engage students andenhance learning. (Working Paper No. 2007-1). Amherst, MA: Department of Resource Economics,University of Massachusetts Amherst. Retrieved from http://courses.umass.edu/resec/workingpapers/documents/ResEcWorkingPaper2007-1.pdf

Mazur, E. (1997). Peer instruction: A user’s manual. Upper Saddle River, NJ: Prentice Hall.Michael, J. (2006). Where’s the evidence that active learning works? Advances in Physiology Education, 30,

159–167. doi:10.1152/advan.00053.2006 1043-4046/06Poulis, J., Massen, C., Robens, E., & Gilbert, M. (1998). Physics lecturing with audience paced feedback.

American Journal of Physics, 99, 439–441. Retrieved from http://www.bedu.com/Publications/PhysLectAPF.pdf

Ramsden, P. (2003). Learning to teach in higher education. New York, NY: RoutledgeFalmer.Simpson, V., & Oliver, M. (2007). Electronic voting systems for lectures then and now: A comparison of

research and practice. Australasian Journal of Educational Technology, 23, 187–208. Retrieved fromhttp://ascilite.org.au/ajet/ajet23/simpson.html

Trees, A. R., & Jackson, M. H. (2007). The learning environment in clicker classrooms: Student processes oflearning and involvement in large courses using student response systems. Learning and Media Tech-nology, 32, 21–40. doi:10.1080/17439880601141179

Yazedjian, A., & Kolkhorst, B. B. (2007). Implementing small group activities in large lecture classes. CollegeTeaching, 55, 164–9.

Yourstone, S. A., Kraye, H. S., & Albaum, G. (2008). Classroom questioning with immediate electronicresponse: Do clickers improve learning? Decision Sciences Journal of Innovative Education, 6, 75–88.doi:10.1111/j.1540-4609.2007.00166.x

Zhu, E. (2007). Teaching with clickers. University of Michigan: Center for research on learning and teaching(Occasional papers, 22). Retrieved from http://tccl.rit.albany.edu/knilt/images/f/fe/Zhu_Teaching_with_Clickers.pdf

Innov High Educ