1. 1. NEERO 45th Annual Confernece Symposium 2.2 | Thursday, 10:15-11:45am Symposium Title: Student Evaluations of Higher and Basic Education Instructors: Three Case Studies Presenters:Tom NeuschaferNichol MurrrayDiane Onorato
2. 2. Objectives of Presentation Secondary Heading 1. To demonstrate that high school students can recognize good teaching and can collaborate to build an assessment tool which parallels professionally tested instruments 2. To demonstrate evidence that college students must have buy-in and confidence that their opinions are valued and to share strategies for increasing the response rates of evaluations. 3. To demonstrate a process by which an online or blended course can be improved by student evaluations using an instrument which correlates to the course design and management
3. 3. Student-Designed Evaluations The Evaluation The need for an evaluation Wrights article The evaluation What they were told / what they werent told Questions on parenthesis Grading system Explanation sections
4. 4. Student-Designed Evaluations The Evaluation Lets take a look at the actual evaluation! Sections of the evaluation About the instructor About the course About the programNow lets hear some of their reasons, in their own words!
5. 5. Student-Designed Evaluations Student Reasons
6. 6. Top Complaints of Faculty General Instructors arriving late or unprepared Instructors that lecture for an entire class period Instructors taking too long to respond to messages and provide feedback on work Instructors who do not provide enough examples or case studies Instructors who are boring Instructors with poor speaking skills/Cannot understand instructor When an instructor thinks that he or she is better and above the students
7. 7. Top Complaints of Faculty Online Courses Instructions that are not clear Waiting too long for grades/feedback Missing or disengaged instructors (lack of support) Poor course design navigation is confusing, hard to find assignments
8. 8. Top Praises of Faculty Professionalism dress, punctuality, behavior A detailed, organized syllabus that clearly conveys expectations Flexibility when an instructor is willing to make changes to the schedule and content when appropriate Interesting and relevant course material Varied methods of instruction When an instructor shows a passion for teaching Ample interaction with students Respect and fairness Source: Butler County Community College Office of Academic Affairs
9. 9. Student Evaluations of Higher Education Faculty Concerns Students are being selective in who they choose to evaluate.Evaluations are now administered online.Ineffective Feedback Students are not taking the evaluations seriously.Instructors do not receive results until after the semester is over.
10. 10. Online Course Evaluation linkActions Taken Discussed concerns with VP of Academic Affairs Looked at the current evaluation instrument for alignment with my teaching Added additional questions to the evaluation instrument: Please rate the following statement as it applies to you: 1. I feel that my instructor cares about my success in this course. 2. My instructor responds to my questions in a timely manner. 3. My instructor takes time to include lessons that are purposeful and can be applied to real-life situations. 4. My instructor values me as an individual.
11. 11. Case Studies Fall 2012Spring 2013 4 course sections (81 students) 4 different approaches to encourage participation Administered ongoing course feedback survey 2 course sections (27 students) Reflective writing activity regarding attitudes toward instructor evaluations Class discussion about evaluations
12. 12. Results Fall 2012 Response Rate 6/25 = 24%Section L01 Announced that the evaluation window has opened9/22 = 41%Section L02 Announced that the evaluation window opened; sent weekly e-mail reminders9/17 = 53%Section C51 Announced that the evaluation window opened; sent students a link to the online evaluation site (through their personal e-mail)13/17 = 76%Section B01 Class discussion on instructor evaluations; gave students opportunity to complete evaluations during class time in the computer lab
13. 13. Respose Rates
14. 14. Ongoing Feedback Survey Administered 2/3 into semester Important to share and discuss comments with students 1. My instructor is available to me in person, by phone, and via e-mail. 2. Participation in this class is valued. 3. I feel comfortable in this class. 4. Which of the following were the most interest and helpful methods of instruction for you? (methods listed) 5. Comment on any suggestions you may have to improve your experience in this course.
15. 15. Spring 2013 Reflective Writing Activity Survey on Faculty and Course Evaluations How seriously do you approach faculty evaluations? When the time comes to complete evaluations, do you complete them for all of your instructors, some of them, or none? Do you think the instructors read and take your comments into consideration? Think of any negative experiences you have had with a course/instructor. What made it a negative experience? How would you describe your ideal instructor? If you could add one question to a faculty evaluation, what would it be? If you were the President of the college, what methods would you use to evaluate instructors?
16. 16. Findings from Survey The majority stated that they complete evaluations for the instructors that they either had a very positive experience with or one that they thought was very bad. I complete them if I had a bad experience with a teacher. If they dont ask, I dont care. If they do ask, then Ill do it. Students do not learn or care about classes that the instructor does not care about.
17. 17. Findings from Survey When describing an ideal instructor: Respectful, on time, easy-to-reach, interesting, varies instruction, posts grades online One who applies it to real-life rather than just making me memorize I like professors who make a lasting impression. Spring 2013: Results will be available after May 7.
18. 18. Strategies to Increase Awareness of Faculty Evaluations Take evaluation questions into consideration when developing your course syllabus, objectives, content, and assessments. Provide opportunities during class time to complete evaluations. Reflective writing activities In-class discussions Mid-semester surveys Share findings with students
19. 19. Evaluation of Blended College Courses Using a Community of Inquiry Survey: Opportunity for Improvement Diane Z. Onorato Indiana University of PA Mercyhurst University, North East, PASurvey retrieved from http://communitiesofinquiry.com/model
20. 20. Babson Survey Research Group and the College Boards 2011 report on the status of online learning in the US, 1. 6.1 million college students took an online course in Fall 2010 2. 31 percent of all college students were taking at least one online course in 2011 3. Online education had 10% growth rate; traditional education, a 2% growth rate 4. 65 percent of all reporting tertiary level schools include online education as part of their strategic plan. 5. According to the most recent distance and online learning report from Converge (2011), in five years, if the percentages of increase stay the same online students will outnumber traditional students. 6. 1.8 million online basic education student in 2009-10 (Converge, 2011, p. 5)
21. 21. Theoretical framework of a Community of Inquiry (CoI) collaborativeconstructivist learning experience through the development of three interdependent elements social, cognitive and teaching presence.
22. 22. Definition of Blended Learning Blended learning is the organic integration of thoughtfully selected and complementary face-to-face and online approaches and technologies. Blended learning is seen as an opportunity to fundamentally redesign how we approach teaching and learning in ways that higher education institutions may benefit from increased effectiveness, convenience and efficiency. (Garrison & Vaughan, 2008)
23. 23. Teaching Presence is the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes (Anderson, Rourke, Garrison, & Archer, 2001, p. 5). CategoriesExamples of IndicatorsFacilitating DiscourseDefining, focusing, motivating with knowledge of students efficacy; Clarifying ideas, shared experiences; Managing netiquette; Providing feedback and directionsDirect InstructionDiscussion; Questioning, Injection of new knowledgeInstructional Design & OrganizationSetting curriculum, standards, targets, scaffolding; Calendar; Designing methods, activities; Use of medium; Offering direction for technical support
24. 24. Teaching Presence: Survey Items Design & Organization 1. Clearly communicated important course goals 2. Clearly communicated important course topics 3. Provided clear instructions on how to participate 4. Clearly communicated important due dates/time frames Facilitation 5. Identifying areas of agreement and disagreement to help me to learn 6. Guiding class towards understanding course topics to clarify my thinking 7. Keeping participants engaged and participating in productive dialogue 8. Keeping the course participants on task 9. Encouraged course participants to explore new concepts 10. Reinforced the development of a sense of community Direct Instruction 11. Focus discussion on relevant issues in a way that helped me to learn 12. Feedback that helped me understand my strengths and weaknesses Retrieved from http://communitiesofinquiry.com/methodology 13. Feedback in a timely fashion
25. 25. Social presence is the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships(Garrison, 2007; van Shie, 2008, retrieved from by way of projecting their http://communitiesofinqu iry.com/sites/community individual personalities ofinquiry.com/files/conce pt-map.gif) (Garrison, 2009, p. 352). CategoriesExamples of Indicatorssense of self and audienceEmotional Expression Using humor, self-disclosure, emoticons; Expressions of feelings; a sense of safety to speak without unfair treatment Open CommunicationContinuing threads; Quoting and referring to each others statements; Complimenting and agreeing; Risk-free ExpressionGroup CohesionEncouraging and including others; Using names and greetings
26. 26. Social Presence: Survey ItemsRetrieved from http://communitiesofinquiry.com /methodologyAffective expression 14. Getting to know other[s] gave me a sense of belonging 15. Able to form distinct impressions of [others] 16. Online . . . is an excellent medium for social interaction Open communication 17. Comfortable conversing 18. Comfortable participating in . . . discussions 19. Comfortable interacting with other[s] Group cohesion 20. Comfortable disagreeing . . . while still maintaining sense of trust 21. My point of view was acknowledged by other[s] 22. Develop a sense of collaboration
27. 27. Cognitive Presence is the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse (Garrison, Anderson, & Archer, 2001, p. 11). CategoryExamples of Indicatorreflection and interactionTriggering EventSense of puzzlement about a topic; Recognizing the problem but not yet beginning an activity on the topicExplorationInformation exchange through discussion; Recognition of divergent views; Offering of suggestions; BrainstormingIntegrationConnecting ideas; Finding points of convergence; Offering tentative conclusions or solutionsResolutionTest and defend solutions; Apply new ideas
28. 28. Cognitive Presence: Survey Items Triggering event 23. Problems posed increased my interest 24. Activities piqued my curiosity 25. Felt motivated to explore content related questions. Exploration 26. Utilized a variety of information sources 27. Brainstorming and finding relevant information helped me resolve content-related questions 28. Discussions help[ed] me appreciate different perspectives.
29. 29. Cognitive Presence: Survey Items (continued) Integration 29. Combining new information helped me answer questions 30. Activities helped [with]. . . explanations/ solutions 31. Reflection . . . helped me understand fundamental concepts Resolution 32. Can describe ways to test and apply [course ideas] 33. Have developed [practical] solutions 34. Can apply the knowledge created in this course to my work or . . . other activities
30. 30. Results from CoI Survey IndicatorClass Class Class Class Class Class Class Class Class #1 #2 #3 #4 #5 #6 #7 #8 #9TP design & organization8%0%0%4%10%11%10%10%15%TP facilitation5%25%4%6%17%12%8%13%17%TP direct instruction13%17%7%13%13%13%9%13%20%SP affective expression3%17%4%8%3%11%7%10%27%SP open communication0%0%4%8%10%11%9%13%20%SP group cohesion0%0%7%9%13%13%11%10%27%CP triggering event17%0%7%12%10%11%11%12%13%CP exploration17%0%4%13%20%14%9%13%20%CP integration0%0%0%11%10%9%7%8%20%CP resolution10%0%7%10%13%15%11%13%33%
31. 31. Findings & Implications Scaffolding Time for reflection Clear expectations, models Active and collaborative learning Manage the learning environment; summarize the conversation/new direction Provide feedback beyond grades Students self-efficacy
32. 32. Findings & Implications Terry Ann Morris (2011) 65% completion for online CC and 72% for traditional. Survey says, 1. Enjoyed discussion activities (multiple days) 2. Instructor feedback was the high point (24 hrs.) 3. Dont like chat rooms, social networking with no instructional purpose 4. Self-efficacy is determining factor in successful completion 5. Varied favorite activities underscore the diversity of the population
33. 33. To enhance social & cognitive presence, use a Wiki using a scaffold to create a pro/con debate
34. 34. Cognitive PresenceArbaugh (2007) most difficult Elder and Pauls Model of Critical Thinking (2010) Carefully design word tasks to promote critical thinking. They developed a model of questions based on standards that improve learners thinking skills and help them question ideas before they decide if they should accept them. Questions include clarification, accuracy, precision, relevance, logic, significance, fairness.
35. 35. To keep learners engaged, Survey (Bonk)
36. 36. Teaching presence Welcome email Scavenger hunt Wiki introductions Once upon a time Guest lectures Templates Scaffolding Multiple discussion forums Pride Wall
37. 37. Importance of Teaching Presence The body of evidence is growing rapidly attesting to the importance of teaching presence for successful online learning The consensus is that teaching presence is a significant determinate of student satisfaction, perceived learning, and sense of community. Dr. Marti Cleveland-Innes, Athabasca University Dr. Randy Garrison, University of Calgary (2007) 13th Annual Sloan-C Conference
38. 38. Students must feel a sense of participation Students know and can articulate what good teaching, learning, and collaboration is at the high school level. If there is no student evaluation procedure, a high school class can collaborate to produce a suitable instrument. Students must have buy-in or an accessible, secure means of participation in course evaluation in order for significant numbers to provide usable feedback.
39. 39. The evaluation must measure class procedures to allow for improvement The evaluation instrument should measure what happens in class and be a standard across similarly constructed classes so all faculty can be evaluated fairly. Results of a course evaluation by a student should provide data for teachers to improve their courses with respect to increased perceived learning and student satisfaction.
40. 40. NEERO 45th Annual Confernece Symposium 2.2 | Thursday, 10:15-11:45am Symposium Title: Student Evaluations of Higher and Basic Education Instructors: Three Case Studies Thank you for your attention!Tom NeuschaferNichol MurrrayDiane Onorato