Download - OPID Conference Presentation
Evaluation of a Flipped Classroom Format for an Introductory Level
Marketing Class
George KruegerAssistant Professor
University of Wisconsin--Platteville
UW-Platteville
Flipped Class Concept
The research is mixed on learning outcome results. Some studies show a positive impact on quantitative students results and qualitative student perceptions. Others show a negative impact or no difference.
Marketing is an application-based subject and largely qualitative in evaluating student results. Would a flipped class result in a more positive student outcomes?
Purpose
The purpose of this study was to evaluate the flipped classroom methodology in an introductory level course.
The professor has taught the course for several years in a conventional lecture format.
What is “Flipped”
There are numerous ways to define flipped as a teaching methodology. • Video presentations• Pre-assigned readings• Assignments to be completed outside of class and
then reviewed during class• Outside of class online discussions • Preparation for student presentations.• Online quizzes
Online Quizzes
• Flipped classroom 10 quizzes at 50 points & due the Monday night BEFORE the Tuesday/Thursday classes
• Control: 10 quizzes at 20 points due the Thursday before Friday class
Allocation of Class Time
Flipped: • Student presentations• In-depth discussions
Control• More lecture, less projects and discussion
Let’s See What Happens at UWP
The Classes
• BSAD 2630 Introduction to Marketing• Section 02: Tuesdays and Thursdays from 9:30 to 11:45 • Section 03: Monday, Wednesday, Friday from 9:00 to 9:52. • The courses used the same textbook, in-class instruction
materials and followed the same course content. • Students in each class was 35 at the start of the semester. • The students were predominantly business majors along
with business minors. Very few were marketing emphasis.
Online Quizzes
• The quizzes were administered online in D2L.• Multiple choice questions were loaded
directly from the textbook testbank.• Quiz questions were randomized for each
student. • Open book/open notes. The quizzes covered
the chapters that were assigned for each week.
Flipped Group Projects
Each group focused on one company and applied the subject matter in discussion and presentation formats. – Oakley Sunglasses– Gatorade– Canon Digital Photography– Monster Beverages– Blackberry– UnderArmour
Control Group Projects
• Sunday Ticket—Direct TV• Netflix• Kwik Trip Convenience Stores• Mountain Dew KickStart• Domino’s
Discussions…what next?
Exams
• 3 exams administered during the semester. • Study guides were provided and the professor
emphasized key items that would be on the exam.
• Exams were similar but not exactly the same for the two classes.
• Approximately 60% multiple choice and 40% short answer/essay. The exams were returned to the students.
Final Exam
• Comprehensive exam covering the entire semester.
• Exams were exactly the same for the 2 groups.• 40% multiple choice and 60% application of
terms related to a case on Subway Restaurant’s marketing.
• The case subject was given to the students along with a study guide before the exam.
Final Individual Project
• An individual marketing plan project was assigned to the students.
• The student to selects a product or service to market and write a marketing plan for their selection.
• Outline and grading rubric was provided to each class.
• Intended to be a capstone application of the marketing concepts.
Summary of Grading Control Class Flipped Class
Online Quizzes 200 500
Group Projects 100 100
Exams 300 300
Final Exam 100 100
Final Individual Project 100 100
Class Participation 100 100
Total Points 900 1200
Results
Online Quizzes
• Conventional class took the quizzes after the course material had been covered in class and averaged 77.4%.
• Flipped class took the quizzes before the lecture they only trailed the conventional class by .7 at 76.7%.
• No statistical difference between the two measurements.
Exams
The results for the exams were not statistically significant. There were no trends apparent in the data. Exam 1 Exam 2 Exam 3 Final ExamFlipped 71.6% 79.0% 77.8% 68.2%Conventional 73.5% 76.6% 75.0% 72.5%• The observations of the Final Exam answers for
the Flipped group was deemed poor by the professor.
Final Individual Project
• The flipped class scored 80.8% on the individual marketing project while the conventional class scored 83.2%. There was no statistical difference.
Overall Class Grades
• The final class percentage achievement for the flipped classroom was 78.9% and the conventional class was 77.7%. No statistical difference.
Pre and Post Survey Results
Flipped Flipped Control ControlQuestion Average
PreAverage Post
Average Pre Average Post
This class has met my expectations so far for content delivery
4.1 4.1 4.3 4.1
This class is taught like most classes that I have taken
2.7 3.2
3.1 3.2
I am satisfied with the way this course is being taught
4.2 4.0
4.0 3.7
I believe that I understand the content that has been covered so far in the class
3.8 3.8 3.8 3.7
I believe that I can apply the content that has been covered so far in the class
4.0 4.0
4.0 3.9
I believe that the learning objectives are clear
4.0 3.9 4.0 3.9
I am satisfied with my learning so far in this class
3.8 3.9 3.8 3.8
I believe that the amount of homework in the course is
2.1 1.9 2.0 1.9
I believe the difficulty of the homework is
1.9 1.9 2.0
1.8
Student Evaluations
Question Flipped Fall 13
Control F 13
Sp 13 Fall 12 Fall 12 Fall 11 Fall 11
3. Uses examples and illustrations that clarify the material.
4.857 4.524 4.367 4.308 4.519 4.548 4.31
4. Explains course material clearly.
4.5 4.05 3.967 3.8 3.926 4.097 4.1797. Incorporates current material into the course.
4.821 4.619 4.533 4.692 4.519 4.452 4.6219. Encourages classroom participation from students.
4.857 4.476 4.4 4.28 4.481 4.677 4.48310. Presentations/ discussions enhance the text.
4.571 4.238 4.367 4.038 4.296 4.258 4.138
15. Uses class time for student discussions, presentations or questions.
4.821 4.238 4.3 4.292 4.296 4.613 4.448
Mean 4.738 4.394 4.361 4.278 4.51 4.498 4.42Responses 28 21 30 26 27 31 29
Student CommentsFlippedPositives
14 Entertaining/enthusiasm/engaging6 Good knowledge6 Liked group presentations/discussions
Improvements 9 More help on projects/expectations/assignments2 Dislike online quizzes4 Go through study guides & reviews1 Less Powerpoint by Professor1 Some student groups were not good presenters
ControlPositives
14 Entertaining/enthusiasm/engaging5 Good knowledge/examples1 Liked group presentations/discussions1 Professor is concerned about students (cares)
Improvements7 More help on projects/expectations/assignments6 Dislike online quizzes1 Go through study guides & reviews3 Lectures can be confusing at times1 Stay on syllabus1 More activities
Professor Observations
They Are Students!
• Several times during the semester the professor had to intervene in the group presentations to correct mistakes and add in omitted subject matter. In addition, some of the presentations were very poor and students attention was lacking.
Unique Challenge
• The flipped format was a challenge with 35 students. Individual attention and a large number of groups can be difficult to manage in a classroom environment. When the professor is focusing on a group of 6 students the other 29 students can be disengaged from the class.
Soaring With Eagles?
Another Observation
• As noted in the research, not having a base of knowledge led to visible frustration from some students as observed by the professor. Often when the subject matter was enhanced in the presentations by the professor the response from the students was something like, “how are we supposed to know that?”
Interesting….
• The flipped format required more work outside of class by the students and this was perceived by the professor to be an issue during the semester. However, this perception did not manifest itself in the evaluations.
Interesting….
• Some students expressed their preference to be presented the information in lecture format first with explanation by the professor. They were then more comfortable and effective with the group work and projects.
Conclusions
Conclusions
• The flipped classroom methodology utilizing the key tools of online quizzes taken before classroom coverage of the subject matter, student preparation for presentations outside of class, student presentations and a much higher level of group work in class did not yield any statistical differences in student outcomes as measured by grades when compared the control class of lecture and class discussion format.
Conclusions
• The students did not perceive a significant level of difference in the way the classes were taught from their previous experiences.
Conclusions
• The flipped classroom did create a higher level of satisfaction for the students numerically and did appear to engage the students more actively as measured by student evaluation scores.
As a Result of This Study…
• Attempted to use the “best of both worlds” in the Spring 2014 for Introduction to Marketing and it worked very well.
• More focused lecture covering key areas. • More focus was put on class presentations
and projects applying the subject matter to their group projects.
Follow-Up Study
• Lessons Learned
– 1. To take the best of both worlds and utilize highly focused lectures on key marketing concepts during the Tuesday class session and then have the students apply the concepts in a weekly discussion format during the Thursday class session.
– 2. Utilize the student’s projects throughout the semester in the context of lecture and application of marketing concepts.
Group Project Discussions
Projects:• Gatorade• NorthFace• Signal Snowboards• Apple I-Phone• Samsung Galaxy Watch
Grading Summary of Grading
Lessons Learned Class Spring 2014
Flipped Class Fall 2013
Online Quizzes 200 500
Group Projects 100 100
Exams 400 300
Final Exam 100 100
Final Individual Project—Marketing Plan
100 100
Class Participation 100 100
Total Points 1000 1200
Table 2. Average scores for the fall 2013 Flipped class and spring 2014 Lessons Learned class.
Fall 2013 Flipped n = 33
Lessons Learned n = 31
t-test results
Mean Std Dev Mean Std Dev t p
Online Quizzes 77% 8.6 78% 16.5 0.344 .37
Total Exams 76% 9.2 76% 9.8 0.035 .49
Final Exam 68% 9.5 77% 16.1 2.62 .005**
Marketing Plan Project 81% 13.6 88% 13.5 2.02 .024*
*Significant at the .05 level, **Significant at the .01 level
Table 3. Student evaluations from fall 2011to spring 2014.
Question Lessons Learned Sp 2014
Flipped Fall 13
Control F 13
Sp 13 Fall 12 Fall 12 Fall 11 Fall 11
3. Uses examples and illustrations that clarify the material.
4.45 4.857 4.524 4.367 4.308 4.519 4.548 4.31
4. Explains course material clearly.
3.69 4.5 4.05 3.967 3.8 3.926 4.097 4.179
7. Incorporates current material into the course.
4.61 4.821 4.619 4.533 4.692 4.519 4.452 4.621
9. Encourages classroom participation from students.
4.66 4.857 4.476 4.4 4.28 4.481 4.677 4.483
10. Presentations/ discussions enhance the text.
4.03 4.571 4.238 4.367 4.038 4.296 4.258 4.138
15. Uses class time for student discussions, presentations or questions.
4.64 4.821 4.238 4.3 4.292 4.296 4.613 4.448
Mean (all responses) 4.39 4.738 4.394 4.361 4.278 4.51 4.498 4.42 Responses 29 28 21 30 26 27 31 29
Table 4. Categorical assessment of student comments including number of responses (n).
Done Well
n 2013 Flipped n Lessons Learned
14 Entertaining/enthusiasm/engaging 20 Entertaining/enthusiasm/engaging
6 Good knowledge 9 Good knowledge/examples
6 Liked group presentations/discussions 4 Liked group presentations/discussions
Improvements
n 2013 Flipped n Lessons Learned
9 More help on projects/expectations/assignments
8 More help on projects/expectations/ assignments
2 Dislike online quizzes 5 Dislike online quizzes
4 Go through study guides & reviews 4 Go through study guides & reviews
1 Less PowerPoint by professor
1 Some student groups were not good presenters
Group Work
• Discussions in the context of the weekly subject matter rather than presentations
Observations
• Improved engagement of students• More consistent engagement• Basic knowledge improvement—fewer
mistakes in subject matter concepts• Improved insight from the student—”why”• Samsung/Apple—excellent compare and
contrast examples• They really hate the online quizzes
Conclusions
• Improved learning outcomes as measured by final exam and individual marketing project scores
• Reverted back to the mean on student evaluations
Thank You!!