ted j. branoff, ph.d. north carolina state university college of education department of...
TRANSCRIPT
Ted J. Branoff, Ph.D.North Carolina State UniversityCollege of EducationDepartment of Mathematics, Science and Technology Education
Large Course Redesign of GC120Moving the Foundations of Graphics Course from a Face-to-face to a Blended Learning Format
GC120 - Content
Previous Structure of GC120
• Daytime sections taught by full-time faculty– Met 2 times per week for 110 minutes
– 2-3 sections rotated into a computer lab during these meetings
• Night sections taught by part-time faculty– Met once per week for 3.5 hours
– Typically had their own dedicated computer lab
Issues with GC120
• Consistency between sections– SolidWorks instruction– Evaluating SolidWorks assignments– Proportion of sketching and SolidWorks assignments
• Access to SolidWorks help outside of class• Large number of students on wait list (about 130
each semeter)
Large Course Redesign Grant
• Summer 2008 - Submitted proposal for DELTA Large Course Redesign Grant.
• Proposed moving all sections of GC120 to a hybrid or blended format.
• Proposed more laptop sections.• Proposed help sessions in Elluminate.• Proposed an automated SolidWorks grader.
Moving Sections to Blended Format
• During the Fall of 2008 content was moved from WebCT to Moodle.
• Currently all day-time sections of GC120 are blended.
• Continue to train night faculty.
New Course Structure• Daytime sections meet in class once per week
– Answer students’ questions– Talk about key concepts for the week– Check homework
• Online content completed outside of class– Voiced-Over PowerPoint Presentations– Voiced-Over SolidWorks Demonstrations– Videos of an instructor sketching– Study Guides– Low-stake Online Assessments in Moodle – 10-20 questions – 2
Attempts
More Laptop Sections
• Gradually moving to more laptop sections.
• Have renovated Poe 106 to handle 60 students.
• Still have many issues to overcome with student owned computers.
Online Help Sessions
• Offered some help Elluminate help sessions during the Fall 2008 semester for exam reviews and project help.
• Students still seem to prefer face-to-face help sessions.
• We have started offering face-to-face help sessions each semester.
Automated SolidWorks Grader
• Began programming in Fall 2008.
• Made some improvements during Fall 2009 with departmental funding.
• Still have issues with ease of use for faculty.
Fall 2008 Survey
• Prior to moving to Moodle, compared 3 pilot sections of blended structure to F2F sections.
• Materials were organized in open web pages.• Conducted in the 13th week of semester.• Questions about:
– Previous experience with online or hybrid courses– Instructional preference– General order when navigating through online materials
Fall 2008 Results
• Some previous experience with online and blended courses.
• Prefer blended over face-to-face.
• Prefer face-to-face over online.
• Students navigated through material in many different ways.
Fall 2008 Results
• No differences between face-to-face and blended sections on the midterm exam.
• Significant difference between the groups for the final exam.
Moving to Moodle
• We can now look at how students are actually navigating through materials.
• What are the best strategies for success?
Fall 2009 Study Research Questions• In what order did students progress through the
materials related to the textbook?
• What was the typical number of attempts at each assessment?
• Did students who attempted all of the online assessments perform better on the midterm and final exams than students who only attempted a few assessments?
Participants
GC120 Section
Frequency %
003 55 34%
004 56 34%
005 53 32%
Total 164 100%
Participants – Academic Year
Year Frequency %
Freshmen 4 2%
Sophomores 115 70%
Juniors 29 18%
Seniors 16 10%
Total 164 100%
Participants – Academic MajorAcademic Major Frequency %
Aerospace Engineering 17 10%
Civil Engineering/Construction Man. 45 27%
Mechanical Engineering 44 27%
Other Engineering Majors 29 18%
Education 7 4%
First Year College 8 5%
Other Majors 14 9%
Total 164 100%
Moodle LMS
Moodle Lesson Format
Voiced-Over Presentations
Sketching Videos
Results
In what order did students progress through the materials related to the textbook?
Results – Order of Navigation
Results
What was the typical number of attempts at each assessment?
Results – Number of Assessment Attempts
Results
Did students who attempted all of the online assessments perform better on the midterm and final exams than students who only attempted a few assessments?
Results – Midterm Means by Assessment Attempts
Results – Exam Means by Assessment Attempts
Analysis – Correlation between Midterm Examand Online Assessment Attempts
Spearman’s RhoAssessment
Attempts Before Midterm
Midterm Exam
Assessment Attempts Before the Midterm Exam
Correlation Coefficient
1.000
Sig. (2-tailed) .
N 164
Midterm Exam
Correlation Coefficient
.233** 1.000
Sig. (2-tailed) .004 .
N 164 164
** Correlation is significant at the 0.01 level (2-tailed)
Analysis – Correlation between Final Examand Online Assessment Attempts
Spearman’s RhoAssessment
Attempts Before Final Exam
Final Exam
Assessment Attempts Before the Final Exam
Correlation Coefficient
1.000
Sig. (2-tailed) .
N 164
Final Exam
Correlation Coefficient
.283** 1.000
Sig. (2-tailed) .000 .
N 164 164
** Correlation is significant at the 0.01 level (2-tailed)
Discussion / Conclusions• While a number of students followed the
explicitly recommended order of material use (i.e., view the streaming video before attempting the quiz), many students took alternate approaches to usage order.
• In addition, this usage order also changed over the course of the semester.
Discussion / Conclusions• The weekly online quiz assessments were only
worth a total of 10% of students’ final grade and the midterm and final worth collectively 40% of the grade.
• It is our assumption that students primarily saw the value in the weekly assessments as preparing them for the larger summative assessments of the midterm and final.
Discussion / Conclusions• The data collected seem to support the
conclusion that those students who attempted more weekly assessments (and/or made use of the streaming videos) did better on the midterm and final exams.
Discussion / Conclusions• Probably the most important finding of this study is
that the logging tools provided in Moodle provides a powerful tool for instructors to gather and analyze data on how students make use of the resources provided online.
• It is now easier to chart student trends and performance in a more accurate way than in past incarnations of this course.
Discussion / Conclusions• This provides instructors with better information to
use in the redesign of course materials for the future.
• Since the production of the multimedia learning resources is a labor-intensive practice, this formative data provides valuable evidence as to whether such material is being used by students and whether it provides real educational value.
Future Research• Develop methods for more fine-grained analysis of log
data.– More data points over the semester to better understand
trends– Analysis of first versus second tries on quizzes– Use of SCORM-compliant learning resources in conjunction
with Moodle that allow richer data collection usage (e.g., how long did they view a video and how many times did they stop and start it)
Questions?