unleashing analytics in the classroom
TRANSCRIPT
Unleashing Analytics in the Classroom
Jessica Rodriguez Gus Roque
Resources
● High Level Analytics ● Course Level Analytics:
o Impact on Course Design o Impact on Student Success & Performance
● Blackboard Analytics ● Q&A
What days of the week are the majority of users logging into Blackboard?
A. Early in the week (M/Tu/W) B. Later in the week (W/Th/F) C. Weekend (S/Su)
Bb Mobile App (Spring 2015)
Bb Mobile App (Spring 2014 to Spring 2015)
Bb Mobile App (Spring 2014 to Spring 2015)
Logins
Activity
Grades
Participation
Analytics & Course Design
Course Design
First, Consider the Purpose of Measuring Student Engagement.
– Identifying which activities not only engage students but also
produce quality analysis, is critical to designing and adapting online courses.
– What are the patterns of performance you are trying to evaluate?
Course Design
─ When are students logging into my course?
Patterns of Performance
─ Which course resources/tools are being used most frequently?
─ Which discussions boards generate the most traffic?
─ What are the patterns of performance in an online assessment?
─ What are some of my opportunities for improvement?
Course Design
Course Reports – Course Activity Overview – Student Overview for Single Course
When are Students Logging into My Course?
Course Design
Course Design
Course Design
– Posting Announcements/Messages – Materials/Resources being Accessed
[Homepage] – Time Allotted for Module Content
Completion
How does this Information Affect my Course Design?
Course Design
California State University identified that the more ‘views’ or visits to the course home page, the higher the final grade.(Whitmer, Fernandes, & Allen, 2012). What material do you house within your course homepage?
Course Design Which Course Resources/Tools are Being used
Most Frequently?
Course Reports ─ Overall Summary of User Activity
Course Design
Course Design
– Content Presentation (Tool Modality) • Video clips, posted documents, etc.
– Supplemental Resources – Group Tools – Student Support Cases
How does this Information Affect my Course Design?
Course Design
Course Reports ─ User Activity in Forums
Which Discussion Boards Generate the Most Traffic?
Course Design
Course Design
– Number of views on discussion boards can be an indicator of level of interest of a given topic.
– Hot Topic / In the News
How does this Information Affect my Course Design?
Course Design What are the Patterns of Performance in an Online
Assessment?
Item Analysis ─ By Question
Course Design
A. Filter the question table by question type, discrimination category, and difficulty category. B. Investigate a specific question by clicking its title and reviewing its Question Details page. C. Statistics for each question are displayed in the table
Course Design
– Re-evaluate questions that may be misleading and/or ambiguous
– Pre-work for assessments – Development of supplemental
resources – Create varied forms of assessment
How does this Information Affect my Course Design?
Course Design
• Enterprise Surveys – Enterprise Surveys should be used to collect data
on student satisfaction about course items and reflect on the changes based on the collected data.
What are Some of My Opportunities for Improvement?
Course Design What are Enterprise Surveys?
• Enterprise surveys are delivered at the system level. • This system level tool does not replace the course level surveys tool
available to instructors in their courses • You can send your survey out to multiple sets of recipients and analyze the
results. • Survey results are compared using response periods, which are sets of
recipients and a time frame that you specify. • You can also compare by enrollments, membership, demographics, or by a
specific survey question.
Course Design Enterprise Surveys Deployed in Course
Course Design How does this Information Affect my Course Design?
• Ease of Navigation • Create actionable items based on feedback (e.g. extra time on exam) • Are the resources within the course sufficient? • Is there sufficient peer to peer interaction/collaboration? • Multimedia preference: Pre-recorded lectures versus live lectures • Are the course and professor expectations clear? • Determine what day/time works best for live virtual meetings every week.
Course Design Sample Questions
Course Design Instructor Dashboard
Analytics & Student Success
Key Question
● How can I identify students who are at risk?
Retention Center
Retention Center
Retention Center
● Missed Deadlines ● Grades ● Course Activity ● Course Access
Retention Center
Retention Center
Key Question
● How can I identify my top performers?
Key Question
● How can I identify my top performers?
● Why? ● Send them kudos for being top performers ● Ask them to serve as peer mentors ● Interview them and find out what makes them
successful; share with the class
Retention Center
Retention Center
Retention Center
Key Question
● How do I know if a particular student has been investing time and energy into my class?
Student Activity
• Find out how much time the student is spending in the course
• Evaluation > Course Reports > Student Overview for Single Course
Course Activity
• Find out how their time spent in the course compares to other students.
• Evaluation > Course Reports > Course Activity Overview
Course Average
Resources
Key Questions
● How do I identify students at-risk? ● Retention Center
● How do I identify my top performers? ● Retention Center with customized rules
● How do I know if a particular student has been investing time and energy into my class? ● Course Reports: ● Student Overview for Single Course ● Course Activity Overview
Blackboard Analytics
● More robust reporting & analytics tool ● Interactive reports ● Export to multiple formats (XML, CSV, Excel,
TIFF, Word)
Bb Analytics
Bb Analytics
• This report provides: – Comparative information against other courses
in the same academic department – How the students in the course are performing
compared to the average of all students enrolled in the same course
Course At a Glance
Course At a Glance
Course At a Glance
• This report plots students enrolled in a course against two variables: the Grade Center External Grade and Course Accesses.
• This report can help analyze how the activity of students does (or doesn’t) relate to the grade recorded in the Grade Center.
Activity & Grade Scatter Plot
Grade Scatter Plot
• This report might be useful to find students who fit certain activity and performance profiles – Instructors can find students that, for example,
are very active in a course, but who are struggling from a grade perspective
Activity Matrix
Activity Matrix
• This report provides a list of students in a course with submission information for each student.
• This report can be used to identify students who are not engaged by displaying trends and numbers of submissions compared to the average across all students in that course.
• Students below the average may not be sufficiently engaged in the course and may need assistance.
Course Submission Summary
Submissions Summary
Blackboard Analytics: Student Report
Student Report
Roll Out Plan
• Spring 2015: – Implementation & Testing
• Summer 2015: – Pilot Group
• Fall 2015: – Full Roll Out
Resources
● http://go.fiu.edu/analytics o Copy of the slides o Digital version of conference poster o Cheat Sheet on navigating course reports o Cheat Sheet for Bb Analytics reports o Sample ‘End of Survey’ questions
• Jessica Rodriguez, Senior Instructional Designer – [email protected]
• Gus Roque, Senior Instructional Designer – [email protected] – -@RoqueGus
Contact your Instructional Designer!!!
Contact Us
Q&A