learnline - early warning system & performance dashboard
TRANSCRIPT
1
EARLY WARNING SYSTEM AND PERFORMANCE DASHBOARD
SubtitleJune 2012
2
Outline of session
My experience Monitoring students and engaging with
large numbers The problems The context
Potential management and engagement tools: EWS and Performance Dashboard
Examples from NUR219 Learnline site Discussion/questions
June 2012
3
My Experience: Self-portrait (circa 2009)!
June 2012
4
The Problems: NUR219 Mental Health Nursing
Disconnection Unable to establish effective interpersonal relationships with students Minor unit management errors on my part became exponentially magnified
through the effect of high volumes, technology, and isolation from campus activity
Complexity of teaching across multiple modes increases risk of confused information
Non-traditional student cohort NUR219: 367 students; 11% internal, 1% full-time (“Area 52” data, 21st May)
Unable to access traditional feedback from students as a check Minimal ability to obtain direct feedback from students; difficult to evaluate
the quality of feedback Difficult to set up expectations and norms as these are not set by on-campus
socialisation or effective face-to-face routine contact. We now compete with other powerful expectation-setting contexts (e.g. employers, past education experiences, workplace inservices, etc.)
June 2012
5
Feedback, Engagement, and Management
Lecturer -> -> Student (large no’s)
Students -> -> Lecturer Student -> -> Peer
Community
June 2012
My view of NUR219 students
6
“Invisible” online students
Students who are losing track or not-progressing for other reasons are not detected in a timely manner.
Leads to a high risk of failure Which then leads to students blaming a
lack of support, guidance, confusion, etc.
June 2012
7
Identifying students who need support
Online Early Warning System Performance Dashboard
Track progression data, then Assertively provide help and support
(asynchronously).
June 2012
“My maths teacher” (circa late-ish 1900’s)!
8
Unit Design Considerations
Tracking progression involves decisions about what “items” that will generate the data needed. This decision will guide how to set up learning
materials and assessment items. EWS and Performance Dashboard…
uses two main types of data: Review Status (Learning Materials) and Due Date Status (Assessment Items)
allows the Unit Coordinator to interrogate students’ progress
the unit coordinator can then identify significant groups of students and respond en masse
June 2012
9
Data-driven Unit Design
Early Warning System uses “due dates” data from any assessment items. Notifications can be sent x days before or after a due date, or if poor unit access, etc.
Performance Dashboard uses “Review Status” data from content items. The number of items marked as Reviewed by the student can be observed.
June 2012
10
EWS example
Example: Notification to students with late attempt at Week 1 self-assessment quiz.
June 2012
11
Performance Dashboard example Example: Identifying students who have
not reviewed Learning Materials.
June 2012
12
Unit Design: Thinking About Data
June 2012
Wk
Topic (Review items) Self-assessment (due date items)
Assessment
1 Orientation to NUR219 Quiz 1
2 Theory as a basis for practice
Quiz 2
3 Neuroscience as a basis for practice
Quiz 3
4 Tools of psychiatric nursing Quiz 4 Census date
5 Culture and ethnic considerations
Quiz 5 Assignment 1
6 Law, ethics and mental health
Quiz 6
7 The client experiencing crisis / anxiety
Quiz 7
8 The client experiencing schizophrenia
Quiz 8 Assignment 2
Etc…
13
Progress and Performance Data for NUR219
weekly self-assessment quizzes with automated marking and feedback provides weekly “due date” data for the EWS can also develop an EWS notification for poor scores if you
wish!
weekly Learning Modules provides a sequential weekly structure With Review Status and Adaptive Release
switched on to generate “Reviewed” content data.
June 2012
14
Example: Early warning notifications
June 2012
15
Early Warning System
A student progression management tool, but EWS Notifications are a mechanism for “assertive engagement” with students
June 2012
16
Performance Dashboard: Tracking progress
Example: Sort on Review Status column (starting at
zero) Show all
June 2012
17
NUR219 Example: Tracking progress (Review status)
June 2012
18
Example: Tracking progress student detail
June 2012
19
Communicating expectations Monitoring progression; who is falling
behind and who is becoming “High Risk”?
Assertive engagement using notifications generated from EWS Frame up helpful notifications that express
clear boundaries
Audit trail
June 2012
20
Example NUR219 notification to students from EWS
June 2012
21
Example NUR219 notification (absence from unit)
June 2012
22
Example NUR219 notification (insufficient progression through Learning Materials)
June 2012
23
Example NUR219 notification (insufficient progression through Learning Materials)
June 2012
24
High Volumes (Advanced Performance Dashboard)
Can’t send notifications directly from Performance Dashboard
Use Grade Centre to send notifications if needed to appropriate students en masse
Copy Dashboard to Excel for advanced filtering and format as a table In Dashboard, select Show All. Select all the cells in the Dashboard table including
column headings. Use “Paste Special” (choose Unicode) to dump data into
Excel sheet. Then Format as Table in Excel
June 2012