real-time visual analytics to evaluate online collaboration (166278128)
TRANSCRIPT
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 1/61
Real-Time Visual Analytics to Evaluate
Online Collaboration
John McCormick | Sarah Krongard
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 2/61
Agenda
1. Learning analytics defined
2. Discourse and collaboration
3. Social Networks Adapting Pedagogical Practice(SNAPP): A tool for visualizing interaction
4. Future of discourse learning analytics tools
5. Q and A
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 3/61
Horizon Report: Emerging Technologies
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 4/61
Mobile
Learning(1-2 yrs)
TabletPCs(1-2 yrs)
Learning
Analytics(2-3 yrs)
iPad
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 5/61
Definitions
The measurement, collection, analysis and reporting of dataabout learners… for purposes of understanding and optimising
learning and the environments in which it occurs
- 1st international Conference on Learning Analytics and Knowledge (LAK 2011)
----------------…selection, capture and processing of data that will be helpfulfor students and instructors at the course or individual level.
…able to adjust content, levels of support and other personalized
services by capturing, reporting, processing and acting on data
on an ongoing basis in a way that minimizes the time delay
between the capture and use of data.
-Elias, T. (2011). Learning Analytics: Definitions, Processes and Potential
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 6/61
Definitions (cont.)
Learning Analytics:
• is the analysis of many kinds of learner-produced and
learner-related data;
•seeks to monitor learner activity and progress and to
predict learner outcomes; and
• enables interventions and decision making about
learning by instructors and students.
- From Educause/ELI Focus Session on Learning Analytics (2012)
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 7/61
Example 1: Signals: Purdue
Source: http://www.itap.purdue.edu/studio/signals/
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 8/61
Signals: Results
“When instructors follow recommended Course Signals bestpractices, grade performance improvement has been consistently
demonstrated at both the course and departmental level.”
• 10% increase in numbers of As and Bs awarded in semester-
long courses, and a 6.41% decrease in Ds, Fs and withdrawals.
• Greatest improvement seen in students who were initially
receiving Cs and Ds in early assignments, and pull up half a
letter grade or more to a B or C.
• “…the overwhelming response from the students is that Course
Signals is a helpful and important tool that aids in their overall
academic success”
Arnold and Pistilli (2012)
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 9/61
The seven dimensions of ‘Learning Power’
1. Critical curiosity
2. Changing and learning
3. Meaning making
4. Creativity
5. Learning relationships (interdependence)
6. Strategic awareness
7. Resilience
Example 2: Effective Lifelong Learning
Inventory (ELLI)
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 10/61
Source: http://www.slideshare.net/Ruthdeakincrick/learning-dispositions-and-transferable-competences-pedagogy-modelling-and-learning-analytics
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 11/61
Source: http://www.heacademy.ac.uk/assets/documents/what-works-student-retention/Northumbria_ELLI_Dec_11.pdf
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 12/61
Spider Diagram
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 13/61
ELLI ResearchFindings from research and development projects indicate that
learners who are aware of their learning power profile using ELLI and
who have structured learning opportunities to improve it, become:
• More reflective and aware of themselves as learners
•
More confident and motivated to learn• More willing and able to take responsibility and ‘own’ their
learning
• More keen to carry on learning
• More likely to attain higher levels and grades• More likely to fulfil their potential
• More sociable and able to learn with and from each other
• More able to manage change and transitions
Source: My Learning Power: Using ELLI in Higher Education
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 14/61
Enquiry Blogger
Source: http://wordpress.org/extend/plugins/eb-enquiryblogbuilder/
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 15/61
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 16/61
Instructor Dashboards
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 17/61
ELLIMent
Source: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent/index.html
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 18/61
Academic vs. Learning Analytics
Long, P. & Siemens, G. (2011). Penetrating the Fog: Analytics in learning and education.
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 19/61
Initial Thoughts about Learning Analytics?
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 20/61
Discourse & Online Collaboration
• What is the value of collaboration in an online
environment?
•
What are the challenges & opportunities doinstructors face in facilitating online
collaboration?
•
To what extent have the roles of instructorand student changed?
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 21/61
The Importance of Collaboration
• To refine and confirm personal understanding
with a community of learners
• To develop, foster, and practice higher-order
thinking skills
• To share and co-construct knowledge to
support emerging collectives
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 22/61
Online Collaboration
Challenges
• Increased Workload
• Difficulty of Encouraging
and Sustaining High-LevelDiscourse
• Asynchronous Interactions
• Lack of F2F Connections
• Challenge of Self-Motivation
• Technical Difficulties
Opportunities
• Time Allocated for
Thoughtful and Reflective
Dialogue• Permanency of the Written
Word
• Tools Available to Facilitate
Interaction and Engagement• Opportunities for Student
Agency and Efficacy
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 23/61
Making the Shift in Roles from…
… Traditional Instructor to Guide and Facilitator
… Presumed Authority to Collective Credibility
… Student as Passive Recipient to ActiveProducer and Co-Creator
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 24/61
“Learning is in social relationships – situations of cooperation with others…
Both the learner and the community in which
they participate are transformed by thecommunity as a complementary unit thatwork in harmony to promote learning…”
(Dooley, Lindner, Dooley, & Murphy, 2005, p. 49)
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 25/61
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 26/61
SNA Concepts: Centrality
Source: http://www.wolfram.com/mathematica/new-in-8/graph-and-network-analysis/degree-centrality-in-social-networks.html
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 27/61
Node Size: Number of Interactions
Source: http://socialmedialab.ca/wp-content/uploads/2011/02/Fig.1.jpg
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 28/61
Information Brokers
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 29/61
Frequency of Interaction
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 30/61
SNA and Online Discourse
• Very little SNA research on online discussion for learning
General Findings:
•
More evenly distributed activity beneficial• Low hierarchical structures better (groups interacting,
teacher not playing central role)
• More interaction not necessarily better
Toikkanen , T. & Lipponen, L. (2011). The applicability of social network analysis tothe study of networked learning. The Journal of Interactive Learning Environments,19 (4), 365-379.
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 31/61
Social Networks Adapting Pedagogical Practice (SNAPP)
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 32/61
Other SNAPP metrics
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 33/61
What SNAPP can tell you
Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). ‘Seeing’ Networks: Visualising and evaluating student learning networks.Australian Learning and Teaching Council, Canberra, Australia.
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 34/61
Dawson et al findings: Instructor use
“extremely effective in promoting reflection onteaching activities and rapidly assessing overalleffectiveness..”
“However, teachers require additional support and training to better understand therelationship between their implemented online
learning activities and the resultant onlinestudent behavior…”
Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). ‘Seeing’ Networks:
Visualising and evaluating student learning networks. Australian Learning and
Teaching Council, Canberra, Australia.
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 35/61
Interpretation and Learning Design
“Future research and professional development
should seek to merge learning design with
learning analytics and data interpretation. A
sound understanding of learning designprinciples will greatly assist academics in better
interpreting the learning analytics data into
informed pedagogical action”.
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 36/61
Action Research Steps
1. Create visual taxonomy
2. Choose instructors
3. Collect data
4. Send “Visual Key” and “Custom Report”
5. Interview instructors
6. Follow- up with instructors
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 37/61
Visual Key
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 38/61
Visual Features (filtering)
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 39/61
Instructor-Led
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 40/61
Learning Community
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 41/61
Combined Patterns
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 42/61
Learning Community with strong instructor presence
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 43/61
Emerging Learning Community
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 44/61
Emerging Learning Community with strong instructor
presence
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 45/61
Weak Learning Community
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 46/61
Learning Community Development
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 47/61
Two Facilitation Styles
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 48/61
Facilitator Role: Central vs Peripheral
Instructors critical to success of discussion• Blignaut & Trollip, 2003; Ferdig & Roehler, 2004; Greenlaw & DeLoach,
2003; Love, 2002; Roblyer & Wiencke, 2003; Kearsley, 2000; Arnold &
Ducate, 2006; Darabi etal., 2011; Ertmer et al., 2011; Liu & Yang, 2012;
Instructors should take a back seat, let students
construct own knowledge• Burstall, 2000; Li, 2003; Mazzolini & Maddison, 2003; Poole, 2000; Rourke
& Anderson, 2002; Paloff & Pratt, 2001; Wang & Chen, 2010, Baram &Correia, 2009
Learner Population: Undergraduate Business
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 49/61
Learner Population: Undergraduate Business
Typical Pattern: Learning Community with strong instructor
presence
Intervention Suggestions: Reduce number of instructor postings,
add summary or “weave” following discussion, use student-led
facilitation, organize discussions with use of titled posts.
Intervention Plans: Reduce instructor posting in some
discussions and better organize initial posts.
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 50/61
d
Instructor: High Centrality, Low In-Degree
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 51/61
Future learning analytics
Source: http://www.studentsoftheworld.info/sites/people/32917.php
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 52/61
Meerkat-ED
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 53/61
Group interaction over time
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 54/61
Networks of Terms
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 55/61
d d l h d l k d
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 56/61
Individual Discussion Thread: Click on edges
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 57/61
Natural Language Processing
https://www.coursera.org/course/nlp
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 58/61
Dashboards: Desire2Learn
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 59/61
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 60/61
“The ties between people make
the whole greater than the sum
of its parts”
Nicholas Kristof, Sociologist
7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)
http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 61/61
Questions, Comments, Answers?