1. Learning Analytics The Science Behind Success UCISA 2014 Conference #D2L, #UCISA14

2. Michael Moore, MSCIS Sr. Advisory Consultant Analytics Desire2Learn, Inc. 3. Please introduce yourself Name & institution Your role What is your experience with learning analytics What is your hope/expectation for this workshop Introductions 4. Society for Learning Analytics Research: Learning analytics is the collection and analysis of data generated during the learning process in order to improve the quality of both learning and teaching. 2013, Siemens, Dawson & Lynch What are Learning Analytics? 5. EDUCAUSE Next Generation Learning Initiative The use of data and models to predict student progress and performance, and the ability to act on that information. 2010, Siemens What is Learning Analytics? 6. The Horizon Report, 2011 The interpretation of a wide range of data produced by and gathered on behalf of students in order to assess academic progress, predict future performance, and spot potential issues. Johnson et. al., The Horizon Report 2011 What is Learning Analytics? 7. Data Gathering Analysis of Data Decisions Based on Data Learning Analytics Process More than just collecting data More than just analyzing data Goal is: Provide deeper insights to make smarter decisions based on facts! Michael Ticknor, July 2012, Teachers College Columbia University https://www.youtube.com/watch?v=SEFmvaBTZ3I 8. InsightandInformationValue D2L Integrated Learning and Advanced Analytics Platform Risk Forecasting Predictive Modeling What will happen? Stage Two Reporting Data Access What has happened? What is happening? Stage One Optimization Strategic What do I want to happen? Stage Three Advanced Predictive Advanced Adaptive What do you want to happen for you?? Stage Four ILP - Analytics Capability and Maturity Model 9. Horizon Report 2014 As learners participate in online activities, they leave an increasingly clear trail of analytics data that can be mined for insights. 10. Any data contained in the VLE can be considered learner data Just having the data does not bring improvement, What you do with the data brings improvement Learner Data 11. Learner Data Valuable insight about: Content consumption Learning behaviors Student interactions Digital breadcrumbs Personalization Data has Value Mine (use/extract) that data Source: HSBC 12. Source: Matthew Aslett, The 451 Group Updated database landscape graphic, Nov 2, 2012 http://blogs.the451group.com/information_management/2012/11/02/updated-database-landscape-graphic/ 13. Dont Under-estimate: The VASTNESS of the data available The VALUE of existing data Dont Over-estimate: The ACCURACY of the existing data Learner Data 14. Educational Data Mining Developing methods to explore the unique types of data that come from an educational context and, using these methods, to better understand students and the settings in which they learn. Romero et. al., The Data Mining Handbook How do we get the data? 15. How do we Analyze the Data? Chatti, A Reference Paper for Learning Analytics 16. Do you know what data your organization tracks, monitors or measures? What types of data matter to you? Do you know the types of data that would matter (provide value) for a progression, retention, or attainment initiative at your institution? Open Discussion 17. Learning Analytics Learning moment Data right where it matters most Mid-course correction 18. Decisions can only be as good as the data they are based upon. Why Bother? Improved decisions Improved student learning Personalized student learning Course improvement Improved learning outcomes Etc. Primary Purpose 19. Objectives Chatti, A Reference Paper for Learning Analytics 20. Data Policies Data Governance Scope of the data Who needs it Who sees it Who uses it Data Consistency Data Policies Enforcement Validation Who owns it Who maintains it Source: http://commons.wikimedia.org/wiki/File:DARPA_Big_Data.jpg Used under public domain Defense Advanced Research Projects Agency (DARPA) 21. Data Policies Data Retention How long do you keep it? Where do you keep it? Who needs access to it? What access is needed? Data Privacy Who can see what data? What can students see about themselves? Source: http://commons.wikimedia.org/wiki/File:DARPA_Big_Data.jpg Used under public domain Defense Advanced Research Projects Agency (DARPA) 22. Institutional Data Governance Program/Department Standards Course Practices Data Policies More Tactical More Strategic Course Offering Instances 23. Communication Data Standards/Policies Data Strategy Data Processes Elements of Data Governance Just because data exists does not automatically mean you can get to it or report on it. 24. Potential Issues Plan for resources Hardware, software, systems Staff!! (plan for people/capacity) Education Focus on self-serve Enable end-users Train staff Source: http://www.flickr.com/photos/dellphotos/11354480054/in/photostream/ Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/ 25. Morning Break Group 1 Discussion - Plan for resources Hardware, software, systems Staff!! (plan for people/capacity) Group 2 Discussion Education Focus on self-serve Enable end-users Train staff Source: http://www.flickr.com/photos/dellphotos/11354480054/in/photostream/ Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/ 26. Mid morning break 27. Group 1 Discussion - Plan for resources Hardware, software, systems Staff!! (plan for people/capacity) Group 2 Discussion Education Focus on self-serve Enable end-users Train staff Discussion Questions 28. How to Use the Data? Belinda Tynan & Simon Buckingham Shum (2013). Designing Systemic Learning Analytics at the Open University 29. User Needs Understand use cases Clearly define requirements Identify stakeholders Understand nuances of the data Provide user-friendly views and reports Source: http://www.flickr.com/photos/dellphotos/11354379865/in/photostream/ Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/ 30. No Report is Perfect Understand the goal, the use, the decisions Use appropriate visualizations where possible Identify relationships, trends, patterns Focus on Key Question to be answered Reporting 31. Tables and Charts 32. Summary and Detail 33. Real-Time and Valuable 34. Q#1 what are some examples in your personal life of a rich visualization done right? Q#2 how would you apply that visualization to learning analytics? Q#3 do instructors have specific visualization needs? Q#4 what about accessibility needs re: visualizations? Open Discussion 35. Personalized 36. Predictive 37. Data Rich Visualizations 38. Data Rich Visualizations 39. Global Graduation Data #1-Switzerland #2-United Kingdom #5-EU-27 #10-United States Pct 34% 29% 23% 15% 0% 10% 20% 30% 40% Source: http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tertiary_education_statistics Ranked by graduation percentage rate. 40. US Higher Ed Problem Student Retention First year attrition rates exceed 25% Some states reach 40% Only 1 in 2 students ever complete a degree Degree Completion 75% students are non- traditional 40% students are not academically prepared 40% are part time Time to Degree 60% FT students complete 4 yr Bachelors within 8 yrs 24% PT students complete 4 yr Bachelors within 8 yrs 20% take more courses than needed Efficacy of Higher Ed Source: Complete College America Time Is the Enemy - Summary http://completecollege.org/docs/Time_Is_the_Enemy_Summary.pdf 41. Adaptive Learning Knowillage LeaP Adaptive learning engine Personalized learning experience What if textbooks could learn? Source: http://www.flickr.com/photos/m00by/2538526391/Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/ 42. Desire2Learn LeaP Adaptive Learning Adaptive analytics and semantic learning engine for personalized learning 43. Semantic learning Data mining analysis Examples: Survey responses Customer feedback Journals and publications Discussion forums Email threads Text Analytics 44. Invited to attend and deliver opening night address Bridging the gap between research and vendor worlds Based on D2Ls Our support of the analytics research community Our delivery of research-backed solutions to market Practical application of our current toolkits Product-based research Brand new ideas Leverage scientific research in the field Vetted proofs of concept that can help solve the challenges educators are facing today LAK 2014 45. Partnership Research Examples of learning analytics research Predictive analytics algorithm and data modeling techniques for Insights S3 and Degree Compass (past research works) Semantic Content Analysis and Big Data (current research works) Learning Objectives and Unstructured Text (future research works) 46. Partnership Research http://go.desire2learn.com/LAKSurvey 47. Questions? Michael Moore Sr. Advisory Consultant Analytics Desire2Learn, Inc. Direct 888.772.0325 x6604 Twitter: @MikeMooreD2L Michael.Moore@Desire2Learn.com Slides available on SlideShare www.slideshare.net/MikeMoore14 Thank You Let the dataset change your mindset 48. Desire2Learn, Campus Life, CaptureCast, Desire2Learn Binder, myDesire2Learn, Insert Stuff, Insert Stuff Framework, Instructional Design Wizard, and the molecule logo are trademarks of Desire2Learn Incorporated. The Desire2Learn family of companies includes Desire2Learn Incorporated, D2L Ltd., Desire2Learn Australia Pty Ltd, Desire2Learn UK Ltd, Desire2Learn Singapore Pte. Ltd. and D2L Brasil Solues de Tecnologia para Educao Ltda. Lets transform teaching and learning, together.