jisc learning analytics update-nov2016

The Open University, 2nd November 20168th UK Learning Analytics Network Meeting

Jisc Learning Analytics 2016

Programme10:25 – 11:15

Update on Jisc’s learning analytics programme

11:15 – 11:30

Tea / coffee

11:30 – 12:30

Learning design meets learning analytics, Dr Bart Rienties, Open University

12:30 – 13:30

Lunch

13:30 – 14:15

Parallel session 1: Legal issues for learning analytics, Andrew Cormack, Jisc

Parallel session 2: Addressing the challenges , Il-Hyun Jo, Ewha Womans University

14:15 – 15:00

Parallel session 1: The potential of blockchain , Prof John Domingue, Knowledge Media Institute, OU

The design and deployment of a learning analytics dashboard, David Evans, North Warwickshire & Hinckley College

15:00 – 15:15

Tea / coffee – Juniper/Medlar Room, The Hub

15:15 – 15:55

The Learning Analytics Community Exchange, Dr Doug Clow, Institute for Educational Technology, OU

Paul Bailey, Senior Codesign Manager, Research and DevelopmentJisc learning analytics service

http://www.slideshare.net/paul.bailey/

Where we started…


Effective Learning Analytics ChallengeRationale»Organisations wanted help to get started and have access to

standard tools and technologies to monitor and intervene Priorities identified»Code of Practice on legal and ethical issues»Develop basic learning analytics service with app for students»Provide a network to share knowledge and experienceTimescale»2015-16—test and develop the tools and metrics»2016-17—transition to service »Sep 2017—launch, measure impact: retention and achievement


Jisc’s Learning Analytics ProjectThree core strands:

Learning Analytics Service

Toolkit Community

Jisc Learning Analytics

Learning Analytics Sophistication Model


Analytics – the bigger picturehttps://docs.google.com/presentation/d/1AdBkYHO3hqEJ7W2McYIsAKzF4EgFNYJM9XGfDOTRYek/edit?usp=sharing

Michael Webb

https://docs.google.com/presentation/d/1AdBkYHO3hqEJ7W2McYIsAKzF4EgFNYJM9XGfDOTRYek/edit?usp=sharing



Descriptive Analyticswhat happened?

Diagnostic Analyticswhy did it happen?

PredictiveAnalyticswhat will happen?

Prescriptive Analyticswhat should I do?

Automated Decision makingIt's done

Analytics maturity

Descriptive Analyticswhat happened? How do I compare?


Predictivewhat will happen?

Automatedit’s done

Data


Ordered Data

Sector Transformation

Awareness

Experimentation

Organisation support

Organisational transformation

Analytics without a national approach


Awareness

Experimentation




Predictive Analyticswhat will happen?



Data


Ordered Data

Standardised Data

Analytics with a national approach


Awareness

Experimentation




Predictive Analyticswhat will happen?



Data


Ordered Data

Standardised Data

Adaptive learning etc.

Recommendation engines etc.

Predictive models, Intervention

management etcData exploration tools, processes etc

Dashboards, Benchmarking etc.

Data Warehouse, data stores

Data connectors

Analytics with a national approach

Descriptive Analytics

Predictive Analytics

Prescriptive Analytics

AutomatedDiagnostic Analytics

Standardised Data

Learning Records Warehouse

xAPI Plugins

Data transformation tools

Data and API Standards

Jisc Services

Other ProviderServices

Basic dashboards

Student App

Analytics Labs

Benchmarking services

College Analytics

Basic predictive modelling and intervention management

Procurement frameworks

Integration tools

Services for researchers

Pilot projects

Services for researchers

Pilot projects

Institutional Dashboards

Data visualisation tools

Data exploration tools

Advanced predictive modelling

Integrated intervention management

??? ???


- Sector Data used in mashups:

- NSS- SCONUL- LiDP- HESA- Open Access Reporting/Deposit, - JUSP / IRUS- IRUS - IMD- Altmetrics - H index- Impact Factor - REF metrics- Jisc Collections bands &

Subscription data

Library Labs: 6 teams, 33 participants drawn from Libraries


Library AnalyticsLibrary Labs

- BUT also analytics on institutional data:

- e-resource usage by type & department

- e-resource cost benchmarking- EZProxy logs- Loans- Gate entries- Acquisitions- Counter reports- Capita Decisions- Journal Citation Reports


Library AnalyticsLibrary LabsBirkbeck, University of LondonSheffield Hallam UniversityUniversity of EdinburghUniversity of WarwickThe University of ManchesterUniversity of SalfordLiverpool John Moores UniversityNewcastle UniversitySouthampton Solent UniversityAnglia Ruskin University LibraryUniversity of South WalesUniversity of NottinghamBrunel University LondonKingston UniversityTeesside UniversityBodleain Libraries, University of OxfordUniversity of WolverhamptonUniversity of LeicesterUniversity of ReadingManchester Metropolitan UniversityUniversity of BathDe Montfort University


Library Analytics- Mashing up Library data was difficult – SCONUL is not HESA- Many different internal systems, comparative analytics difficult - Proof of concept dashboards stimulating institutions (traffic lights)- More interest and contributions to recipes at http://github.com/jiscdev

/xapi-lib- New verbs! Eduroam, presence- Data Sharing Agreements and an experimental area in the Heidi Lab- Scope for more librarians alongside planners on Jisc’s beta BI project

http://github.com/jiscdev/xapi-lib





Where are we now…


Community: Project Blog, mailing list and network eventsBlog: http://analytics.jiscinvolve.org – over 30 blog postsMailing: [email protected] – 422 members (182 organisations)8th Network Meeting ~600+ participants

http://analytics.jiscinvolve.org/



mailto:[email protected]


http://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics

Code of Practice

http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf





Library Analytics Service

Learning Analytics Service Architecture


Learning analytics products and toolsLearning records warehouse – activeData Explorer – basic visualisations Student Unified Data Definition – version 1.2.7 and examples major SRS and validation tooVLE – xAPI recipe and plugins for Blackboard and MoodleAttendance tracking – xAPI recipe (being piloted soon)Student App – release 1 Dec 2016

Tribal Student Insights (10)Open Learning Analytics Processor (4)Further learning analytics product pilots (tbc)


UDD Validator Tool• Customer-side UDD validation (web-based, secure access)• UDD data preparation tool for institutions• Jisc will load the historical data (once validated)• Covers current & future UDD - 1.2.7, 1.2.x, 1.3.0 etc• Links directly to UDD GitHub site (dynamic updates)• Agile approach to software functionality/ release• V1.0 - hard validation (UDD structure, optional/ mandatory fields, field contents)• Relational entities – integrity checks• Soft validation - data quality and concentration/ coverage (working with Tribal/ Unicon Marist)• Focus on key fields for predictive modelling purposes, student app• Gives control & flexibility to our members – rapidly quick data validation (Azure Cloud)


ImplementationsProfile Aims Tools N

oData Sources

Teaching and research led Universities

Student retention and success

Tribal student insight/data warehouse

7 VLE (Moodle and Blackboard), student records and attendance

Teaching and research led Universities

Success and engagement

Student app 4 VLE (Moodle and Blackboard), student records

Teaching led Universities

Student retention

Open source processors/data warehouse

4 VLE (Moodle and Blackboard), student records and attendance

FE Colleges Student retention

Tribal student insight

2 VLE (Moodle), student records and attendance

Getting on-board…

https://analytics.jiscinvolve.org/wp/on-boarding/


On-boarding Process

Stage 1: OrientationStage 2: DiscoveryStage 3: Culture and Organisation SetupStage 4: Data IntegrationStage 5: Implementation Planning

https://analytics.jiscinvolve.org/wp/on-boarding/


Stage 1: Orientation

Stage 1. Orientation

1. Sign up to the analytics mailing listEvidence required:A list of people in your institution signed up to the mailing list

2. Review the learning analytics blog post and relevant reportsEvidence required:Notes on useful articles and posts you have found

3. Attend a Jisc webinar, network meeting or workshopEvidence required:Notes from attending a recent event


Stage 2: Discovery ReadinessStage 2. Discovery 4. Decide on institutional aims for learning analyticsEvidence required: A prioritised list of your aims for learning analytics

5. Strategic alignment, senior management approval and you have a nominated project lead Evidence Required: Named sponsor from the senior management team, Named project lead and contact details, Named technical lead and contact leaded, A list of members of your working/management group

6. Undertake the readiness assessmentEvidence required :A completed readiness assessment questionnaire with your commentary on the answers

7. Arrange a verification meeting with Jisc to discuss the outcomes and possible next stepsEvidence required: Date of meeting, documentation to share and a list of people attending


Discovery readinessTopic I

DQuestion Commentary Response Score

Leadership

1 The institutional senior management team is committed to using data to make decisions

Please provide a commentary on you response to each question where appropriate

0 - Hardly or not at all1 - To some extent2 - To a great extent

Leadership

2 Our vice-chancellor / principal has encouraged the institution to investigate the potential of learning analytics


Leadership

3 There is a named institutional champion / lead for learning analytics

0 - No2 - Yes

Vision

4 We have identified the key performance indicators that we wish to improve with the use of data


A supported review of institutional readiness

https://analytics.jiscinvolve.org/wp/on-boarding/step-6-readiness-assessment/


Stage 3: Culture and Organisation SetupStage 3. Culture and Organisation Setup

8. Start to address readiness recommendationsEvidence required: Action plan to address readiness recommendations

9. Legal and ethical policy considerations in handEvidence required: List of institutional policies relevant to learning analytics; Plan to update/create policies to cover learning analytics

10. Decision on learning analytics products to pilotEvidence required: A documented list of products with an agreed rational for choices

11. Data processing agreement signedEvidence required: Signed Data Processing Agreement

12. Select student groups for the pilot and engage staff/studentsEvidence required: List of student groups/cohorts and numbers of students involved


Stage 4: Data IntegrationStage 4. Data Integration 13. Undertake a data and systems audit 14. Contact Jisc to start data integration 15. Install and evaluate the VLE data plugin(s) on a test system at your institution

16. Extract student data, transform to UDD and validate.

17. Extract historical VLE (or other activity) data

18. Install VLE (or other activity) data plugin(s) on live system, activate for live data upload to LRW

19. View uploaded LRW data using data explorer to check quality


Stage 4: Data collection

About the student Activity data

TinCan (xAPI)ETL


Stage 5: Implementation Planning

Stage 5. Implementation Planning

20: Move to implementation StageEvidence required: An implementation plan with agreed timescales


On-boarding Process

Data Visualisation Dashboards

Ready to implementReady to

implement


On-boarding – get started

Stage 1: Orientation – review/doneStage 2: Discovery – mostly self-supportStage 3: Culture and Organisation Setup – Jan 2017Stage 4: Data Integration – slots from early 2017Stage 5: Implementation Planning - slots from early 2017

Further exploration…

https://www.jisc.ac.uk/rd/get-involved


Co-design challenges 2017Explore our co-design challengesHelp steer our innovation work by exploring the next big ideas for technology in education and research.


Data driven

learning gains

Next generation research

environment

Digital skills for

research

Should we gather more data on students, staff and buildings that would allow us to deliver

better experiences?

We think it is time for a new type of learning

environment, but what would this look like?

We think it is time for a new type of learning

environment, but what would this look like?

What would a truly digital apprenticeship look like?

Can we make better use of data to improve learning,

teaching and student outcomes?

How do we equip researchers and related staff with the skills

they need for the future of research?

The intelligent

campus The digital apprentice

Next generation

learning environment


1Discuss

emergingchallenges

2

Prioritise

ideas

3Announce successful

ideas

4Report progre

ss

Identify ideas

31st Oct – 24th Nov

4th Jan– 30th Jan 6th Feb Apr/May

Release 6 challenge areas and invite Jisc members and

other experts to discuss

Audience: managers,

consumers, some leaders, other

experts

Present ideas for activities Jisc

could do and ask members which

they support

Audience: managers,

consumers, some leaders



Audience: everyone who followed the

challenge



Audience: everyone who followed the

challenge

Contacts

Paul Bailey [email protected]

Further Information: http://www.analytics.jiscinvolve.org

Join: [email protected]


mailto:[email protected]

http://www.analytics.jiscinvolve.org/

http://www.analytics.jiscinvolve.org/

http://elevator.jisc.ac.uk/