olat log analysis

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Outline

A Log-based Learning Content Creation (Part I)OLAT Course Log Analysis

Yi GuoSupervised by: Prof. Harald Gall

Universitat ZurichInstitut fur Informatik

SEAL IFI Soft Talks, 2009

1 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

1 IntroductionMotivationObjective

2 Mapping Log To MXMLRaw LogsMXML FormatMapping

3 AlgorithmsProcess Mining OverviewHeuristic MiningFuzzy Mining

4 Case Study

5 Discussions

2 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

MotivationObjective

Motivation

Requirement of Legacy LMS

The monitoring solution of legacy LMS is incomplete

To analyze course activities it is necessary to correctly setup the data recording when creating a new OLAT course.

— OLAT 6.1 User Manual

Abstract the course schema from course contentsgo to olat courses table

Next generation e-learning courses need clearer reference andschema

3 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

MotivationObjective

Objective

Have an accurate view of the ”learning patterns”

Construct a clearer model of user behaviors

4 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Raw LogsMXML FormatMapping

Raw Logs

5 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Raw LogsMXML FormatMapping

MXML Format

Figure: MXML Class Diagram

6 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Raw LogsMXML FormatMapping

Mapping

Assumptions

1 Singleuser

2 Singlesession

3 No noisydata

Figure: OLAT Log and MXML7 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

Process Mining

software system

process/systemmodel

eventlogs

modelsanalyzes

discovery

records events, e.g., messages,

transactions, etc.

specifiesconfiguresimplements

analyzes

supports/controls

conformance

“world”

people machines

organizationscomponents

business processes

verification

Figure: Process mining: link logs to models

8 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

A Heuristic Algorithm

Advantage

Less sensitive for noise and the incompleteness of logs

can handle some limitations of the α-algorithm

9 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

Construction of the dependency/frequency table

A, B: sample event

B #B #B<A #A>B $A→L B $A→ B

Metric Calculation

$A→L B = (#A>B −#B>A)/(#A>B + #B>A + 1) (1)

$A→ B = $A→L B × δn (2)

δ: fall factorn: the intermediary event number

DS(X ,Y ) = (($X →L Y )2 + ($X → Y )2) (3)

10 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

Dependency/Frequency graph

Figure: D/F graph

11 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

Fuzzy Mining

Why Spaghetti-like ?

1 Less-structured process2 2 assumptions

1 reliablity2 existence

12 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

Fuzzy Mining Algorithms

3 principles

1 Aggregation

2 Abstraction

3 Emphasis

Metric Matrix

Unary Binary

Significance Frequency FrequencyRouting Distance

Correlation x Proximityx Endpointx Originatorx Data Typex Data Value

Table: Metric matrix

13 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Process Mining OverviewHeuristic MiningFuzzy Mining

Result

Figure: A fuzzy graph example

nodes incluster 32

14 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Data Collection

Course Name Event No. Inst No. Time

CareOL CBZ Home 1329 111 2009eCF Basic I 623648 585 HS08eCF Advanced II 97551 427 FS08GEO 112 Humangeographie I 49794 278 2007-2009PTO - Psychologie Taught Online 441126 1286 2008-2009Sprachliche Interaktion im Raum 2243 25 2009

Table: Courses collected from University of Zurich

15 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Corporate Finance II

Figure: eCF II Schema Figure: eCF Fuzzy models

16 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Algorithm Improvement

Mapping log case to process instance supporting collaborativelearning activities

Supporting multiple sessions

Result Evaluation

What are proper thresholds?

Result Application

How to reflect the analysis result to the course creation

Other Perspectives

Social network

Performance analysis17 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions andsuggestions?

Thank you!

18 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions andsuggestions?

Thank you!

19 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

20 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

21 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

22 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

23 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

24 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

25 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

26 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

27 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

28 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

29 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

30 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

31 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

32 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

33 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

34 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

35 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

36 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

37 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

38 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

39 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

40 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

41 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

42 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

43 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

44 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

45 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

46 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

47 / 48

IntroductionMapping Log To MXML

AlgorithmsCase StudyDiscussions

Discussions

Questions and suggestions?

Thank you!

48 / 48