indicators project @ cquniversity

Ken Clark and Colin Beer

How it began

http://farm4.static.flickr.com/3196/2804845277_1829975f23_o.jpg

http://farm4.static.flickr.com/3579/3677384272_d79654abb6_o.jpg

What is it?• Create a project database• Data-mining for user behaviour• Examine the data according to some

models• Create something useful for staff, students

and administrators• Develop a model to benchmark the LMS

against the seven principles

Who is involved?

http://farm3.static.flickr.com/2257/3534516458_48e4e8595f_b.jpg

Who to blame for the Indicators Project

Ken Clark (FABIE) Colin Beer (Curriculum Design Unit)

Thanks also to:Rolley TicknerNathaniel Fitzgerald-HoodDavid Jones

http://farm4.static.flickr.com/3111/2402890105_27d2bd2377_b.jpg

How can the Indicators data be used?

• Academic ReflectionTeacher Reflection

http://farm3.static.flickr.com/2017/2206733790_ac6328b1c8_b.jpg

http://farm1.static.flickr.com/113/286709039_105881e4b9_o.jpg

Benchmarking

Frameworks

http://farm2.static.flickr.com/1106/1387142898_807e2ef311_o.jpg

Frameworks

http://farm2.static.flickr.com/1106/1387142898_807e2ef311_o.jpg

Dawson & Heathcoate

Frameworks

http://farm2.static.flickr.com/1106/1387142898_807e2ef311_o.jpg

Dawson & Heathcoate

Malikowski

Seven principles of good practice in undergraduate

education(Chickering & Gamson)

http://farm2.static.flickr.com/1119/1328426678_1b31dfb17d_o.jpg

Student engagement, activity and grades

http://farm4.static.flickr.com/3616/3407202210_ef9aa8257d_o.jpg

Staff engagement and activity

http://farm1.static.flickr.com/96/224456132_12a1791aea_o.jpg

Data sources

http://farm4.static.flickr.com/3045/2920562020_e808543f0b_o.png

Indicators Data Sources

Blackboard PeopleSoft• 5147 Courses• Approx 300,000,000 database rows• 184,000,000 hits

• 2,065,000 records in total• 135,584 Individual Student Results

http://www.flickr.com/photos/zigazou76/3636704534/

“Utilising a systems view to codify designer and user behaviour is ‘indistinct’, but can play in the refinement, ratification and benchmarking of broader evaluation strategies.” (Heathcote & Dawson 2005)

Limitations!

Ongoing use of the process of reflection is essential for building knowledge, and increasing knowledge increases one’s ability to use reflection effectively and to develop as a teacher.(McAlpine & Weston, 2004)

http://farm3.static.flickr.com/2017/2206733790_ac6328b1c8_b.jpg

A model for research into course management systems that equally considers technical features and research into how people learn.

(Malikowski et al., 2007)

The Malikowski Model

Feature adoption over time at CQUni

Feature adoption over time at CQUni

A tool for reflection

Average hits on Blackboard by student grade

Sample size = 311,025 students

Average hits on Blackboard by student grade

Flex (70690)

Total (311025)

Average hits on Blackboard by campus

Average forum reads by grade

Flex (70690)Total (311025)

How would you interpret this?

AVG Files VS AVG hits on the course site

Why benchmark?

Benchmarking

http://farm3.static.flickr.com/2352/2364272931_61a6c53b85.jpg?v=0

A critical lens that we can use to evaluation teaching and learning in Learning Management Systems

The Seven Principles

1. Encourages Contacts Between Students and Faculty

2. Develops Reciprocity and Cooperation Among Students

3. Uses Active Learning Techniques

4. Gives Prompt Feedback

5. Emphasizes Time on Task

6. Communicates High Expectations

7. Respects Diverse Talents and Ways of Learning

Good Practice:

Chickering, A. W., & Gamson, Z. F. (1987)

Summary

• Provide tools and information to aid in teacher reflection

Summary

• Provide tools and information to aid in teacher reflection

• Develop a benchmarking model for LMS evaluation based on the 7PGPUE

Summary

• Provide tools and information to aid in teacher reflection

• Develop a benchmarking model for LMS evaluation based on the 7PGPUE

• Opportunity for publications

Summary

• Provide tools and information to aid in teacher reflection

• Develop a benchmarking model for LMS evaluation based on the 7PGPUE

• Opportunity for publications

• Ken and Col’s Masters project.

http://farm4.static.flickr.com/3309/3424831795_5bf50cb49c_b.jpg