Nov 9th 2006

Determinants of Engagement in an Online Community of Inquiry

 Jim Waters

College of Information Science and Technology

Drexel University

Philadelphia

james.waters@drexel.edu

Nov 9th 2006

Background

• Problem of maintaining student engagement.

• Online learning creates separation

• Alienation, lack of commitment and antisocial behavior ?

• Community of Inquiry ?

Nov 9th 2006

• Pragmatism: Dewey and Addams

• Problematic situation, scientific attitude and community as participatory democracy

• Inquiry is controlled or directed transformation of an indeterminate situation

• There is a community engaged in inquiry. Inquiry is an open-ended process with positive feedback.

Dewey (1916,1933)

Nov 9th 2006

Community of Inquiry

Garrison et al 2000

Nov 9th 2006

Cycles of Inquiry

Garrison et al 2000

Nov 9th 2006

Building on the Garrison et al Model

• Content Analysis of online Discussion Board

• Graduate Information Systems Students

• Open-ended debate

• Practical and Theoretical questions

• Derived behaviors that incorporated different elements of the Garrison model

Nov 9th 2006

Student RolesRole Analogy Main Behavior Types

Initiator Spider Social

Facilitator Middleman Social, Teaching

Contributor Journeyman Social, Cognitive

Knowledge-eliciter Seeker Social, Cognitive

Vicarious-acknowledger

Me-too Social, Cognitive

Complicator Reframer Teaching, Cognitive

Closer Synthesizer Social, Teaching, Cognitive

Passive-Learner Freeloader Cognitive

Waters and Gasson 2005

Nov 9th 2006

Research Questions

1. Are there noticeable patterns of interactions between participant roles?

2. Do patterns of interaction change over time?

3. Does the online learning environment support critical inquiry ?

4. What interactions generate greatest student engagement

Nov 9th 2006

Study Post-Hoc analysis of online learning archive:

10 week graduate IS Management course at a US university

23 students, experienced professionals & managers.

3 - 4 open-ended questions posted to discussion board weekly:

1063 discussion-board messages

951 student responses (analyzed)

112 instructor postings (not analyzed).

Content analysis of postings and responses:

Each student contribution message assigned to single response type, reflecting dominant mode of behavior.

Nov 9th 2006

Raw results

25,937 individual reads of discussion board message (range 331 – 2179 reads per student)

951 student postings (range 1 – 154 per student)

Most active period weeks 1 & 2 (157 posts and 162 posts)

Then steady pattern of ~ 70-80 posts per week.

Nov 9th 2006

Student behavior Contributor (61%)

Facilitator (22%)

Fluid patterns of class behavior

Students adopt different behaviors from week to week

Popularity and volume were unrelated

Possible connection between facilitation and popularity/reference to poster.

Nov 9th 2006

Detailed Analysis

• Nine typical threads analysed• Three threads each for weeks 3, 6 and 9• The most productive debate produced 30 messages

with a maximum thread depth of 7. • The least productive produced 14 messages with a

thread depth of 2. • The mean number of messages on a discussion was

22• Four discussions had a thread depth of greater than 3. • Pattern of responses analysed

Nov 9th 2006

Are there noticeable patterns of interactions between participant roles?

From To Frequency Percentage

Acknowledger Contributor 3 1.7%

Eliciter Contributor 4 2.3%

Complicator Contributor 4 2.3%

Facilitator Faculty 4 2.3%

Closer Faculty 6 3.4%

Complicator Faculty 6 3.4%

Complicator Facilitator 7 4.0%

Facilitator Contributor 16 9.2%

Facilitator Facilitator 18 10.3%

Contributor Faculty 106 60.9%

TOTAL 174 100.0%

Nov 9th 2006

Senders

Acknowledger 3 1.7%

Eliciter 4 2.3%

Closer 6 3.4%

Complicator 17 9.8%

Facilitator 38 21.8%

Contributor 106 60.9%

Faculty

Total 174 100%

Receivers

Acknowledger 0 0.0%

Eliciter 0 0.0%

Closer 0 0.0%

Complicator 0 0.0%

Facilitator 25 14.4%

Contributor 27 15.5%

Faculty 122 70.1%

Ratio of receive to send Contributor = 27/106 = 0.25Facilitator = 25/38 = 0.65Complicator = 0/17 = 0.00

Nov 9th 2006

Do patterns of interaction change over time?

Week 3 (n = 63) Week 6 (n=51)

Week 9 (n = 60)

Nov 9th 2006

Does the online learning environment support critical inquiry ?

Stahl 2006

Muukkonen et al 1999

Nov 9th 2006

Does the online learning environment support critical inquiry ?

• Few threads reached a definitive conclusion• Closer synthesizes and ends debate

– Closer often ignored• Elements found

– Information Gathering – Synthesis – Concrete experience – Reflective observation. – Critical evaluation– Deepening questions– Generating subordinate questions – Refining given knowledge– Generating hypotheses

• Open-ended debate ?• Not problem centered ?

Nov 9th 2006

What interactions generate greatest student engagement

• Analysis of all 951 student messages • Analysis of Read frequency for different message

types• Knowledge-elicitation messages (asking

questions) generated significantly more (24) reads pre message than any other type of message.

• Average reads per message for all messages is 16.78

• Some participants messages are read more frequently than others

Nov 9th 2006

Who are the most attended to posters ?

Nov 9th 2006

Why are some posters more engaging ?

• Does frequency of posting messages affect popularity?

• Does length of message affect frequency of reads? • Does position of messages affect frequency of reads ?• Does type of “participant” affect frequency of reads ?

Nov 9th 2006

Is frequency of posting related to popularity?

• Correlation between number of messages and total reads of a persons messages is 0.97,

• Weak -0.21 correlation between frequency of posting and reads/message.

• Most frequent poster posted 136 messages which attracted an average of 15.65 reads per message.

• The average messages per person was 37• Top three most attended to participants posted an

above average number but subject 20 did not. • Two of the least attended to participants posted

well above average numbers of messages.

Nov 9th 2006

Does length of message relate to read frequency

• Correlation between length of post and reads for that post = 0.011

• Grouping messages into very short (< 101 words), Short (101—200 words), medium (210—300 words) and long (>301 words)

• One-Way ANOVA on frequency of reads gives an f value of .373 and a significance level of .773, no apparent significant effect

Nov 9th 2006

Does position of message affect frequency of reads

• Messages posted in the first 2 days of a thread are read significantly more frequently (f=36.339, p= 0.000) than later messages.

• Messages posted after the third day are read by less than 50% of participants.

• If a message is one of the first 10 posted it is much more likely to be read than later messages (f=22.564, p = 0.000).

• However only two of the most attended to participants are “early” posters.

Nov 9th 2006

Does type of participant affect frequency of reads

• The most attended to participants posted more facilitation messages (39% of messages posted)

• The least attended to participants typically posted far fewer facilitation messages. (23% of messages posted).

Nov 9th 2006

Conclusions

• Peer Facilitation does work

• Students quickly identify valuable contributors

• Early stages crucial

• Changing Contributor to Facilitator

• Identification of thought leaders

• Asking questions gets responses

• Fluid patterns of behavior within the community

• Volume is not the same as quality

Nov 9th 2006

Future Work

Small, exploratory study Initial framework Open to debate

Limitations

Influence of prior online learning-experience on patterns of behavior

Larger sample size

Deeper analysis of content

Explore vicarious learning contributions more fully

Explore why patterns change

Compare ill-defined vs. well-bounded questions.

Nov 9th 2006

Questions?