Peter Shea

Jason VickersSuzanne Hayes

University at Albany

Revisiting the Community of Inquiry Framework of Online Learning

Outline

Introduction Status of CoI Research Overview of project phases and design Findings and Implications

Changes in Higher Education

Shift underway in undergraduate education from providing classroom instruction to producing learning

With growth in online learning raises parallel question “What matters in online learning?”

CoI is the widely accepted framework for explaining and predicting how people learn in online environments in the absence of face to face instruction

Community of Inquiry Framework

Social PresenceThe ability of participantsto identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships by way of projecting their individual personalities.

Cognitive PresenceThe extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry.

Teaching PresenceThe design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes

Current CoI Research

Focused in two areas: Large scale surveys of student perceptions

of SP, TP and CP Quantitative content analysis of online

discussions

What is QCA?

Research methodology used to examine transcripts of online interactions to see what participants say to each other

Use SP, TP and CP codes to classify these statements

Results used to look for patterns within course

How has QCA been used?

Have focused primarily on threaded discussions

Has constrained us from seeing the broader picture of activity within a course

TP in threaded discussions only accounts for 20-30% of TP activity

Some limitations of past research using QCA in CoI

Early studies relied on samples of threaded discussions

No studies examining an entire online course Most studies focus on one form of presence at a

time Many studies based on a single course Early studies did not fully document IRR

measures

Our purpose

Address prior methodological shortcomings Use a holistic approach Explore expansion of CoI framework to make it

more encompassing and descriptive of all online course activities

Determine if previous research results can be verified, refined or extended

Improve and enhance CoI model and its constructs

Research Design

Two identical courses Based on template developed by content experts

and instructional designers Instructors were not the developers Five modules with discussions, mini lectures,

case studies, scaffolded research project submitted in stages

Upper level business management course from a state college in northeast

Discussion: 25% of the final grade

Debates: 10%

Other Course Activities: 65%

Purposive Sample

Four sections offered in Fall 2007 Two selected based initial assessment to

identify instructors with different approaches Classes comparable in size

Instructor A Instructor B

Students enrolled at start of the term 19 20

Students completing course 17 16

Data Analyzed in Multiple Phases

Phase One: May 08 – March 09 Online discussions in each of the 5 main course

modules in each course (SP, TP, CP)

Phase Two: Jan – June 09 Examined student case studies for CP & SOLO

taxonomy (Biggs and Collis, 1982) to measure high order thinking

Phase Three: Aug 09 + Coding for SP and TP* that take place in course

communications (outside of online discussions) Coding of all course documents for SP and TP*

* CP coding not meaningful in this context

P1

P2

P3

CP SOLO

Case Study Mod 1 X X

Case Study Mod 2 X

Case Study 3 X X

Case Study 4 X X

Description of Term Project    

Research Paper Outline    

Final Research Paper    

Discussions SP TP CP

Module 1 X X X

Module 2 X X X

Module 2 Debate** Coding Underway

Module 3 X X X

Module 4 X X X

Module 5 X X X

SP TP CP

Orientation & Syllabus X X X

Introduction Module (w/ Icebreaker) X X

Module 1 X X  

Module 2 X X  

Module 3 X X  

Module 4 X  

Module 5  

Course Instructions & Documents

Course Material CodedP1

Communicative DocumentsP3

P3

P2Course AssignmentsDiscussions

Coding Process

Paired coders Coded random module for practice Established and documented

guidelines to assure consistency Coded target modules Met to negotiate, identify and resolve

disagreements, when possible Recorded IRR prior to and following

negotiation

Coding Process*

Message as Unit of Analysis

Recorded presence/absence of category e.g. for SP we marked AF, OC, CH or NC

Did not count occurrences of each indicator within each message

Discussions & Course Communications

Whole paper as Unit of Analysis in SOLO

Paragraph as Unit of Analysis in CoI course documents coding(presence/absence of category e.g. for SP we marked AF, OC, CH or NC)

Did not count occurrences of each indicator within each message

Assignments &Course Documents

*Coding for only CoI and SOLO in discussions and case studiesrequired more than 6300 individual decisions.

Cognitive Presence Coding Sheet

Phase Code Indicator Socio-Cognitive Process

Triggering Event CP-TE-1 Recognize problemPresenting background information that may culminate in a question or presents a problem/issues

  CP-TE-2 Sense of puzzlementAsking questions or Messages that take discussion in a new direction

Exploration CP-EX-1Exploration within the online community

Unsubstantiated agreement or disagreement/contradiction of previous ideas

  CP-EX-2 Exploration within a single message Many different ideas/themes presented in one message

  CP-EX-3 Information exchange Personal narratives or description

  CP-EX-4 Suggestions for consideration Author explicitly characterizes message as exploration

  CP-EX-5 Leaps to conclusions Offers unsupported opinions

Integration CP-IN-1Integration among groups members; Building on, adding to others' ideas

Reference to previous message followed by substantiated agreement or disagreement (I agree/disagree because…)

  CP-IN-2Integration within a single message (response to prompt) Justified, developed, defensible, yet tentative hypotheses

  CP-IN-3 Connecting ideas, synthesisIntegrating information from one or more sources - textbook, articles, personal experience, other posts or peer contributions.

  CP-IN-4 Creating solutions Explicit characterization of message as a solution by participant

Resolution/Application CP-RE-1

Vicarious application to real world testing solutions; Providing examples of how problems were solved  

  CP-RE-2 Defending solutions Defending why a problem was solved in a specific manner

Solo Taxonomy Coding

0 = no response

1= prestructural: misses the point

2 = unistructural: minimal response

3 = multistructural: source facts only

4 = relational: substantiated positions

5 = extended abstract: higher level of abstraction

Used low and high indicators for levels 2-4. 

Inter-Rater Reliability Reported Two Ways

Cohen’s kappa Corrects for chance or

random agreement between 2 coders

Conservative May be artificially low

(even with high level of agreement)

1.0 perfect agreement; .75+ excellent; .40 poor

Measures the extent to which two or more coders agree

Holsti’s Coefficient Expressed as a

percent Less Conservative Minimum of 80%

required 90% is excellent

Findings: Locus of TP

Not just limited to threaded discussion Majority of TP found outside of discussion in

communicative processes such as: Course e-mail Private folders Comments on written assignments

Distribution of Teaching Presence within Each Course

Findings: CP Measures in Discussions Consistent with prior research Majority of student postings are at exploration

stage Low levels of integration and resolution

Findings: SOLO Measures in Case Studies

SOLO: used because it measures outcomes instead of process

Students did not "excel" on these measures (i.e. reach 4s and 5s)

Student did "average" or slightly below (2s-3s)

Mismatch between culture of grading and the ideals of SOLO framework

Findings: SOLO Measures in Case Studies cont.

However, SOLO scores combined with CoI metrics do explain a majority of variance in teacher assigned grades

The two models combined therefore show promise in describing and explaining learning in online environments - prime goals of a theoretical framework

More research using this model is needed

Alternate models for measuring outcomes should be considered

Implications

Design of course learning activities needs to be well integrated to encourage: integration of public discourse in discussion AND documentation of student’s learning in their

private written artifacts When this does not occur there is limited

opportunities for students to draw associations or reinforce course concepts to learning activities

Findings

Disconnect between instructor’s grades and student learning outcomes based on SOLO and CP

Failure of meaningful learning or difference Who’s wrong: The instructor or the

framework?

Implications

Failure of meaningful learning? Difference in cultures of measurement

between researchers and instructors?

Thank you!

Peter SheaJason Vickers

Suzanne HayesUniversity at Albany