Qualitative analysis: Qualitative analysis: Steps, Tips, and Traps to AvoidSteps, Tips, and Traps to Avoid

Laurie DrabbleLaurie Drabble

Feb, 2012Feb, 2012

Overall Approach to Analysis: Overall Approach to Analysis: PhenomenologyPhenomenology

Looks at the lived experience of the Looks at the lived experience of the phenomena of interestphenomena of interest

Analysis: Analysis: Identify descriptions of the Identify descriptions of the phenomenon; cluster into discrete phenomenon; cluster into discrete categories; taken together, these categories; taken together, these describe the “essence” or core describe the “essence” or core commonality and structure of the commonality and structure of the experience (p. 1373).experience (p. 1373). Starks & Trinidad (2007) Choose your method: A

comparison of phenomenology, discourse analysis, and grounded theory.

Process: InductiveProcess: Inductive The “story” of the research comes from The “story” of the research comes from

multiple reading & interpretation of multiple reading & interpretation of datadata

Purpose of this process:Purpose of this process: Condense and summarize extensive and Condense and summarize extensive and

varied raw datavaried raw data Establish link between research questions Establish link between research questions

and findingsand findings Develop a model (or theory) about the Develop a model (or theory) about the

underlying structure of experience or underlying structure of experience or processes processes (Thomas, 2003)

http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf

Overall Steps (Cresswell)Overall Steps (Cresswell)

1. Preparing and organizing the data (transcripts, field notes, etc.)

2. Reducing the data into themes through a process of coding and condensing the codes

3. Representing the data in figures, tables or a narrative

What is a theme?What is a theme?

A theme is a category of information A theme is a category of information that you as the researcher identify that you as the researcher identify and name and that may include a and name and that may include a variety of coded information. This variety of coded information. This involves a process of “coding” the involves a process of “coding” the data and then condensing or data and then condensing or classifying the codes into larger classifying the codes into larger families of themes. families of themes.

Cresswell, 2002

What’s the overall process?What’s the overall process? Qualitative analysis is an iterative Qualitative analysis is an iterative

process—meaning that a final analysis of process—meaning that a final analysis of the data is achieved by repeatedly the data is achieved by repeatedly reading and re-reading the data and reading and re-reading the data and repeatedly developing themes and repeatedly developing themes and refining themes. refining themes.

Developing themes involves a process of Developing themes involves a process of classifying, describing and interpreting classifying, describing and interpreting the data. the data.

The data are reduced into meaningful The data are reduced into meaningful “chunks” of information“chunks” of information

During this process, some information will During this process, some information will inevitably not be included in the analysisinevitably not be included in the analysisCresswell, 2002

Specific StepsSpecific Steps Transcribe data (important part of Transcribe data (important part of

“immersion”) & prepare for analysis“immersion”) & prepare for analysis Preliminary exploration Preliminary exploration

Close reading transcript(s)Close reading transcript(s) Making notes/recording ideas Making notes/recording ideas

Creating categories Creating categories General categories (using research aims)General categories (using research aims) Sort into themes & subcategories (via coding)Sort into themes & subcategories (via coding) Continue to revise and refine; select quotes Continue to revise and refine; select quotes

that capture the core categorythat capture the core category Interpret & report findingsInterpret & report findings

Review and describe themes (usually 3-8)Review and describe themes (usually 3-8) Use quotes to illustrateUse quotes to illustrate

A bit more about codingA bit more about coding Select a process to “sort” text segments Select a process to “sort” text segments

into categories (e.g. computer)into categories (e.g. computer) One segment of data may end up in two One segment of data may end up in two

categories; some may be uncategorized categories; some may be uncategorized (but keep “uncategorized” data to review(but keep “uncategorized” data to review

Within categories, looks for sub-topics, Within categories, looks for sub-topics, contradictionscontradictions

Categories can be combined when they Categories can be combined when they have similar meaninghave similar meaning

Expect to code across interview questions. Expect to code across interview questions.

Overview of process Overview of process

(Adapted from Creswell, 2002 as cited in Thomas, 2003)http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf

Initial readthrough textdata

Identify specificsegments ofinformation

Label thesegments of data tocreate categories

Reduce overlap/redundancyamong thecategories

Create a modelusing mostimportantcategories

Many pages of text

Many segments of text

30-40 categories

15-20 categories

3-8 categories

Share & TellShare & Tell

Examples of organizing dataExamples of organizing data Examples of codingExamples of coding Examples of themesExamples of themes Examples of how themes may be Examples of how themes may be

depicteddepicted

Ways to assess Ways to assess trustworthinesstrustworthiness

Independent coding (code from raw text)Independent coding (code from raw text) Coding consistency check (independent Coding consistency check (independent

coder given coding scheme and text coder given coding scheme and text segment…can calculate inter-rater segment…can calculate inter-rater reliability)reliability)

Stakeholder checks Stakeholder checks Summarize and correct and end interviewSummarize and correct and end interview Verify interpretations/date in later Verify interpretations/date in later

interviewsinterviews Share preliminary or near-final results Share preliminary or near-final results

(Thomas, 2003)http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf

IllustrationsIllustrations Collaboration between Child Welfare and Collaboration between Child Welfare and

Addiction Treatment in Canada (note-Addiction Treatment in Canada (note-taking, coding, themes & using quotes)taking, coding, themes & using quotes)

NCRP – Funding for Social Change in NCRP – Funding for Social Change in California (saturation, allowing, emerging California (saturation, allowing, emerging themes, avoiding bias, member-themes, avoiding bias, member-checking)checking)

Educating Students for Collaborative Educating Students for Collaborative Practice – 298 project (summarizing and Practice – 298 project (summarizing and reporting). reporting).

Project: Collaboration between Child Project: Collaboration between Child Welfare and Addiction Treatment: Welfare and Addiction Treatment:

Canadian ContextCanadian Context

Qualitative interviews - purposive Qualitative interviews - purposive sample of 24 professionals (12 CW, 12 sample of 24 professionals (12 CW, 12 AOD)AOD)

Research questions:Research questions: Contextual factors influencing policyContextual factors influencing policy Factors that facilitate/impede collaborationFactors that facilitate/impede collaboration Perceived opportunities for improvementPerceived opportunities for improvement

What we didWhat we did Initial review of Each looked at the a Initial review of Each looked at the a

sample of 2 AOD and 2 CW interviews, sample of 2 AOD and 2 CW interviews, made separate notes about what we made separate notes about what we saw as provisional codes saw as provisional codes

First major round of coding, that First major round of coding, that generated about 40 categoriesgenerated about 40 categories

Used consensus process to sort through Used consensus process to sort through initial categories and modify codinginitial categories and modify coding

Organized codes into bigger “themes”Organized codes into bigger “themes”

The Analysis Process: The Analysis Process: Inductive Inductive

1.1. Initial “sorting” by interview questionsInitial “sorting” by interview questions Context, Help/Hinder, OpportunitiesContext, Help/Hinder, Opportunities

2.2. Initial “open-coding” to Initial “open-coding” to Conceptualize, Conceptualize, Compare,Compare, Categorize dataCategorize data

3.3. Iterative process to create then revise Iterative process to create then revise over 40 categories categories, which over 40 categories categories, which eventually became our “themes.”eventually became our “themes.”

What Helps?: Cross-cutting What Helps?: Cross-cutting Enabling FactorsEnabling Factors

Shared purpose and Shared purpose and mandatemandate

Respectful relationships Respectful relationships

Leadership (individual or Leadership (individual or group advocates)group advocates)

Knowledge translation Knowledge translation

Knowledge TranslationKnowledge Translation

““When I came here, people weren’t When I came here, people weren’t talking about outcomes and talking about outcomes and benchmarks and things like that. benchmarks and things like that. Now, they’re starting to move in that Now, they’re starting to move in that direction, and [asking] how do you direction, and [asking] how do you improve your outcomes and what are improve your outcomes and what are we measuring?...That’s working.”we measuring?...That’s working.”

What Helps: Program and What Helps: Program and Practice InnovationPractice Innovation

Proactive safety and relapse Proactive safety and relapse prevention supportprevention support Focus on safety & relapse Focus on safety & relapse

preventionprevention ““not closing the door” on not closing the door” on

familiesfamilies Adoption of specific innovative Adoption of specific innovative

programsprograms

Focus on safety & Focus on safety & relapse preventionrelapse prevention

“ “What it’s designed to do is really What it’s designed to do is really to have the kind of three-way to have the kind of three-way agreements between addictions agreements between addictions services, child protection, and the services, child protection, and the client in taking a look at yes, client in taking a look at yes, relapse may happen. What are the relapse may happen. What are the kinds of things that I as a parent kinds of things that I as a parent need to put in place in order to need to put in place in order to ensure the safety of my child?”ensure the safety of my child?”

Cross-cutting enabling factors

•Shared purpose•Leadership•Respectful relationships•Knowledge translation mechanisms

Principles and Values•Working from principles and values•Translating principles and values into practice guidelines•Animating principles in collaborative planning

Processes and Protocols Planning in partnershipCommunication protocols and guidelinesMechanisms for conflict resolutionContinuous learning

Program & Practice Innovation

•Proactive support for safety and relapse prevention/ reducing harm•Innovations in practice as forming a nexus for collaborations

Shared Outcomes•For clients: Child safety, healthy mothers and families•For systems: Continued improvements, attention to social determinants, increased community level supports, new approaches for Aboriginal mothers

Reflective of specific contexts

Figure 1: Summary of factors that facilitate collaboration between systems

Illustration : NCRP: Funding Illustration : NCRP: Funding for Social Change in for Social Change in

CaliforniaCalifornia PURPOSE: Examine the role of PURPOSE: Examine the role of

progressive and mainstream foundation progressive and mainstream foundation funding in structuring, influencing, funding in structuring, influencing, enabling or constraining the policy and enabling or constraining the policy and advocacy work of nonprofitsadvocacy work of nonprofits

METHOD: Telephone interviews with METHOD: Telephone interviews with representatives from 73 nonprofit representatives from 73 nonprofit organizations involved in state level organizations involved in state level public policypublic policy

““Gaps” in the Public Gaps” in the Public Policy Nonprofit Policy Nonprofit

LandscapeLandscape Need for greater strategic planningNeed for greater strategic planning Foundation hesitance to fund policyFoundation hesitance to fund policy Confusion about policy work (what it Confusion about policy work (what it

is and who does it?)is and who does it?) Conflict between mission and moneyConflict between mission and money Tension between research Tension between research

organizations/think tanks and organizations/think tanks and grassroots organizationsgrassroots organizations

Assessing Trustworthiness: Assessing Trustworthiness: Member CheckMember Check

Discussion groups with 20 nonprofit Discussion groups with 20 nonprofit and foundation representatives and foundation representatives (Northern and Southern CA)(Northern and Southern CA)

Purpose…Purpose… Validate initial findingsValidate initial findings Provide input for interpretationProvide input for interpretation

Added Study Component & Added Study Component & Report ComponentsReport Components

Interviews with 8 foundation Interviews with 8 foundation representativesrepresentatives How foundation defines policy related How foundation defines policy related

grantmakinggrantmaking Advantages of funding policy workAdvantages of funding policy work Barriers overcome in order to fund the Barriers overcome in order to fund the

policy work of nonprofitspolicy work of nonprofits Lessons learned for other foundationsLessons learned for other foundations

Added “success” case storiesAdded “success” case stories

Driving forces for Driving forces for foundation support of foundation support of

policypolicy Leveraging larger results for issues Leveraging larger results for issues

areas and communitiesareas and communities Creating a voice for those most Creating a voice for those most

impactedimpacted Increasing community assets and long Increasing community assets and long

term capacityterm capacity Building a multifaceted response to Building a multifaceted response to

community problemscommunity problems

Tips for writing resultsTips for writing results

Organize findings by research questionsOrganize findings by research questions Do NOT organize by interview guide Do NOT organize by interview guide

questions.questions. Name, summarize the theme (and Name, summarize the theme (and

subcategories), illustrate with quotes.subcategories), illustrate with quotes. Consider providing a visual that Consider providing a visual that

captures the “big picture.” captures the “big picture.”

Cross Cutting Themes Themes related to Values, Systems, Outcomes & Service Planning

Knowledge of family violence Dynamics of family violence Culturally competentAdaptability Creative problem solving Open to new systems and

perspectives FlexibilityCommunication Skills Listening skills Negotiation skills Engaging clients/ colleagues Finding your voice at the

collaborative table

Values Demonstrate respect Know personal and professional valuesSystems Understand and know the systems of

each role and partner Understand differences between

systemsOutcomes Define/ create attainable outcomes Reviewed shared outcomesService planning Knowledge of resources Knowledge of safety planning Ability to process resources/ prioritize

services

Sample Summary (298 project) Sample Summary (298 project) Visual– What students need to Visual– What students need to

learn for collaborationlearn for collaboration

Learning Strategies for Learning Strategies for Developing Competence in Developing Competence in

Collaborative PracticeCollaborative Practice

Community Projects

Field Experience

Mentorship Reflection

Classroom Learning

Observation

SocializingNetworking

Learning Strategies