participatory multi-regional qualitative research: methods ... · pdf...
TRANSCRIPT
Participatory Multi-Regional Qualitative Research: Methods
and Models
Carol S. Camlin and Nicolas Sheon
University of California at San Francisco (UCSF)
Summer Institute on Migration and Health
24 July 2015, Oakland, CA
Workshop Agenda
1. Multi-regional participatory qualitative research: scientific
and political rationale (1:00-1:10)
2. Problems/challenges in doing collaborative cross-site
qualitative work (1:10-1:30)
• How each of the products do or don’t address the challenges
1. One example: SEARCH Qualitative Study (1:30-2:00)
• Training content: initial and periodic
• Process, tools, methods
2. Advice/ trouble-shooting participants’ study concepts, plans
or challenges (2:00-2:25)
• Different project management models
Multi-regional participatory qualitative research
• What is it?
• Many forms and models (CBPR, etc.), but principles are to involve
communities under research to participate in that research
• Can involve hiring local people and training them to conduct
research – not just data collection but ALSO data interpretation, i.e.
all levels of staff participate in production of scientific knowledge
• Can be time-intensive: training, mentorship, capacity-building
• Why do it? Ethical and scientific reasons
• Better science: Scientific validity demands cultural competence
• ‘Local’ researchers can especially draw upon culturally relevant
theory and experiences in the interpretation of findings, understand
contexts, and develop trustworthy communication (language)
• Builds team motivation, morale and investment in study success
• Promotes global north-south equity in research processes
Challenges of conducting
collaborative cross-site
qualitative research
Single Investigator Qual Project
• Collect interviews
• Transcribe
• Code: find and categorize segments
• Search for relevant and illustrative segments
• output best segments to word
• Write report/article
Opportunities and Challenges in
Collaborative Qualitative Projects
• Triangulation of various cross-cultural differences such as
• types of linguistic and cultural expertise
• disciplinary perspectives on discourse
• interpretive approaches
• Identifying and resolving discrepancies in definition and application
of codes (political as well as scientific question)
• Assessing and reporting inter-coder agreement
• Division of coding labor and write up of results
Division of Labor • Data collection
• Coding
• Checking coder agreement
• Merging of coding by multiple coders
• Analysis/Summaries of cases/themes
• Searching and Retrieving relevant data
• Assignment of manuscript topics
• Writing and submitting manuscripts
Deciding on collaborative coding strategy
• Broad vs. Detailed Codes
• Index vs. thematic coding
• Grounded vs. a priori codes
• What is the unit of analysis?: Lumpers vs. Splitters
• Overlapping vs. exclusive coding
• Coding on paper vs. coding with software, or some combination
• Meet often, review double coded transcripts and resolve coding discrepancies
• Who will merge coding and have the final say on code definitions and their application?
Supervision and training
of interviewers/transcriptionists
• Should begin during pilot phase
• Are interviewers able to adequately explain and contextualize questions?
• Are you getting rich data to answer the research question?
• Are initial responses explored adequately?
• Are you including enough detail in the transcripts?
• Do you need to revise the interview guide? It should evolve with the research questions.
Qualitative software can help
• Ensure that all transcripts are up to date
• Ensure that all code lists are up to date
• Merge coding work done by multiple coders
• Provide an audit trail - who added what, when
• Supervise coding work in real time
atlasti.com
• $670 per educational license/$99 for 2 years per student license
• Has excellent text search, query, and geocoding functions
• Windows and Mac Versions not interoperable at the moment (v. 7.5)
• Merging HU’s is done manually and requires care and expertise
• Based in Berlin. Support from ATLAS is not great.
• Does NOT require internet connectivity
dedoose.com • Browser/Cloud based (works the same on mac and
windows) – requires internet connectivity
• Designed for mixed methods (e.g. use survey data to add variables to filter qualitative responses.
• Merging of work by multiple coders is automatic.
• All contributions to the database are signed and time stamped.
• Pay as you use it. Monthly fee of $9/user.
• Based at UCLA - very responsive support
ATLAS.ti vs. Dedoose
ATLAS.ti Dedoose
Mixed methods support No Yes
Search text database Yes No
Cost per user $670 $9/month
Calculate Coder Agreement No Yes
Internet Required No Yes
Password Protection No Yes
One example: the SEARCH trial Qualitative Evaluation Study
Sustainable East Africa Research in Community Health (SEARCH): http://www.searchendaids.com/
• SEARCH will test the hypothesis that anti-retroviral therapy (ART) treatment of all persons (HIV “test and treat”) can bring a community's collective viral load to 0 to stop the spread of HIV
• and can improve overall health, education and economic productivity of the community
• includes 32 communities of roughly 10,000 persons each, in Uganda and Kenya
Overall purpose of the qualitative evaluation of SEARCH
To evaluate how the introduction of universal HIV testing
and treatment (i.e., “test and treat”) influences beliefs,
attitudes and behaviors regarding HIV and, in turn, how
these changes influence the uptake and success of “test
in treat” in East Africa
To understand why the intervention works, how it works
in different settings, and why it failed if it does in some
communities
Qualitative Data Analysis
Approach: • Data collection designed on basis of implicit hypotheses re:
social & implementation processes that may influence how SEARCH achieves its aims
• We explore evidence for rather than test hypotheses
• We pay close attention to contradictory evidence that
suggests need for adaptation of theory or development of new theory
8 communities (matched)
Tom Mboya & Sena
Ongo & Othoro
Kazo & Nyamuyanja
Kameke & Kadama
Team Structure
• Three local study teams: one per region • Project Coordinator
• Qualitative Research Coordinator
• Qualitative Research Assistant
• Team building
• Research capacities • Capacity building activities
• Language capacities
Training content (beyond study-specific) • Purpose and value of qualitative research
• Types of research questions that can be answered with qual methods
• Qualitative data collection methods • In-depth interviews, Focus Group Discussions (roles and skills in facilitation
and recording), Participant Observation (including preparation of field notes)
• Data collection skills training • Principles of ethical conduct in human subjects research
• Establishing rapport, maintaining appropriate boundaries, effective question types/ spontaneous probes, projecting neutrality, handling difficult participants, ethical quandaries; managing group culture in FGD dynamics, ensuring balanced participation, etc.
• Data analysis skills training • Using theory, using software, from data collection to interpretation
A snapshot: culmination of data analysis training, right
before software training
Qualitative Data Analysis • process leading up to publication?
• Reading scientific literature, learning, observing,
listening, and thinking about the research topic… so
that you begin with some expectations of what the data
may reveal
• Carrying out data collection using guides developed to
explore the topic
• Reading transcripts, with an open mind, alert to findings
that contradict your assumptions and to new,
unexpected information
• Coding transcripts
• Developing code reports and analytical memos to
organize findings by key themes
Qualitative Data Analysis
Intro. to Atlas.ti
Preparing & Coding
Transcriptions
A process: Snapshots 1. Get transcripts ready
2. Read transcripts
3. Develop list of codes
4. Load and code in Atlas.ti
Process: Snapshots 5. Generate code reports
6. Write analytical
memos
7. Write &
publish
findings
Communication Structure
• Expectations named in preliminary training
• Weekly team meetings on (Webex, Skype, GoToMeeting)
• Local RC & RAs, UCSF PI & RC
• Poor internet connectivity sometimes a challenge
• Full team in-person meetings 2x per year
• Team building
• Training
• Shared document server
Clear Document Structure
• Weekly updated documents
• Meeting minutes
• Progress reports
• Server
• Consistent file structure
• Naming conventions established for all file types
Workplan
Weekly Progress Reports
Team Coding Process
• UCSF maintains updated Atlas.ti HU
• New data files added at UCSF
• Copy bundles updated and distributed to teams with new coding
assignments
• Merging process for all coded team data
• Coding process
• Master code list developed by research team
• Limited addition of new codes informed by qualitative data or new
quantitative SEARCH needs – iterative process
Team Coding Process
• Coding assignments distributed every few months
• 6-10 transcripts to code per person per round
• Coding prioritized for local teams who have collected data (also
UCSF)
• Tracking in Excel
• Reports automated in Stata
Tracking Team Coding
Group Analysis
• Project-dependent • Products identified for UCSF lead
• New concepts proposed by team members • Baseline: 2 manuscripts being led by local team
• Group thematic analysis and discussion at in-person team meetings or on scheduled calls • Selection of targeted code reports per team (QC/QA)
• Presentation and discussion of themes with larger group
• Feedback
Questions & Discussion
Closing
• Our e-mail addresses:
• Qualitative Working Group:
• email Nicolas to be added to mailing list