Focus Groups

Ciaran B. TraceMAC 2010 Symposium: Archival User StudiesDayton, Ohio, October 21-23, 2010

Outline of the presentation

Introduction to focus groups

Before you begin Carrying out the

focus group Data analysis Sum up Questions?

Resources used in this presentation

Richard A. Krueger and Mary Anne Casey, Focus Groups: A Practical Guide for Applied Research, Sage: Los Angeles, 2008.

David W. Stewart, Prem N. Shamdasani, Dennis W. Rook, Focus Groups: Theory and Practice, Sage: Thousand Oaks, 2007.

Denise Troll Covey, Usage and Usability Assessment: Library Practices and  Concerns Washington, D.C.: Digital Library Federation Council on Library and Information Resources, January 2002. Available at http://www.clir.org/pubs/reports/pub105/pub105.pdf)

Daniel L. Morgan, Focus Groups as Qualitative Research, Sage: Thousand Oaks, 1997.

What are focus groups?

“A focus group is an exploratory, guided interview or interactive conversation among seven to ten participants with common interests or characteristics. The purpose of a focus group is to test hypotheses; reveal what beliefs the group holds about a particular product, service, or opportunity and why; or to uncover detailed information about complex issues or behaviors from the group’s perspective. Focus group studies entail several such group conversations to identify trends and patterns in perception across groups. Careful analysis of the discussions reveals insights into how each group perceives the topic of discussion.” [Covey 2002]

Key elements

Notion of an interactive conversation among a small number of participants Not focused on the experiences and opinions of any one

participant [would need to do interviews for this] Evidence of the differences and similarities in the group

More than one focus group held in order to properly explore trends and patterns in perception across groups (each focus group will last anywhere between one and two hours)

Need participants and also a moderator and an observer

Need to record data and subsequently analyze that data

What can focus groups be used for? Determining patron satisfaction with

resources and services Helping to shape the future direction

of an archive as part of the strategic planning process

Developing and evaluating web and catalog interfaces

Developing and evaluating user education or reference services

Things to note…

A form of qualitative research method (with all that that entails) Moderator has a distinctive role Interaction occurs in a group setting There isn’t a very high degree of naturalism

Generally, you are recording not what people do, prefer etc….. But what they say they do, say they prefer, etc… Data is verbal and self-reported

Small number of participants, selection criteria, and frequent use of convenience sampling mean that you cannot generalize the results

The results are generally intelligible and accessible to all audiences

Things to note…

Can be used as a self-contained method, as an initial/ supplementary source of data, or as part of a multi-method approach (paired with surveys, individual interviews, participant observation etc) Paired with surveys – focus groups as a means to construct

questionnaires (domains, dimensions, wording) Paired with individual interviews – focus groups as a means

for devising the interview schedule for unfamiliar topics and vice versa

Paired with participant observation – focus groups as a tool to help you chose among potential field sites

As you can see…. focus groups can be used as a means to initially identify appropriate language, questions, or tasks, and as a follow-up tool to get clarification

Before you begin…

Articulate the purpose of the focus group Why is there a need for a study such as

this? What is the problem that the study seeks to

address? What kinds of information do you want to

get from the study? What wants the information? How will the information be used?

Before you begin…

Figure out what types of people can give you the information that you want Who has the greatest amount of

insight on this issue? What particular characteristics

should these people have?

Before you begin…

Figure you how many focus groups you need to hold and how they should be sequenced Usually hold 3 or 4 focus groups – allows you to be

sure that you have reached saturation point and allows for data to be compared across groups

If you want to compare and contrast how certain types of people talk about an issue separate them into different groups

If you anticipate that participants will have conflicting opinions ,hold focus groups in two stages (separate the groups initially and then bring them together)

Before you begin…

Determine how you will capture the focus group data▪ At minimum, will have an observer taking notes▪ Can also audio tape the focus groups

(transcribe partially or in toto?)▪ Can also video tape the focus groups

(transcribe partially or in toto?)▪ Video taping is rarely done – it is considered too intrusive

▪ Can take a combined approach – notes and audio, for example

Before you begin…

Think about the ethical issues and determine if you need to obtain human subjects clearance…▪ Nature of the group setting brings up privacy

concerns – limits the kind of topics the researcher can pursue

▪ Will need to get explicit permission from participants to tape (audio and video)

▪ Decide up front who will be able to listen to the audio tapes (should limit access to research staff)

▪ Make decisions about how the data will be used so that this information can be conveyed to the participants

Recruit and schedule participants

Using the criteria you have already identified, recruit and schedule participants. ▪ Not looking for generalizability but to minimize bias ▪ Not random sampling but theoretically motivated sampling or

segmentation▪ Remember that focus groups are composed of people who are

similar to each other in a way that matters to the researcher – homogeneity is important but you need enough diversity to allow for contrasting opinions

▪ Involves scheduling six to twelve participants in designated demographic groups

▪ Usually avoid mixing people with different levels of power or expertise

▪ “Ideally” participants should not know each other▪ May want to “pre-screen” to make sure the participants have the

requisite characteristics

Prepare interview questions

Prepare interview questions….▪ Think about the types of questions you need to ask ….and how you are going to sequence the questions▪ Also think about what makes for good questions…..▪ Normally ask about a dozen questions for a two hour focus group▪ Remember that the interview guide will have a direct bearing on the content of the group discussion

Good questions…

Good questions are… Open ended and one dimensional Sequenced (e.g. general before specific,

positive before negative etc) Evoke conversation and allow for depth (ask

participants to think back, avoid asking them “why”)

Engage participants (can go beyond oral questions and engage people in doing things – listing, rating, choosing among alternatives)

Good questions…

Good questions… Use words that participants would

use when talking about the issue, are easy to say and easy to understand, are short, are clear, include well thought out directions…..

Types of questions used in focus groups

Types of questions used in focus groups 1. Opening question – icebreaker, for all

participants, question should be designed to be easily and quickly answered (30 seconds or less), shouldn’t be used to highlight differences among the group (age, occupation, educational level), generally not included in the data analysis

Types of questions used in focus groups

Types of questions used in focus groups 2. Introductory questions – open ended

questions that introduce the topic of discussion, get people thinking about their connection to the topic▪ Should be questions that interest all participants▪ Method - ask people to remember when they first

encountered the organization, service etc and to describe the experience. Or “what is the first thing that comes to mind when you hear the phrase….”

Types of questions used in focus groups

Types of questions used in focus groups 3. Key questions - focus on the key

areas of concerns. Most of the questions fall into this category.

4. Ending question(s) - E.g. is there anything we should have talked about but didn’t.

Role of moderator

Recruit, schedule, and train a moderator and observer(s) for each focus group Skilled moderator is key - develop rapport with the

participant, remain impartial, and keep the discussion moving and focused on the research objectives

Moderator must have knowledge of the background topic Moderators must be able to handle the human element… Moderators and observers must avoid doing anything that

may bias the results of the study Goal is to encourage conversation among the participants

– NOT between the moderator and the participants

Role of observer

Main task is to observe the session and to take extensive notes

Not only capturing what is said but also documenting nonverbal signs and body language – tells you a lot about how the group feels about the topic as well as giving some indication of how many people hold the same idea

Notes should therefore include meta-comments, e.g. “Jim seemed surprised at Angela’s response…”

Handling logistical issues

Handle all logistical issues associated with running focus groups Securing compensation?, scheduling rooms,

ordering food/drink, having copies of focus group questions for moderator and observer, testing the audio- or videotape equipment, creating name placards for all those taking part in the focus group

On the day - getting the room ready (light and temperature), setting up and testing equipment, bringing supplies (handouts, name placards etc)

On the day…introduction, welcome, background

The actual focus group itself…role of the moderator Welcomes the participants and thanks them for

participating, introduces and explains their role as moderator and the role of the observer, reiterates the purpose of the focus group, confirms human subject requirements (anonymity, taping, how data will be used etc), briefly describes the ground rules and how the focus group will be conducted, emphasizes that the goal of the focus group is not to reach consensus - disagreements and discussion are important

NCSU sample focus group script (introduction, welcome, background)

North Carolina State University Libraryhttp://search.ebscohost.com.www.lib.ncsu.edu/userstudies/documents/

Thanks for taking time to join us to talk about some library tools, services, and resources that can help students with their class assignments. We here at the library are hoping to develop a suite of web tools to assist students with their learning experience. We really want to make these web pages as useful for students as possible. That's why we've asked you to come talk with us. You've probably noticed our microphone. We're going to tape record this session because we don't want to miss any of your comments. People often say helpful things in these sessions that we can't write down fast enough plus we don't want you to feel that you need to talk at a pace we can write. Please feel free to talk casually and naturally. (Co-moderator) and I will take what we learn from your tonight and share your ideas with our colleagues working on these web pages. Again, this is meant to be a relaxed discussion so we will be on a first name basis for the duration of this session. Each of you as you can see has a name placard in front of you. Hopefully the names on them are correct. We assure you that your names will not be linked in any way to any written materials produced from this focus group.

NCSU sample focus group script (introduction, welcome, background)

We're going to ask you look at some content and tools that we are thinking about including on these web pages. First we will provide a static version and would like you to rank them by your first impression of their function, and we will discuss as a group which of the pieces of content seem most interesting and helpful from your perspective and how you might envision yourself using each piece. You can even suggest new pieces of content that we might have not have thought of. After this initial exercise and discussion we will have each of you revisit each piece of content and explore their functionality as currently designed using the laptops in front of you. Try and note mentally how using these tools reinforces or dilutes the power of the tool you had initially assessed. We would like you to once again rank your top fives content pieces, singling out one as the most valuable. We will then discuss each other's rankings and how things may have changed. We feel that these activities and discussions will help us dig into what you think about tools, services, and resources that are helpful to you or that you can imagine being helpful to other students. There are no wrong answers for what we're going to discuss. We know you might have differing points of view and we value everyone's ideas so we really want to hear them.

First and subsequent questions….

Sometimes the first question is asked round-robin and is a question that each participant can answer. In this way each participant responds and gets comfortable talking.

Subsequent questions are answered less formally, and more conversationally.

Moderator clarifies anything that participants do not understand.

The moderator may invite further comments to better understand what the participants are saying, For example, “Would you explain that further?” or “Please give me an example.”

The moderator uses verbal and body language to invite comments from shy participants and to discourage others who may be trying to dominate the conversation.

Subsequent questions….

The probe should also be paired with another important tool…. the pause. Pause used as a way of eliciting a response from the participants.

Moderator must be comfortable with these silences.

Data analysis

Data analysis and reporting

You might or might not transcribe the focus group tapes - cost of transcribing may exceed the benefits of having a full transcription At minimum, can use the tapes to annotate

the observer’s notes Ideally, at least two people analyze the data

—the moderator and observer—high inter-rater reliability

Goal is to examine, categorize and make sense of the data.

Data analysis and reporting

Review – doing a thorough read through of the data – notes, transcripts, etc

Conduct content analysis on the data▪ Initial coding – involves reading the data and assigning labels

or codes to each piece of relevant information▪ Can be done by hand, using note cards, or using a computer.▪ Choice of strategies - note all mentions of the code, note

whether each individual mentioned a given code, note whether each groups discussion mentioned a particular code

▪ Reduce this information to manageable aggregates by creating memos for key themes that emerge from this initial coding

▪ Test the themes by going back to look at the data

Data analysis and reporting

Reporting…. Which topics to emphasize in the report▪ How many groups mentioned the topic▪ How many people within each group

mentioned the topic▪ Degree of enthusiasm displayed for the topic▪ Notion of “group-to-group” validation

Blend of summary of discussions and direct reporting of comments

To sum up…

ADVANTAGES OF FOCUS GROUPS

A way to gather insight into behavior and experience from the participants’ perspective

Allows for the gathering of rich data, in the participants own words

Produce a concentrated amount of data on a precise topic quickly and at relatively low cost

ADVANTAGES OF FOCUS GROUPS

Group discussion creates synergy and facilitates recall and insight

Direct interaction with participants allows for clarification of responses, probing of responses, and follow-up questions

Results of focus group research are user friendly and easy to understand

To sum up…

DISADVANTAGES OF FOCUS GROUPS

Quality of the responses depends on clarity of the questions, the moderator’s skills, and the participants’ understanding of the goals of the study and what is expected of them

Rely heavily on the observational skills of the moderator and observer

DISADVANTAGES OF FOCUS GROUPS

Quality of the information is impacted if participants are unduly influenced by others, stray off topic, or seek consensus

Hawthorne effect - cannot observe something without changing it

Findings have to be interpreted at the group level and are not generalizable

Questions…