method mini-toolkit

Method Mini-Toolkit

Learning Collaboration Leadership

Ontario Centre of Excellence

for Child and Youth

Mental Health

Bringing People and Knowledge Together to Strengthen Care.

Case studies: An Overview


Helpful tips More information Other Mini-Toolkits

Case studies: An overview

www.excellenceforchildandyouth.ca Method Mini-Toolkit— Case studies: An overview 2

CONTENT

What is a case study?

When to use

Some limitations

Where to start

Helpful tips

Ethical considerations

For more information

Other available Mini-Toolkits

About the Centre

As a leader for child and youth

mental health in Ontario, we build

connections, share knowledge

and draw upon our expertise to

enhance the skills of front-line

service providers. We passionately

promote and share the benefits

of both organizational learning

and evidence-informed practice.

Together with our partners we are

working to strengthen Ontario’s

mental health programs and services

for all children, youth families and

caregivers.

Introduction

What is a case study?A case study is a qualitative research method that thoroughly examines one or more defined cases through the collection of data from multiple sources (e.g. interviews, surveys, document

review and observations).

http://www.excellenceforchildandyouth.ca/resource-hub/toolkits

http://www.excellenceforchildandyouth.ca






CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

When would I use this research method?

• To reveal an in-depth experience

• To learn what happened and/or how something happened. Particular case study approaches may also tell us more about any causal links thought to exist and tell us why something may have happened, particularly where traditional surveys or experimental methods will not adequately capture the richness of data that is needed. For more information on other research methods click here

• To understand the implementation of a program in a new context (e.g. new organization, new city, new country, etc.)

• To better understand a case(s) that is considered an outlier among other similar cases

• To better understand cases in environments characterized as unpredictable and complex

• To explore a topic in greater depth

• To strengthen areas of practice where there are significant gaps in knowledge

• To examine phenomena that cannot be replicated in a research setting

• To enhance an experimental study

• To produce rich and detailed research others can use to judge the suitability of a unique program for their own context

• To examine the process and how it impacts the outcomes




http://www.excellenceforchildandyouth.ca/introduction-qualitative-and-quantitative-methods-and-analyses

http://www.excellenceforchildandyouth.ca/introduction-qualitative-and-quantitative-methods-and-analyses





CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Some limitations of the case study

• Can be complex to plan and carry out (i.e., the time and effort of collecting, compiling and analyzing detailed case study data is extensive)

• Validity may be compromised as it might be hard to ensure that the data is well-founded and accurate (See Table 3 for ways of controlling reliability and validity issues)

• May be difficult to relate findings to other individuals, programs or organizations given the narrow focus








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Where do I start?Before conducting a case study it is important to determine exactly what you want to know about your case(s). When the case study is part of an evaluation, it is a good idea to start by identifying your evaluation questions. These are questions you would like answered about your case(s), as well as hypotheses formulated from a literature review on relevant studies.

Engaging key stakeholders is a good way to formulate your specific evaluation questions and ensure you have their best interest in mind (Morra, 1999). (Refer to the Centre’s program evaluation toolkit for a comprehensive guide to identifying your evaluation questions).

STEP 1: Determine if a case study is an appropriate methodology

A case study is appropriate when:

• Your evaluation question involves investigating outcomes for a unique group of individuals participating in a program.

• You want to know how or why an intervention is having a certain effect.

• You are looking to supplement your quantitative data with a thorough qualitative understanding of the different components of your program.

• You would like to gain a deeper understanding of your evaluation results.









CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

STEP 2: Identify your case(s)/ units of analysis and design

In some case study designs it can be useful to differentiate the terms case and unit of analysis. A case refers to the main phenomenon under study and unit of analysis refers to a smaller sub-unit under study within the case. For example, in an evaluation of a mental health organization that offers various services, the organization itself could be the case, and the various services could be the units of analysis. Your case or unit of analysis is what you are looking to study in-depth. In this mini-kit, the word case refers to the main phenomenon under study and unit of analysis is used to describe sub-units in certain case study designs.

Your case(s) may include a single individual, a group of individuals, an organization, a program or an event. Identifying your case can sometimes be challenging but getting feedback from a variety of stakeholders can help you narrow your focus and select the best case for your study.

The focus of your case(s) is dependent upon what you would like to investigate. For example, if you are interested in looking at how an individual is affected by the implementation of a program, then your case is the individual. Another possible case in the evaluation of a program implementation is the program itself as well as the investigation of details and processes of the implementation.

Table 1 can help in choosing the best type of case study for your evaluation.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Table 1 - Types of case studies

Case study design

Types of case studies General question

Explanatory:Explains the rela-tionship between different aspects of a program

Program Implementation:Investigates the processes tak-ing place within a site or multiple sites.

How is the program implementation work-ing?

Program Effects: Examines causality using multiple methods of assessment. Usually involves investigating multiple sites.

Why is the program having the observed effect?

Descriptive:An in-depth description with a more narrow focus than the explanatory style

Illustrative:Intended to add real and extensive examples to other information about a case.

What does this case look like in-depth?

Exploratory: Intended to generate hypotheses for future investigation.

What does this case look like and what hypotheses can be brought forward for future investigation?

Critical Instance: Examines a distinctive case and attempts to understand its uniqueness. Can also be used to test an assertion of a case under study.

a) What is contributing to the uniqueness of

this case? ORb) Is the program achieving its anticipated outcomes?

Combined Methodology:A mix of different case study methodologies

Cumulative: Brings together findings from past case studies to answer a current hypothesis.

What do the findings of past case studies on this program suggest about the current program?

For all case study designs, the case(s) can be individuals, groups, programs, events or

organizations.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

STEP 3: Select methods and collect your data

Once you have selected a case study type and design, you can now determine what methods you will use to collect your data. It is a good idea to be aware of methods your program or organization is already using and to look to the literature to guide your selection.

Typically case studies use multiple methods and data sources to ensure rich detail and comprehensive insight into the case(s) studied. The goal with data collection is to try and answer basic who, what, where, when and why questions. For example:

a) Who are the key individuals involved?

b) What is their role? And what did they do? (their actions, during their involvement)

c) Where did the program events take place?

d) When did the program events take place? What period in time are you basing your evaluation?

e) Why did the individuals in the program do what they did? What do their actions mean?








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Table 2

Commonly used data collection methods

Individual interviews with key informants

Interviews are the foundation of any case study. They provide detailed information on who was involved and their reactions. Interviews may be structured, semi-unstructured, or completely unstructured.

Focus groups Focus groups provide a wealth of knowledge about the opinions of a key group of individuals relevant to the study. For example, it may be appropriate to hold a staff based focus group with various members of an interdisciplinary team in order to gain an in-depth understanding of staff attitudes and concerns.

Observations Observations allow you to gain direct experience with the case you are studying compared to formalized interview data. During an observation, the evaluator should take detailed notes of events and quotes from relevant individuals. It is common practice to record observation sessions. This will ensure you have captured all pertinent information that can be later analyzed for any missing information or quotes. Of course, it is important to obtain consent from all participants prior to observing and recording the sessions.

Archival records and documents

Archival records and documents allow you to generate information on historical opinions, decisions and information relevant to the current case study. They should be reviewed critically through a qualitative analysis to uncover patterns and themes within the data.

Surveys Surveys containing open-ended questions are useful for generating opinions of individuals involved in the evaluation. Standardized questionnaires are not as popular for case study data collection, as the interviewer generally doesn’t know enough about the situation being investigated to create an appropriately comprehensive questionnaire.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

STEP 4: Analyze and interpret the data

It is important to consider the intended uses of the data. If the findings of your evaluation are intended to help guide specific program or policy decisions, keep these in mind while synthesizing your data. Also, organizing and conducting a preliminary exploration of the data earlier in the collection can help draw out important information, themes and patterns throughout the evaluation process. It can also help to inform any adaptations to data collection methods that may be necessary. Organizing information from each source immediately following collection will also significantly reduce the amount of time you will spend deciphering notes taken months earlier. It will also allow you to more easily identify missing and inconsistent data which can be conveniently addressed in the earlier stages of the evaluation. Keep in mind that data collected early may include outliers that cannot be addressed until all of your data are available.

It is also very important to maintain clear records of your data collection methods and the steps you are taking in analyzing and interpreting the data throughout your evaluation. It is equally important that you engage a range of stakeholders in the analysis and interpretation of your data to ensure a range of perspectives are taken into account. When you begin interpreting your data it is a good idea to refer back to your evaluation questions and organize the information accordingly. Look for evidence in the data that helps you to answer your questions and tell a story about your case(s). Finally to help facilitate writing the final report, put your findings together in an organized and easy to understand manner.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

STEP 5: Write your case study report

• Begin drafting your report early in the evaluation process and continue the writing process throughout. You will be able to draft some sections (e.g. the literature review and methodology) before you even start collecting your data.

• Ensure you include enough details in the report about your case(s). Giving your audience a detailed outline of the case(s) and context(s), allows them to evaluate the conclusions you have drawn and form their own conclusions.

• Ensure your report includes a chain of evidence that clearly links your evaluation questions, specific methods and data sources and your conclusions. In other words, make sure to adequately reference data sources within your report and clarify specific pieces of evidence you used along the way to draw your conclusions.

• Tailor your report format, style and content to the needs of your audience(s).

• Refer to previously conducted case study reports (e.g. within your organization, community partners or relevant literature) to get a sense of how you may wish to present your case study.

• To increase levels of construct validity (see table 3), have various stakeholders (including participants of the study) review your draft report. If major disagreements arise, this may mean additional evidence is required to resolve these issues. Input from various stakeholders regarding the draft report may also be incorporated as part of the case study and presented in the final report.

For a guide to writing case study reports please click here.




http://www.excellenceforchildandyouth.ca/sites/default/files/docs/minikit_appendix_case_study_evaluation_report_outline.pdf





CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

You can refer to the table below for more information on how to maintain high levels of reliability and validity in your case study evaluation.

Table 3 - Techniques in enhancing validity and reliability in case studies

Test Case study techniques

Construct validity:Measures are used to correctly assess operationally defined concepts under study

Triangulation: Using several sources of informa-tion and data measuring techniques to capture an accurate and comprehensive representation of the case.

Chain of evidence: The sequence of observa-tions to conclusions. A strong chain of evidence is a thorough and accurate process of observation, data analysis and drawing of conclusions that, if conducted by a second evaluator, would produce the same results.

Critical review: An in-depth critical evaluation of the case.

Internal validity:Establishing a causal relationship (not appropri-ate for descriptive or exploratory studies)

Pattern matching: The use of past experience or knowledge, logic and theory to make predictions about what you believe you will find. This technique matches data to existing hypotheses.

Explanation building: Creating hypotheses based on collected data.

Time series analysis: The organization of infor-mation analyzed in chronological order. Within a multi-site evaluation, information would be analyzed chronologically within each individual site.

External validity:The ability to generalize your study

Multiple case studies: Case study looking at more than one case.

Reliability: Study operations can be repeated and will generate the same results.

Case study protocol: Outline a study protocol with all the components of the evaluation arranged in chronological order and follow accordingly.

Case study database: Create a case study data-base that holds all of your data in an organized and easy to manage way.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

STEP 6: Share your findings

It is important to share your case study findings with all of your key stakeholders and to tailor the information so it is accessible and relevant to the needs of the different groups. Findings have the potential to influence decision makers, and are therefore very valuable and important to share. You may also consider sharing your findings at a relevant conference with a poster and/or presentation. Other ideas for sharing findings include lunch-and-learn events, newsletters, electronic bulletins and even social media.

Your case study report is a comprehensive document that can be used as a springboard for other knowledge exchange products. In some instances, it is appropriate to share your entire final report with a group of stakeholders, whereas others may benefit from receiving a summary, presentation or other method that presents your main findings and next steps in a clear and more concise format.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Developing a case studyHelpful tips:

Do:• Ensure there is common understanding of the term case

study among core members of your evaluation team and other stakeholders.

• Define the goals of your evaluation and ensure that a case study methodology suits your purposes.

• Discuss existing barriers and facilitators for your evaluation to determine if you have the necessary resources to conduct a case study (e.g. time to gather and analyze large amounts of data, knowledge of case study designs, access to resources detailing how to conduct case studies, etc.).

• Clearly define your case(s) and seek agreement from the evaluation team on which case or cases to select.

• If possible, examine multiple cases rather than a single case to facilitate more complex and rigorous comparisons, increase generalizability and clarify program impacts.

• Use multiple data sources or tools (e.g. collect data from various individuals, use observation, document reviews, etc.).

• Create a case study database that contains all of the various forms of raw data you will collect along with information such as the timing and location of data collection.

• Put together a data analysis plan detailing the types of analyses to be performed and the roles of various stakeholders to help ensure analyzing large amounts of data runs as smoothly as possible.

• Begin to organize and look at your data early in the data collection process.

• Gather input from a variety of stakeholders when you interpret what your case study data means for your program,








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

organization, evaluation practices as a whole, etc.

• If the data you collect reveals that it may be beneficial and if it will not decrease the rigor of your evaluation, modify or adapt the design of your case study (e.g. move from a single- to multiple-case design, change your focus to a different case(s) if it is not as similar or unique a case as initially thought, etc.).

• Seek continuous stakeholder feedback throughout the evaluation process.

Don’t: • Select a case study methodology without first considering its

trade-offs with non-case study methods, your goals for the evaluation and your available resources.

• Investigate all possible cases. Narrow your focus to the most relevant case(s) to ensure your evaluation remains informative yet feasible.

• Delay the involvement of various stakeholders whose input is crucial in the planning stages of your case study evaluation.

• Make major case definition, methodological decisions or form case conclusions without the input of various stakeholders.

• Use one data source from which to gather your evidence.

• Wait until large amounts of data have been collected to begin organizing.

• Draw conclusions from your case(s) without explicitly linking these back to your evaluation questions and specific data sources.

• Wait until the late stages of your evaluation to outline the format of your final report.








CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

What are some ethical con-siderations for conducting case studies?Many of the same ethical considerations that are relevant to non-case studies also apply to case study evaluations, with some ethical issues of particular relevance.

• It is important to obtain ongoing informed consent from key informants whom you may be gathering data from multiple times and through various methods during the evaluation.

• Establishing sensitive practices around confidentiality or anonymity for key informants is essential for maintaining the integrity of the data and good relationships throughout the evaluation.

• Evaluators and key informants are responsible for negotiating specifics on what mutually beneficial confidentiality standards look like for a particular evaluation (i.e. at the end of an interview, giving participants the chance to exclude information they feel uncomfortable sharing publicly).

• Some important questions to keep in mind during your case study evaluation include:

• Are my interview questions too intrusive?

• Am I invading the privacy of participants during my observations?

• Do I have informed consent from participants to film or record their behaviour?

• Have I accurately captured participants’ responses?

• Have I represented participants fairly in my report?

For more resources on ethics in evaluation click here




http://www.excellenceforchildandyouth.ca/sites/default/files/docs/webinars/_attach/ethics_presentation_110612.pdf





CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Where can I go for more information?

References

Albright, A., Howard-Pitney, B., Roberts, S., and Zicarelli, J. (1998). Tell Your Story: Guidelines for Preparing an Evaluation Report. Sacramento, CA: California Department of Health Services.

Balbach, E.D. (1999). Using case studies to do program evaluation. Califor-nia Department of Health Services.

Bates College; Burton, Neil et al. Doing Your Education Research Project. Los Angeles, CA: SAGE, 2008; Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College. Colorado State University. (2013). Case studies. Retrieved April 16, 2013, from http://writing.colostate.edu/guides/page.cfm?pageid=1284.

Burton, Neil et al. Doing Your Education Research Project. Los Angeles, CA: SAGE, 2008; Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College.

CIRCE, 1999. Center for Instructional Research and Curriculum Evaluation. Retrieved from http://education.illinois.edu/circe/

Creswell, J.W. (2007). Qualitative inquiry and research design. (2nded). Thousand Oaks, CA: Sage Publications. Flyvbjerg, B. (2006). Five misun-derstandings about case-study research. Qualitative Inquiry, 12(2), 219-245. Gerring, J. (2004). What is a case study and what is it good for? American Political Science Review, 98(2), 341-354.Islam, M. n.d. Case study guide. Retrieved from: http://community.eldis.org/.59b8e89c/GUIDELINE%20AND%20TOOLS%20FOR%20CASE%20STUDY.pdf

Davey, L. (1991). The application of case study evaluations. Practical As-sessment, Research & Evaluation, 2(9) Retrieved April 30, 2008 from http://PAREonline.net/getvn.asp?v=2&n=9.

GAO. (1990). Case study evaluations. Transfer Paper 10.1.9. Washington: Program Evaluation and Methodology Division.




http://writing.colostate.edu/guides/page.cfm?pageid=1284

http://education.illinois.edu/circe/

http://community.eldis.org/.59b8e89c/GUIDELINE%20AND%20TOOLS%20FOR%20CASE%20STUDY.pdf

http://community.eldis.org/.59b8e89c/GUIDELINE%20AND%20TOOLS%20FOR%20CASE%20STUDY.pdf

http://PAREonline.net/getvn.asp?v=2&n=9

http://PAREonline.net/getvn.asp?v=2&n=9





CONTENT


When to use

Some limitations

Where to start

Helpful tips




About the Centre














caregivers.

Goodson, P., Gottlieb, N. H., & Smith, M. N. (1999). Put prevention into practice: Evaluation of program initiation in nine Texas clinical sites. Ameri-can Journal of Preventative Medicine, 17(1), 73-78.

Issel, L.M. (2004). Health program planning and evaluation: A practical, systematic approach for community health. Toronto: Jones and Bartlett Publishers.

Mind Matters. (n.d.). Case studies. Retrieved from http://www.mindmatters.edu.au/about/evaluation/evaluation_case_studies.html

Morra, L. & Friedlander, A. (1999). Case study evaluations. Washington: World Bank Operations Evaluation Department.

Neale, P., Thapa, S., & Boyce, C. (2006). Preparing a case study: A guide for designing and conducting a case study for evaluation input. Retrieved from www2.pathfinder.org/site/DocServer/m_e_tool_series_case_study.pdf

Simons, H. (2009). Whose data are they? Ethics in case study research. Case Study Research in Practice. Thousand Oaks, CA: Sage Publications.

Spath, R. & Pine, B. (2004). Using the case study approach for improved program evaluations. Child and family social work, 9, 57-63.

Stake, R.A. (1995). The art of case study research. Thousand Oaks, CA: Sage Publications. Yin, R.K. (2003). Case study research: Design and Methods,3rdedition. Thousand Oaks, CA: Sage Publications.

Tellis, W. (1997, July). Introduction to case study [68 paragraphs]. The Qualitative Report [On-line serial], 3(2). Available: http://www.nova.edu/ssss/QR/QR3-2/tellis1.html

Torres, R.T., Preskill, H.S. & Piontek, M.E. (1997). Communicating and re-porting; practices and concerns of internal and external evaluators. Evalua-tion Practice, 18, 105-125.

Yin, R.K. (2012). Applications of case study research, 3rd edition. Thou-sand Oaks, CA: Sage Publications.– Chapter 1 available at http://www.sagepub.com/upm-data/41407_1.pdf




http://www.mindmatters.edu.au/about/evaluation/evaluation_case_studies.html

http://www.mindmatters.edu.au/about/evaluation/evaluation_case_studies.html

http://www2.pathfinder.org/site/DocServer/m_e_tool_series_case_study.pdf

http://www2.pathfinder.org/site/DocServer/m_e_tool_series_case_study.pdf

http://www.nova.edu/ssss/QR/QR3-2/tellis1.html

http://www.nova.edu/ssss/QR/QR3-2/tellis1.html

http://www.sagepub.com/upm-data/41407_1.pdf

http://www.sagepub.com/upm-data/41407_1.pdf

method mini-toolkit

Documents