naturalistic evaluation2

Naturalistic EvaluationJasmine Ervins M.Ed.Shika Hershel M.Ed. Ilda Gonzalas M.Ed.

Naturalistic Evaluation An approach to assessment that evolved from

the work of researchers at the Indiana Center for Evaluation Wolf Tymitz Guba Lincoln

Has been used in numerous applications including schools, social programs, museums, and health care

Model’s Design

Two Streams

Naturalistic Evaluation has evolved from two parallel streams ‘Responsive evaluation’ from the evaluation field Naturalistic methodologies from the area of qualitative

‘inquiry’ used by anthropologists and sociologists practicing ‘ethnography’

Combining the two makes Naturalistic Evaluation an intensive endeavor

Model’s Purpose

Purpose Generally used to discover what is wrong

(generally) and how to fix it (in a manner that will last), or how something is working (generally) and why (so you know how to keep doing it, or for possible use by others)

This is more rigorous than finding out what is wrong (according to the audience) and deciding what to do (according to the audience)

Philosophical Perspective

Major Role of Evaluation Guba and Lincoln believe it is to respond to audience

requirements for information in ways that take account of the different value perspectives of its members

Naturalistic Evaluation puts the evaluator in the role of learner – the informants teach

However, criteria for scientif ic r igor still apply!

General Phases Familiarization phase

Gaining entry Understanding the Circumstance and the players

Three C’s Collection of Data Classification and Analysis of Data Confirmation of Propositions

Synthesis Presentation of findings and debriefing

Naturalistic Evaluation Pros

When done correctly, it can provide a very complete picture of a program’s context, stakeholders, operations, effects (qualitative and quantitative), and systems of relationship

A participant-oriented ‘gold standard’ Cons

When done correctly, can Take a prohibitively long time Cost a lot Be very labor intensive

When done incorrectly, can give a false sense of knowledge

History of Development

Background of Naturalistic Evaluation Approach

O The Naturalistic approach gives the evaluator freedom to choose the methods used to collect, analyze, and interpret their data.

Major Developer(s)

Responsive EvaluationO Stake (175) is one of the

major developers of the naturalistic evaluation approaches.

O Stake was concerned that conventional approaches were not sufficiently receptive to the needs of the evaluation client.

O Stake believed that evaluators should use whatever data-gathering schemes seem appropriate relying heavily on human observers and judges.

Robert StakeO First evaluation theorist to introduce Participant

Oriented Evaluation into the field of education. O Introduced the theory of Responsive Evaluation

which focuses on “re-directing data gathering and interpretative efforts around emerging issues of importance to program practitioners in the evaluation setting” (Abma, 2005)

O Created the Countenance Framework in 1967. The model refers to the two faces of evaluation: description & judgment.

O Description includes the evaluator’s observation and list of benchmarks for the activities being evaluated. Judgment is the evaluator’s overall rating of merit.

Uses of the Model

Implementation Techniques

Implementation Techniques