owen perkins - cipp

7/28/2019 Owen Perkins - CIPP

1/32

June 25, 2013Owen Perkins

The CIPP Approach toEvaluation


2/32

2

Meant to serve decision makers at all levels

Good decision making is dependent on evaluative

information A systems approach to evaluation

Decisions are made about inputs, processes, andoutputs

Approach was a shift from program objectives to the

decision(s) of program managers Evaluation of the decision(s) the administrator(s) mustmake

Management-Oriented Evaluation Approach


3/32

3

Daniel Stufflebeam, supported a decision-oriented evaluationapproach to assist administrators make good decisions

His approach was developed in the 1960s

His Evaluation method proposed that administrators face fourkinds of decisions

Context evaluation

Input Evaluation

Process evaluation

Product evaluation Emphasis is on decision making by program administrators

The CIPP Evaluation Model


4/32

4

Determine needs to define objectives of theprogram

Asks what should we do? Collect data to determine goals of program

Example: A standardized test is given toevery student in 6th grade across a state.

What are the objectives of this program?Raise level of education, standard to evaluateteacher performance, etc.

Context Evaluation


5/32

5

Determine resources, alternative strategies,

and determine plan that has best potential to

meet the needs of the program

Asks how should we do it?

Involves determining the steps and resources

needed to accomplish goal or objective

Input Evaluation


6/32

6

To evaluate how well the plan has been

implemented Asks are we doing it as planned?

Continuously monitor the program, allows

decision maker to see conflicts, strengths and

weaknesses of program and materials, howwell it is following the established plan, etc.

Process Evaluation


7/32

7

Determine program attainments

What results were obtained? How well were needs reduced?

How did the program work?

Compare actual outcomes to anticipated outcomes

Should the program being evaluated be continued,

modified or dropped?

Product Evaluation


8/32

8

1. Focus the evaluation

2. Collection of Information

3. Organization of Information

4. Analysis of Information

5. Reporting of Information

6. Administration of the Evaluation

Stufflebeam suggested evaluators followthese general steps:


9/32

9

Identify the level of decision making being addressed

For each level of decision making analyze potential

decisions Define criteria for each decision situation by

specifying variables for measurement and standardsfor use in the judgment of alternatives

Define policies within which the evaluator mustoperate

Focusing the Evaluation


10/32

10

Specify the source of information to be

collected

Specify the instruments and methods for

collecting the needed information

Specify the sampling procedure to be used

Specify the conditions and schedule forinformation to be collected

Collecting Information


11/32

11

Provide a format for the information that

is to be collected Designate a means for performing the

analysis

Organization of Information


12/32

12

Select the analytical procedures to be

used Designate a means for performing the

analysis

Analysis of Information


13/32

13

Define the audiences for the evaluation

reports Specify a way to present information to

audiences

Develop format for evaluation reports Schedule the reporting of information

Reporting of Information


14/32

14

Summarize the evaluation schedule

Evaluate the potential of the evaluation design forproviding information that is valid, reliable,

credible, timely, and pervasive

Specify and schedule means for periodic

updating of the evaluation design Provide a budget for the total evaluation program

Administration of the Evaluation


15/32

15

Holistic approach:

Broad, vague, and summative Good or bad?

Idealized notion of process and program:

What the process should be rather than itsactual form

Too top-down and managerial in approach

Strengths and Weaknesses of CIPP


16/32

16

Imprisonment and Career Development:

An Evaluation of a Guidance Program for Job Finding

By: Gemma Filella-Guiu and

Angel Blanch-Plana


17/32

17

This study describes the evaluation results of PORO;a three level occupational guidance program

Designed to improve offenders job findingpossibilities and to reduce reoffending afterimprisonment

Used Stufflebeams CIPP model, considering the

efficiency, effectiveness and impact of the program Evaluation outcomes and practical suggestions for

career development applications addressed to prisonpopulations are reported

Article Abstract


18/32

18

The Guidance Program for Job Search

(PORO) Go figure?

Wikipedia: Poro is a fraternal secret

society of Sierra Leone and Liberia

PORO Definition


19/32

19

All internees (prisoners) are categorized bytheir individual judicial situation

PORO comprises three levels of guidance tointernees:1. Internees given professional and occupational

training in prison

2. Internees given ordinary parole permits

3. Has two distinct sections: Internees go into community on daily basis

Probation and freed internees in community

Program Description


20/32

20

Context Evaluationis the delineation and specification of the programsenvironment, the population and sample of individuals to be used, andthe program objectives

Input Evaluation identifies and assesses system capabilities,determines methods that meet objectives and needs, and studies therelations between cost and potential benefits of each proposed method

Process Evaluation helps to provide feedback to the teachers on

progress of the program

Product Evaluation allows decisions to be taken regarding thecontinuity, termination or modification of the program

CIPPs Use in Study


21/32

21

What should we do?

Collect data to determine goals ofprogram

Context Evaluation


22/32

22

145 people in the experimental group

51 in the control group

(N = 196) 11% females and 89% males

Between 25 and 35 year old

Serving sentences of less than 4 years.

Specialties taken in the occupational courseswere car mechanics, graphic arts, industrialdressmaking, building, electricity, binding,carpentry and computing

Context Evaluation: Collect Data


23/32

23

The main objectives of this intervention were:a) To improve individual competence in job searching

b) To facilitate the acquisition of useful job searchknowledge and skills

c) To increase professional self-esteem

The accomplishment of these specific aims is as

fundamental for enabling ex-convicts to adjust tothe job market and the community on a long-termbasis

Context Evaluation: Determine Goals


24/32

24

How should we do it?

Determine resources, alternativestrategies, and determine plan that has

best potential to meet the needs of the

program

Input Evaluation


25/32

25

The contribution of an expert group was requested toassess the design and methodological validity of PORO inorder to achieve the best possible adaptation to aconfinement context

The potential benefits that might be obtained were

rewarding enough considering the assignment ofoperational, material and human resources = Cheap Program

Input Evaluation: Determine plan that has bestpotential to meet the needs of the program


26/32


27/32

27

Feedback from all course teachers and 52 randomly

selected participants on the program were used toevaluate the process dimension

No serious difficulties arose during the teachingprogram on the majority of occupational courses

Internees were not very enthusiastic about the programin its initial stages. However, in the end, this lack ofmotivation was gradually overcome during the programdevelopment, final opinions being highly positive

Process Evaluation: How well has theplan been implemented?


28/32


29/32

29

Increase in job knowledge and skills scores

Increase in professional self-esteem scores

Based on the findings it should be expected that other

internees would reach these same employment rates whentheir sentences have been served

The program had a high level of efficiency: The material, human and operational resources used in the

[program] development did not pose any additional expense

The program did not have any adverse effects: Undesirable contingencies were not noted once the investigation

finished, nor were unforeseen relevant consequences due to theoccupational guidance [program] detected

Product Evaluation: Determine Program Attainments


30/32

30

The resultsgive further support to theidea that career intervention addressed atimprisoned people in adverse social andeconomic circumstancesis of paramountimportance for improving the employment

possibilities of people whose [behavior] andlack of skills are troublesome to them and tosociety

Conclusion


31/32

31

This case study is an excellent example

of the CIPP evaluation approach Each step of the approach was carefully

considered

Conclusion, contd


32/32

June 25, 2013GO HOKIES!

The End

owen perkins - cipp

Documents