4ik003 - assignment 3 - group 11a

8/12/2019 4IK003 - Assignment 3 - Group 11a

1/4

Linnaeus University

Informatics

Programme: Master of Information systems

October 12th2013

Assignment 3Presentation of an Article and Comments to an

Article

Choosing Evaluation Models - A Discussion on

Evaluation Design, Hanne Foss Hansen, 2005,Evaluation Journal, vol. 11 (4), pp. 447-462.

(Athens Group 11a)

Authors: Kanelakopoulou Marianna, Manikas Konstantinos, Rekleitis DimitriosE-mails:[email protected]/[email protected]

[email protected] / [email protected]

[email protected]

Course: 4IK003

Teacher / tutor: Christina Mrtberg/ Thomas Ivarsson
mailto:[email protected]:[email protected]:[email protected]:[email protected]


2/4

1

PAPER REVIEW

WHAT: What i s the key question that the author addresses in her text?

In the evaluation literature, many evaluation models are presented. The most

discussed are six and some subcategories. So, the logical question that arises is: "How

can evaluation sponsors and evaluators decide how to design an evaluation with somany models to choose from?".

WHY: How does she motivate (i.e. justi fy) the importance of the question?

This question is crucially important because the volume of evaluation models

is so large that causes confusion to sponsors and evaluators in which one to choose in

order to have the desired effect. Moreover, the authors of evaluation literature claim

that their own work is the most appropriate, but the author of the paper suggests that

alternative approaches of evaluation should be studied comparatively and meta-

models should be developed in order to help us consider the designing evaluations.

HOW: How does she go about to produce her arguments to the question addressed?Firstly, the author presents the Typology of Evaluation Modelswhich includes

Result models, Explanatory process model, System model, Economic model, Actor

model and Programme theory model, and then she escalates her reasoning by

presenting three Criteria which are three different types of logic. The first Criterion

suggests that The purpose of Evaluation should Determine Design. The second

Criterion suggests that Characteristics of the Evaluand should Determine Design. In

this reasoning there are two variants. ThePossibility Reasoning, which means that the

characteristics of the evaluand should be determined in order to select a model, and

the Legitimacy and Justice Reasoning, which means that the characteristics of the

evaluand should be considered, as well as what they justify, in order to select a model.

The third Criterion suggests that The problem to be solved by the Evaluated Object

should Determine Design. Apart from the Criteria presented, the author claims that

there are also some other factors that influence evaluation design. These are the

Negotiation Process, namely the different requests and interests of the actors can

become objects of negotiation, theAppropriateness, namely what the designers expect

to be appropriate considering the image of the actor and the situation, the What is

Usually Done, namely past experiences that influence the evaluation sponsors and the

evaluands choices and opinions, and the What Can be Done, namely the designers do

this in which they are competent.

MESSAGE: What is the answer to the question stated, or what i s the key messageof the paper that the author tr ies to send to the readers?

In the article we can see that by different types of reasoning are produced

different recommendations. Different recommendations are produced when design is

determined by the purpose of evaluation, when design is determined by the

characteristics of the evaluand and when design is determined by the problem to be

solved.

BENEFI T: What are the potenti all y posit ive benefi ts, that the author state herself ,

of the key message of the text?

The evaluation literature is large, chaotic and confusing and it needs further

clarifications. Through the authors' research and her suggestions, evaluators can bebenefited in order to choose the most appropriate evaluation models.


3/4

2

CRITICAL ASSESSMENT

What are the strengths and the limi tation of the paper that you as the student would

like to bring forward? (i.e. what do you think is clear and unclear with the papers

content?)

The article of Hanne Foss Hansen is quite important because it demonstrates

the problems in the existing evaluation literature. The author performs a good analysis

of the models and the criteria which mustbe set for the modelschoice, and the other

factors which affect the evaluation design. However, we have also noticed some

drawbacks. The first is that the writers reasoning rests on a theoretical level and has

not been implemented in practice. The same observation is mentioned by her in her

conclusion by saying that we do not have systematic knowledge in relation to how

important these logics and procedures are in practice. The second drawback of this

article is that, in some parts of it, the writer widens the theories more than needed and

it s difficult for the reader to follow her thoughts and stay focused on the main issue.

Therefore, the article could be more readable and keep the reader in a logical order. Ingeneral, the text is quite interesting and helps the reader to understand the complexity

of selecting and designing an evaluation model based on the different needs.


4/4

4ik003 - assignment 3 - group 11a

Documents