monitoring and evaluation and followup of training programmes prashanth

8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth

1/43


2/43

MONITORING

AND

EVALUATION

OF

TRAINING PROGRAMMES

P.PRASHANTHPhD SCHOLAR

Dept. of Agricultural ExtensionI.D. No. RAD/11-10


3/43

2 WHAT IS MONITORING?

5 FOLLOW UP TRAINING PROGRAMME

MODELS, TYPES, LEVELS OF TRAINING EVALUATION4

IMPORTANT MONITORING TECHNIQUES 3

.

6 CONCLUSION

WHAT IS TRAINING NEED ANALYSIS?1

Presentation Outline


4/43

TRAINING

MANAGEMENTCYCLE

3Implementation

4Monitoring

1

Problems & Training needs analysis

5

Evaluating

2Planning, Designing

and Organising


5/43

MONITORING & EVALUATION

OF TRAINING PROGRAMME

Assessment and control of

the progress of trainingprogramme for its effective

organisation


6/43

A continuous or periodic review and surveillance

(overseeing) by management at every level of the

hierarchy of the implementation of an activity to

ensure that input deliveries, work schedules,targeted outputs and other required actions are

proceeding according to plan


7/43

Indicators are defined as specific (explicit) and objectivelyverifiable measures of changes or results brought out by anactivity. Monitoring indicators are required to be :

Valid-they should actually measure what they are supposed tomeasure;

Reliable--verifiable or objective-conclusions based on them

should be the same It measured by different people at different times and under

different circumstances;

Relevant-they should be relevant to project objectives

Sensitive-they should be sensitive to changes in the situationbeing observed;

Specific-they should be based on available data; Cost effective the results should be worth the time and money it

costs to them; and

Timely it should be possible to collect the data reasonablyquickly.


8/43

Training performance has dimensions, namely

(i) Participation (commitment to training), (ii) Utility (used of training to trainees), and

(iii)Effectiveness (usefulness of training to ext

organisation)

Based on these three dimensions, the followingthree training performed indicators are suggested.

TPI

TUI

TEI


9/43

Assessment of training needs

Pre tests

Daily- tests

Alternate -day- tests

Weekly tests

Recall sessions

Fortnightly testsReviews

Back home application

IMPORTANT MONITORING TECHNIQUES


10/43

Evaluation is a process of investigating how far the objectives

of the training programme have been achieved.

It also involves analyzing the strengths and weakness of the

programme, reactions of the participants, and its over all impact

on behavioural change and job performance of the participants Attempt to obtain information (feedback) on the effects of a training

programme

To assess the value of the training in the light of that information

A systematic processes of collecting information for and about a

training activity which can then be used forguiding decision-

making and forassessing the relevance and effectiveness of

various training components.


11/43

Formative training evaluation -Pertains to the

implementation stage of the training programme and helps

in day-to-day decision-making.

Summative training evaluation relates to final

evaluation of the objectives of the training programme

Ongoing training evaluation, as the term indicates, is the

evaluation undertaken during the course of a training

activity.

Ex post training evaluation is the evaluation undertaken

after completion of a training project, usually after full

project benefits and impact are expected to have been

realised.


12/43

WHY DO YOU

EVALUATE ?

1. To see what has been achieved

2. To measure the progress

3. To improve

monitoring

4. To strengthen th

programme

5. To see whether

the effort is

effective6. To see cost

benefit ratio

7. To collect information

for planning and

managing programme

8. To share

experiences

9. To improve

effectiveness

to have more

impact


13/43

Assessment of training needs

Development of training objectives

Development of course design

Developing training methodology

Organizational climate for setting training

Actual conduct of training course

Monitoring & evaluation of training course

Follow -up of training programme

ESSENTIAL AND IMPORTANT COMPONENTS

OF TRAINING PROGRAMMES


14/43

Time of planning

Quality of training course

Coverage of the syllabus/content

Extent of use of teaching aids

Mid term reviews

Training methods Training skills

Field visits/study tours

Use of Audio-Visual equipment

ESSENTIAL ITEMS TO JUDGE THEEFFECTIVENESSOF A TRAINING

PROGRAMME


15/43

Training faculty/technicians

Availability of funds for training

Change in the behavior of trainees

Field applicability

Trainees opinion

Daily / alternate / weekly tests

Lodging and boarding facilities

Pre and post evaluation

ESSENTIAL ITEMS TO JUDGE THEEFFECTIVENESSOF A TRAINING

PROGRAMME


16/43

KNOWLEDGE SKILLS

ATTITUDE

FACTORS TO BE MEASURED IN

TRAINING EVALUATION


17/43

Objective-oriented Training Evaluation

Decision-oriented Training Evaluation

Transactional Training Evaluation

Goal-Free Training Evaluation

Adversary Training Evaluation

Illuminative Training Evaluation


18/43

REACTION LEARNING PERFORMANCE IMPACT

Purpose - To know training

participants attitude

towards the various

components of

training

To know the knowledge,

attitude and / or skills

gained by the participants

To know the trainees

performance in their

organisationsor place of

work

To assess the long term

effects of training on

the community or

society

Data Gathered Impression or attitude

toward the topics,

methods, aids

administrative support

Knowledge, attitude and

skills

Skills and outputs Demographic, socio-

economic Data

Sources of Data Trainees Trainees Superior and subordinate Members of the

trainees, community or

society

Tool for

gathering Data

Questionnaire Questionnaire Checklist and score card Interview schedule

Time to gather

Data

During and

immediately after

closing of training

Immediately after closing

of training or before and

after training

After trainees return to

work and / or during

training

Many years after

trainees have resumed

work or employment in

their community or

society

Person in-charge

of Data

gathering

Trainer Trainer Trainer or superior Independent evaluation

group

Use of Data Improving trainingenvironment

Improving trainingcontents, methods and

Improving skills andoutputs

Improving trainingstrategy

LEVELS OF TRAINING EVALUATION


19/43

The planning stage

The implementation stage

The stage immediately after

termination of training

The impact stage

TYPES OF TRAINING EVALUATION -

STAGES OF TRAINING PROCESS


20/43

Planning Evaluation Consists of two phases. In the first phase-TNA-

through one of the following four approaches:

Performance analysis Task analysis

Competency study

Training needs survey

The skill gap analysis

In the second phase, an assessment ofinstructional methods is done

The most appropriate training methodology andinstructional strategy is worked out.


21/43

It helps in solving the problems arise in implementation ofprogramme

According to Raab et al. (1987) as reported by Misra (1990),

"process evaluation is performed to detect or predict defects in the

procedural design of a training activity during the implementation

phase".

Investigation is made on the following:

Monitoring of training to know whether it goes on as per the plan.

Appropriateness of training methods or audio visual aids used.

Effectiveness of delivery or presentation

Effectiveness with regard to proper mix of theory and practical

Changes in the knowledge, skill and attitude among participants

Satisfaction of trainees with boarding and lodging facilities

Effective utilization of finance


22/43

Concerned with learner performance

Analyzed by two approaches i.e.

Norm-referenced and criterion-referenced.

Under the norm- referenced evaluation,The gains in learning are determined based

on pre and post training scores.

The criterion referenced evaluation is

Related to investigation of what was taughtand what was learned.


23/43

Refers to assessing the programme impact on the Jobperformance of the participants.

It attempts to find answer to the following questions:

What are the improvements in the performance of job asthe results of training

What are the benefits accrued to the participants andorganisations as the result of improved job performance?

What are the problems in applying the skills learned in realwork situation?

What can be done to enhance the application of newly

learnt knowledge and skill in the work place? It can be carried out at two levels

At the level of immediate objectives

At the level of long-range objectives


24/43

TRAINING EVALUATION ACCORDING TO STAGES OF TRAINING PROCESS

EVALUATION FOR

PLANNING

PROCESS EVALUATION TERMINAL

EVALUATION

IMPACT

EVALUATION

What is it? Instructional strategies

and means

Feedback from trainees and

others regarding training

Effectiveness of

training after its

completion

a) Immediate objectives

b) Long range

objectives

Who evaluates? Trainers Trainers, also external

evaluators

Trainers, also external

evaluators

a) Trainers

b) Specialized units ofthird party evaluators

When evaluated? Before and during

training

During training After completion of

training

After completion of

training

Methods of

evaluation?

1.Determining most

appropriate instructionalprocedures

2.Pilot testing

i) Pre-assessment

ii) Interim assessment

iii) Other sources of

assessment information

i) Measurement of

change intrainee(KSAOs)

ii) Measurement of

trainees perceptions

about training

i) Trainee follow ups

ii) ii) Employer

surveys


25/43

Decide on the method of evaluation

Decide on the type of evaluation

Plan the evaluation

Prepare the evaluation tools

Implement the evaluation

Report the evaluation results

Utilize the evaluation results

STEPS IN EVALUATION


26/43

Step 1: Identify and described the training to be evaluated. Step 2 : Identify and clarify the purposes for the evaluation.

Step 3: Identify the customers of training and evaluation and

clarify their needs for information.

Step 4 : Determine the key questions that the evaluation mustaddress.

Step 5 : Determine the over all design approach that will best

meet customer needs and answer the evaluation questions

identified.

Step 6 : Identify the best data collection sources and procedure

to address the evaluation questions.

Step 7 : Plan reporting and communication actions


27/43

(i)Constraints imposed by time dimensions as the sponsors arealways in a hurry to get the quick results for decision makingand other uses,

(ii) Constraints of ethical necessity for continuous feedback;which remains a costly affair,

(iii) Constraints imposed by disciplinary boundaries whichhave separate evaluation methodologies,

(iv) Constraints of openness of the social system which theproject is operating, thereby creating problem of control andmanipulation at will,

(v) Constraints of long-standing tensions between theprofessional personnel and evaluators who tend to view eachother with suspicion and even hostility. However, theseconstraints are showing signs of easing,


28/43


29/43

Activities related to finding out the utility of thelearning outcome of the programme in job

performance

Analysis of reasons for non-utilization of new

skills or knowledge acquired Providing help and reinforcement for application

of learning as well as assessing future training

requirements


30/43

To what extent the content covered /

taught during a particular traininghave been adopted by the trainees /

sponsoring institutions


31/43

To find out the impact on individual job performanceand the organisational environment is essential tojustify the continuance of the programme in future

Feedback about the utility of programme from the

trainees after they resume their job and theirexperimentation with new knowledge and skills learnt

is essential to validate the impact of the programme

The feedback from the participants will greatly help

in making suitable changes in the contents andmethodology of the programme

To tell us about the quality of training and the effect

that training has created on the learners


32/43

To know the extent of learning by the trainees

To know the application of the content covered

during the training in field situation

To find out reasons for poor

TOLearning

To be aware of the problems in adoption of the

technology in field situation

To know the impact of training on production level

To get the feedback about the quality of training

To know the future training requirements of clients


33/43

Evaluation of training Evaluation of behavioural change

Results of application

Further Personal Development

On the job Further training courses

Help in Practical Applications External services including consultancy

Help by superiors and colleagues

Liasion with former participants Personal contacts

Associations

Information, conferences and conventions

Alumnis help in training


34/43

Visit oftrainers to

work place

of trainees

Application

workshops

Follow up

through mailed

questionnaires


35/43

Decide the place of visit.

Make plan for follow-up.

Develop a interview schedule for follow-up.

Inform the trainees/sponsoring institution about

date of your visit to them.

Visit the trainees/sponsoring institution.

Observe the trainees while working.

Make notes on observation.

Prepare and submit report.


36/43

Plan for the application workshop well in advance.

Decide date of application workshop.

Develop day-to-day programme/ design of

application workshop.

Inform/Invite trainees/sponsoring institutions well

in advance to participate in application workshop.

Conduct application workshop.


37/43

Cheaper method of conducting the follow-

up of training Develop a detailed questionnaire based on the contents covered in a training

course.

Send the questionnaire to trainees/sponsoring institutions after 6 month/ one

year.

Write a letter to all the trainees/ sponsoring institutions to return thequestionnaire after filling it.

Send reminders to those, who have not returned the complete

questionnaires.

Compile all the questionnaires.

Process the data received.

Analyse the responses received from the trainees.

Prepare a detailed report.

Send the report to sponsoring institutions for information.

Take appropriate action on the suggestions given by the trainees for

improvement in the quality of training programmes.


38/43

Systematic follow-up can point out weaknesses inapplication of learning so that they could be correct

and guided to apply the learning properly. It can also tell us the extent to which trainers have

transferred this learning to the work situation and theresults yielded there after.

Follow up during post training evaluation is done

during a period ranging from 3 months to 1 year afterthe training the ideal period is being six months;

In this period trainee would have got the opportunity, ifeverything including the organizational climate is inthe favour, to implement what is learned in the training

phase and Results to be visible to trainee.

To assess whether, trainees have applied their learningin the form of changed behaviour on the job.


39/43

The percentage of getting the response of mailed

questionnaire is very less.

Some times no response is received even by

repeated reminders. Some times the questionnaire is not completely

filled up.

Some times the answer to some of the questions is

given out of concern. Some of the responses are received very late.


40/43

Plan for Application workshop well in

advance

Decide date of Application workshop Develop day to day programme

Inform the trainees / sponsoring institute

about the Application workshop

Conduct Application workshop

Prepare and submit report


41/43

Develop a detailed questionnaire based on the contents coveredin a training

Send the questionnaire to trainees / sponsoring institutions after6 months / one year

Write a letter to all the trainees / sponsoring institutions to

return the questionnaire after filling it Send reminders, if necessary

Compile all the questionnaires

Process the data received

Analyze the responses received from the trainees Prepare a detailed report

Send the report to sponsoring institution for information

Take appropriate action on the suggestions


42/43

1. An analysis of utility of the programme.

2. Finding out reasons for non-adoption of newly acquired skills and knowledge.

3. Reinforcing the motivation and skills of the participants through:

(i) Post - training contact through regular Newsletter or Journal.

(ii) Organizing refresher courses.

(iii) Personal contact.4. Providing help for adoption of newly acquired skills through:

(i) Clarification of doubts and solving technical problems arising out of transfer of

learning.

(ii) Making arrangements for availability of needed equipments materials needed for

learning.

(iii) Providing positive feedback.

5. Encouraging communication among the trainees through:

(i) Formation of Alumni association.

(ii) Development of e-mailing list, for net work among the trainees.

(iii)Informal meetings.


43/43

Thank You forur kind attention

monitoring and evaluation and followup of training programmes prashanth

Documents