3-1 copyright © 2013 mcgraw-hill education (australia) pty ltd hill, hill, perlitz, professional...

3-1Copyright © 2013 McGraw-Hill Education (Australia) Pty Ltd Hill, Hill, Perlitz, Professional Training & Assessment, 1e

CHAPTER 3

TAEDES505A

Evaluate a training program


Plan and prepare evaluation

For an evaluation undertaken to be of any real value it must be properly planned: you need to be clear about the purpose of the evaluation and what criteria will be used to determine outcomes.


Aim and scopeAim

• Examine the effectiveness of the training

• Is there a demand for this qualification?

• Will training ensure an employment outcome for the learner?

• Return on investment

• Increase in productivity

Scope

• Examine the whole of the organisation

• Examine only specific parts of the organisation

• Examine only specific qualifications

• Examine operational issues


Information sources

• Analysis of work performance/observation

• Analysis of workplace productivity

• Feedback from employers, students and trainers

• Secondary research from government, ISBs or other industry bodies

• Organisational documents: policies and procedures

• Legislative documents


Who are the stakeholders?

• Employers• Training graduates• Workplace supervisors• Trainers and assessors


The project plan

• Roles and responsibilities of review team members• Constraints on the project• Project risk assessment• Ground rules for the project


Collect and collate data

With the project plan prepared and task roles and responsibilities allocated, the review process can begin.

In the initial stages this means gathering all relevant documents and information and collating the results for evaluation.


Evaluation instruments

• Surveys• Questionnaires• Direct examination of outcomes• Diaries or logs• Workplace observation


Example of an observation/competency checklist

In responding to the scenario, the student: Yes No

Described the value of interpersonal and communication skills in building a rapport with customers

Described how seeking assistance from others can overcome limitations in customer service delivery

Discussed methods by which they can determine and address customer needs and expectations

Described methods by which customer complaints could be dealt with quickly, effectively and courteously

Explained how gathering feedback can assist an organisation in improving its customer service standards

Explained the steps involved in building a positive relationship with customers Discussed methods by which customer service levels could be monitored to ensure

consistency Described methods of reporting deficiencies in service standards to management

Described the difference between different customer satisfaction levels

Successfully identified opportunities to enhance the service given to their customer Demonstrated problem solving skills in overcoming customer complaint and negative

reactions by:o using positive listing skillso using positive body language o asking effective questionso offering solutions and listening to customer reactions


Analysing information

Regardless of whether you are reviewing the RTO’s entire operation or only selected segments, the process will be the same: the actual documents and procedures will need to be crosschecked and compared to the legislative or organisational standards to determine if they are compliant and meet requirements.

Examples of what to look for follow ...


Learning and assessment strategy• Have these strategies been developed in line with the relevant qualification’s

packaging rules?

• Has the latest version of the qualification summary been used so that all unit codes and benchmarks are correct?

• Have all the core units of the qualification been included?

• Have the elective units been chosen with workforce development outcomes in mind?

• Have all necessary prerequisite units been observed and included? Are these units being delivered in the right order?

• Is the strategy aimed at a specific target group and if so:

– does the strategy detail all the necessary resources needed for these learners to complete the program successfully?

– are the delivery methods relevant to this target group?

– is the duration of the training program sufficient to allow the target group to complete successfully?

– does the program address workforce development needs?


Assessment tools• The review of these tools should be conducted with the

unit/s of competency close to hand.

• The assessment tool should be compared to and should encompass the unit’s performance criteria as well as any other requirements as outlined in the individual units.

• Other aspects of assessment review will include the level of moderation and validation the assessment tool has received:

– how often is moderation and validation undertaken?

– what recommendations were made and what action was taken?

• Check that the assessment is appropriate to the AQF level at which it is aimed.


Program documentation

• Have student details been entered correctly?

• Is the record of their progress through the qualification accurate and up to date?

• Have completed assessment tools been marked and stored appropriately?

• Have all necessary forms and documents been completed accurately and in a timely manner?

• Are reporting requirements (such as AVETMISS) being completed accurately and on time?


Compliance with ASQA Standards

In all cases, regardless of what is being reviewed, the RTO must measure its performance against the ASQA Standards.

A condition of registration is that these standards be upheld and met at all times. This can be achieved through adherence to a continuous improvement program and the development of checklists or forms that can be used during an evaluation process.


Documenting findings

ASQA STANDARD SNR 4: The applicant must have strategies in place to provide quality training and assessment across all of its operations4.1 The applicant has a defined continuous improvement strategy that allows for the collection and analysis of data. The strategy includes implementation of continuous improvement activities for training and assessment.

Evidence Result

The RTO does not currently have a formalised method of ensuring continuous improvement. No records are kept of student feedback and there is no evidence that the RTO discussed feedback during staff meetings.

CompliantNon-compliantNot audited

Table 3.4 Self-audit document with non-compliances shownSource: www.training.com.au/pages/menuitem5cbe14d51b49dd34b225261017a62dbc.aspx


Factors that affect performance

• Job redesign

• Change in supervision arrangements

• Change in work support/mentoring

• Communication and information campaigns

• Job aids

• Standard operating procedures


Reporting and recommendations

The final part of the review process is the formal report on the findings of the evaluations undertaken.

This report should then, ideally, form part of the organisation’s continuous improvement program.

The report should include information on any breaches of compliance identified as well as suggestions for their rectification.


Making recommendationsAny suggestions made must be discussed with relevant stakeholders prior to formalising the report. There is little point in making a prescribed recommendation if the organisation is unable to meet the suggested pathway or alternative.

While compliance with the ASQA standards is not negotiable, there are generally a number of methods that can be used to ensure that these standards are indeed met and discussions with stakeholders at the end of the evaluation process can identify the best ways of rectifying any areas identified for improvement.


Evaluation report

ASQA STANDARD SNR 4: The applicant must have strategies in place to provide quality training and assessment across all of its operations4.1 The applicant has a defined continuous improvement strategy that allows for the collections and analysis of data. The strategy includes implementation of continuous improvement activities for training and assessment.

Evidence Result

The RTO does not currently have a formalised method of ensuring continuous improvement.

No records are kept of student feedback and there is no evidence that the RTO discussed feedback during staff meetings.

CompliantNon-compliantNot audited

Rectification required:Produce and file minutes of all meetings to ensure that continuous improvement issues are discussed with staffProduce action plans, based on feedback received from students, employers and staffKeep a written continuous improvement log

Table 3.6 Rectification report (extract)

3-1 copyright © 2013 mcgraw-hill education (australia) pty ltd hill, hill, perlitz, professional...

Documents