tf – sp capability development programme for competency-based assessment
Post on 28-Jan-2016
48 Views
Preview:
DESCRIPTION
TRANSCRIPT
TF – SP Capability Development Programme for Competency-Based Assessment
Facilitator: Dennis Sale
Programme Objectives
• Identify the main components of a competency-based qualification system (e.g., Singapore Workforce Skills)
• Explain the process and principles of good assessment practice
• Analyse the standards for developing and conducting competency-based assessment
• Produce an assessment plan for a element of competence
• Produce assessment methods and tools for an element of competence
• Validate an assessment plan and accompanying assessment tools
• Produce a portfolio of evidence for meeting the competency standards for assessment
• Identify the importance of verification in the quality assurance of assessment practices
• Identify the role and duties of an Internal Verifier
Competency Based Approach
Key features:
• Meeting specific work related functions
• Meeting a defined standard (at different levels) of competence
• Assessment is mainly through direct performance in real work activity
Singapore Workforce Skills Qualifications (WSQ) - Competency-Based System
Competency Unit
CompetencyElement 1
CompetencyElement 2
CompetencyElement 3
Underpinning KnowledgePerformance CriteriaRange and ContextEvidence Sources
Underpinning KnowledgePerformance CriteriaRange and ContextEvidence Sources
Underpinning KnowledgePerformance CriteriaRange and ContextEvidence Sources
Competency Unit 1: Develop A Competency-Based Assessment
Competency Elements
1.Prepare an assessment plan
2.Develop competency-based assessment tools
3.Validate assessment plan
Competency Unit 2: Conduct Competency-Based Assessment
Competency Elements
1.Prepare for conduct of assessment
2.Prepare candidate for assessment
3.Carry out conduct of assessment
Conduct Competency-Based AssessmentCompetency Element Title1. Prepare for Conduct of Assessment
• How to prepare for conduct of assessment
Underpinning KnowledgeA competent individual needs to know and understand:
Performance CriteriaA competent individual must be able to successfully perform the following:
1.1 Confirm details of assessment plan with relevant stakeholders1.2 Ensure resources stipulated in the assessment plan are available, safe and in proper working conditions1.3 Ensure the set up of assessment site is in accordance with assessment plan and workplace health & safety guidelines1.4 Ensure the assessment tools required for assessment are available for use
Conduct Competency-Based AssessmentCompetency Element Title1. Prepare for Conduct of Assessment
Range & Context• Confirm details of assessment plan with relevant stakeholder (e.g., evidence requirements, resources needed for assessment, organisational requirements)• Ensure availability of resources ( e.g., tools equipment , technical support, etc) and assessment tools (assessment instruments, tools, etc)
Evidence SourcesExamples of evidence needed to meet the standards required for this element
ProcessObservation of work, which includes• confirming details of assessment plan with stakeholders• checking assessment site is appropriately set up and assessment instruments are available for use
Product• Assessment site is set up according to assessment plan
A Structured Assessment Process
Stage 1 Stage 2a Stage 2b Stage 3 Stage 4
Planning for assessment with candidate
Reviewing performance and knowledge evidence
Judging all evidence against agreed standards
Recording decisions and giving feedback
Contributing to quality assurance processes
Technical Principles of Assessment
Assessment must comply with the following Technical Principles of
assessment:
• Validity• Reliability• Flexibility• Fairness
These Technical Principles of Assessment must be fully addressed in the design and conduct of the assessment process as well as the
development of assessment items and instruments
Validity
In basic terms, validity refers to the ‘truth’ of the assessmentsmade. A valid assessment is one that measures accurately what itis we intend to measure.
For example, a valid driving test is one in which driving skills(Performance) is measured in typical traffic (Conditions) againstthe criteria established by the Motoring Authority (Standard)
Reliability
Reliability refers to the capability of a test to produce the same scores with different assessors (persons scoring the test)
Grade A
Grade A
Grade A
Assessor 1 Assessor 2
Assessor 3
Flexibility
Flexibility is concerned with the process of assessment, not the standard of the assessment.
For example, candidates can display the competence in a range of situations (e.g., on the job, simulated conditions) at mutually convenient times, etc), provided the competence is validly and sufficiently demonstrated.
Fairness
Fairness relates to a number of considerations in assessment. However, they are all concerned with ensuring that candidates, when being assessed, are provided with appropriate access to the assessment activities and are not unfairly discriminated against in the assessment process.
Unfair discrimination typically means discrimination based on criteria unrelated to the assessment activity itself, for example, gender or racialcharacteristics.
Fairness is a general concern throughout assessment, relating as much toproviding candidates with sufficient knowledge and time for assessment, to non-discriminatory processes in marking their work.
Rules of Evidence
Evidence of competence must be:
• Valid• Sufficient• Current • Authentic
Sufficient
Sufficient refers to ‘how much assessmentevidence’ do we need in order to feel confidentthat a candidate is competent in the area assessed?’
Current
Currency of evidence refers to how recent the evidence is generated and whether it fully relates to the most up-to-date knowledge, skills and practices for the work function being assessed.
This needs to be checked against the industry standardand any specific policy guidelines stated.
Authentic
Quite simply this refers to how sure we are that the work produced has been done by the candidate
In an examination, we can be more confident of authenticity
However, with assignments done by candidates in their own time, authenticity becomes a concern
Efficiency
Assessment can be time consuming. It is important to:
Identify assessment opportunities that enable the assessment of as wide a range of performance criteria as possible
Use methods that are efficient for what we are assessing
Avoid over-assessing – be clear about what is Sufficient evidence
Identifying relevant evidence
Assessors and candidates need to be clear on:• What types of evidence are needed to meet the specific
performance criteria
• What underpinning knowledge can be assessed either through performance or questioning (written or oral)
• Arrangements for the assessment (e.g., equipment, materials required)
Collecting Evidence of Competency 1
• Assessment of competence must involve a full demonstration of the competence – and should involve avariety of evidence types and contexts where possible
• At least one form of direct evidence should be used in judging practical performance (e.g., observation of simulated performance)
• Supplementary (indirect evidence) should be used in support of direct evidence to:– Extend range of evidence and possible performance in rare
situations (e.g., theft, breakdown) – Facilitate transfer to new situations/contexts– Assess underpinning knowledge
Collecting Evidence of Competency 2
• Evidence gathering methods should be culturally inclusive and take into account the language, literacy and numeracy skills of the participant
• Reasonable adjustments may be considered for participants with physical disabilities or special learning needs
• Evidence of competency must include occupational safety and health and other legislative aspects of the job
• Prior learning can be recognized providing it can be validated by accepted certification.
Assessing Underpinning Knowledge
• While to some extent underpinning knowledge is implicit in performance – it should be separately assessed to ensure understanding (the why behind the what and how)
• The use of verbal questioning (especially immediately after demonstrating competence) is an effectively way to assess underpinning knowledge
• Use oral or written work related questions in assessing underpinning knowledge (avoid MCQ’s)
Assessment Context
The assessment context is the participant’s current place of work and job role. A judgement of competence denotes that the participant is able to produce the desired outcomes reflected in the competency element within the requirements of the work role and environment.
Assessment may be undertaken on and off the job – the essential reference for the collection of evidence is the actual Competency Unit
Competency Unit 1: Develop A Competency-Based Assessment
Competency Element 1:
Prepare An Assessment Plan
Purposes of the Assessment Plan
• Show how the Unit is to be assessed - detailing actual assessment process, methods and instruments
• Calibrating assessment to actual competency elements and performance criteria, etc
• Provides explicit guide to trainers, trainees and other stakeholders as to how the assessment is to be carried out in relation to Principles & Rules of Assessment
Prepare An Assessment Plan
Meeting this competency requires you to:• Establish the purpose and context of assessment in consultation
with relevant people • Obtain information on the characteristics of candidates • Identify the types of evidence to be gathered based on the
competency standard• Determine what competences will be assessed (e.g., elements,
performance criteria, etc)• Select appropriate assessment methods and tools to collect the
required evidence• Identify assessment location, physical resources and specific
materials required for conduct of assessment
Competency Unit 1: Develop A Competency-Based Assessment
Competency Element 2:
Develop Competency-Based Assessment Tools
Develop competency-based assessment toolsMeeting this competency requires you to:• Develop holistic and integrated assessment activities based on the
assessment plan• Develop specific assessment tools that are appropriately
contextualized for the target audience• Develop clear instructions for assessor and candidate on the
conduct of assessment• Develop assessment tools that meet the Principles of Assessment
and the Rules of Evidence
Competency Unit 1: Develop A Competency-Based Assessment
Competency Element 3:
Validate Assessment Plan
Meeting this competency requires you to:• Identify the purpose, focus and approach to validation• Conduct the validation activities and address the critical
findings • Produce a report recommending improvements to the
assessment strategies, process and tools
Validate Assessment Plan
Competency Unit 2: Conduct Competency-Based Assessment
Competency Element 1:
Prepare For Conduct ofAssessment
Meeting this competency requires you to:
• Confirm details of assessment plan with relevant stakeholders• Ensure resources stipulated in the assessment plan are available,
safe and in proper working conditions• Ensure the set up of assessment site is in accordance with
assessment plan and workplace health & safety guidelines• Ensure the assessment tools required for assessment are available
for use
Prepare For Conduct Of Assessment
Competency Unit 2: Conduct Competency-Based Assessment
Competency Element 2:
Prepare Candidate forAssessment
Meeting this competency requires you to:• Explain the purpose and context of assessment to candidate
• Brief candidate on the assessment requirements and process, including clear instructions on the assessment to be taken
• Brief candidate on his rights and process of appeal to the assessment outcome
• Communicate in a clear and supportive manner to create a conducive assessment environment
• Confirm with candidates on any special needs he/she may have and explain how those needs will be addressed during the assessment
• Seek feedback from candidate and clarify his/her understanding of the assessment requirements
Prepare Candidate for Assessment
Competency Unit 2: Conduct Competency-Based Assessment
Competency Element 3:
Carry Out Conduct of Assessment
Meeting this competency requires you to:• Use assessment plan and tools to carry out the conduct of assessment• Use evidence gathered to decide if the relevant criteria are met and make
assessment decision• Deal with limitations of evidence required, where necessary• Comply with the Code of Practice for Assessors when conducting assessment
and making assessment decisions• Record assessment outcomes promptly and accurately in accordance with
assessment system, policies and procedures• Provide clear and constructive feedback to candidate regarding assessment
decision and advice on how performance gaps can be overcome• Review assessment plan and process and provide feedback to appropriate
personnel for modification, if required
Carry Out Conduct of Assessment
When giving feedback to candidates
1. Let candidates give their opinion on the assessment first 2. Identify positives before areas for improvement3. Limit what you cover4. Be specific, not general 5. Focus on what can be changed/improved6. Give the candidate time to think and respond7. Keep to the standards8. Check for understanding9. Listen to how the feedback is received (including observation of non-
verbal communication)10. End on a positive (with encouragement)
Verification: Quality Assurance in Assessment
Verification is the main quality assurance process associated with assessment systems and practice - whether competence-based or otherwise
Quality Assurance is a vital part of any credible qualification system. It must ensure that everyone involved in the assessment of candidates is assessingcorrectly, working to agreed procedures and to an agreed standard of performance
Verification: The Range of Roles & their Relationships
External Verifier
Internal VerifierCoordinator
Internal Verifiers(programme level)
Assessors
Awarding Body
Centre
Centre
Centre
Workplace
Internal Verifier (IV): The key Role in Quality AssuranceThe internal verifier(IV) is the key person(s) for ensuring quality
of assessment practices at the institutional level (Centre)
The IV conducts internal quality assurance of the assessment
process by:
1. Carrying out and evaluating internal assessment and quality assurance systems
2. Supporting assessors
3. Monitoring the quality of assessors’ performance
4. Meeting external quality assurance requirements
Carrying out and evaluating internal assessment and quality assurance systems
This will involve:• Comparing own institution requirements with those of the external
awarding body• Identifying the outcomes needed by the agreed standards and their
consequences for internal auditing• Auditing existing administrative and recording arrangements -
modify/ change if necessary - to meet external audit requirements• Carrying out assessment standardization arrangements (e.g.,
sampling strategy)• Ensuring a procedure for complaints and appeals is in place (note:
consider this in local context)• Identifying problems, developing improvement plans
Sampling strategy
This will vary from institution, according to needs (e.g., size,
programmes offered, resources, etc) but it must be agreed with the
External Verifier.
The following need to be taken into account:• Size of sample to ensure reliability
• Assessment methods used by the centre (e.g., to ensure good principles of assessment are practice, etc)
• Assessors – number, experience, workload, location
• Candidate cohorts (e.g., full-time/part-time, different employees, etc)
• The programmes offered – particularly if new/revised
• All units must be included, but there may be a greater focus on any problem unit(s)
Standardizing assessment decisions
It is essential to ensure, as far as is possible, that:
• Assessors consistently make valid decisions• Assessors make the same decision on the same evidence base• All candidates are assessed fairly
This involves:• conducting moderation activities (e.g., a number of assessors
judging the same candidate’s evidence and comparing their perceptions and decisions)
Supporting assessors
This will involve:• Ensuring assessors have appropriate technical and
vocational experience• Ensuring that assessors are competent to assess• Identifying the developmental needs of assessors and
facilitating any necessary training• Monitoring progress and development of assessors
Monitor the quality of assessors’ performanceThis will involve:
• Ensuring that assessors are preparing and planning assessment effectively and efficiently (e.g., understand the standards and applying principles of good assessment, etc)
• Observing and questioning individual assessors to ensure good practice and consistency over time
• Checking records of the assessment process (from assessment planning to summative decisions)
• Giving assessors accurate and helpful feedback
Meet external quality assurance requirementsThis will involve:• Identifying how internal assessment will be checked externally and
ensuring these requirements can be met• Planning, collecting and analyzing information on internal
assessment decisions (note: this involves the secure keeping of all records relating to the assessment of candidates)
• Maintaining contact and meeting with the external verifier(s)• Providing feedback on the outcomes of meetings with the external
verifier to centre assessors• Acting on the feedback given by external verifiers
Checklist of the range of information needed for purposes of External VerificationIt is the Internal Verifiers responsibility to have the following information
quickly accessible:• Number of registered candidates per qualification level• Assessor details (e.g., CV’s, Assessor qualifications, development plans)• Candidate progress reviews and achievements• Assessment records and plans• Candidate support resources available (e.g., guide for candidate assessment)• Assessments sites used – where the assessments are carried out• Internal verification sampling strategy• Internal Verifier records, including feedback to Assessors and feedback from
Candidates• Record of claims for certification• Candidate assessment evidence (e.g., portfolios)
Dennis Sale can be contacted at:
Email: dennis_sale@sp.edu.sg
Telephone: (65) 6772 1490
Blog: dennissale.com
top related