chair academy perkins iv sandy

of 57 /57
Perkins IV Perkins IV Assessment of Assessment of Technical Skill Technical Skill Attainment Attainment Wisconsin Technical College System Sandy Schmit WTCS Foundation – WIDS Judy Neill

Upload: sandyschmit

Post on 17-Dec-2014

735 views

Category:

Education


2 download

Embed Size (px)

DESCRIPTION

This is the powerpoint from the presentation I gave at the Chair Academy in Nashville on Technical Skill Attainment in Wisconsin

TRANSCRIPT

Page 1: Chair Academy Perkins Iv Sandy

Perkins IV Perkins IV Assessment of Assessment of Technical Skill Technical Skill

AttainmentAttainment

Wisconsin Technical College System

Sandy SchmitWTCS Foundation – WIDS

Judy Neill

Page 2: Chair Academy Perkins Iv Sandy

Overview of the Wisconsin Technical College System

What does Carl Perkins 2006 (Perkins IV) require?

How does curriculum fit in? How is WTCS using performance

assessment? What will WTCS require for 3rd Party

Assessments? How can WIDS help?

Overview

Page 3: Chair Academy Perkins Iv Sandy

The Wisconsin Technical The Wisconsin Technical College System (WTCS)College System (WTCS)

Page 4: Chair Academy Perkins Iv Sandy

16 Technical Colleges

349 unduplicated Career and Technical Education Programs

1179 Duplicated Career and Technical Education Programs

390,272 enrolled students in 2008, 69,632 full time equivalent

23,617 Degrees granted in 2008

WTCS

Page 5: Chair Academy Perkins Iv Sandy

How do you know when How do you know when your students “get it”?your students “get it”?

Page 6: Chair Academy Perkins Iv Sandy

Assessment?Assessment?

Page 7: Chair Academy Perkins Iv Sandy

Summative Summative Assessment?Assessment?

Page 8: Chair Academy Perkins Iv Sandy

Carl Perkins IV?Carl Perkins IV?

Page 9: Chair Academy Perkins Iv Sandy

What does Carl Perkins What does Carl Perkins 2006 (Perkins IV) 2006 (Perkins IV)

require?require?

Page 10: Chair Academy Perkins Iv Sandy

“Student attainment of challenging career and technical skill proficiencies, including student achievement on technical assessments that are aligned with industry-recognized standards, if available and appropriate.” (U.S. Department of Education, 2006).

The Law

Page 11: Chair Academy Perkins Iv Sandy

Focuses on industry aligned skill sets.

Students know they are acquiring skills that have value & portability.

Increases odds that students will be successful in their employment.

Ensures that WTCS is educating students to meet the needs of employers.

Big Picture Perspective

Page 12: Chair Academy Perkins Iv Sandy

Increased accountability to improve teaching and learning

Accreditation (NCA, AQIP, industry)

Carl D. Perkins IV legislation and funding

Drivers to Process

Page 13: Chair Academy Perkins Iv Sandy

StudentsFeedback about skills needed on the jobAdded value for education

 CollegesImprovement of teaching and learningSupports overall mission

 EmployersAssurance that WTCS graduates have attained technical skills needed on the job

Benefits

Page 14: Chair Academy Perkins Iv Sandy

Carl D. Perkins 2006

Newly revised Career and Technical Education (CTE) Law

Key Concept - Demonstrate that program graduates have attained the knowledge and skills that are industry validated career and technical skills

Page 15: Chair Academy Perkins Iv Sandy

Preparation for postsecondary education and employment

Preparation not only to get a ‘job,’but ‘academic and technical preparation’

Increased emphasis on achievement of a degree, certificate or credential

Perkins IV Areas of  Emphasis

Page 16: Chair Academy Perkins Iv Sandy

Increased Accountability

New accountability provisions are a reminder that:

Funds are not an entitlement

Use of funds is flexible & responsive to the accountability data

Page 17: Chair Academy Perkins Iv Sandy

Separate secondary and postsecondary measures

Performance levels at state and local levels

Negotiations and redirection of funds

Increased Accountability

Page 18: Chair Academy Perkins Iv Sandy

Indirect Measures Data that support

inferences about the quality of teaching and learning and give insight into why achievement is high or low:• Enrollment• Retention rates• Course completion• Student/graduate/

employer satisfaction surveys

• Placement statistics

Direct Measures Data that provide

evidence that students have achieved the intended outcomes:• Performance

assessments with rubrics• Portfolios• Artifacts• Performances• Outcome referenced

tests

Increased Accountability for Direct Measurement of Learning

Page 19: Chair Academy Perkins Iv Sandy

Denominator: number of CTE concentrators who took technical courses during the three year reporting period

Prior accountability required only indirect measurement:

Numerator: Number of CTE concentrators who passed 80% of the technical courses they attempted during the three year reporting period

Measure =

Increased Accountability for Direct Measurement of Learning

Page 20: Chair Academy Perkins Iv Sandy

•Denominator: Number of CTE concentrators who took technical skill assessments during the reporting year

New OVAE Guidance for Perkins IV Measurement requires direct measures of skill attainment:

Numerator: Number of CTE concentrators who passed technical skill assessments that are aligned with industry-recognized standards, if available and appropriate, during the reporting year

Measure =

Increased Accountability for Direct Measurement of Learning

Page 21: Chair Academy Perkins Iv Sandy

Department of Education goal is to have end of program outcomes assessments that are:• Direct measures of outcomes• Independent• 3rd-Party• Employer validated

By 2015 most WTCS assessments should be external or approved assessments.

Increased Accountability for Direct Measurement of Learning

Page 22: Chair Academy Perkins Iv Sandy

WTCS TSA Assessments External TSA Assessment

(ex. NCLEX, Barber/Cosmetology, etc.)

Indirect Assessments

Wisconsin Standards for Assessment - WTCS

Increased Accountability for Direct Measurement of Learning

Page 23: Chair Academy Perkins Iv Sandy

WTCS Assessment StandardsWTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation.

May be developed and/or implemented by the state system.

May be developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval.

Must be valid and reliable.

External Assessment Standards Indirect Assessment Standards

Wisconsin Standards for Assessment - WTCS

Increased Accountability for Direct Measurement of Learning

Page 24: Chair Academy Perkins Iv Sandy

WTCS Assessment Standards External Assessment Standards (ex.

NCLEX, ASE, Barber/Cosmetology)

External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation.

Indirect Assessment Standards

Wisconsin Standards for Assessment - WTCS

Increased Accountability for Direct Measurement of Learning

Page 25: Chair Academy Perkins Iv Sandy

WTCS Assessment Standards External Assessment Standards Indirect Assessment Standards

Non-assessment related indicators such as indirect measures.GPA and/or Course CompletionProgram Completion – Graduation

Teacher-developed exams that do not meet WTCS standard (not WTCS approved)

Wisconsin Standards for Assessment - WTCS

Increased Accountability for Direct Measurement of Learning

Page 26: Chair Academy Perkins Iv Sandy

WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments

WTC

S Assessm

ent

External A

ssessment

Indirect

2012-2013

New Accountability for Direct Measurement of Learning

Page 27: Chair Academy Perkins Iv Sandy

How does curriculum fit How does curriculum fit in?in?

Page 28: Chair Academy Perkins Iv Sandy

Plan•Establish outcomes•Build assessments•Design learning

DO• Deliver• Assess• Document

CHECK• Analyze assessment

data• Propose

improvements

ACT•Adjust design

•Adjust delivery

•Adjust assessment

Assessment Process and Curriculum

Page 29: Chair Academy Perkins Iv Sandy

WIDS Model 3 of 4 components answer assessment questions

What do they need to be able to achieve?

Who are the learners?

How will we know and show when they’ve achieved it?

How will they get there?

WHO

WHAT

HOW

Page 30: Chair Academy Perkins Iv Sandy

Program Outcomes Measurable, observable, and field-

specific skills – major outcomes• Number 5-7 per program (guideline not rule)

Originate from:• Current DACUMs• Accrediting Agencies• National (or other) Skill Standards• Advisory Committees

Threaded through courses Performance verified with

summative assessment of skill performance

Page 31: Chair Academy Perkins Iv Sandy

Integrating Outcomes

Linked Competenci

es(performance assessments)

Exit Learning Outcomes

Program Outcomes(summative assessment)

Core Abilities

(summative assessment)

Gen Ed Outcomes(summative assessment)

Job Task Analysis(DACUM)

Job Task Analysis(DACUM)

External Standards

(Research-based)

External Standards

(Research-based)

Linked Competenci

es(performance assessments)

Linked Competenci

es(performance assessments)

Formative assessment of program outcome

Summative assessment of learning outcomes

External

Program-Level

Course-Level

Drives Learning Design

Assesses Learning Results

Page 32: Chair Academy Perkins Iv Sandy

Drives Learning Design

Assesses Learning Results

External

Program-Level

Course-Level

Dental Hygiene Example

Graduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services.

Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies

Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist.

ADA Dental Hygiene Education Standard

Dental Hygiene Program Outcome

Dental Hygiene CompetencyFormative assessment of program outcome

Summative assessment of Learning outcomes

Page 33: Chair Academy Perkins Iv Sandy

Summative/Formative Assessment

Page 34: Chair Academy Perkins Iv Sandy

Sound Assessment

Valid Outcomes based on standards

(industry) Measures intended outcomes Measures application and critical

thinking

Reliable Performance assessment based on

consistent rubrics, scoring guides, and rating scales

Consistent process - each learner is assessed in same way as other learners

Fair Valid Reliable Learners informed of expectations

up front! Feedback to learners

Page 35: Chair Academy Perkins Iv Sandy

Standards for WTCS Standards for WTCS Approved TSA Approved TSA AssessmentsAssessments

Page 36: Chair Academy Perkins Iv Sandy

WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments

WTC

S Assessm

ent

External A

ssessment

Indirect

2012-2013

New Accountability for Direct Measurement of Learning

Page 37: Chair Academy Perkins Iv Sandy

is valid and reliable;meets WTCS standards/guidelines; is approved by industry (advisory

committee); is developed and/or implemented

collaboratively by the WTCS colleges; ordeveloped and/or implemented locally

according to system guidelines and with industry (advisory committee) approval.

WTCS Assessment Standards

WTCS approved assessment objectively measures student attainment of industry recognized skills upon graduation and:

Page 38: Chair Academy Perkins Iv Sandy

Available in your handout packet

Page 39: Chair Academy Perkins Iv Sandy

WTCS Standards forWTCS Standards for33rdrd-Party External -Party External

AssessmentsAssessments

Page 40: Chair Academy Perkins Iv Sandy

3rd Party TSA Also referred to as:

• External• Direct Method• Credential• License

Generally controls entrance into an occupation

Page 41: Chair Academy Perkins Iv Sandy

Validity / Reliability Alignment to Standards Availability of data Value to the Student Development /

Administrative Costs Pooling / Sharing Resources

Considerations for selecting 3rd Party External Assessments

Page 42: Chair Academy Perkins Iv Sandy

Can we obtain the results/data?

Are cost and administration feasible?

Does it add value or control entry to the student/occupation?

Is it valid, reliable, reputable, recognized?

Is it summative, cumulative or a partial measurement of skills?

Considerations for selecting 3rd Party External Assessments

Page 43: Chair Academy Perkins Iv Sandy

Does the assessment align with program outcomes?

Does the assessment measure what needs to be measured?

Does the data help improve and strengthen our programs?

Considerations for selecting 3rd Party External Assessments

Page 44: Chair Academy Perkins Iv Sandy

WTCS 3rd Party Assessments Currently in Place

Occupations regulated by the Department of Regulation and Licensing• Barber/Cosmetology • Funeral Director • LPN • Registered Nurse• Dental Hygienist• Massage Therapist• Occupational Therapy Assistant• Physical Therapy Assistant• Respiratory Care Practitioner• Veterinary Technician• Wisconsin Registered Interior Designer

Page 45: Chair Academy Perkins Iv Sandy

Guidelines for Selecting 3rd Party Assessments

This is a guiding document to assist districts in determining direct technical skill assessment appropriateness.

Districts will work with the state office staff in the determination.

Keep in mind this is for a small number of programs.

Page 46: Chair Academy Perkins Iv Sandy

Guidelines for Selecting 3rd Party Assessments

Page 47: Chair Academy Perkins Iv Sandy

Concerns: Only measures part of the program

outcomes-”All Code-No Application” Does not indicate job performance Does it help employers? Focused only on technical skills,

what about academic skills and soft skills?

Guidelines for Selecting 3rd Party Assessments

Page 48: Chair Academy Perkins Iv Sandy

3rd Party Assessment - Example

Automotive Service Excellence (ASE) Long-standing credential of

automotive technicians A series of 8 certification

areas/assessments Would it work for Perkins?

Page 49: Chair Academy Perkins Iv Sandy

Logistics Professional CPL/DL-International Society of Logistics

CPIM/CSCP-Association for Operations Management

PLS/CTL/DLP conferred by the American Society of Transportation & Logistics

3rd Party Assessment - Example

Page 50: Chair Academy Perkins Iv Sandy

Federal Aviation Administration (FAA) Airframe and Powerplant Mechanic Certificate

Administered by the FAA Written, practical and oral examination

Required for entry

3rd Party Assessment - Example

Page 51: Chair Academy Perkins Iv Sandy

How Can WIDS Help?How Can WIDS Help?

Page 52: Chair Academy Perkins Iv Sandy

The Learning & Assessment Design Technology Tool Kit

Tra

cDat

B

lack

board

iWebfo

lio

eFo

lio

Quest

ionm

ark

People

Soft

B

anner

Data

tel

Page 53: Chair Academy Perkins Iv Sandy

Create/maintain Program Outcome Summaries for all new and existing programs.

Develop/maintain Course Outcome Summaries for all courses.

Design Performance Assessment Tasks for course level (formative) and summative assessments.

Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses.

Present WIDS documentation as a central feature in accreditation.

How can WIDS help?

WIDS Model and Software Tools:

Page 54: Chair Academy Perkins Iv Sandy

Facilitate collaborative development of:• DACUM or Industry Review Focus

Groups• Program Outcomes• Program Outcome Summaries• Summative Performance

Assessments Produce

• Industry Approval Reports• Analyzer Reports• Updated WIDS files and reports

How can WIDS help?

WIDS Team

Page 55: Chair Academy Perkins Iv Sandy

How do we communicate WTCS long-standing commitment to industry input and standards?

How do we ensure that we use quality assessments? (That we don’t regress to testing regurgitation of information?)

How do we ensure that we actually are doing what our model and assessment plans imply?

How do we collect and report data?

Challenges

Page 56: Chair Academy Perkins Iv Sandy

Current ProjectsCurrent Projects

Page 57: Chair Academy Perkins Iv Sandy

Automotive Technology/Technician NATEF end of program evaluation

Accounting WIDS

Early Childhood WIDS

Criminal Justice WIDS

Welding WIDS

Current Projects