Field Test of Counselor

System

July 2000

AlabamaProfessionalEducationPersonnelEvaluationProgram

Provides feedback from those who are being evaluated and those who are evaluators

Provides data for setting standards

Alabama Counselor Evaluation SystemField Test

Provides a sense of how well the instruments and procedures are working

Provides information on what areas of the evaluation system need changing

Alabama Department of Education Counselor Field Test Evaluation

Alabama Counselor Evaluation System

The evaluation system incorporates the Alabama Comprehensive Counseling and Guidance Model

Guidance curriculumIndividual planningResponsive servicesSystem support

Alabama Department of Education Counselor Field Test Evaluation

Foundations

The evaluation system considers the primary interventions of counselors as approved by the American School Counselor Association in June 1999

CounselingLarge Group GuidanceConsultationCoordination

Eight Competencies: Structure parallels other evaluation systems

1.0 Preparation for Delivery of Services

2.0 Delivery of Counseling and Guidance Services

3.0 Evaluation of Counseling and Guidance Services

4.0 Management of the Counseling/Guidance Program

5.0 Positive Counseling and Guidance Climate

6.0 Communication

7.0 Professional Development and Leadership

8.0 Performance of Professional Responsibility

Alabama Counselor Evaluation SystemCompetencies

Alabama Department of Education Counselor Field Test Evaluation

Alabama Counselor Evaluation SystemData Collection Instruments/Forms

Alabama Department of Education Counselor Field Test Evaluation

• Self-Assessment

• Interview (oral or written)

• Supervisor Review Form

• Observation of Guidance Sessions

• Professional Development Plan (generates Professional

Development/Leadership Competency score during next cycle)

• Pre/Post Observation Conferences

• Evaluation Summary Report

• Portfolio• Surveys

Generates data for all competency areas

Optionally used by the counselor and shared with evaluator only if counselor agrees

Used by the counselor to:

1) Identify areas that need improvement

2) Compare the counselor’s perceptions ofperformance with the results of theevaluation

3) Determine areas for professional growthactivities to be included in aprofessional development plan

Alabama Counselor Evaluation SystemSelf Assessment

Alabama Department of Education Counselor Field Test Evaluation

Two types of interviews: A Structured Interview with 5 questions commonly asked of

all counselors (oral and written options) A Clarifying Interview with probing questions focused on

clarifying or bringing understanding to the written interview and portfolio (questions differ from counselor to counselor)

Clarifying Interview conducted after the evaluator analyzes the written interview and portfolio

Modified scripting is used to record responses in oral option

Alabama Counselor Evaluation SystemThe Interview

Alabama Department of Education Counselor Field Test Evaluation

Alabama Counselor Evaluation SystemThe Observation

Counselor is observed while conducting guidance sessions

Alabama Department of Education Counselor Field Test Evaluation

Counselor may be working with individual students or a group of students

Each counselor is observed on two occasions (three, if needed)

Each observation is accompanied by pre- and post-observation conferences

Generates data for competencies in Management,Communication, and Professional Responsibilities

Provides data where a supervisor should have objective, experiential information

Alabama Counselor Evaluation SystemSupervisor’s Review Form

Alabama Department of Education Counselor Field Test Evaluation

Alabama Counselor Evaluation System

Portfolio

There is no one specific form of packaging required for the portfolio, but there are organizational requirements

Each entry (piece of evidence) is accompanied by a Portfolio Entry Form

Total portfolio requires approximately 15 pieces of evidence

One piece of evidence might serve 2-3 purposes

With multiple purposes, a portfolio may have 8-10 entries (pieces of evidence)

Counselor is responsible for providing some evidence of his/her knowledge and skills

Generates data for 4 Competencies and 7 Indicators

Alabama Department of Education Counselor Field Test Evaluation

Alabama Counselor Evaluation SystemPortfolio

Explicit guidelines for developing evidence in each area are provided in the evaluation/orientation manual

"Extra" entries will not be considered by the evaluator

Evaluated on the quality of materials rather than the quantity of materials in the portfolio

Portfolio

Alabama Department of Education Counselor Field Test Evaluation

Indicator Evidence

2.1 Guidance Survey results2 artifacts around components ofcomprehensive school guidance

2.2 Counseling Survey results1 artifact around consultation

2.3 Coordinates Assessment Survey results1 artifact

3.1 Evaluates Program 1 artifact around evaluation1 artifact around use of results

6.2 Written Communication 2 artifacts around written work

8.3 Cooperates with Peers Survey results, 1 artifact

8.4 Cooperation with School/Comm Survey results, 1 artifact

Alabama Counselor Evaluation SystemPortfolio

Alabama Department of Education Counselor Field Test Evaluation

Surveys are being field tested to determine whether they can be used in the Portfolio and Supervisor’s Review Form as part of the holistic scoring process

Surveys are provided for teachers, students in grades 4-6 and 7-12, and parents

Survey questions address the Competencies and Indicators

Alabama Counselor Evaluation SystemThe Survey

The counselor will distribute, collect, and analyze the surveys

Alabama Department of Education Counselor Field Test Evaluation

Respondents are to remain anonymous and represent a cross-section within the group

Used to present the scores developed from each of the instruments

To complete the ESR, the evaluator:

1. Completes demographic information requested

2. Records the scores from each instrument

3. Writes comments, explanations, or justifications in the space provided that support the score for the specific area

4. Identifies 1 - 2 areas for focus in planning professional development for the coming year(s)

Alabama Counselor Evaluation SystemEvaluation Summary Report

Alabama Department of Education Counselor Field Test Evaluation

Activities focus on areas of improvement as identified in the Evaluation Summary Report

Activities focus on the improvement of student achievement

Alabama Counselor Evaluation SystemProfessional Development Plan

Used to evaluate Competency 7: Professional Development and Leadership after first year of evaluation

Professional

Development

Plan

Alabama Department of Education Counselor Field Test Evaluation

Questions?

Alabama Department of Education Counselor Field Test Evaluation

SYSTEM COMPARISONS

Teachers Special Ed. II Speech/Lang. Counselors Librarians

Observation Observation Observation InstructionalObservation

ManagementObservationInstructionalObservation

Interview 1 added question 1 added question slight differences slight differences

SRF SRF SRF SRF SRF

PDP(Competency 7)

PDP(Competency 7)

PDP(Competency 7)

PDP(Competency 7)

PDP(Competency 7)

Portfolio(7 indicators)

Surveys(Students,

Parents, Faculty/Staff)

Surveys(Students,

Faculty/Staff)

ESR ESR ESR ESR ESR

Teacher System Counselor System

Eight Competencies Eight Competencies

COMPETENCIES

COUNSELORS

Preparation for Instruction

Presentation of Instruction

Assessment of Student Performance

Classroom Management

Positive Climate

Communication

Prof. Development/Ldrship

Prof Responsibilities

Preparation for Delivery of Services

Delivery of counseling and guidance Services

Evaluation of counseling and guidance

Management of Counseling/Guidance Program

Positive Counseling/Guidance Climate

Communication

Prof. Development/Ldrship

Prof Responsibilities

Teacher System Counselor System

Classroom Classroom or Counseling/Guidance Centers

COMPARISONS

COUNSELOR SYSTEM

Academic Content Academic Content, Educational Skills (e.g., decision making), Career Knowledge and Awareness, Knowledge of Self

(e.g. prevention and intervention, skill development)

Equipment/Materials Equipment/Materials, Activities, and Techniques

Management of Behavior Management of Program/Services

Climate Counseling/Guidance Climate

Teacher System Counselor System

Self Assessment Self Assessment(Same format)

Interview Interview

Observation

(3 questions, 5 indicators.Focus around instructionalpreparation, studentassessment)

(5 questions, 6 indicators. Focus aroundpreparation of services and evaluation ofthose services

(17 indicators) Observation

(Focus on presentation of guidanceservices. Some indicatorsare the same)

Pre/Post Observation Pre/Post Observation (Same procedures, first two questions adapted for counselors)

INSTRUMENTS/DATA SOURCESCOUNSELOR

Teacher System Counselor System

(Adds program management, rest sameas teacher system)

Supervisor’s Review Form Supervisor’s Review Form

Evaluation Summary Report ESR is same except for adding portfolioscoring

PDP PDP has no changes

(Not scored in field test since no pastPDP has been developed)

Competency 7 Scoring Competency 7 Scoring is same

INSTRUMENTS/DATA SOURCESCOUNSELOR

Portfolio and surveys (No Equivalent)