colorado department of education, educator effectiveness winter 2014

Colorado Department of Education, Educator Effectiveness Winter 2014

Implementing the Model System for

Evaluating Colorado’s Educators

Legislative Overview, Professional Practice and Measures of Student

Learningfor CACTA

Introductions

How many of you have attended a CDE training or a training in your

school/district on the State Model

System?

How Do You Feel About SB 10-191? Choose a picture from below that best reflects your feelings

regarding SB 10-191. Why did you select the picture you did?

Using the Post-It notes in front of you, write down any questions you have regarding the implementation of Senate Bill 10-191. Be sure to write one question per Post-It note!

Place your questions under the most applicable posters hanging around the room. SB 10-191 Legislation Professional Practice/Rubric Measures of Student Learning Decision Framework Other Questions

Understanding Your Needs

Purposes of S.B. 10-191

A system to evaluate the effectiveness of licensed personnel and continually improve the quality of education and student outcomes.

Provide meaningful feedback for professional growth and continuous improvement.

Provide a basis for making decisions in the areas of hiring, compensation, promotion, assignment, professional development, earning and retaining non-probationary status, dismissal, and nonrenewal of contract.

Critical Effects of S.B. 10-191

Requires statewide minimum standards for what it means to be an _________ teacher or principal

Requires ______ evaluation of all teachers and principals

Requires that all teachers and principals be evaluated at least ____% on the academic ______ of their students

annual

“effective”

50 growth

Critical Effects of S.B. 10-191Changes non-probationary status from one that

is ______ based upon years of ______ to one that is ______ based upon three consecutive years of demonstrated ___________

Provides that non-probationary status may be ___ based upon two consecutive years of

____________Makes non-probationary status ________ Prohibits _____ placement of teachersforced

“portable”

lostineffectiveness

earnedearned

effectiveness

service

Priorities of Implementation

Human judgment Data should inform decisions, but human judgment will always be

a part of the process Processes and techniques are recommended to improve individual

judgment and minimize errors and bias

Embodiment of continuous improvement by monitoring Data from pilot and rollout intended to capture what works and

what doesn’t Changes in assessment practices and tools Emerging research and best practices

Providing credible and meaningful feedback with: Actionable information Opportunities for improvement Idea that this is a process and not an event

Involves all stakeholders in a collaborative process Families, teachers, related service providers, administration,

school board, etc. Educators involved throughout development process


Takes place within a larger, aligned and supportive system All components of the system must focus on increasing the

number of educators and students who are successful


Timeline for Implementation of New Requirements for Personnel Evaluation

Systems2013-14: New performance evaluation system based on Qual i ty Standards wi l l be

implemented statewide per State Board rule. Teachers wi l l be evaluated based on qual i ty standards. Demonstrated effectiveness wi l l begin to be considered in the acquis ition of

non-probationary status.

2014-15: Continued implementation. Demonstrated effectiveness or ineffectiveness wi l l be considered in the

acquisition or loss of non-probationary status.

2015-16: First year that non-probationary status can be lost based on 2 consecutive

years of demonstrated ineffectiveness.

Definition of Teacher Effectiveness

Effective Teachers in the state of Colorado have the knowledge, ski l ls , and commitments needed

to provide excel lent and equitable learning opportunities and growth for a l l students. They str ive to support growth and development, c lose achievement gaps and to prepare diverse student populations for postsecondary and workforce success. Effective Teachers faci l i tate mastery of content and ski l l development, and employ and

adjust evidence-based strategies and approaches for students who are not achieving mastery and students who need acceleration. They also develop in students the ski l ls , interests and abi l ities necessary to be l i fe long learners, as wel l as for democratic and civ ic participation. Effective Teachers communicate high expectations to students and their famil ies and util ize diverse strategies to engage them in a mutual ly supportive teaching and learning environment. Because effective Teachers understand that the work of ensuring meaningful learning opportunities for a l l students cannot happen in isolation, they engage in col laboration, continuous reflection, on-going learning and leadership within the profess ion.

STATE COUNCIL FOR EDUCATOR EFFECTIVENESS

Framework for System to Evaluate Teachers


I. Know Content

50% Professional Practice Standards 50% Student Growth MeasuresWeighting: How Much Does

Each Standard Count Towards Overall Performance?

Observations of Other Measures Teaching Aligned with

CDE Guidelines

State Other Assessments Other MeasuresSummative for Non-tested Aligned with Assessments Areas CDE Guidelines

Match of test to teaching assignments

Weighting:Scoring Framework: How Do Measures of Quality Standards

Result in a Determination of Individual Performance?

Performance StandardsIneffective Partially Effective Effective Highly Effective

Quality StandardsII. Establish

EnvironmentIII. Facilitate

LearningIV. Reflect on

PracticeV. Demonstrate

LeadershipVI. Student

Growth

Appeals Process

Definition of Principal Effectiveness

Effective Principals in the state of Colorado are responsible for the collective success of their schools, including the learning, growth

and achievement of both students and staff. As schools’ primary instructional leaders, effective Principals enable critical discourse and data-driven reflection about curriculum, assessment, instruction, and student progress, and create structures to facil itate improvement. Effective Principals are adept at creating systems that maximize the utilization of resources and human capital, foster collaboration, and facil itate constructive change. By creating a common vision and articulating shared values, effective Principals lead and manage their schools in a manner that supports schools’ abil ity to promote equity and to continually improve their positive impact on students and families.


Framework for System to Evaluate Principals

Definition of Principal Effectiveness

I. Strategy II. Instruction III. CultureV.

ManagementIV. Human Resources

VI. External Development

VII. Student Growth

50% Professional Practice Standards 50% Student Growth MeasuresWeighting: How Much Does


Number and Percentage Other Measures of Teachers Aligned with CDE

Guidelines

School Performance Other Measures Framework Aligned with CDE Guidelines

Weighting:Scoring Framework: How Do Measures of Quality Standards


Performance Standards

Ineffective Partially Effective Effective Highly Effective

Quality Standards

Principal Standards Teacher Standards

Principal and Teacher Quality Standards

What did you notice about thealignment between the Principal and Teacher standards?

2. Annual

Orientation

3.Self-

Assessment

4.Review of

Annual Goals and

Performance Plan

5.Mid-Year Review

6.Evaluator

Assessment

7.End-of-Year

Review

8.Final Ratings

9.Goal-Setting

and Performance

Planning

1.Training

Evaluation CyclePrincipal/AssistantPrincipals and Teachers

May 15Within the first two weeks of

school.

End of September.

Prior to the beginning of

Spring Semester

Train: Prior to the beginning of

School.Orient: Within

the first week of School.

End of MayMid-JuneEnd of June

State Model Rubric Basics

Standards basedCumulative in contentEach level of the rubric represents an increase

in the quality, intensity, consistency, breadth, depth, and complexity of practice

Outlines the practices that you must meet to be at standard

Scoring the Rubric

Determining the teacher ’s professional practices rating is a three-step process that involves rating the individual elements and standards and using those to determine the overall rating on professional practices.

1. Rating the Elements2. Rating the Standards3. Determining the Overall Professional Practices

Rating

Rubric Structure and Rating Level Focus

The focus of the Basic rating is the educator whose

performance does not meet state quality standards. The

educator rated as Basic is typically performing at a foundational level. Every

educator is expected to perform Basic professional practices in

their day-to-day work.

The focus of Partially Proficient and Proficient levels is what educators do on a day-to-day basis to achieve state

performance standards and assure that students are achieving at expected levels.

The focus of Accomplished and Exemplary ratings shifts to the outcomes of the educator’s practices, including

expectations for staff, students, parents and community members, as a result of practices exhibited under rating levels

2 and 3.

Teacher Quality Standard Performance

Rating Levels

Element of the Standard

Professional Practice is Not Observable Professional

Practice is Observable

Components of the Rubric

Starting in the Basic rating level and moving across, look for the first unchecked professional practice. Move one column back to identify the rating for the element.

Understanding the Scoring “Business” Rule

Determining the Element Rating

Starting in the Basic rating level and moving across, look for the first unchecked professional practice. Move one column back to identify the rating for the element.

Rubric Rating LevelsStandard

Basic Partially Proficient Proficient Accomplished Exemplary

Element

Professional Practices





0Educator’s

performance on professional practices is significantly

below the state performance

standard.

1Educator’s

performance on professional practices is

below the state performance

standard.

3Educator

exceeds state standard.

2Educator meets

state performance

standard.

4Educator

significantly exceeds state

standard.

3

2

1

2210

Proficient

First, determine each Standard’s contribution to the overall professional practice rating by using the following formula:

(Total Pts. Earned for Std.) X (Std. Weight X No. of Stds.)(Number of Elements Associated with Standard)

Determining Overall Professional Practice Rating

= 2

Example: 10 X (.20 X 5)

5 Once you’ve completed this step for every Standard, you are

then able to determine the overall professional practice rating.

Determining Overall Professional Practice Rating

Measures of Student Learning and Outcomes

29

2. Annual

Orientation

3.Self-

Assessment

4.Review of

Annual Goals and

Performance Plan

5.Mid-Year Review

6.Evaluator

Assessment

7.End-of-Year

Review

8.Final Ratings

9.Goal-Setting

and Performance

Planning

1.Training

Evaluation CyclePrincipal/AssistantPrincipals and Teachers

May 15Within the first two weeks of

school.

End of September.

Prior to the beginning of

Spring Semester

Train: Prior to the beginning of

School.Orient: Within

the first week of School.

End of MayMid-JuneEnd of June

Review and finalize compiled results of outcome measures

Include any new district or school

decisions for measuring Student Learning Outcomes

Confirm measures used to determine Student Learning

Outcomes – Finalize baseline data

Review available data from measures

to determine if students are on

track


Framework for System to Evaluate Teachers


I. Know Content

50% Professional Practice Standards 50% Student Learning Outcome MeasuresWeighting: How Much Does


Observations of Other Measures Teaching Aligned with

CDE Guidelines

State Other Assessments Other MeasuresSummative for Non-tested Aligned with Assessments Areas CDE Guidelines

Match of test to teaching assignments

Weighting:Decision Matrix: How Do Measures of Quality Standards


Performance RatingsIneffective Partially Effective Effective Highly Effective

Quality StandardsII. Establish

EnvironmentIII. Facilitate

LearningIV. Reflect on

PracticeV. Demonstrate

LeadershipVI. Student

Growth

Appeals Process

Teacher User’s Guide, Page 13

From Student Academic Growthto

Measures of Student Learning

Measures of Student Learning Using multiple measures to

determine student learning over time.

Colorado Growth Model (CGM)

establishes technical

measure of “growth.”

Requirements for Teacher Evaluation

50% Student

Academic Growth

VI. Responsibility for student academic growth

Evaluated using the following:(1) a measure of individually-attributed growth, (2) a measure of collectively-attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model.

Refers to outcomes on a measure that are

attributed to an individual licensed person,

e.g. DRA2 outcomes for a 1st Grade Teacher’s

students

Refers to outcomes on a measure attributed to two or more licensed

personnel, e.g. 10th gr. Math TCAP –

All Secondary math teachers in school

Approaches for Selecting and Using Multiple Measures in Teacher

Evaluation

Step 1: Begin with the Colorado Academic Standards

Step 3: Group available assessments according to teacher type

Step 2: Identify available assessments used to evaluate student learning throughout the year

Step 4: Select and preliminarily weight assessments for use in educator evaluations

Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth

Step 6: Combine weighted scores into a measures of student learning rating

Measures of Student Learning Guidance for Districts V 2.0

http://www.cde.state.co.us/sites/default/files/MeasuresofStudentLearningFINAL081413.pdf

Begin with the Colorado Academic Standards The Colorado Academic Standards impact curriculum development which should then inform assessment development

Step 1: Measures of Student Learning


Evaluation









Determine how student learning is currently measured in your districtConduct an assessment inventory to identify

what is currently being used to measure student learning

Identify where gaps exist


Districts, BOCES and schools may decide after completing the Assessment Inventory that a locally created assessment would likely be included as a measure in evaluation.

Districts and BOCES can use the Assessment Review Tool to show they are taking steps to ensure that the locally created measurements they select are fair, valid, and reliable.

Using Local Assessments in Evaluation

Assessment Review Tool

Criteria used in this tool:Alignment and DOK Scoring Fair and UnbiasedOpportunities to Learn

Assessment Review Tool

Assessment Support

Content Collaboratives P-12 educators from around the state gathered to identify and

create a high-quality assessment resource bank, which is aligned to the new Colorado Academic Standards and may be used in the context of Educator Effectiveness evaluations.

The Content Collaboratives, CDE, along with state and national experts, will establish examples of student learning measures within each K – 12 content area including:

Cohort I

Dance Drama & Theatre Arts Music Reading, Writing and Communicating

Social Studies Visual Arts

Cohort IIPhysical Education Science World Languages Comprehensive Health

Mathematics CTE


Evaluation









Group available assessments according to teacher typesThis type of categorization helps make

expectations clearer to all teachers on how different types of assessments will be considered in their evaluations.



Possible example of teacher/assessment grouping:


Evaluation









Select and weight multiple measures of student learning to be included in educator evaluationsUse your completed Assessment Inventory to

determine what measures to include in evaluation

Student Learning Outcomes Tool can help districts visualize selections



Evaluation









Set student learning targets and scales based on data from the selected measures.Use the student learning objective process

to measure student outcomes based on various types of assessments.

Step 5: Student Learning Objective Process

What is the Student Learning Objective Process? Enables educators to utilize academic standards

to establish learning outcomes for students, monitor progress and evaluate the degree to which students achieve those outcomes

In educator evaluation systems, this process serves as a method of measuring student growth, progress or mastery of the standards

Student Learning Objective Process

The essential steps of the Student Learning Objective Process include: Understanding the Colorado Academic Standards in order to

select learning outcomes Collecting baseline information to inform target and scale

setting Assessing quality, attainment level and rigor of student

learning targets and scales Monitoring student learning (formative practice) Determining attainment of student learning targets and

scales Reflecting and refining the Student Learning Objective

Process



Reflecting and Refining Student Learning Objective Process

Determining Attainment of Student Learning Targets and Scales

Monitoring Student Learning (Formative Practice)

Assessing Quality, Attainment Level and Rigor of Student Learning Targets and Scales

Collecting Baseline Information Setting Student Learning Targets Setting Appropriate Scales for Measurement

Understanding the Colorado Academic Standards Selecting Learning Outcomes Procuring or Developing Assessments Using Quality Criteria


Turn and Talk:• Why is it important to start the Student Learning

Objective process with the Colorado Academic Standards?

• How does this first step support the work done in professional practice?

• What tool is in place to support you with part of this process?

Understanding the Colorado Academic Standards

Selecting Learning Outcomes for Target Setting

Procuring or Developing Assessments Using Quality

Criteria


THE PROCESS OF: Collecting Baseline Information

Setting Student Learning Targets Setting Appropriate Scales for Measurement

THE PROCESS OF: Understanding the Colorado Academic Standards Selecting Learning Outcomes for

Target Setting Procuring or Developing

Assessments Using Quality Criteria

Student Learning Objective Process:

Strategies for Target Setting Possible strategies to consider when determining how

students will be captured when target setting: Individual – Teacher sets individual goals for students and then

determines how many students met their individual goals Whole Class – Teacher sets a goal that would be applied to all

students in a class Subgroup of Students – Teacher sets a goal for a group(s) of

students with similar baseline data levels Case Load – Teacher sets a goal for the specific students he or she

supports

Some of these strategies may work better with some teaching assignments. Can you think of which teaching assignments might

work best with which strategy based on your local context?

Possible Student Learning Targets and Scales

Examples of Student Learning Targets and Scales:16 of my 21 students will increase their reading

proficiency by 1-2 grade levels. 9 more students will receive + at the end of the

year. The median score on my end of course

assessment last year was 73%. At the end of the course for this year, the median score will be at least 78%.


Strategies for Target Setting Possible strategies to consider when setting learning targets for

students: All targets are indicators of growth using a variety of data sources: Proficiency data Targets – students meeting grade level

expectations Examples: a % of students will pass the end-of-course exam; a % of

students will score a 3 or better on the AP exam Growth data Targets – students growing over the course of

instruction Examples: a % of students will progress one fitness level; a % of

students will make projected growth or better on the MAP assessment Averaging data Targets – students’ average score on an

assessment Example: Students will answer, on average, 80% of the questions

correctly on the end-of-course exam

Possible Student Learning Targets

What are the targets in these examples?16 of my 21 students will increase their reading


year. The median score on my end of course

assessment last year was 73%. At the end of the course for this year, the median score will be at least 78%.


Strategies for Scale SettingTeachers set ranges (how many students are

expected to meet the set target) across 4 rating levels: above expected, expected, less than expected, and much less than expected

Teachers collaboratively set ranges with their evaluator Ranges will vary from year to year and possibly class to

class as baseline data will vary due to student beginning points

Collaboratively setting ranges will allow teachers and evaluators to calibrate about expected student learning outcomes

You can use the Measures of Student Learning Tool to record the targets and scales you have established.

When Colorado Growth Model data is available the scale indicating expected student learning outcomes has been established for you.


Strategies for Scale Setting

Possible Student Learning Scales

What are the scales in these examples?16 of my 21 students will increase their reading


year. The median score on my end of course assessment

last year was 73%. At the end of the course for this year, the median score will be at least 78%.



THE PROCESS OF: Collecting Baseline Information Setting Student Learning Targets Setting Appropriate Scales for Measurement

Understanding the Colorado Academic Standards Selecting Learning Outcomes for Target Setting Procuring or Developing Assessments Using

Quality Criteria


Determine Target and Scale Quality Will your evaluators establish quality criteria for

the use of Student Learning Targets and Scales?Criteria for establishing quality targets and

scales might include:Approval processesCollaboratively establishedLevel of attainmentRigor Based on previous dataMeet or exceed past student performance






Understanding the Colorado Academic Standards Selecting Learning Outcomes for Target Setting Procuring or Developing Assessments Using Quality Criteria


Reflecting and Refining Student Learning Objective Process





Understanding the Colorado Academic Standards Selecting Learning Outcomes for Target Setting Procuring or Developing Assessments Using Quality Criteria

Assessment literacy of educators creating targets Data literacy of educators Time and effort required of teacher and evaluator Focus of school or district (all students and/or

subgroups of students) Available information or data in school or district Ease or readiness level of teachers and evaluators Possible approval process of strategies selected Number of targets wanted as a district/building

Considerations when Setting Student Learning Targets


Evaluation









Combine the results of multiple measures to get a single student learning outcomes rating.

Step 6: Student Learning Outcomes

Putting It All Together The Decision Framework helps districts

aggregate the 50% Professional Practices and the 50% Student Learning Outcomes to reach a final rating of one of the following: Highly EffectiveEffectivePartially Effective Ineffective

Decision Framework

Prof

essio

nal P

racti

ces R

epor

t

A FREE educator calibration & professional development system available for all school districtsFall 2013 – Master scored videos for evaluators to

calibrate on the professional practices in the rubric2013-14 – Master scored Teaching Cases created to

include multiple videos of a teacher and supporting artifacts that mimics the evaluation cycleVideo clips aligned to the rubric for professional

development for all educators

Inter-Rater Agreement and My Learning Plan (MLP)

A FREE educator evaluation performance management system will be available for school districts

Four modules will be available for districts: 1. Professional practice2. Aggregating of Student Outcomes3. Decision Framework4. Reporting

The RFP has been awarded to RANDA Solutions RANDA has 13 years of experience doing this level of

work

Educator Evaluation Performance Management

System

Resources Available

CDE Educator Effectiveness website : http://www.cde.state.co.us/EducatorEffectiveness/ FAQ’s Fact Sheets Policy and State Model Evaluation System Info

CDE Educator Effectiveness e-newsletterhttp://www.cde.state.co.us/scripts/communications/EE

Newsletter.aspStay informed by signing up for this monthly newsletter

http://www.cde.state.co.us/EducatorEffectiveness/

http://www.cde.state.co.us/scripts/communications/EENewsletter.asp

http://www.cde.state.co.us/scripts/communications/EENewsletter.asp

Contact Us

EE Leadership Katy Anthes: Executive Director

[email protected] Toby King: Director

[email protected] Jean Williams: Rubric Evaluation

Specialist [email protected]

Colorado Legacy Foundation Mike Gradoz: Director

[email protected]

Communications Amy Skinner

[email protected] Katie Lams:

[email protected]

Britt Wilkenfeld: Data Fellow [email protected]

Tricia Majors: Project Mgr. [email protected]

mailto:[email protected]








Implementation Support and Development

Mary Bivens [email protected]

Courtney Cabrera [email protected]

Sed Keller [email protected]

Dawn Pare [email protected]

Bob Snead [email protected]

Chris Vance [email protected]

Contact Us







colorado department of education, educator effectiveness winter 2014

Documents

student growth

professional growth

new system

student outcomes

nonprobationary status

new evaluation system

intent of sb

messaging sb