Finding CoherenceFocus, Measure, & Connect

The Measure Sessions!November, 2014

http://digitallearningforallnow.com

http://www.slideshare.net/jpcostasr

costa@educationconnection.org

Jonathan P. Costa

Overall Purpose and Goals• To create a matrix that reflects the least amount of assessments necessary to

sufficiently inform instructional planning and delivery at all grade levels and in all content and program areas.

• Challenges – Limitations – Needs:

1. Need to balance the amount of available teaching/learning time with the need to obtain adequate diagnostic information

2. Limited funding availability

3. Need to identify an affordable data warehousing system that is capable of running all necessary reports in real time

4. Need to address CSDE assessment mandates

5. Need to reduce the amount of potential confusion with regard to who needs to do what assessment, when, and why

6. Need to ensure sustainability of assessment protocols over time

Coherence Pathways

G = Goals P = Practices M= Measures Jonathan P. Costa S= Students A = Adults O = District/Building

Mission

To develop in all children the

knowledge, skills, attitudes and

values...

Theory of Action

FocusMeasureConnect

Student DataDriving

InstructionalPractices &

DecisionMaking

InstructionDriving

ImprovedMeasures

ProfessionalPracticeDriving

ImprovedMeasures

SystemsDriving

ImprovedMeasures

Adult DataDriving

ProfessionalLearning &

DecisionMaking

OrganizationalData

DrivingSystemsDecisionMaking

Three Principles of Coherence

Measure what you

value

Value what you measure

Priority Student Learning

Priority Adult

LearningPriority Systems Learning

Student Learning

Adult Learning

Systems Learning

MeasureFocus Connections

S-G Goals for Student Learning

• What are your most important goals for learning?

How will you know if you are improving (Measure)?

Using the identified skills, create an assessment plan that includes; a. existing

standardized data (CAPT/CMT), b. identifies potential Smarter Balanced/Common Core

assessments, and c. creates local assessments that can be used for formative

measurement of student performance of these skills.

S-M

Student Measures

G = Goals P = Practices M= Measures Jonathan P. Costa S= Students A = Adults O = District/Building

Leadership

FocusMeasureConnect

Mission

To prepare every student for

learning, life, and work in the 21st

century.

Coherence Pathways – Essential Connections

Goals

MeasuresPractices

Focus

Critical Stance Based on Evidence

M3/E4

Construct and engage in viable arguments based on evidence and critique reasoning of others.

SMP 3 – Distinguish correct logic in reasoning from that which is flawed. SMP3 –Build a logical progression of statements to explore the truth about

conjectures. SMP 3 -Use stated assumptions, definitions, and previously established

results in constructing arguments. SMP 3 - Recognize and use counter examples. SMP 3 – Contextualize and decontextualize quantitative relationships E4 - Question an author’s or speaker’s assumptions and premise 21 - Effectively analyze and evaluate evidence, arguments, claims and beliefs 21 - Synthesize and make connections between information and arguments 21 - Interpret information and draw conclusions based on the best analysis

Define and Solve Problems

M1/M2/M8/21

Make sense of problems and persevere in solving them

SMP 1 – Explain the meaning of a problem and look for entry points to its solution.

SMP 1 – Analyze givens, constraints, relationships, and goals. SMP 1 – Make conjectures about the form and meaning of a solution. SMP 8 – Maintain oversight of the process while attending to details. SMP 2 – Create a coherent representation of a problem. 21 -Identify and ask significant questions that clarify various points of view

and lead to better solutions 21 -Reflect critically on learning experiences and problem solving processes 21 -Solve different kinds of non-familiar problems in both conventional and

innovative ways

Reflections on Assessment…

Before we review your inventory and next steps, let’s consider…

• Balance factors

– Reliability and value

– Formative vs. summative

– Content/department vs. skills

– Classroom, building, district needs

• Quality factors

– Just because it happens, does not mean it should or that its good

• Improvement opportunities

Coherence Is Supported by Assessment

Jonathan P. Costa

Value

Reliability

Reflections on Assessment… • To be worth anything, all assessment must do one of these two things….

1. Improve student learning (formative)

2. Create both student and teacher accountability (summative)

• How would you define/characterize these two categories of assessment?

– Formative:

• Why –

• How -

– Summative:

• Why –

• How -

Reflections on Assessment…

- Finding the Balance Points -

- Given what we have discussed about the purpose and process of formative and summative assessments… what do you think is the most appropriate and productive % time split between these two assessment purposes (of the total time devoted to assessment – how much for formative and how much for summative)?

-Form / Sum -

- Given what we have discussed about the purpose and process of formative and summative assessments… what do you think is the CURRENT PRACTICE between these two assessment purposes in your district (of the total time devoted to assessment – how much for formative and how much for summative)?

-Form / Sum -

Align Your Systems With Your Goals for Learning

Type of Assessment

Required

Subject Area Responsibilities

Everyone’s Responsibility

Content(Declarative)

Facts

Content Skills(Procedural)

Discrete Skills

CC/21st Cent. Skills(Contextual)

Applied Understandings

Type of Knowledge

Desired

Type of InstructionRequired

Lecture, video, films, assigned readings and

memory activities.

Classroom or textbook problems, experiments, discussions, practice and

repetition.

Complex projects,real time explorations,authentic and relevant

skill applications.

Amount of Time

Required

Discrete units,spiraled and predictable.

Ongoing, systemic and without a finite

or predictable end.

Discrete units,spiraled and predictable.

Recall & recognitionbased quizzes, tests,

and activities. Multiplechoice, matching, etc.

(SAT/AP/Exams)

Checklists, analytic rubrics,

or other agreed upon skill standards

(AP/SB/CAPT/Exams)

Holistic and, analytic rubrics,

or other agreed upon standards of rigor

(Portfolios, Exhibitions, SB)

Align Assessments With Goals for Learning

Appropriate

Assessment

Instruments

Recall & recognition

based quizzes, tests,

and activities. Multiple

choice, matching, fill

In the blanks, etc.

(SAT/AP/Exams)

Checklists,

analytic rubrics,

or other agreed upon

skill standards

(AP/Smarter B/Exams)

Holistic and,

analytic rubrics,

or other agreed upon

skill standards

(Portfolios, Demonstrations,

Exhibitions, Etc)

Content(Declarative)

Facts

“Define this term”

Content Skills(Procedural)

Discrete Skills

“Factor this equation”

Meta Skills(Contextual)

Applied Understandings

“Skilled Information Consumer”

Type of

Knowledge

Desired

Appropriate

Assessment

Standards/Scales

Take this quiz, test,

or other appropriate

content focused

assessment instrument.

Answering/responding to

classroom or text based

problems, experiments, and

writing assignments.

Complex projects,

real time explorations,

authentic problem based

skill applications.

Appropriate

Assessment

Tasks

Percentages of

right and wrong

answers on an agreed

upon numeric or letter

grade scale.

1-100/A-B-C-D

Specific criteria tied

to an agreed upon

numeric or

descriptive scale.

1-100/Pass-Fail

Insufficient-Excellent

Specific criteria tied

to an agreed upon

descriptive scale.

Insufficient-Sufficient

Proficient-Excellent

Overlap between columns can and does occur.

TeachersAs

StudentCoaches

DataSupportingReflectivePractice

DataSupportingReflectivePractice

DataSupportingReflectivePractice

AdministratorsAs

InstructionalCoaches

DistrictStaff

AsAdministrator

Coaches

Reflections on Assessment…• To effective, formative or summative assessments should be:

1. Generally Reliable (as opposed to statistically so)

2. Valid (reflects what it is intended to measure)

3. Helps the learner improve his/her performance (formative mostly)

4. Can be tracked and analyzed over time (summative mostly)

• What is your general sense of the state of quality of these elements in your school/district?

Formative Summative

Warm Cool Warm Cool

1. Philosophy – the role that assessment plays in your system – differences of formative and summative – thinking about the differences between growth and accountability and how to balance them in practice.

2. Technical Capacity

• Constructing effective rubrics – measuring what you value

• Designing rubric aligned tasks – connecting it to practice

• Scoring protocols – build capacity to judge student work

• Data collection and analysis tools – how to use the results

3. Systems effectiveness

• Organization of the weight of your assessments – inventory

4. Other?

Building Assessment Capacity

The ParetoLeverage Principle

VitalFew

Some things are more

important than others.

20/80

Pareto Is Not a New Idea

I served with General Washington in the Virginia Legislature and with Dr. Franklin in Congress. I never heard either of them speak to any but the main point that was to decide the question. They laid their shoulders to the great points, knowing that the little ones would follow themselves.

Thomas Jefferson

The ParetoLeverage Principle

With these things in mind, what can you:

Reduce/eliminateImprove/integrate