comprehensive needs assessment reviewer training...

Future Ready Schools

Presented by

The North Carolina Department of Public Instruction

District and School Transformation Division

Comprehensive Needs Assessment Reviewer Training & Capacity Building Program


Introductions

Salutations


Creating Ground Rules


Ground Rules for the Training

• A room of equals• Be open• Bring your contribution with confidence• Ask all the questions you need to ask• Listen to each other with respect• One voice at a time• Minimize distractions - cell phones/laptops• Keep to time• And/or …?


Overview of the CNA


• Provides districts and schools with a clear view of their strengths and areas for improvement, challenges and successes

• Enables a systematic review of practices, processes, and systems within a school district

• Assists district and school leadership in determining needs, examining their nature and causes, and setting priorities for future action

• Guides the development of a meaningful district or school improvement plan and suggests benchmarks for evaluation

• Provides a corner stone for continuous improvement –ensuring the best possible education for all students

The Purpose of the Comprehensive Needs Assessment


Code of Conduct

• All reviewers sign a Code of Conduct.

• Please consider this, and if appropriate, sign and return it by the end of the day


Connectivity and the CNA: A Critical Issue

• Family member at the school

• You were principal or senior member of staff at the school in last 5 years

• Any reason participating in the review at the school might compromise the process

• Declaring connectivity

• If you are concerned – discuss!

• It’s all right – until it isn’t !!!


Stages of the CNA


There are 3 stages of assessment:

• Preparation

• Site Visit

• Final Report


Stage 1: Preparation

• Good preparation assists the assessment process

• Training and development• Data• Self-Evaluation Form• Notes of Guidance


Stage 2: Site VisitSample School Assessment Schedule

Day 1 Homework


Stage 3: Final Report

• Analysis of strengths and areas for improvement

• Review of how well the organization meets each of the Assessment Dimensions - text and judgment for each Dimension


The Process and the Foundation

• The foundation of the CNA, the rubric upon which judgments are made, was developed by DPI.

• The CNA process was developed by Cambridge Education.


Expectations, Questions & Concerns

What are your thoughts at this time?


The Comprehensive Needs Assessment

Rubric and Dimensions


• Focus on IMPACT and OUTCOMES• Professional judgments on quality of the work of a school/district and the education it is providing

• Based on rubrics• Not compliance driven

The Comprehensive Needs Assessment Model


Framework for Comprehensive Needs Assessment

Leadership Capacity Professional Capacity

Planning & Operational Effectiveness

Family & Community Support

Instructional Excellence & Alignment:

Part I: Teaching and Learning Part II: Support for Student

Achievement


A1. Instructional Excellence and Alignment Part I – Teaching and Learning

A2. Instructional Excellence and Alignment Part II – Support for Student Achievement

B. Leadership Capacity

C. Professional Capacity

D. Planning and Operational Effectiveness

E. Families and Communities

6 Dimensions of the Rubric:


Key Skills of Reviewers

• Creating a collegial atmosphere

• Gathering and analyzing evidence

• Interpreting evidence and making judgments

• Communicating judgments orally and in writing

• Code of Conduct


Evidence


What is Evidence ?In small groups -

• Identify the nature and type of evidence you would seek for each dimension/sub-dimension from the school rubric.

• What is the most pertinent, relevant evidence ?• Consider and record how you would gain this

evidence from the range of activities within the school assessment schedule ……..

Think about the question – so what is evidence ?


So What is Evidence?

• School schedule • Meetings with principal, teachers,

students, administration, parents• Collaborative meeting• Lesson observations• Other first hand observations –

documents, touring the school, the environment

• Self - Evaluation Form


Key Sources of Evidence Beforeand During the School Review

• Performance and other data about students• Documents supplied by the school

• School Self-Evaluation Form

• Direct observation of teaching and learning

• Discussions with adults and students

• Looking at students' work

• Direct observation of the school environment and culture


Evidence – so WHAT?

• What’s the impact ?

• What’s the outcome ?

• So what ?

• How do you know ?


Triangulation

Direct Observation


Professional Judgment

• Examination of evidence leads to judgment

• Informed judgments are not checklists

• Use terminology that is accurate and understandable


Judgments

• LE - Leading

• EM - Embedded

• E - Emerging

• LA - Lacking


Be back on time!!!!!!


Stage 1: Preparation


Preparation for the Assessment

• The timeline• Responsibilities• Why prepare?• The Assessment Preparation Form

(APF)

• Initial data from school/ Self-Evaluation Form (SEF)


Analyzing Data


Types of Data

• There are many sources of data available for any school:

• State web site• District web site• School home page (website) • Statistical Information: attendance, budget • Annual School Report Card• Other evidence which the school keeps on

student progress and attainment• Schools are data rich


Interpreting Data

A reviewer interprets data into relevant information from which s/he can make hypotheses and judgments on the quality of education provided by the school.


Achievement Data

• K-8 2013-14 Report Card Profile

• K-8 2013-14 Disaggregated Data

• 9-12 2013-14 Report Card Profile

• 9-12 2013-14 Disaggregated Data


Investigate the Reading 3 Year Trend

Now we see a far more complex picture


Interpreting Data Reading 3 Year Trend


Interpreting Data

• Does this information change your initial hypotheses?

During the site visit:

• What questions would you ask? • What would you examine?• What evidence would you expect the

school to be able to provide?


The Self Evaluation Form

(SEF)


Analyzing the SEF

• Read the SEF.• Discuss the document as a group and pick 5 key issues to raise when going into the school.

• Frame questions to ask – consider whom you would ask these questions.


Putting Together the SEF and Initial School Data


Review of School Data & Preparing Basis for Initial Questions during Visit

• In pairs or trios: • Review the following sample pre-visit data:

– School Self-Evaluation Form (SEF)– NC School Report Card– OTMS Disaggregated Data– Results of Teacher Work Conditions Survey

• What preliminary issues can you glean from the school’s self-evaluation or the parental survey?

• What might you follow up on in the school?

• What questions might you ask? When? Who?


Stage 1: Preparation Continued

Making Contact with the School- Role and responsibility of the lead

reviewer- Initial phone call- Critical element of the

preparation stage- Developing the professional

relationship


Initial Phone Call to the School

• What do you want to talk about?

• The principal of the middle school you are about to review has returned the SEF

• Observe initial phone call role play


Critique the Phone Call

• Did the lead reviewer cover all the important points?

• How was the call handled?

• Were all the principal’s questions answered?

• How well did the call prepare the school for a smooth visit?


The School Initial Phone Call • Build the relationship• Introduce self and team• Outline the process and key purposes of the

assessment• Explain how the team will work with the school• Confirm receipt of the SEF• Agree on the schedule and key terms• Discuss the arrangements for the exit interview • Confirm a working space• Respond to questions


Review of the Day• What are the three most valuable

points I will take away from today ?

• How will they help me as a reviewer ?

• Spend 5 minutes reflecting.

• Discuss with your neighbor.– Sharing learning– Tomorrow


Comprehensive Needs AssessmentTraining – Day 2

Please sit with someone new today !


Assessment Preparation Form (APF)

• Prepared by the Lead Reviewer

• Shared with team members prior to site visit

• Sample APF


APF

What do you notice about the tone and style of the example?


Preparing an APF

• Review the Self-Evaluation Form (SEF)

• Review the initial school data

• In your pairs, agree on key matters that arise from your scrutiny

• Begin writing an APF for the school

• Be ready to share with the whole group


Ensuring a Smooth Beginning

• Review Team • School


Before Arriving at the School …

The day before:• Read the SEF

• Read the APF and prepare initial thoughts/ questions

• School phone number for emergency contact

• Clear directions for getting to the school


The Day of the CNA . . .

• TURN OFF CELL PHONE

• Dress professionally

• Arrive on time

• Don’t check email during the CNA

• Complete the entire CNA process for each day


• Greet everyone warmly and professionally (Zip it on!)

• Be aware of body language and facial expressions

• Laptops are not needed

• Bring pens, highlighter, and Notes of Guidance

The Day of the CNA …


Stage 2: Site Visit

Collecting Evidence


The Record Book

• What are the purposes?

• What are the responsibilities?

• Keeping it organized!

• Review the blank record book. What do you notice about it?


Preparing for the Meeting with the Principal


Dimension A Sub-Dimensions 3-4

• Reference: – Sub-dimensions 3-4– SEF– School data

• What questions will you ask the principal?

• Work in pairs and record your questions


Role Play

• Role play the meeting between the lead reviewer and the principal!

• Use the brief provided


Outcomes of the Role Play

In your group, discuss:

• Which questions worked well and why?

• What issues emerged?

• How were any sensitive issues handled?

• What evidence emerged?

• How well were you able to record?

Future Ready SchoolsBuilding the Relationship with the Principal

• First impressions of the review team are critical!

• Arrange to meet with the principal morning/evening

• Lead reviewer and principal will undertake a joint lesson observation and calibrate outcomes

• Keep in touch with the principal to ensure all is proceeding well


Building the Relationship with the Principal

• Maintain the dialogue with the principal

• Feedback on key issues for learning and teaching each day

• If the ongoing relationship/dialogue is maintained, there should be few surprises by the end of the day

• Quality assurance – the school’s perception of the review experience


Asking Questions

• In groups, agree on the questions to ask:• Teachers• Team leaders ( e.g. data team leader, senior colleagues.)• Students• Parents / guardians

• To evaluate:• Dimension A Part 1 Sub-dimensions 1 and 2• Dimension B Sub-dimension 7• Dimension C Sub-dimension 8

• Think about range, style and type of questions

Future Ready SchoolsTeacher Focus Group Observation

Take notes in Record Book of Teacher Focus Group Meeting


Teacher Focus Group

• Reviewers should ask questions previously prepared or about any aspect of the dimensions they choose.

• Role Play the meeting – 1 Lead Reviewer– 2 Teachers – could have specialists

roles– 1 Observer


Teacher Focus Group

The staff is well-established;

however, the principal is in the

first year of leadership at Our

Town Middle School.


Observing a Collaborative Meeting

• This meeting will occur some time after your initial meeting with the principal and perhaps after a few class observations

• What might you be able to get evidence of or begin triangulating while observing this meeting?

• Highlight on the rubric sub-dimensions you might observe


Student Focus Group

• How do you encourage students to feel secure so they can answer honestly?

• Are there any other sensitivities we should be mindful of when interviewing?

• How can you avoid asking leading questions and avoiding students giving you the answers they think you want?


Building Your Bank of Questions

• Most questions will emerge during the conversation

• Now, begin building a bank of questions that could apply to different parts of the rubric


The Parent Meeting

• Review excerpts from a record book

• Annotate the evidence from the parents’ meeting showing which sub-dimensions are addressed.

• Summarize the evidence at the foot of each page –for the Families and Communities Dimension

• What else would you want to/need to find out about to support the team’s overall judgment about this dimension?


Review of the Day

• Complete the statement:

“ The most important things I learned today are …”

• Share with your partner and the group


Evaluating Teaching and Learning

Consider the following:

• What is good teaching?

• What are the elements of a good lesson?

• What does learning look like?


Evaluating Teaching and Learning

Use teaching and learning criteria to:

• help you make sense of what you see

• help you to record evidence and outcomes clearly

• know what or how much students are learning


Teaching and Learning Evidence.

• In groups read Dimension A – Part 1 –Teaching and Learning.

• Based on the descriptors for each sub-dimension create a list of criteria for effective teaching and learning.

• What evidence will you examine?


Learning and Teaching

• Teaching ……………….Learning• Cause ………………….......Effect• Teaching …………………Impact


Recording Evaluative Evidence

• If a teacher is demonstrating skills, how would you know by observing cause and effect?

• If a teacher is demonstrating knowledge and understanding of content, how would you know by observing cause and effect?

• If a teacher is facilitating student learning of accurate and appropriate content, how would you know by observing cause and effect?

Cause ………………………..EffectTeaching …………………….Learning?

Future Ready SchoolsTeaching and Learning Effective Evidence Based Recording

• Work in pairs and choose one of the questions from the previous slide.

• Record cause and effect statements for your chosen question.

• Record boldly on a flip-chart to display to the group.


Evaluating Teaching and Learning: Lesson Observation 1

• Lesson observation

• Observe 5 minutes of a lesson.

• Jot down salient points and – or just watch and consider.

• Feedback to the group


Lesson Observation 1- continued

Take notes in the Record Book as the lesson observation continues


Lesson Observation 1- continued

Continue taking notes in theRecord Book as the lessonobservation continues


Evidence: Student Work


Scrutinizing Students’ Work in School

• Take 5 minutes:

• Jot down everything you can learn from students' work…


Scrutinizing Students’ Work

• Grading, assessment target setting

• Editing and feedback by teachers• Use of rubrics • Impact of feedback on progress• Consistency of school practice• Meeting student needs, differentiation• Progress• Student motivation and attitudes• Implementation of the curriculum


Evidence from Student Work

• Use a highlighter to identify the elements of the rubric which can be evidenced by scrutinizing student work

• Review the whole picture!


Finding Out About Students’ Work(aka: Snooping in the School)

• When?

• Talk to students in the classroom, when they are working … if appropriate

• Look at displays around the school and in classrooms

• Ask to see samples


Lesson Observation 2

Observe the DVD extract

Record your evidence in relation to Teaching andLearning

Trios

• Discuss the quality of teaching and learning in this lesson as if you were the assessment team

• Draw up summary of key issues

• Be ready to defend your views


Giving Effective Feedback


Golden Tips for Giving Feedback

• Individually, identify your 5 tips for giving effective feedback

• Now - share with your partner – agree, between you, the top 5 Golden Rules.

• Be ready to share with the group


Feedback Activity

• Form a trio and number yourselves 1, 2, 3

• Number 1 – reviewer - provides oral feedback to principal

• Number 2 - principal - receives feedback from the reviewer

• Number 3 – observer - observes the feedback session and leads debrief

• Feedback to the principal

• Discuss and debrief the feedback


CNA Judgments


Making the Judgment

• The team approach to making judgments

• Sharing evidence and calibrating judgments


Synthesis Activity

Cause ……………………….Effect

Teaching ……………………Impact


Review of the Day

Wows & Wonders


CNA Judgments


Making the Judgment

• The team approach to making judgments

• Sharing evidence and calibrating judgments


Judgment Summary Sheet Activity

• Review evidence- principal meeting- teacher meeting- parent meeting notes- classroom observations

• Make a judgment for each sub- dimension• Share evidence with your group • Calibrate judgments.


Sharing Outcomes

• The role of the lead reviewer

• Ongoing dialogue with the principal

• The exit interview


Sharing Outcomes Continued

- Summary Feedback to Principal

Things the school does well:

Things the school needs to improve:


Summary Feedback Sample

• Review the key issues arising from the assessment of the sample school

• What do you know about the school from reading these?

• What do you notice about the tone and style of the bullet points?


Summary of Key Judgments

Based on your judgment summarysheet write 3 things the “school” doeswell and 3 things the “school” needs to improve.


Stage 3: The Report


The Report

• Reviewing the CNA report template

• Looking at an example of a final report


Quality Writing: Remember the 6 C’s

• Consistent

• Convincing

• Clear

• Concise

• Complete

• Correct


What does the report say?

• Review the “mythical” reports

• Review the excerpt in light of the previous slide.

How well does the writing meet the QA criteria?


Writing Activity: Cause and Effect

Working with your partner, underline all examples of cause and effect statements in all your report samples/excerpts.


Writing Activity: Families and Communities

• Individual writing

• Write a Families and Communities section, using the record book excerpt or a school you know from the past.

• Exchange with someone – can you offer any improvements?


Writing the report:

• The writing process

• The Quality Assurance (QA) process


What Did You Mean to Say?

This is a school where positive and productive relationships with families and the communities is a growing priority, lead by the principal and her leadership team. The principal has a clear understanding of the current position. ???


Did you really mean to say that?

• The school nurse is used for sex education.

• The school has a magnificent new library and librarian.


What’s wrong here?

The school’s work reflects its strong sense of professional commitment. Teachers are eager for their students to succeed. The majority of teaching provides students with a good range of curriculum-focused and stimulating opportunities. In this way, teachers foster students’ success. Students’ attendance and behavior have been key priorities since the principal’s appointment. Both have clearly improvement. However, the school recognizes that it needs to do more and it has appropriate plans in place for this.


Impact of Self-Editing


Lesson Observation 3

• Observe the lesson

• Record the evidence using cause and effect

• Complete “Key Strengths” and “Key Areas for Improvement”


Activity

• What would you ask the students?

• What would you ask the teacher?

• What would you ask the teacher assistant?

• What is one more thing you would want to do if you were in the room?

• What would you want to ask the principal?


Please complete the sentence:

“Before my first assessment I will…”


Roles, Responsibilities and Accountabilities

• Code of Conduct

• Lead reviewer and the team

• Quality assurance – on site, documentation, record books, feedback from the school/district


Connectivity and the CNA: A Critical Issue

• Family member at the school

• You were principal or senior member of staff at the school in last 5 years

• Any reason participating in the review at the school might compromise the process

• Declaring connectivity

• If you are concerned – discuss!

• It’s all right – until it isn’t !!!


Additional Information

• Logistics

• Feedback forms


CNA Test

Mark each statement true, false or maybe!


•Good Bye!!! •See you in the field. •Look for your deployment orders soon!