making sense of data to inform professional

MAKING SENSE OF DATA TO INFORM PROFESSIONAL

DEVELOPMENT AND PBC DECISIONS

WE ARE GOING TO..

• Discuss key ideas for using data to help inform decision about PD and PBC

• Explore child data and teaching practices for BEACON and discuss how the data might inform decisions about PD and PBC

• Consider what data your program might

have available to use to inform PD and PBC decisions

KEY IDEAS

DATA-BASED VS. DATA-DRIVEN DECISION MAKING

• Who is “driving”?

• Not the data!

• Team is the driver in data-based decision making

• Team is using data as information to “fuel” decisions

DIGGING INTO DATA

Data level: • Grantee

• Broken out by cluster sites or programs

• Classrooms

What do they tell us? How are we doing overall?

How are sites or programs doing (look for patterns)?

How are teachers and

children doing?

Think about…

Where do we start?

SCHOOL READINESS GOALS

• Broad statements that articulate expectations for child learning

• Help staff to choose and implement

curricula, assessments, and effective teaching practices

• Set the stage for professional development

Select Goals from BEACON’s School Readiness Plan

BEACON School Readiness Goal TS Gold Indicators

Measurable Outcome

Soci

al &

Em

otio

nal D

evel

opm

ent

• Children will develop and demonstrate positive interactions and relationships with adults and peers.

• Children will develop and demonstrate the ability to recognize and regulate emotions, attention, impulses, and behavior.

• Children will develop confidence and positive self-awareness.

Obj 2a, 2c, 2d Obj 1a, 2b Obj 1c

85% of children proficient by end of year 85% of children proficient by end of year 85% of children proficient by end of year

Lang

uage

& L

itera

cy

• Children will utilize language to express their wants and needs, engage in conversations, and follow directions.

• Children will be able to demonstrate knowledge of print and develop the awareness that print conveys meaning.

• Children who are DLL will demonstrate competency in their home language while acquiring beginning proficiency in English.

Obj 8b, 9a, 10a Obj 17a, 17b Obj 37, 38

85% of children proficient by end of year 85% of children proficient by end of year 80% of children who are DLL proficient by end of year

WHAT DOMAIN WILL YOU WORK WITH?

We are asking you to work through planning activities for one of your school readiness goal domains.

Discuss and confirm with your table which domain

and goals you will work with while we are here. 1. Do you know how you will measure these

goals? 2. Have you set your measureable outcome for

these goals? 3. Complete the grid on Handout 1

TEACHING PRACTICES

Framework for Effective Practices

TEACHING PRACTICES

• Specific statements of the actions and behaviors of teachers that support child learning

• Help plan targeted professional development

IDENTIFYING TEACHING PRACTICES

Source: https://www.teachingstrategies.com/national/creative-curriculum-preschool-system-planning.html

Social Emotional Goals for Children

Examples of Teaching Practices that Teachers Can Use to Support This Goal

Children will develop and demonstrate positive interactions and relationships with adults and peers.

• Initiate frequent and extended conversations with children, actively listen, contribute relevant responses, and ask related questions.

• Provide opportunities for children to interact with their peers.

Children will develop and demonstrate the ability to recognize and regulate emotions, attention, impulses, and behavior.

• Clearly teach, explain, and review the classroom rules and behavior expectations with children.

• Teach children about emotions, how to recognize feelings in themselves and others, and how to express their feelings.

Children will develop confidence and positive self-awareness.

• Use clear descriptive positive feedback, more than statements that provide general praise.

• Incorporate pictures, stories, and artifacts from children’s home life into classroom activities, have visual representation of children’s family on display in the classroom.

Let’s Look at An Example

EVIDENCED-BASED TEACHING PRACTICES

Where to find specific teaching practices that have evidence for effectiveness:

• Classroom observation measures (CLASS,

TPOT, ELLCO, ECERS-R)

• Practice lists or checklists

• Instructional strategies guides or briefs

GENERATE A LIST OF KEY PRACTICES

• SR Goal: Children will develop and demonstrate positive interactions and relationships with adults and peers

• Possible Sources: – NCQTL Tools for Supervisors – CLASS observation tool – CSEFEL practice list – ECERS-R observation tool

SR Goal: Children will develop and demonstrate positive interactions and relationships with adults and peers.

Screenshot from: NCQTL Tools for Supervisors – Fostering Connections In-


Screenshot from: Classroom Assessment Scoring System (CLASS; Pianta et al., 2008).


Screenshot from: CSEFEL Inventory of Practices


Screenshot from: Early Childhood Environment Rating Scale (ECERS-R; Harms et

GENERATE A LIST OF KEY PRACTICES


Possible Priority Teaching Practices: • Teachers join in children’s play activities • Teachers have a positive affect and show warmth

to children through smiles and appropriate touch • Teachers acknowledge children’s emotions • Teachers call children by name

LISTS OF TEACHING PRACTICES CAN BE USED TO CREATE A NEEDS ASSESSMENT FOR PBC

CONNECT TO PBC

APPLY IT

Think about the School Readiness Goal you identified. List possible teaching practices.

• How do those teaching practices support child learning ?

• What sources did you use to select teaching practices ?

• Write these practices on Handout 2; only complete the first 3 columns.

CONNECTING TEACHING & LEARNING

PROFESSIONAL DEVELOPMENT

PD Puzzle Pieces developed by McLaughlin & Snyder, Center for Excellence in Early Childhood Studies at the University of Florida

THEORY OF CHANGE

Pd w/Coaching Use of Effective Child Learning Teaching Practices

Teacher needs guide PD Child needs guide what teachers do in classrooms

WORK BACKWARDS

PD w/Coaching USE OF EFFECTIVE CHILD LEARNING TEACHING PRACTICES

THEORY OF CHANGE FOR BEACON



Teaching Practices that Teachers Can Use to Support This Goal


• Provide opportunities for children

to actively interact with their peers.

Current T/TA Plan

Training led by BEACON Education Specialist, Written/Visual reference materials

Possible PD Strategies to Support Teachers

• Training in Pyramid Model

• Online resources and videos

• Teacher-to-teacher support groups

• Practice-Based Coaching

APPLY IT

Work with your table to add possible PD strategies to your own “Theory of Change.”

Consider: • The SR goal you selected • The aligned teaching practices • Your current professional development efforts • What PD strategies will you add in the last column of

handout 2?

BRINGING IT BACK TO DATA

REVIEW OF BEACON’S SOCIAL-EMOTIONAL SR GOALS, ALIGNED TEACHING PRACTICES, AND

PD

BEACON’S SOCIAL-EMOTIONAL SCHOOL READINESS GOALS: GRANTEE LEVEL

0

10

20

30

40

50

60

70

80

90

100

Program-Wide Child Outcomes TS Gold: Social Emotional Scores in

September

CURRENT PROGRAM DATA

GOAL: 85%

69


SE 1: Children will develop and demonstrate positive interactions and relationships with adults and peers.

Teaching Practices that Teachers Can Use to Support This Goal


• Provide opportunities for children

to actively interact with their peers.

T/TA Plan

Training led by BEACON Education Specialist, Written/visual reference materials

PD Strategies to Support Teachers

• Training in Pyramid Model

• Online resources and videos

• Teacher-to-teacher groups

• Practice-based coaching at 4 sites

Example from BEACON’s T/TA Plan and PD Strategies: Linkages with social-emotional SR goals

and teaching practices

EXPLORING SR CHILD LEARNING DATA FOR BEACON

QUESTIONS TO CONSIDER AS REVIEW DATA

‘Do we think what BEACON is doing related to PD and PBC

appears to be making a difference with respect to social-emotional school readiness goals?’

‘What else might be helpful to know?’

0

10

20

30

40

50

60

70

80

90

100

September January May

Program-Wide Child Outcomes TS GOLD: Social Emotional Scores Over the School Year

GOAL: 85%

WHOLE PROGRAM

69 74 85

0

10

20

30

40

50

60

70

80

90

100

% c

hild

ren

prof

icie

nt

TS Gold -Social Emotional Scores Over School Year

September

DATA BY SITE

GOAL: 85%

0

10

20

30

40

50

60

70

80

90

100

% c

hild

ren

prof

icie

nt


September

January

DATA BY SITE

GOAL: 85%

0

10

20

30

40

50

60

70

80

90

100

% c

hild

ren

prof

icie

nt


September

January

May

DATA BY SITE

GOAL: 85%

0

10

20

30

40

50

60

70

80

90

100

TS Gold: Positive Interaction Scores Over the School Year

September

DATA BY CONTENT AREA: Social Emotional Goal #1

GOAL: 85%

0

10

20

30

40

50

60

70

80

90

100


September

January

GOAL: 85%


0

10

20

30

40

50

60

70

80

90

100


September

January

May

GOAL: 85%


LET’s try it!

• Work with your team (you can do it!)

BEACON SOCIAL-EMOTIONAL GOAL #2

• How are the different sites performing in relation to the benchmark at each time point?

• Have they made progress? • How do these data inform decisions

about PD and PBC? • What other information do we need?

0

10

20

30

40

50

60

70

80

90

100

TS Gold: Emotions Scores Over the School Year

September


GOAL: 85%

0

10

20

30

40

50

60

70

80

90

100


September

January


GOAL: 85%

0

10

20

30

40

50

60

70

80

90

100


September

January

May


GOAL: 85%

BY CLASSROOMS IN A PROGRAM

TS Gold - Social Emotional Scores in May % of Children Proficient

Mean for Location

St. Francis Center (72%

proficient overall)

BEACON

benchmark = 85%

Location 1

C1 84

81 C2 76

C3 85

C4 80

Location 2

C1 75

74 C2 74

C3 74

Location 3

C1 62

61 C2 64

C3 60

C4 56



Mean for Location

Wendell (79%

proficient overall)

BEACON

benchmark = 85%

Location 1

C1

68

71

C2

74

Location 2

C1

88 86

C2

84



Mean for Location

Washer Lake (90%

proficient overall)

Location 1

C1 82

81 C2 79

C3 81

Location 2

C1 93

97 C2 98

C3 97

C4 97

Location 3

C1 95 89

C2 83

SUMMARY

‘Do we think what BEACON is doing related to PD and PBC at these three sites appears to be making a difference with respect to

social-emotional school readiness goals?’

‘What else would be useful to know to answer this question?’

EXPLORING TEACHING PRACTICES DATA FOR BEACON

EXAMPLE DATA SOURCES

• Program quality indicators/measures • Classroom quality indicators/measures* • Quality teaching and learning practices

indicators/measures • Counting intentional teaching trials • Curricular Implementation checklists • Other indicators or sources of data?

BEACON Grantee-Level Data: CLASS – FALL AND SPRING

1

2

3

4

5

6

7

Emo. Supp. Class. Org. Inst. Supp.

Clas

s Sc

ores

Fall

Spring

BEACON Grantee Level Data: CLASS – FALL AND SPRING

1

2

3

4

5

6

7

Emo. Supp. Class. Org. Inst. Supp.

Clas

s Sc

ores

Fall

Spring

All HS Spring

DATA BY SITE

1

2

3

4

5

6

7

CLAS

S Sc

ore

CLASS Domain Scores in Spring

Emo. Supp.

DATA BY SITE

1

2

3

4

5

6

7

CLAS

S Sc

ore


Emo. Supp.

Class. Org.

DATA BY SITE

1

2

3

4

5

6

7

CLAS

S Sc

ore


Emo. Supp.

Class. Org.

Inst. Supp.

BY FOCUS AREA

1

2

3

4

5

6

7

CLAS

S Sc

ore

Emotional Support Scores Fall/Spring

Emo. Supp. Fall

Emo. Supp. Spring


CLASS Emotional Support Scores in May Mean for Location

St. Francis Center

(Child SE outcome

=72% proficient overall)

Location 1

C1 6.8

6.6 C2 6.9

C3 6.6

C4 6.3

Location 2

C1 6.5

6.5 C2 6.4

C3 6.7

Location 3

C1 5.2

4.6 C2 5.5

C3 4.3

C4 3.5


CLASS Emotional Support Scores Mean for Location

Wendell (Child SE outcome

=79% proficient overall)

Location 1

C1 3.7 3.9

C2 4.0

Location 2

C1 5.5 5.6

C2 5.7

• How did they do over time? • How are they doing overall? • How are different locations and classrooms

doing? • What are some recommendations for PD

and PBC?


CLASS Emotional Support Scores Mean for

Location

Washer Lake (90%

proficient overall)

Location 1

C1 4.8

4.6 C2 4.5

C3 4.6

Location 2

C1 6.8

6.8 C2 6.7

C3 7

C4 6.4

Location 3

C1 6.6 6.4

C2 6.2

SUMMARY

Do we think what BEACON is doing related to PD and PBC for the

sites we have looked at appears to be making a difference with respect to social-emotional teaching practices?

‘What else would be useful to know to answer this question?

DOCUMENTING PD AND PBC IMPLEMENTED AS INTENDED

WHAT CAN BEACON DO?

• Brainstorm ideas about what kinds of data BEACON can collect regarding their implementation of PD and PBC

DOCUMENTING IMPLEMENTATION OF PD AND PBC

Related to:

Did we do what we said we would do related to PD and PBC to support implementation of effective

teaching practices and attainment of school readiness goals?

Example PD/PBC data BEACON gathered: • Number and duration of Pyramid Model trainings • List and cost of materials or resources purchased • Training content implemented as intended (workshop fidelity

forms) • Amount (dose) of coaching provided • Coaching implemented as intended (coaching logs)

WRAP UP: MAKING SENSE OF DATA TO

INFORM PD AND PBC

OVERARCHING QUESTIONS TO CONSIDER

• Is what we are doing with PD and PBC making enough of a difference to support implementation of effective teaching practices and attainment of school readiness goals?

• Are we implementing PD and PBC as intended?

• What other implementation supports will be needed to support implementation of effective teaching practices and attainment of school readiness goals? (Link to 3Ps)

IMPLEMENTATION NOTES

• What data do you have related to school readiness goals that could help inform decisions about PD and PBC?

• What data do you have related to effective teaching practices that could help inform decisions about PD and PBC?

• What data do you have related to whether you implemented PD and PBC as intended?

ADDITIONAL RESOURCES

• National Center on Program Management and Fiscal Operations (PMFO)

http://eclkc.ohs.acf.hhs.gov/hslc/tta-

system/operations/center

http://eclkc.ohs.acf.hhs.gov/hslc/tta-system/operations/center




hg Handout 1: School Readiness, Teaching Practices, and Professional Development

Part 1

Use the table below to describe the domain and goals you will work with during our time together.

What Domain Area?

Which school readiness goals are you focusing on?

How are these goals measured?

What is your measurable outcome?

Part 2

Handout 5.1: Apply It: School Readiness, Teaching Practices, and Professional Development 1

BEACON used several sources to identify evidence-based teaching practices (EBP) that will support children’s schools readiness goals related to social and emotional development. They wrote a list of key teaching practices that teachers can use to support children’s learning.

Social Emotional Goals for Children Examples of Teaching Practices that Teachers Can Use to Support This Goal Sources* for EBP


Initiate frequent and extended conversations with children, actively listen, contribute relevant responses, and ask related questions.

Provide opportunities for children to actively engage with their peers, activities in which they

work together.

House Framework CLASS TPOT ECERS-R Incredible Years *sources for EPB will vary based on domain area.

Children will develop and demonstrate the ability to recognize and regulate emotions, attention, impulses, and behavior.

Clearly teach, explain, and review the classroom rules and behavior expectations with children.

Teach children about emotions, how to recognize feelings in themselves and others, how to

express their feelings.

Children will develop confidence and positive self-awareness.

Use clear descriptive positive feedback, more than statements that provide general praise, so children know exactly what they are doing well.

Incorporate pictures, stories, and artifacts from children’s home life into classroom activities,

have visual representation of children’s family on display in the classroom.

Think about your agency. Select one or two goals to work with and complete the chart on Handout 2. Write possible teaching practices that will support the children’s goal. Consider:

• What teaching practices support child learning? • What sources did you or might you use to select teaching practices for that goal? • For this activity, complete the columns for Children’s Goals, Teaching Practices that Teachers Can Use to Support this

Goal and Sources for Evidence-Based Practices (EBP)


Handout 2: School Readiness, Teaching Practices, and Professional Development Chart

Children’s Goal Evidence-Based Teaching Practices that Teachers Can Use to Support This Goal Sources for EBP Possible PD Strategies to Support

Teachers

Handout 5.2: School Readiness, Teaching Practices, and Professional Development Chart


Making Sense of Data: Notes for Back Home

Question to Consider Data We Already Have Available Data We Might Want to Gather

What data do we have related to school readiness goals that could help inform

decisions about PD and PBC?

What data do we have related to effective teaching

practices that could help inform decisions about PD

and PBC?

What data do we have related to whether we

implemented PD and PBC as intended?


MAKING SENSE OF DATA

NOTES FOR BACK-HOME

Question to Consider Data We Already Have Available

Data We Might Want to Gather

What data do we have related to

school readiness goals that could

help inform decisions about

PD and PBC?


effective teaching practices that

could help inform decisions about

PD and PBC?


whether we implemented PD

and PBC as intended?

NCQTL Leadership Academy – August 2013

making sense of data to inform professional

Documents