enhancing equity in school discipline 1: using discipline data to assess and address...

Enhancing Equity in School Discipline 1:

Using Discipline Data to Assess and Address Disproportionality

Kent McIntosh

Kelsey Morris

University of Oregon

Handouts:http://pbis.sccdoe.org

PBIS Center Disproportionality Workgroup

Acknowledgements

Aaron Barnes Alondra Canizal Delabra Yolanda Cargile Erin Chaparro Tai Collins Bert Eliason Erik Girvan Milaney Leverson Steve Goodman Clynita Grafenreed Ambra Green Rob Horner

Don Kincaid Tim Lewis Kent McIntosh Kelsey Morris Rhonda Nese Vicki Nishioka Heidi von Ravensberg Jen Rose Russ Skiba Kent Smith Keith Smolkowski

Equity 1: Data (Session B)District and school teamsExternal coaches

Equity 2: Implicit Bias (Session C)School staffThose supporting school staff

(coaches, teams, etc.)

Who is the audience for the presentations?

1. I am aware of my personal biases.

2. I am concerned about the consequences of bias in education.

3. I have effective strategies for reducing bias in educational decisions.

Starting Questions:How much do you agree?

(Devine et al., 2012)

1. Describe the challenge of disproportionality in school discipline

2. Share an intervention approach for enhancing equity in school discipline

3. Guide you though a process for using data to:1. Assess levels of disproportionality

2. Identify causes and solutions

3. Plan to monitor progress

Overview of Today’s Session

Handouts: http://www.pbis.org

http://pbis.org/

http://pbis.org/

Disproportionality in School Discipline (Losen et al., 2015)

Poverty plays a role, but racial disproportionality remains, even when controlling for povertyAmerican Psychological Association, 2008Skiba et al., 2005Wallace et al., 2008

Addressing Common Questions

“Isn't it all really about poverty?”

No evidence of different base rates of behavior for any subgroupsBradshaw et al., 2010Losen & Skiba, 2010Skiba et al., 2014


“Aren’t Black boys just more violent?”

No! Our research indicates that disproportionality comes from unconscious bias – that we’re not even aware of.Banaji & Greenwald, 2013Greenwald & Pettigrew, 2014van den Bergh et al., 2010


“Are you saying that all teachers are racist?”

We are aware of the extent of disproportionality

We are committed to enhancing equity in school discipline

This work is uncomfortable There are a wide range of approaches

that could work to enhance equity

A few assumptions…

A 5-point

Intervention

Approach to Enhance Equity in School Discipline

http://www.pbis.org/school/equity-pbis


1. Use engaging academic instruction to reduce the achievement gap

2. Implement a behavior framework that is preventive, multi-tiered, and culturally responsive

3. Collect, use, and report disaggregated student discipline data

4. Develop policies with accountability for disciplinary equity

5. Teach neutralizing routines for vulnerable decision points

5-point Intervention Approach



Teacher presents student with grade level academic task

Teacher removes academic task or removes student

Student engages in problem behavior

Student escapes academic task

Student’s academic skills do not improve

1. Why a focus on engaging academic instruction?

(McIntosh et al., 2008)

Explicit instruction High rates of opportunities to respond Quality performance feedback Progress monitoring and data-based

decision making

What do we mean by engaging academic instruction?

(Hattie, 2009)

2007-08 2008-09 2009-10 2010-11 2011-12 2012-130%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

43% 47%36%

28% 24%11%

81% 84%88%

94% 91% 94%

38% 37%

52%

66% 67%

83%

White

Latino

Perc

ent M

eetin

g or

Exc

eedi

ng S

tand

ards

Effects of Engaging Instruction on the Achievement Gap

Tigard-Tualatin School District (Chaparro, Helton, & Sadler, in press)

1. Proactive, instructional approach may prevent problem behavior and exposure to biased responses to problem behavior

2. Increasing positive student-teacher interactions may enhance relationships to prevent challenges

3. More objective referral and discipline procedures may reduce subjectivity and influence of cultural bias

4. Professional development may provide teachers with more instructional responses

2. Why start with a foundation of SWPBIS?

(Greflund et al., 2014)

Effects of SWPBIS onDiscipline Disproportionality(Vincent, Swain-Bradway, Tobin & May, 2011)

200506 200607 2007080%

5%

10%

15%

20%

25%

SWPBISNo SWPBIS

Bla

ck-W

hite

Dis

cip

line

Ga

p

How inviting are we for all?

Examined change in Black-White Relative Risk Index for suspensions in 46 schools

Two key predictors of decreased disproportionality:Regular use of data for decision making Implementation of classroom SWPBIS

systems

Which SWPBIS Features are Most Related to Equity? (Tobin & Vincent, 2011)

Expected behaviors defined clearly Problem behaviors defined clearly Expected behaviors taught Expected behaviors acknowledged regularly Consistent consequences CW procedures consistent with SW systems Options exist for instruction Instruction/materials match student ability High rates of academic success Access to assistance and coaching Transitions are efficient and orderly

Which features predicted decreased disproportionality?

Ensure equitable access to praise and acknowledgment systems

Develop and revise school-wide systems with active involvement of families, students, and the community

Use regular student and family surveys to assess acceptability and fit

Culturally Responsive SWPBIS Implementation

Student Input &SatisfactionSurvey

Aligned directly with SWPBIS Tiered Fidelity Inventory (TFI) Tier I Scale Identifies SWPBIS critical feature Identifies cultural responsiveness conceptProvides non-examples, examples, activities,

and resources

PBIS Cultural Responsiveness Companion

PBIS Cultural Responsiveness Companion

http://tinyurl.com/ncn8fmf

http://tinyurl.com/ncn8fmf

Disproportionality Data Guide

3. Using disaggregated data to assess and address equity




4. Implement policies with accountability for equity How could policy work fit in to enhancing

equity?Could highlight a common priorityCould reduce effects of explicit biasCould enable implementation of other

aspects of equity interventionsCould reduce use of discriminatory practices

Enacting policies that nobody knows about Enacting policies that don’t change

practice Policies without accountability for

implementation

What does not work in policy

Include a Specific Commitment to Equity Create mission statements that include equity Enact hiring preferences for equitable discipline

Install Effective Practices Require clear, objective school discipline procedures Support implementation of proactive, positive

approaches to discipline Replace exclusionary practices w/ instructional ones

Create Accountability for Efforts Create teams and procedures to enhance equity Share disproportionality data regularly Build equity outcomes into evaluations

Equity Policy Recommendations

Stay tuned (next session…)

5. How can we reduce implicit bias in our decision making?

Discipline Data Systems Needs Required features:

Consistent entry of ODR data and student race/ethnicity

School enrollment by race/ethnicity

Instantaneous access for school teams (not just district teams)

Capability to disaggregate ODRs and patterns by race/ethnicity

Discipline Data Systems Needs Recommended features:

Standardized ODR forms with a range of fields (e.g., location, time of day, consequence)

Clear definitions of problem behaviors

Clear guidance in discipline procedures (e.g., office vs. staff-managed)

Instantaneous graphing capability

Capability to show graphs by race/ethnicity

Automatic calculation of disproportionality data

The School-Wide Information System (SWIS) meets these criteriaAvailable at http://www.pbisapps.org

Discipline Data Systems Needs

http://www.pbisapps.org/

What data sources will you be using? Options:

School-level data systems (e.g., SWIS) Which school(s)?

State-level data systemsNothing? Use the SWIS demo account data

Worksheet Activity

Discipline Data System Example

SWIS Demo Data: http://www.pbisapps.org

http://www.pbisapps.org/

Discipline Data System Example

General Problem Solving Model

2. Problem Analysis

3. Plan Implementation

4. Plan Evaluation

1. Problem Identification

Is there a problem?

Why is it happening?

What should be done?

Is the plan working?

2. Problem Analysis


4. Plan Evaluation


Step 1: Problem Identification

Is there a problem?

Step 1: Problem Identification General problem-solving approach:

Use valid and reliable metrics that assess outcomes of concern

Quantify the difference between current outcomes and expected outcomes (goals)

For disproportionality:Compare outcomes (e.g., discipline rates) across

racial/ethnic groups (e.g., Black vs. White)Quantify these differences Multiple metrics are recommended

(IDEA Data Center, 2014)

Step 1: Problem IdentificationCommon Metrics Risk Index

Percent of a group that receives an ODR or suspension (i.e., risk for that outcome)

# of Enrolled Students

# of Students With Referrals

% of Students Within Ethnicity With Referrals

Risk Index

Native 5 2 40.00% 0.4

Asian 21 10 47.62% 0.48

Black 70 42 60.00% 0.6

Latino 123 101 82.11% 0.82

Pacific 5 3 60.00% 0.6

White 255 165 64.71% 0.65

Unknown 0 0 0.00% 0

Not Listed 0 0 0.00% 0

Multi-racial

21 14 66.67% 0.67

Totals: 500 337

# of students with 1+ ODRs

# of students in the group

# of Latino/a students with 1+ ODRs

# of Latino/a students enrolled

# of White students with 1+ ODRs

# of White students enrolled

Step 1: Problem IdentificationCommon Metrics Risk Ratio

Risk index for one group divided by risk index for comparison group (usually White students)

1.0 is equal risk

> 1.0 is overrepresentation

< 1.0 is underrepresentation

Risk Index of Target Group

Risk Index of Comparison Group

Risk Index of Latino/a Students

Risk Index of White Students

.82

.65= 1.27

Step 1: Problem IdentificationCommon Metrics Risk Ratio

Risk index for one group divided by risk index for comparison group (usually White students)

1.0 is equal risk

> 1.0 is overrepresentation

< 1.0 is underrepresentation

Risk Index of Target Group

Risk Index of Comparison Group

Risk Index of Latino/a Students

Risk Index of White Students

.82

.65= 1.27

Available for free at http://goo.gl/mNcgVS

http://goo.gl/mNcgVs



Step 1: Problem IdentificationCommon Metrics Composition/Difference in Composition

Compares proportion of students in a group to the proportion of ODRs from the same group

Assesses whether the number of ODRs from one group is proportionate to the group’s size

Step 1: Problem IdentificationProcedure

1. Select metrics to use2. Calculate metrics and compare to goals

• Previous years from same school• Local or national norms

• 2011-2012 U.S. public schools using SWIS with at least 10 Black and 10 White students

• Median Black-White ODR risk ratio = 1.84 (25th percentile = 1.38)

• Logical criteria• U.S. Equal Employment Opportunity Commission (EEOC)

• Disparate impact criterion (goal risk ratio range between .80 and 1.25)

School Example: Rainie Middle School

White Black Latino/a Asian0%

10%20%30%40%50%60%

48%

31%20%

10%

School Enrollment

School-wide Information System (SWIS)

• Metric: risk ratio• Goal: All groups with a

risk ratio < 1.25

STEPS

• Calculate risk indices

• Calculate risk ratios

• African American = 3.2 (significant)

• Latino/a = 1.1

1. Complete STEP 1 (pp. 1-2)1. Select metrics to use

2. Calculate metrics

3. Compare to goals

2. Reflect on your data: to what extent is there a problem?

How clear are you on these steps?

Step 1: Worksheet Activity

Share out…

How were the steps? What metrics did you select?


2. Problem Analysis


4. Plan Evaluation


Step 2: Problem Analysis

Is there a problem?


Step 2: Problem Analysis General problem-solving approach:

Identify underlying causes of the problemFocus on variables that can be changed

For disproportionality: Identify whether disproportionality is consistent

across all situations or more pronounced in some situations

Assess other causes, such as: Achievement gap Fidelity of implementation of discipline or equity

interventions

Explicit Bias (conscious) Pattern: Consistent disproportionality across all

situations Implications: Address through strong policy,

regular reporting of data, and accountability for change

Implicit Bias (unconscious) Pattern: Peaks and valleys of disproportionality

depending on the situation Implications: Clarify discipline procedures,

provide strategies for decision making

Identifying Patterns of Explicit vs. Implicit Bias

A specific decision that is more vulnerable to effects of implicit bias

Two parts:Elements of the situationThe person’s decision state (internal state)

What is a Vulnerable Decision Point (VDP)?

Levels of specificity:

1. All ODR/suspension decisions (general self-instruction routine)

2. Identify VDPs through national data

3. Use school or district data

Options for Identifying VDPs for Intervention

National SWIS Data(2011-12)

3,026,367 ODRs

6,269 schools

47 states, plus DC

Langu

age

Defian

ce/ Disr

espect

Disruption

Fighting

Forge

ry/ Th

eft

Harassm

ent/

bullying

Lying/

chea

ting

M - Defi

ance/

Disresp

ect

M - Disr

uption

M - Dres

s code

M - Lan

guag

e

M - Other

M - Physi

cal Contac

t

M - Pro

perty M

isuse

M - Tard

y0

5

10

15

20

25

5

21

9

51

31

11

7

1 2 3 31

3

Black

Langu

age

Defian

ce/ Disr

espect

Disruption

Fighting

Forge

ry/ Th

eft

Harassm

ent/

bullying

Lying/

chea

ting

M - Defi

ance/

Disresp

ect

M - Disr

uption

M - Dres

s code

M - Lan

guag

e

M - Other

M - Physi

cal Contac

t

M - Pro

perty M

isuse

M - Tard

y02468

1012141618

4

14

5

21

31

15

10

12

6 6

2

4

WhiteOffice Referrals by Problem Behavior

Art Room

Bathro

om/ Rest

room Bus

Bus Load

ing Zone

Cafeter

ia

Classro

om

Commons/ Common Area

s

Computer La

bGym

Hall/ B

reeze

way

Librar

y

Locke

r Room

Music Room

Off-Campus

Office

Other Lo

cation

Parking L

ot

Playgro

und

Speci

al Ev

ent/

Assembly/

Field

trip

Stadium

Unknown Lo

cation

Vocational

Room0

10

20

30

40

50

60

70

1 25

0

5

56

1 03

7

1 0 1 13 2

0

9

0 0 1 0

BlackWhite

Office Referrals by Location

12:00AM

12:30AM

1:00AM

1:30AM

2:00AM

2:30AM

3:00AM

3:30AM

4:00AM

4:30AM

5:00AM

5:30AM

6:00AM

6:30AM

7:00AM

7:30AM

8:00AM

8:30AM

9:00AM

9:30AM

10:00AM

10:30AM

11:00AM

11:30AM

12:00PM

12:30PM

1:00PM

1:30PM

2:00PM

2:30PM

3:00PM

3:30PM

4:00PM

4:30PM

5:00PM

5:30PM

6:00PM

6:30PM

7:00PM

7:30PM

8:00PM

8:30PM

9:00PM

9:30PM

10:00PM

10:30PM

11:00PM

11:30PM

0.01.02.03.04.05.06.07.08.09.0

Black

12:00AM

12:30AM

1:00AM

1:30AM

2:00AM

2:30AM

3:00AM

3:30AM

4:00AM

4:30AM

5:00AM

5:30AM

6:00AM

6:30AM

7:00AM

7:30AM

8:00AM

8:30AM

9:00AM

9:30AM

10:00AM

10:30AM

11:00AM

11:30AM

12:00PM

12:30PM

1:00PM

1:30PM

2:00PM

2:30PM

3:00PM

3:30PM

4:00PM

4:30PM

5:00PM

5:30PM

6:00PM

6:30PM

7:00PM

7:30PM

8:00PM

8:30PM

9:00PM

9:30PM

10:00PM

10:30PM

11:00PM

11:30PM

0.01.02.03.04.05.06.07.08.09.0

White

Office Referrals by Time of Day

Subjective problem behaviorDefiance, Disrespect, DisruptionMajor vs. minor

Non-classroom areasHallways

Afternoons

VDPs from national ODR data

ambiguit

y

LACK OF

contact

fatigue

SWIS Drill Down (www.swis.org)

Add demographic group of interest as a filter (click to

“Include in Dataset”).

Click each graph and compare to overall patterns.

http://www.swis.org/

• Assess PBIS implementation• TFI indicates successful

implementation

• Improve office vs. staff-managed systems

• Improve consequence systems

• SWIS Drill Down for precise problem statement

White Black Latino/a Asian0

20406080

100

7144 52

100

Reading

Met

Tiered Fidelity Inventory (TFI)


1. Complete STEP 2 (pp. 3-4)1. Assess PBIS fidelity

2. Identify vulnerable decision points

3. Assess achievement gap



Stage 1 Behavior: White Students

Stage 1 Behavior: Black Students

Stage 1 Location: White Students

Stage 1 Location: Black Students

Stage 1 Time: White Students

Stage 1 Time: Black Students

Stage 1 Grade: White Students

Stage 1 Grade: Black Students

Share out…

How were the steps? What did you find?


2. Problem Analysis


4. Plan Evaluation


Step 3: Plan Implementation

Is there a problem?



Step 3: Plan Implementation General problem-solving approach:

Use the information from Step 2 (Problem Analysis) to select strategies

Create a plan to ensure adequate implementation of the strategies

For disproportionality:No differences from general approach

Step 3: Plan ImplementationOptions All issues

Calculate and share disproportionality data regularly

Inadequate PBIS implementation Implement core features of PBIS to establish a foundation of support

Clarify ODR definitions and procedures

Misunderstandings regarding school-wide expectations Enhance cultural responsiveness of PBIS with input from families,

students, and community

Significant academic achievement gap Use effective academic instruction

Disproportionality across all settings (indicating explicit bias) Enact strong equity policies that include accountability

Disproportionality in specific settings (indicating implicit bias) Teach neutralizing routines for vulnerable decisions

1. Complete STEP 3 (pp. 5-6)1. Identify strategies to implement

2. Create a detailed action plan



Share out…

How were the steps? What are you planning…

Now? After that?


2. Problem Analysis


4. Plan Evaluation


Step 4: Plan Evaluation

Is there a problem?



Is the plan working?

Step 4: Plan Evaluation General problem-solving approach:

Assess whether the plan is implemented Calculate metrics (from Step 1) regularlyCompare outcomes to predetermined goals

For disproportionality:Disproportionality metrics may not be as

sensitive to rapid change as other measures Consider monthly assessment of implementation and

quarterly assessment of disproportionality metrics Avoid using risk indices (will rise throughout year)

Step 4: Plan EvaluationProcedure

1. Identify the time periods for evaluating disproportionality data

2. Assess fidelity of plan implementation

3. Calculate metrics selected in Step 1

4. Compare to the goal determined in Step 1

5. Share results with relevant stakeholders

October January March June0

0.51

1.52

2.53

3.54

Black-White ODR Risk Ratio

Risk Ratio

• 6th grade team may need a refresher on office vs. staff-managed behaviors

• Revise action plan for next year

• Continue evaluation cycle


1. Complete STEP 4 (p. 7)1. Identify the time periods for evaluation

2. (complete later)

3. (complete later)



Share out…

How were the steps? What are your time periods going to be?


Disproportionality in school discipline is one of the biggest challenges in education today

We can use data to assess and monitor how we are doing If you don’t have the data you need at hand,

advocate for it The same steps we have for solving

discipline problems work for disproportionality

Big Ideas

Contact Information

Kent McIntoshSpecial Education Program

University of Oregon

[email protected]

@_kentmc

Handouts: http://kentmcintosh.wordpress.com

Cannon Beach, Oregon © GoPictures, 2010

mailto:[email protected]

https://twitter.com/@_kentmc

Greflund, S., McIntosh, K., Mercer, S. H., & May, S. L. (2014). Examining disproportionality in school discipline for Aboriginal students in schools implementing PBIS. Canadian Journal of School Psychology, 29, 213-235.

McIntosh, K., Barnes, A., Morris, K., & Eliason, B. M. (2014). Using discipline data within SWPBIS to identify and address disproportionality: A guide for school teams. Eugene, OR: Center on Positive Behavioral Interventions and Supports. University of Oregon.

McIntosh, K., Girvan, E. J., Horner, R. H., Smolkowski, K., & Sugai, G. (2014). Recommendations for addressing discipline disproportionality in education. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports.

Vincent, C. G., Swain-Bradway, J., Tobin, T. J., & May, S. (2011). Disciplinary referrals for culturally and linguistically diverse students with and without disabilities: Patterns resulting from school-wide positive behavior support. Exceptionality, 19, 175-190.

References

enhancing equity in school discipline 1: using discipline data to assess and address...

Documents

school discipline http

school discipline losen

student discipline data

student student

disciplinary equity

racial disproportionality

extent of disproportionality

levels of disproportionality