disproportionality data guide

Download Disproportionality Data Guide

Post on 30-Dec-2015




1 download

Embed Size (px)


Disproportionality Data Guide. Using Discipline Data within SWPBIS to Identify and Address Disproportionality. Kelsey R. Morris, EdD —University of Oregon Aaron Barnes, PhD—Minnesota Department of Education. Session B9. Purpose Describe a framework and steps for: - PowerPoint PPT Presentation


Disproportionality Data Guide

Disproportionality Data GuideUsing Discipline Data within SWPBIS to Identify and Address Disproportionality

Session B9Kelsey R. Morris, EdDUniversity of OregonAaron Barnes, PhDMinnesota Department of EducationPurposeDescribe a framework and steps for:Identifying levels of disproportionalityAnalyzing data to determine solutionsMonitoring effectiveness of action plans in addressing disproportionalityAudienceSchool/District teams seeking to reduce racial and ethnic disproportionality in school disciplineMaximizing Your Session ParticipationWhere are you in your implementation of the concepts presented?Exploration & AdoptionInstallationInitial ImplementationFull ImplementationWhat do you hope to learn?What new learning do you take away from the session?What will you do with your new learning?BackgroundRacial and ethnic discipline disproportionality is both long-standing and widespread.In 1973 African American students almost twice as likely to be suspended than white peers. By 2006, more than three times more likely (Losen & Skiba, 2010).African American students risk suspension for minor misbehavior and suspension/expulsion for same behavior as other students from other racial/ethnic groups (Skiba et al., 2011).These differences have been found consistently across geographical regions of the United States and cannot be adequately explained by the correlation between race and poverty (Noltemeyer & Mcloughlin, 2010). 4Moving ForwardEducators must address this issueIdentify rates of discipline disproportionalityTake active measure to reduce itMonitor the effects of interventions on disproportionality Rigorous collection and analysis of data aids:Understanding the need for changeIdentifying areas for improvementDetermining appropriate actionEnsure that efforts to reduce disproportionality are effectiveGuide necessary system adjustments

Data SourcesRequired features:Consistent entry of ODR data and student race/ethnicitySchool enrollment by race/ethnicityInstantaneous access for school teams (not just district teams)Capability to disaggregate ODRS and patterns by race/ethnicityCapability to calculate risk indices and risk ratios by race/ethnicityData SourcesRecommended features:Standardized ODR forms and data entryODR forms with a range of fields (e.g., location, time of day, consequence)Clear operational definitions of problem behaviorsClear guidance in discipline procedures (e.g., office vs. classroom managed)Instantaneous graphing capabilityCapability to disaggregate graphs by race/ethnicityAutomatic calculation of disproportionality graphs, risk indices, and risk ratiosIs there a problem?Why is it happening?What should be done?Is the plan working?The organization of this guide is based on a four-part problem-solving model commonly used in educational settings (as described by Tilly, 2008). This model provides an effective set of steps to use in using data for discipline decision making. 8Step 1: Problem IdentificationIs there a problem?Identify the difference between what is currently observed (performance) and what is expected our desired (goals).Example: 62% of students have 0-1 ODRs80% is the goalProblem is identified with 18% difference between what is observed and what is expected. Defining the problem with objective measures makes the process more effective and allows accountability for improvement.Requires multiple metricsThe Problem Identification process can occur either whenever a problem is suspected or as part of a planned, recurring evaluation period (e.g., beginning of the year screening, summative end-of-year reporting). 9Step 1: Problem IdentificationRisk IndexPercent of a group that receives a particular outcomeEquivalent to the likelihood of someone from that group receiving that outcomeNecessary to calculate and compare risk indices for each racial/ethnic group

# of Enrolled Students# of Students With Referrals% of Students Within Ethnicity With ReferralsRisk IndexNative5240.00%0.4Asian211047.62%0.48Black704260.00%0.6Latino12310182.11%0.82Pacific5360.00%0.6White25516564.71%0.65Unknown000.00%0Not Listed000.00%0Multi-racial211466.67%0.67Totals:500337# of students with 1+ ODRs# of students in the group# of Latino/a students with 1+ ODRs# of Latino/a enrolled# of White students with 1+ ODRs# of White enrolledDisproportionality may be hidden if only one metric (i.e., way of counting data) is used.

For example, different groups of students may have the same overall risk of receiving ODRs (risk index), but a specific group who receives ODRs may receive many more than others (composition).

As a result, it is important to examine multiple metrics instead of just one (IDEA Data Center, 2014).10Step 1: Problem IdentificationRisk RatioRepresent the likelihood of the outcome (e.g., ODRs) for one group in relation to a comparison group. Comparison group most commonly used is White studentsRisk index for all other groups is sometimes usedRisk Ratio = 1.0 is indicative of equal riskRisk Ratio > 1.0 is indicative of overrepresentationRisk Ratio < 1.0 is indicative of underrepresentation

Risk Index of Target GroupRisk Index of Comparison GroupRisk Index of Latino StudentsRisk Index of White Students.82.65= 1.27

It is possible to use Excel spreadsheets to quickly calculate risk ratios like this example from the Wisconsin PBIS Network.

Available for free at http://goo.gl/mNcgVs

11Step 1: Problem IdentificationCompositionComparison of the proportion of students within a racial/ethnic group to the proportion of ODRs from the same groupAllows for evaluation of whether the number of ODRs from one group is proportionate to the groups size

Composition metrics are a useful addition because in some cases, risk indices and ratios may show that a similar percent of each group has received an ODR, but students from a specific group with ODRs may receive many more than students from other groups.12Step 1: Problem IdentificationRegardless of the specific discipline data system, the following general steps are used:Select metrics to useRisk ratios and composition reports are recommended Calculate metricsCompare to goalsPrevious years from same schoolLocal or national norms2011-2012 U.S. public schools using SWIS with at least 10 African American and 10 White studentsMedian risk ratio (African American to White) = 1.84; 25th percentile = 1.38Logical criteriaU.S. Equal Employment Opportunity Commission (EEOC)Disparate impact criterion Goal risk ratio range between .80 and 1.25Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria.

One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).13Step 2: Problem AnalysisWhy is it happening?By finding the specific cause of the problem, teams can identify more effective solutions.Focus: identifying variables that can be changed, not individual traits or variables that are beyond the control of the systemKey: is the disproportionality identified in Step 1 consistent across all situations or more pronounced in some situations?Disproportionality across all settings indicates explicit biasDisproportionality in specific settings indicates implicit biasThe key outcome in Problem Analysis for disproportionality is to identify whether the disproportionality identified in Problem Identification is consistent across all situations or more pronounced in some situations (McIntosh, Girvan, Horner, & Smolkowski, 2014).

A pattern where disproportionality is consistently high across all situations indicates the effects of explicit bias, or systematic discrimination. A pattern where disproportionality is higher in some situations and not as high in others may indicate the effects of implicit bias, the unconscious and unintended use of stereotypes in decision making (Lai, Hoffman, Nosek, & Greenwald, 2013).

We refer to situations that are more likely to lead to disproportionality as vulnerable decision points because it is at these times where bias is more likely to affect decisions to refer a student to the office or suspend the student (McIntosh et al., 2014). 14Step 2: Problem AnalysisVulnerable Decision Points (VDPs)What problem behaviors are associated with disproportionate discipline?Where is there disproportionate discipline?When is there disproportionate discipline?Times of day, days of the week, months of the yearWhat motivations are associated with disproportionate discipline?Perceived function of problem behaviorWho is issuing disproportionate discipline?Disparities do not indicate racism, but rather contexts where additional supports are necessary.

In addition, it is also worthwhile to examine the achievement gap to assess whether lower academic skill are related to problem behavior for certain groups (i.e., the achievement gap may be exacerbating the discipline gap; Gregory, Skiba, & Noguera, 2010). 15Step 2: Problem AnalysisThe following steps can be used:Assess PBIS fidelityIdentify vulnerable decision points (VDPs)Assess achievement gapIdentified SubgroupLocationTime of DayProblem BehaviorMotivationAfrican American students are receiving ODRs in the hallways after 2:30 PM for disrespect and the behavior is maintained by peer attentionRemove all filters and include all subgroups to confirm whether this statement is unique to this subgroup.Identify vulnerable decision points. Specific situations that are more susceptible to disproportionality can be identified by examining the metrics used in Problem Identification (

View more >