tips data decisionmaking-in school coach training summer institute 2012
TRANSCRIPT
USING DISCIPLINE DATA TO SOLVE PROBLEMS
1
In-School Coach TrainingSummer Institute 2012
Exceptional Children DivisionBehavior Support & Special Programs
Positive Behavior Intervention & Support Initiative
2
Desired Outcomes
The goal of this training is to help schools move beyond simple collection of discipline data to using the collected data to solve problems.
3
Rationale
Data should be used regularly to problem solve because it…
provides an objective viewpoint of the current state.
Increases efficiency by making needed action items clear.
provides information about effectiveness of practices so modifications can be made in a timely manner.
ensures that strategies and interventions are based on precise problem statements.
4
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Data Sources
5
Data Sources
Discipline data is collected in many ways:
Office referrals In and out of school suspensions Classroom management systems Behavior contracts
6
Data Sources: Office Referral Information
Office referrals are the most commonly used discipline data source.
Regardless of the collection method, it is important to gather enough information to determine patterns and design effective solutions.
Ensuring accurate collection of data is an important component of the PBIS team’s role.
7
Average Referrals Per Day 8
Types of Problem Behavior9
Student Referrals
2 or more referrals
10
Location11
Data Sources: Other Data To Consider12
Time of dayProbable motivationAdministrator decisionODR rate compared to national average
Others involved
SWIS summary 2010-11 (Majors Only)4,634 schools; 2,394,591 students; 1,802,178 ODRs
Grade Range
Number of Schools
Mean Enrollment per school
MeanODRs per 100 stud/ school day
Median ODRs per 100 per school day
25th PercentileODR/100/ school day
75th PercentileODR/100/ school day
K-6 2979 456 .32 (.41) .21 .11 .39
6-9 889 626 .65 (.81) .46 .25 .79
9-12 390 818 .85 (.86) .62 .34 1.07
PreK-8 254 438 .49 (.49) .32 .18 .65
PreK-12 50 455 1.1 (3.0) .37 .18 .71
Data Sources: Questions to Consider
Is our database accurate and reliable? Are we capturing information about behavior
in classrooms? What modifications does our system need (if
any)? What training does our staff need regarding
tracking and accessing data? How can we ensure our teachers are utilizing
the data in an interactive manner to design effective solutions to problem behavior on a regular basis?
15
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Meeting Foundations
16
Meeting Foundations: Elements
Team purpose Defined agreements about
processes Established roles and
responsibilities Electronic meeting minutes
17
Meeting Foundations: Overview1. Meeting starts and ends on time
2. Consistent attendance by team members
3. Agenda is used to guide meeting topics
4. Process is in place to monitor progress of implemented solutions (review previous meeting minutes)
5. System is used for documenting decisions
6. Team members prepare for and meet responsibilities during meeting
7. Next meeting is scheduled
8. All team members (absent or present) are given minutes within 24 hours of the meeting
9. Decision makers are present when needed
10. Protocol is established for when administrator is unable to attend
11. Efforts are making a difference in the lives of children/students
18
1. Inform facilitator of attendance issues before meeting
2. Avoid side talk3. Remind each other to
stay focused4. Start and end on time5. Be an active participant
19
Meeting Foundations: Roles
Core roles Facilitator Minute taker Data analyst Communication coordinator Time keeper Administrator Active team member
Backup for each role
20
Can one person serve multiple roles?
Are there other roles needed? EX:
communication coordinator, timekeeper
Meeting Foundations: Roles21
Facilitator: Secures date, time, and location for meetings Manages room set-up (projector, chairs,
smartboard, etc.) Guides team members in the meeting Ensures participation by all members, using
facilitative techniques Works with the time keeper and the minutes
taker to guarantee agenda is followed and time is honored
Acts as liaison between team and administration
Meeting Foundations: Roles22
Minutes Taker: Arranges for back-up minutes taker, if absent Manages the Meeting Minutes and Problem-
Solving Action Plan Captures the discussions and decisions made by
the team in a concise, accurate method Is comfortable reviewing, evaluating and
revising items with the team Sends minutes of meeting to team members
within established time limit Ensures minutes/action plan is posted and/or
shared with appropriate staff members
Meeting Foundations: Roles23
Data Analyst: Competent using technology Has access to necessary data Able to perform cursory analysis of data
to identify possible problems Comfortable mining data live in meeting Capable of creating custom reports and
graphs as team digs deeper in the data
24
25
Meeting Foundations: Minutes
Documentation Logistics
date time location roles
Agenda today’s items next week’s items
Discussion items, decisions made, tasks and timelines assigned
Problem statements, solutions/decisions/tasks, responsible people, timelines assigned, and an evaluation plan
Review Meeting minutes
An effective process check Baseline for current
meeting
Visual tracking of focus topics during and after meetings
Prevents side conversations
Prevents repetition Encourages completion of
tasks
26
Meeting Foundations: Environment27
Problem
SolutionOut of
Time
Use Data
A key to collective problem solving is to provide a visual context that allows everyone to follow and contribute.
Consider using one form to guide the activities of the meeting and using a
projector so that all team members view the content and participate.
Langley Elementary PBIS Team Meeting Minutes and Problem-Solving Action Plan Form
Today’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:
Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:
Team Members (bold are present today)
Today’s Agenda Items Next Meeting Agenda Items01. 02. 03.
1. 2.
Information for Team, or Issue for Team to Address
Discussion/Decision/Task (if applicable) Who? By When?
Administrative/General Information and Issues
Implementation and EvaluationPrecise Problem Statement, based on review of
data(What, When, Where, Who, Why)
Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,
Safety)Who? By When?
Goal, Timeline, Decision Rule, & Updates
Problem-Solving Action Plan
Our RatingYes So-So No
1. Was today’s meeting a good use of our time?2. In general, did we do a good job of tracking whether we’re completing the tasks we agreed on at previous meetings?
3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?4. In general, are the completed tasks having the desired effects on student behavior?
Evaluation of Team Meeting (Mark your ratings with an “X”)
Where on this form would you place:
1. Planning for PTA meeting
2. Too many students in the “intensive support” for literacy
3. Schedule for hallway monitoring for next month
4. There have been five fights on playground in last month
5. Next meeting report on lunch-room status
28
Meeting Foundations: Recording Relevance
Minors-what would we like to do about communicating the minors with families? There is inconsistency among staff, not all teachers use the minors as a teaching tool in the same way. Is this a problem? What should be do?
Discussion: Perhaps we create a little blurb that goes out to families that
teachers will use when sending them home. Sending them home creates a problematic situation, can be an issue with communication with families. Perhaps we need to just say to staff a general reminder about what is going on with the minors for families of multiple students or friends, etc. We will wait until next year to re-train staff and discuss how to use WHOAS and how to communicate them with parents.
Issue: families are not signing and returning minor incident reports
Possible hypotheses: multiple students in household bringing minor incident reports home? parent gets upset with student &
students not giving form to parents to sign?
Decision: re-examine the process being used to document and communicate about minor
incidents
Information for Team, or Issue for Team to Address
Discussion/Decision/Task (if applicable) Who? By When?
Minor incident reportsRe-examine the process being used to document and communicate about minor incidents
team 2-15-10 team meeting
Take proposal to staff Team 2-15-10 staff meeting
29
Implementation and Evaluation
Precise Problem Statement, based on review of data(What, When, Where, Who, Why)
Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,
Safety)Who? By When?
Goal, Timeline, Decision Rule, & Updates
October : We have way too many ODRs and we have a problem with aggression/fighting and disrespect on the playground during K,1,2 morning and lunch recess
BB talked about the 3 students who are starting CICO system. AT is skeptical about parent involvement. TP saw the program work fine without parent involvement at the previous school. AA walked in and asked if another student could be on CICO. JJ asked if the supervisors were moving around, he had seen them talking together in the middle of the playground once last week.We should plan to reteach playground expectations .Older students could teach primary students basketball game rules – contingent on D.C. coming to school on time
Cico team
LL analyze referral data
11/16/2009
This month’s precise problem statement: We have three high fliers, K-2 and a 6th grader on the playground, at 10:00 and 11:45/12:00 46/69 number of total major and minor referrals on the playground
-Reteaching game and playground rules-Reinforce proper lining up-Reinforce exiting and entering building
-MM will ask MA if she can reinforce/teach playground rules
Problem-Solving Action Plan
Last month’s example: We are above the national average of ODR’s and we have a problem with aggression/fighting and disrespect on the playground during K,1,2 morning and lunch recess with 4 students.
3 students are starting CICO systemOlder students teach primary students basketball game rules – contingent on D.C. coming to school on timeReteach playground expectations
Cico teamLL analyze referral dataNN/MM
11/16/2009January PBS meeting11/16/09
CICO Team check-in every two weeks to see if students are meeting 80% of their goal.Decrease of playground referrals by 25% by January PBS meeting. CICO team will reportDecrease of playground referrals by K-2 students and D.C. defiance/disruption referrals
30
Meeting Foundations: Structure/Flow
1. Attendance, roles for meeting2. Next meeting date3. Review agenda for meeting4. Review/status update of previous meeting
minutes 5. Review data and use problem solving model to
prompt the development of a comprehension action plan
6. Reports needed for next meeting7. Team assessment of meeting 8. Dissemination of meeting minutes
31
33
Identify Problem
Develop Hypothes
is
Discuss and
select solution
s
Develop and
implement action
plan
Evaluate and revise action plan
04/12/2023Newton, J.S., Todd, A. W., Horner, R.H., Algozzine, B., &
Algozzine K., 2010
34
Any tasks assigned get copied to the
meeting minutes of the next meeting as
a follow up item
Meeting Agenda Item: Meeting
Foundations Tasks: What, by whom, by
when
35
Completed example
36
Meeting Foundations: Coaches’ Checklist
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Problem Solving Process
38
Problem Solving Process: Using Data
In the previous section, you looked at data that gave an overall view of patterns in a school, but didn’t provide enough data to move forward.
Collecting data is pointless if it is not used by all staff members to solve both school-wide and classroom problems.
The ultimate goal is to improve the school experience for students, staff and families.
Data should help us clearly identify problems and lead us to specific possible solutions.
We need to move past “admiring the problem.”
39
Problem Solving Process: Using Data
In order to solve problems successfully, schools need to use a specific process to define precise problems and design solutions.
The Team Initiated Problem Solving (TIPS) model was created for PBIS teams.
PBIS teams should use the TIPS model for school-wide data, grade level, and for individual students.
All staff members need to have access to the data and be taught how to use the information to solve everyday challenges.
*The TIPS model can easily be used for academic or corporate problems as well.
40
41
Problem Solving Process: Definitions Problem: Difference exists between
expected/desired student behavior and current student behavior.
Problem identification: Difference is discovered and significance is determined.
Problem solving: A plan is created to reduce or eliminate difference.
42
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Identify Problems
43
44
Identify Problems: Broad Problem Statements Typically, schools use behavioral data to
define broad problems. Broad problem statements do not give
enough information to design effective solutions.
The purpose of broad problems statements is to elicit questions that can be answered with data to better define the problem.
45
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Develop Hypothesis
48
Develop Hypothesis: Determine Cause of Problem
Determining the cause, or uncovering why a behavior is occurring, is essential to developing interventions.
The goal is to help students learn to appropriately meet their needs.
When collecting data, it is important for staff to make their best, most informed guesses about why students are engaging in problem behaviors.
49
50
Develop a Hypothesis: Things to Consider
A hypothesis is an explanation for what the data and your experience tell you.
Data can only give part of the picture. Staff with the most direct experience
with the problem need to add their insight to accurately define cause.
An accurate hypothesis is crucial to designing solutions that will be effective.
51
Develop a Hypothesis: Identify a Precise Problem
Creating strategies without knowing the precise problem leads to inefficient solutions and wasted time.
In order to ensure precise problem statements, go back to the data to answer the following: What is the problem? How often is it happening? Where is it happening? Who is engaged in the behavior? When is the problem most likely to occur?
52
Activity: Identifying Precise Problem Statement
Which partial statement is more precise? 53
Too many ODRs
15 instances of disrespect
24 ODRs between 1:00 and 1:30 Too many ODRs in the afternoonToo many ODRs outside the
classroom25% of students have at least 2 ODRs
Too many ODRs on the playground
6 ODRs on the playground
Many students have ODRs
Total of 12 ODRs for aggression on the playground in the last month;
this number is more than last year and shows an increasing trend for
this year; these incidents are occurring during the first recess, and there are different students involved
each time.
Which statement is a complete Precise Problem Statement?
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Discuss and Select Solutions
54
55
Discuss and Select a Solution: Creating a Goal
Prior to designing solution strategies, a clear goal needs to be set that can be evaluated using data.
Goals can be measured in the following ways: Percent reduction Absolute reduction Satisfaction level
The best practice is to use more than one measure of evaluation.
56
Discuss and Select Solutions: Design
Prevent: • How can we modify the context in which the behavior occurs? • Can we remove or alter the triggers for problem behavior?
Teach:• Do we need to define the expected behaviors more clearly?• Can we teach a replacement behavior that meets the same need?• Do we need to provide additional demonstration and/or practice of the
desired behaviors?
Respond:• In what ways can we immediately prompt the correct behavior?• Does our system frequently reward the desired behavior ?• Can we extinguish problems by withholding reinforcement of the
problem behavior? • Can we develop meaningful consequences to correct the problem
behavior that provide a learning or practice component of the desired behavior?
• How will we effectively respond to behaviors that compromise safety while ensuring the student’s dignity is maintained?
57
58
Discuss and Select Solutions: TIPS Worksheet
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Develop Action Plan
59
60
Develop and Implement Action Plan: Design The most effective plans utilize multiple
strategies that affect all causes of the problem-prevent/teach/respond (prompt, reward, correction, extinction, safety).
To ensure fidelity of implementation, specific tasks need to be assigned to individuals with a timeline for checking back.
Change takes time, so implement the plan for at least one month before evaluating.
61
Develop and Implement Action Plan: After Creation
Once strategies are selected, specific tasks need to be delegated and tracked using an action plan.
Action plans should drive the creation of future meeting agendas as well as generate clear action steps at the end of each meeting.
Effective action plans are: Used regularly Frequently reviewed and updated Accessible to all staff Made of specific, manageable action steps with clear
timelines Developed using data from staff and teams
62
Fidelity
Use weekly 1-5 survey from teachers to assess implementation of plan.
04/12/2023Newton, J.S., Todd, A. W., Horner, R.H., Algozzine, B., &
Algozzine K., 2010
63
Are we doing the plan?
1 ….. 2 …..3 ….. 4 ….. 5No Yes
Develop and Implement Action Plan: Ensuring Fidelity
USING DISCIPLINE DATA TO SOLVE PROBLEMS
Evaluate and Revise Action Plan
64
65
Evaluate and Revise Action Plan
Use data to answer the following:
Has the goal been met? If yes, choose one of the following options:
1. Increase goal for the same problem.
2. Choose another problem to address.
3. End use of the plan.
If no, choose one of the following options:
1. Evaluate fidelity of implementation.
2. Consider the accuracy of the hypothesis.
3. Alter the timeline.
4. Find ways to adjust the solutions.
66