visual performance feedback: effects on targeted and nontargeted staff

http://bmo.sagepub.com/Behavior Modification

http://bmo.sagepub.com/content/36/5/687The online version of this article can be found at:

DOI: 10.1177/0145445511436007

2012 36: 687 originally published online 28 March 2012Behav Modifand Keith D. Allen

Raymond V. Burke, Monica R. Howard, Jane L. Peterson, Roger W. PetersonStaff

Visual Performance Feedback : Effects on Targeted and Nontargeted

Published by:

http://www.sagepublications.com

can be found at:Behavior ModificationAdditional services and information for

http://bmo.sagepub.com/cgi/alertsEmail Alerts:

http://bmo.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://bmo.sagepub.com/content/36/5/687.refs.htmlCitations:

What is This?

- Mar 28, 2012OnlineFirst Version of Record

- Apr 27, 2012OnlineFirst Version of Record

- May 3, 2012OnlineFirst Version of Record

- Sep 13, 2012Version of Record >>

at UNIV OF NEBRASKA MED CTR on January 7, 2013bmo.sagepub.comDownloaded from

http://bmo.sagepub.com/

http://bmo.sagepub.com/content/36/5/687

http://www.sagepublications.com

http://bmo.sagepub.com/cgi/alerts

http://bmo.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://bmo.sagepub.com/content/36/5/687.refs.html

http://bmo.sagepub.com/content/36/5/687.full.pdf

http://bmo.sagepub.com/content/early/2012/04/27/0145445511436007.full.pdf



http://online.sagepub.com/site/sphelp/vorhelp.xhtml


Behavior Modification36(5) 687 –704

© The Author(s) 2012Reprints and permission:

sagepub.com/journalsPermissions.navDOI: 10.1177/0145445511436007

http://bmo.sagepub.com

436007 BMO36510.1177/0145445511436007Behavior ModificationBurke et al.

1Behaven Kids, Omaha, NE, USA2University of Nebraska, Lincoln, NE, USA3Munroe-Meyer Institute for Genetics and Rehabilitation at the University of Nebraska Medical Center, Omaha, NE, USA

Corresponding Author:Raymond V. Burke, Behaven Kids, 8922 Cuming Street, Omaha, NE 68114, USA Email: [email protected]

Visual Performance Feedback: Effects on Targeted and Nontargeted Staff

Raymond V. Burke1,2, Monica R. Howard3, Jane L. Peterson1, Roger W. Peterson1, and Keith D. Allen3

Abstract

This study used a multiple baseline with reversal design to assess whether visual performance feedback (VPF) influenced targeted and nontargeted staffs’ use of behavior-specific praise (BSP) in a day-treatment program. This study expands on the typical VPF audience and assesses whether VPF can be effec-tive with noncertified staff in a day-treatment program for young children with behavior disorders, an environment in which it is difficult to maintain high rates of BSP. In previous school-based studies, VPF has been collected by research-ers and provided to targeted teaching staff. In the current study, rather than relying on researchers, the authors used staff instructors to collect VPF and assessed how that experience influenced the instructors’ use of BSP. Results suggest that VPF provided, on average, a doubling in rates of BSP use by directly targeted staff and more than a 50% increase in rates of BSP in nontargeted in-structors who collected BSP data. Furthermore, three of the four participants had substantially higher praise-to-correction ratios during the VPF interven-tion when compared with baseline and reversal conditions. Implications for improving treatment fidelity and reducing supervision time are discussed.



688 Behavior Modification 36(5)

Keywords

behavior-specific praise, day-treatment program, visual performance feedback

If feedback were as effective as it is frequent, our world would be a very different place. Feedback, the procedure of recording, summarizing, and dispensing infor-mation about an individual’s performance, is the most frequently used approach for improving staff training and management programs (Jahr, 1998). One key benefit of performance feedback is that it is an efficient means for improving intervention fidelity, which is especially critical given challenges with school (e.g., Burke, Oats, Ringle, O’Neill Fichtner, & DelGaudio, 2011), social service (Fixsen, Naoom, Blasé, Freidman, & Wallace, 2005), and psychotherapeutic intervention fidelity (Perepletchikova, Treat, & Kazdin, 2007). Findings consis-tently indicate that programs implemented with greater fidelity are associated with more positive outcomes (Fixsen et al., 2005). Feedback is one approach used to promote adherence to protocols and has been associated with superior treatment fidelity when compared with other procedures (Gresham, 2005).

In practice, schools have been especially prolific with the use of perfor-mance feedback to improve staff implementation of programs involving social and academic behavior (DiGennaro, Martens, & Kleinmann, 2007; DiGennaro, Martens, & McIntyre, 2005; Hagermoser Sanetti, Luiselli, & Handler, 2007; Mortenson & Witt, 1998; Noell et al., 2000; Witt, Noell, LaFleur, & Mortenson, 1997). For example, Codding, Feinburg, Dunn, and Pace (2005) used immediate performance feedback to increase teacher use of antecedent and consequent strategies in a classroom. Auld, Belfiore, and Scheeler (2010) used performance feedback to increase student–teachers’ use of differential reinforcement of student alternate behaviors in elementary, middle, and high schools from urban, rural, and suburban school districts. Duhon, Mesmer, Gregerson, and Witt (2009) used performance feedback during weekly response-to-intervention team meetings when teacher inter-vention fidelity fell below acceptable levels. Once performance feedback was introduced, teachers’ use of the intervention improved. Performance feed-back has also been extended to other school staff, as seen with Luiselli (2008), who reported that use of performance feedback led to increases in clinical directors’ supervision frequency in targeted classrooms, and Burns, Peters, and Noell (2008), who used performance feedback to help problem-solving teams increase the number of strategies from a 20-item checklist designed to help team functioning. Team members represented all areas of the school including general and special education teachers, administrators, school



Burke et al. 689

psychologists, and other support staff. One clear advantage to performance feedback is its versatility and applicability to a wide range of individuals and target behaviors.

The use of performance feedback has not only improved fidelity with class-room protocols but has also shown subsequent improvements on student behav-ior (DiGennaro et al., 2007; Martens, Hiralall, & Bradley, 1997). Noell, Witt, Gilbertson, Ranier, and Freeland (1997) demonstrated that while a consultation-only model was inefficient in improving adherence to protocols, the incorpora-tion of performance feedback not only improved teacher adherence but also had a positive effect on student academic performance for two out of three targeted students. Likewise, Ingham and Greer (1992) found that both teacher perfor-mance and student acquisition improved with the delivery of performance feed-back. In an earlier study, Drabman and Lahey (1974) provided feedback to a particular student and found that disruptive behavior of nontargeted students changed despite focusing the intervention on the particular student. Thus, although most research reports change in behavior of the targeted individual, there actually may be ancillary effects on others in the environment.

There are numerous ways to dispense feedback (e.g., verbal consultation, written descriptions, graphic representations), but there may be advantages to using certain approaches over others. For example, Hagermoser, Sanetti, and colleagues (2007) found that delivering verbal and graphic performance feed-back was more effective than delivering verbal feedback alone. Studies have shown verbal feedback to take up to 10 and 12 min (e.g., Codding et al., 2005; Codding, Livanis, Pace, & Vaca, 2008), whereas other approaches, such as a hybrid form of performance feedback called visual performance feedback (VPF), require less time. VPF is an approach in which results of a targeted behavior are graphed and provided to an individual in lieu of verbal or written explanation. As a result, the exchange of feedback takes seconds. By provid-ing specific, concrete information that is relevant and meaningful to the per-son, VPF provides feedback in a clear, private, and time-efficient manner by increasing the efficiency of the performance feedback methodology.

Most studies have focused on use of VPF with teachers (e.g., Reinke, Lewis-Palmer, & Martin, 2007; Reinke, Lewis-Palmer, & Merrell, 2008). Reinke and colleagues (2007) evaluated the effectiveness of VPF in a school setting to increase the rates of behavior-specific praise (BSP) in three class-rooms. Prior to use of VPF, teachers used relatively low rates of BSP despite group consultation meetings. Not only did rates of BSP toward target chil-dren increase with VPF, but teachers also increased their delivery of BSP to other children in the class. In addition, disruptive behavior exhibited by




targeted students decreased substantially with the introduction of VPF. VPF has also been extended to paraprofessionals’ use of behavior management practices (Hall, Stadnik Grundon, Pope, & Balderama Romero, 2010).

While VPF has provided meaningful increases in staff BSP rates and tem-poral improvements in student behavior, there are other aspects of higher BSP rates that may be of practical interest. For example, many authors sug-gest that a praise-to-correction ratio of 4:1 or higher is beneficial for student social and academic success (e.g., Walker, Ramsey, & Gresham, 2004) and as part of effective classroom management practices (Burke et al., 2011). However, to date, studies have not examined the effects of VPF with BSP on adult participants’ praise-to-correction ratios.

Despite the efficiency of VPF, its practicality in a teaching setting is rela-tively unknown. In the majority of VPF studies, performance data collection has depended on research assistants (e.g., Hall et al., 2010). As a result, the ability to incorporate VPF as a routine procedure in a classroom or treatment setting has not been examined yet. In the current study, classroom instructors collected data on rates of BSP delivered by aides and shared it with the super-visor, who then graphed the data and presented it to the aides in the class-room. This provided a unique opportunity to determine the feasibility of VPF as a standard procedure as well as to assess whether VPF had an influence on targeted aides and on the classroom instructor who collected the data—a phe-nomenon that has not been reported in the literature prior to this study.

The current study used two instructor–aide dyads to answer three research questions:

Research Question 1: Will daily VPF (i.e., a cumulative graph of fre-quency rates) increase targeted staff (i.e., aides) delivery of BSP?

Research Question 2: Will collecting performance data on aides’ delivery of BSP increase nontargeted staff (i.e., instructors) delivery of BSP?

Research Question 3: Will collecting or receiving daily feedback on use of targeted strategies influence aide or instructor praise-to-correction ratios?

MethodParticipants

Four day-treatment staff were asked to participate in a project to test a new supervision method. The four participants included two classroom instructors and their two aides from two separate classrooms. Staff were female, had an



Burke et al. 691

average age of 26.4 years, worked at the day-treatment center for an average of 38.8 months, and had a college degree. The two instructors were responsible for conducting daily activities, implementing the classroom academic curriculum, and overseeing treatment implementation for all children in the classroom. Each instructor had an aide in her classroom. Both instructors and aides interacted directly with all children in the classroom.

SettingThis study was conducted at a day-treatment program that serves children, ages 2 to 8 years, the majority of whom have a mental health diagnosis (see Burke, Kuhn, Peterson, Peterson, & Badura Brack, 2010, for a thorough description of the program). While the center is open from 6:00 a.m. until 6:00 p.m., chil-dren are typically at the center for 7 to 8 hr per day. Approximately, one third of referred children have prenatal exposure to alcohol, drugs, or tobacco, one third have a history of exposure to one or more types of maltreatment, and three fourths of parents report a family history of substance abuse. Prior to day-treatment referral, more than half of the children have participated in treat-ment with a psychiatrist or therapist, whereas 1 in 10 have had multiple home placements and have been prescribed psychotropic medication.

The day-treatment program focuses on building social competencies through a combination of verbal reinforcement (Maag, 2001), modeling (Bandura & McDonald, 1963), problem solving and social skill instruction (Gresham, Sugai, & Horner, 2001), and a contingency-based point system (Axelrod, 1971; Christophersen, Arnold, Hill, & Quilitch, 1972; Wolf, Giles, & Hall, 1968). A time-out (TO) hierarchy is used to respond to inappropriate behavior that is atten-tion maintained (e.g., Larzelere & Kuhn, 2005). Social skill instruction for all children is a critical program component and begins with skills of accepting neg-ative and positive consequences and continues with teaching of adaptive replace-ment skills that are based on skill deficits specified by the parent in each child’s Individualized Treatment Program.

Dependent MeasuresThe primary dependent measure to assess effects of VPF was frequency of BSP delivered by participants. Secondary dependent measures included general praise (GP) and TO, and were used to calculate adult praise-to-correction ratios. The first two authors collected frequency counts of instructors’ and aides’ use of BSP, GP, and TO events during daily 10-min observation probes while the class was engaged in a morning semistructured activity (e.g., learning centers,




snack time). Results from these probes were analyzed to provide data for frequency of BSP and an overall praise-to-correction ratio for each partici-pant (by combining behavior-specific and GP rates and dividing the com-bined rate by the rate of TO events).

Behavior-specific praise was operationally defined as a verbal description of children’s positive behavior that may or may not be coupled with a super-lative or affectionate physical contact such as a high five. For example, “That is great! You have been sitting in your seat while you put the puzzle together!” was recorded as one BSP event.

General praise included verbal statements or physical gestures that indi-cated teacher approval for a child’s behavior. GP included affirmations for correct responses to academic questions such as, “Yes!” “Right!” and repeat-ing the answer after a student provided a correct answer. Affirmative state-ments that did not contain a specific description of behavior (e.g., “Way to go!”) were coded as GP. Physical gestures such as a high five, thumbs up, and fist bump were also coded as GP (Reinke et al., 2008).

Time-out was delivered as a consequence for noncompliance and/or disrup-tive behavior. A three-phase hierarchy of TO was applied with all children and ranged from a 10-s self-release (i.e., the child says “Okay,” sits cross-legged on the floor, counts to 10, and then rejoins the activity) to a behavior-contingent teacher release (i.e., the child sits quietly on the floor for at least 30 s until the teacher instructs the child to return to his or her seat). TO was operationally defined as a one-word instruction by the teacher indicating that the child should remove himself or herself from the activity for a designated period of time.

Research DesignThis study used a multiple baseline with reversal design (Kazdin, 1998). In this design, intervention was gradually extended to each aide over time in the usual multiple-baseline design and then removed to demonstrate a functional rela-tionship between intervention and behavior. Thus, introduction and withdrawal of the intervention varied across aides. Instructors, who collected observation data used as VPF for aides, continued to collect data during the aides’ reversal condition, and, therefore, did not experience a reversal condition.

ProcedureThe first author met individually with two instructors and their aides to dis-cuss their interest in participating in the study. Participants were told that the



Burke et al. 693

purpose of the study was to (a) address instructors’ concerns about lack of time to provide support for aides while maintaining other responsibilities in the treatment center and (b) assess whether there were time-efficient strate-gies to provide supervision at the day-treatment program. Participants were given the opportunity to participate or opt out of the study without it affect-ing their job. All staff who were asked agreed to participate in the study. Staff at the day-treatment program routinely collect data on child and staff behav-ior to evaluate program and treatment effectiveness, so a request to partici-pate in the collection of additional data was not unusual.

Baseline. Ten-minute probes were conducted by the first two authors with each instructor and aide. Frequency data of BSP, GP, and TO were collected during semistructured center time or during snack time each day.

Intervention. The first two authors continued to collect data during 10-min probes as described during baseline. Just prior to starting the intervention, the first and second authors spoke with the two instructors privately about collecting BSP data on their aide for 5 min each day during center time in the classroom. We chose 5 min as a time frame for the instructors’ observations because we wanted to assess a minimal amount of time that would be feasible on a daily basis but still obtain enough data to be sensitive to change in BSP rates. Subtle data collection was encouraged to minimize aide reactivity and occurred during the time the instructor typically completed her paperwork each day. Instructor data collection occurred at a different time than researcher data collection occurred. After collect-ing data, the instructor gave the BSP frequency results to her manager, the supervisor.

The supervisor plotted the BSP frequency data on a cumulative graph of BSP rates and gave the graph to the aide. To ensure that the graphed data were solely responsible for influencing performance, the supervisor did not provide any discussion or praise when she gave the graphed results to the aide. The supervisor said, “Here’s your graph of the frequency for behavior-specific praise for today. Be sure to give me the graph when you are finished looking at it,” and nothing more. The intervention continued until there was stability in the data. Multiple intervention phases were used with each aide to demonstrate experimental control.

Reversal. In this phase, data collection by the authors and instructors con-tinued, but the aides did not receive VPF. Thus, reversals only occurred with the aides, not instructors. The second intervention phase began once there was stability in the aide’s data. The second intervention phase mirrored the first in the way in which data were collected by the instructor and then graphed by the supervisor, who shared the VPF with the aide.




Maintenance. We assessed maintenance of VPF effects on instructors’ and aides’ rate of BSP after 1 week without VPF and after 1 month without VPF.

ReliabilityIntervention integrity. Each week during the study period, the first author

visually reviewed the graphs provided to the aides by the supervisor to ensure that data were recorded accurately and conducted a verbal check-in with the supervisor to confirm that graphed data were provided to the aides on a con-sistent basis during the intervention condition.

BSP interobserver agreement. The first and second authors conducted obser-vations together on 21% of instructors’ probes and 26% of aides’ probes. Mean interobserver agreement for BSP was 96% for instructor probes and 91% for aide probes.

ResultsThe purpose of this study was to examine the influence of VPF on directly tar-geted aides and indirectly targeted instructors in a day-treatment setting. Visual inspection of data suggests that VPF resulted, on average, in a doubling of the aides’ rates of BSP and a 50% increase in the instructors’ rates of BSP. Rates of BSP remained high at 1-week and 1-month follow-up. In addition, praise-to-correction ratios, on average, doubled during the intervention condition.

Frequency of BSP for the aides indicated a relatively low rate of BSP dur-ing baseline (Figure 1). Jayne showed a decreasing trend of BSP while Jolanda showed a more variable rate of BSP delivery. Results indicate that following the introduction of VPF, both participants showed a dramatic increase in mean rate of BSP. Jayne’s mean rate increased from 7.3 to 20.0 BSP statements per 10-min interval, whereas Jolanda’s mean rate of BSP increased from 13.1 to 23.9 BSP statements per 10-min interval, despite a decreasing trend during the first intervention phase. These rates declined when VPF was temporarily discontinued and resumed at higher rates when VPF was reintroduced. For example, when VPF was withdrawn, Jayne and Jolanda’s mean BSP rates declined to 17.0 and 12.5 BSP statements per 10-min interval, respectively. When VPF was reintroduced, Jayne and Jolanda’s mean BSP rates increased to 22.9 and 23.5, respectively. During the maintenance condition at 1-week postintervention, both aides’ mean BSP rates increased slightly to 24.0 and 27.5. During maintenance at 1-month postintervention, Jayne’s mean BSP rate declined by 29% but remained more than twice as high as the initial baseline condition, whereas Jolanda’s mean



Burke et al. 695

BSP rate declined by 8% and was close to twice her original baseline rate (Figure 1).

Results from instructors who collected VPF data indicated that there were similar increases in mean BSP rates for the instructors as there were for the aides (Figure 2). As compared with initial baseline rates, the introduction of collecting VPF data resulted in a 58% increase of mean BSP rates for Nancy (from 18.7 to 29.5 BSP per 10-min observation) and a 120% increase for Karen (from 13.6 to 29.9 BSP per 10-min observation). Note that there was no reversal for instructors in that they continued to collect observation data on their aides during the aides’ reversal period. During maintenance at 1-week postintervention, Nancy’s mean rate of BSP increased another 20% and was followed by a slight increase during the 1-month maintenance condi-tion despite having discontinued collecting data at the end of the intervention phase. Karen’s mean BSP rate remained virtually the same (29.9 and 29.8 BSP per 10 min) at 1-week maintenance but increased by 14% (to 34.0 BSP per 10 min) at the 1-month maintenance period (Figure 2).

Figure 1. Daily rate of BSP by aides. The horizontal dotted lines denote the mean rates of BSP during each experimental condition.Note: BSP = behavior-specific praise; VPF = visual performance feedback.




Praise-to-correction ratios are shown in Table 1. Both aides showed increases in praise-to-correction ratios consistent with the implementation of VPF. When VPF was withdrawn, Jayne’s ratio returned to baseline levels, whereas Jolanda’s ratio returned to a rate even lower than baseline. Reintroduction of VPF almost tripled both aides’ ratios of praise-to-correction and was followed by additional increases during the 1-week maintenance condition, before returning to baseline rates during the 1-month maintenance condition.

Changes in praise-to-correction ratios for one instructor, Karen, followed a pattern similar to changes in the aides’ ratios with an almost threefold increase in the praise-to-correction ratio when VPF was introduced. Karen’s praise-to-correction ratio remained fairly stable as she continued to collect VPF data, increased by 66% during maintenance at 1 week, and decreased to a level con-sistent with her intervention praise-to-correction ratio during maintenance at 1 month. The second instructor, Nancy, showed a modest increase in her praise-to-correction ratio during intervention and a decline in the praise-to-correction

Figure 2. Daily rate of BSP by instructors. The horizontal dotted lines denote the mean rates of BSP during each experimental condition.Note: BSP = behavior-specific praise; VPF = visual performance feedback. Arrows indicate the start and end of reversal phases for the aides in each instructor’s classroom.



Burke et al. 697

ratio during the maintenance condition at 1 week and 1 month. However, despite the decline, Nancy’s praise-to-correction ratio during both mainte-nance conditions remained at recommended levels (Walker et al., 2004).

DiscussionThis study assessed the effects of VPF on instructor and aide use of BSP and their praise-to-correction ratios. Typical of many staff training efforts, the staff in this study had completed more than 40 preservice hours and countless in-service training sessions during which BSP, among other topics, was taught, modeled, and practiced. However, although praise rates and praise-to-correction ratios overall were positive for participating staff, instructors reported that they wanted targeted aides to increase their rates of BSP with children. Results indicate that VPF is an effective strategy for increasing staff use of BSP, a key component to improving adult–child relationships (e.g., Gable, Hester, Rock, & Hughes, 2009) and children’s academic engagement during lessons (e.g., Sutherland, Wehby, & Copeland, 2000).

This study’s unique contribution to the use of VPF was twofold. First, VPF has been used in school settings, primarily with schoolteachers (e.g., Reinke et al., 2007; Reinke et al., 2008), and on one occasion with paraeduca-tors (Hall et al., 2010). The current study expands on the typical audience and indicates that VPF can be effective with staff in a day-treatment program for

Table 1. Mean Praise-to-Correction Ratios Across Conditions

Maintenance

Aide Baseline VPF Reversal VPF 1 week 1 month

Jayne 2.86 9.14 3.50 10.06 19.05 5.73Jolanda 6.52 9.65 3.92 15.94 20.58 3.07Mean 4.69 9.40 3.71 13.00 19.82 4.40

Maintenance

Instructor Baseline VPF 1 week 1 month

Nancy 6.07 6.79 5.21 4.25Karen 5.00 16.28 27.00 17.76Mean 5.54 11.54 16.11 11.01

Note: VPF = visual performance feedback. Baseline includes an average of baseline data points. VPF includes all VPF data points in that series. Maintenance results include data from the 1-week and 1-month Maintenance sessions, respectively.




young children with behavior disorders, a setting in which it is not unusual to find low rates of BSP coupled with high rates of children’s disruptive behav-ior (e.g., Stormont, Smith, & Lewis, 2007).

Teachers in many classrooms rely on correction more often than praise during adult–student interactions. For example, in the Stormont et al. (2007) study with Head Start teachers, one third of teachers used correction more frequently than praise and provided BSP, on average, less than once per 15-min observation period. In Baker’s (1999) study, students experienced a 1:3 or 1:6 praise-to-correction ratio, depending on whether they had a high or low level of satisfaction with their elementary school, respectively. More recently, in a study (Burke et al., 2011) assessing the effects of fidelity to a schoolwide classroom management intervention and student academic engagement, discipline infractions, and academic performance, approxi-mately one third of the participants were low-fidelity elementary school teachers who had praise-to-correction ratios that averaged 1:2, which is con-sistent with rates found in some special education classrooms (Sutherland, Wehby, & Yoder, 2002), and far below the 4:1 or higher praise-to-correction ratio that is typically recommended (Walker et al., 2004).

The second unique feature is that, unlike prior studies that have relied on researchers to collect VPF observation data, the current study trained instruc-tors to collect VPF data. Results indicate that rates of BSP increased substan-tially for those collecting VPF observation data. This phenomenon has not been discussed previously in the literature, increases the likelihood that staff will continue with VPF after the study is completed, and holds promise for development of new treatment strategies across settings (e.g., classroom and home) and interventionists (e.g., treatment staff and parents).

This finding has interesting implications for increasing treatment breadth and potency. For example, studies have found a strong relationship between negative, coercive parenting practices and child behavior problems (e.g., Marchant, Young, & West, 2004; Patterson, Reid, & Dishion, 1992). To improve parents’ attention to their children’s prosocial behavior, parents could be invited to the day-treatment program and asked to collect BSP fre-quency data on treatment staff. If results from parent data collection mirror those from this study, a brief training about BSP followed by parent data col-lection during observations in the classroom may result in increases in parent use of BSP in the home setting, thus increasing the child’s exposure to treat-ment beyond the typical time spent at the day-treatment program. This trans-fer of effects holds promise but requires additional studies to assess its value.

As a management tool, VPF represents an effective and efficient interven-tion to improve responding in underperforming staff and help all staff maintain



Burke et al. 699

a positive focus with children who present challenging behavior problems. The supervisor in this study estimated that she spends a minimum of 10 min in indi-vidual supervision meetings per week per staff member to discuss the importance of remaining positive with children, provide examples of GP and BSP, and have staff practice or give examples of BSP that they could use with children. Conservatively, this represents approximately 2 hr per week that she spends working with her staff on providing praise and positive attention. In the current study, the instructor collected BSP data on the aide and gave the BSP rate to the supervisor. The supervisor took approximately 30 s daily to graph the aides’ data and hand each aide her cumulative graph. This 30 s takes care of two staff (each instructor and aide pair); when extrapolated across five instructor/aide pairs, this results in a rough estimate of 2 to 3 min per day or 10 to 15 min per week with her staff, a savings of approximately 90 min per week for the supervisor.

In addition to expanding treatment options and reducing supervision time, VPF appears to help with compliance to treatment protocols, a critical benefit given the lack of attention to intervention fidelity in applied and clinical set-tings (Fixsen et al., 2005; Perepletchikova et al., 2007). In the current study setting, verbal praise (i.e., GP and BSP) is a key component of the program’s staff training protocol (Burke et al., 2010). The majority (73%) of children served in this day-treatment program present clinically significant behavior problems; the most prominent diagnoses at intake are oppositional defiant disorder, disruptive behavior disorder not otherwise specified, and attention deficit hyperactivity disorder. Due to the challenges presented by children who use high rates of aversive behavior to get their needs met (Patterson et al., 1992), program staff need frequent reminders and encouragement to focus on children’s strengths and approximations of alternatives to aggres-sive and disruptive behaviors. Results demonstrate that VPF can be a viable and efficient strategy for increasing adherence to treatment protocol despite the tendency for adults to use aversive responses when faced with children’s challenging behavior (Patterson et al., 1992).

Limitations of this study include a small sample size and the lack of a reversal for the instructors because they continued collecting data on aides’ rate of BSP during aides’ reversal conditions. Future studies will benefit from replicating the use of VPF with a larger sample of treatment staff and the withdrawal of VPF for instructors to further demonstrate experimental con-trol with the effects of VPF on nontargeted staff.

Another key focus for future study will be to assess how often VPF needs to occur to maintain gains over time. In prior studies, teacher use of BSP dropped to near-baseline rates between 20 and 30 days after VPF was discon-tinued (e.g., Reinke et al., 2007). In the current study, VPF results were




maintained at 1 week and 1 month after the initial intervention; however, we did not assess how much longer effects would last without VPF or what VPF intervals would help staff maintain positive gains past 30 days—both of which would be helpful in future studies.

In addition, future studies are needed to (a) replicate findings with other target behaviors such as the amount of staff time spent working with children at their eye level, the number of higher order thinking questions asked by staff, and the number of proactive strategies used by staff prior to the start of activities; (b) assess whether we obtain similar results when colleagues, rather than instructors, collect data on each other; and (c) evaluate whether training parents to collect data at the day-treatment program results in improvements in targeted parenting practices at home.

In conclusion, results indicate that VPF can be an effective and efficient method for increasing rates of targeted behavior and improving adherence to treatment protocol. Furthermore, VPF can be a positive, proactive strategy for supervising staff that requires less time and produces substantial improve-ments in targeted behaviors. Finally, VPF’s residual effects with those col-lecting performance data hold promise for development of additional interventions for a host of other populations and challenges.

Acknowledgment

The authors would like to thank Jess Wachtler and her staff for their willingness to participate in this study and for giving us their permission to publish their data.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interests with respect to the research, authorship, and/or the publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project was supported in part by Project #8188 from the Maternal and Child Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services and by grant 90DD0533 from the Administration on Developmental Disabilities (ADD), Administration for Children and Families, Department of Health and Human Services.



Burke et al. 701

References

Auld, R., Belfiore, P., & Scheeler, M. (2010). Increasing pre-service teachers’ use of differential reinforcement: Effects of performance feedback on consequences for student behavior. Journal of Behavioral Education, 19, 169-183.

Axelrod, S. (1971). Token reinforcement programs in special classes. Exceptional Child, 37, 371-379.

Baker, J. A. (1999). Teacher–student interaction in urban at-risk classrooms: Differen-tial behavior, relationship quality, and student satisfaction with school. Elementary School Journal, 100, 57-70.

Bandura, A., & McDonald, F. J. (1963). Influence of social reinforcement and the behavior of models in shaping children’s moral judgments. Journal of Abnormal and Social Psychology, 67, 274-281.

Burke, R. V., Kuhn, B. R., Peterson, J. L., Peterson, R. W., & Badura Brack, A. S. (2010). Don’t kick me out: Day treatment for two preschool children with severe behavior problems. Clinical Case Studies, 9, 28-40.

Burke, R. V., Oats, R. G., Ringle, J. R., O’Neill Fichtner, L., & DelGaudio, M. (2011). Implementation of a classroom management program with urban, high risk, elemen-tary students: Does program fidelity affect behavior and academic performance? Journal of Education for Students Placed at Risk, 16, 201-218.

Burns, M. K., Peters, R., & Noell, G. H. (2008). Using performance feedback to enhance implementation fidelity of the problem-solving team process. Journal of School Psychology, 46, 537-550.

Christophersen, E. R., Arnold, C. M., Hill, D. W., & Quilitch, H. R. (1972). The home point system: Token reinforcement procedures for application by parents of children with behavior problems. Journal of Applied Behavior Analysis, 5, 485-497.

Codding, R. S., Feinburg, A. B., Dunn, E. K., & Pace, G. M. (2005). Effects of immedi-ate performance feedback on implementation of behavior support plans. Journal of Applied Behavior Analysis, 38, 205-219.

Codding, R. S., Livanis, L., Pace, G. M., & Vaca, L. (2008). Using performance feed-back to improve treatment integrity of classwide behavior plans: An investigation of observer reactivity. Journal of Applied Behavior Analysis, 41, 417-422.

DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of per-formance feedback procedures on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447-461.

DiGennaro, F. D., Martens, B. K., & Mcintyre, L. L. (2005). Increasing treatment integrity through negative reinforcement: Effects on teacher and student behavior. School Psychology Review, 34, 220-231.




Drabman, R. S., & Lahey, B. B. (1974). Feedback in classroom behavior modification: Effects on the target and her classmates. Journal of Applied Behavior Analysis, 7, 591-598.

Duhon, G. J., Mesmer, E. M., Gregerson, L., & Witt, J. C. (2009). Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance. Journal of School Psychology, 47, 19-37.

Fixsen, D. L., Naoom, S. F., Blasé, K. A., Freidman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network. (FMHI Publication #231)

Gable, R. A., Hester, P. H., Rock, M. L., & Hughes, K. G. (2009). Back to basics: Rules, praise, ignoring, and reprimands revisited. Intervention in School and Clinic, 44, 195-205.

Gresham, F. M. (2005). Treatment integrity and therapeutic change: Commentary on Pere-pletchikova and Kazdin. Clinical Psychology: Science and Practice, 12, 391-394.

Gresham, F. M., Sugai, G., & Horner, R. H. (2001). Interpreting outcomes of social skills training for students with high-incidence disabilities. Exceptional Children, 67, 331-344.

Hagermoser Sanetti, L. M., Luiselli, J. K., & Handler, M. W. (2007). Effects of verbal and graphic performance feedback on behavior support plan implementation in a public elementary school. Behavior Modification, 31, 454-465.

Hall, L. J., Stadnik Grundon, G., Pope, C., & Balderama Romero, A. (2010). Training paraprofessionals to use behavioral strategies when educating learners with autism spectrum disorders across environments. Behavioral Interventions, 25, 37-51.

Ingham, P., & Greer, R. D. (1992). Changes in student and teacher responses in observed and generalized settings as a function of instructor observations. Journal of Applied Behavior Analysis, 25, 153-164.

Jahr, E. (1998). Current issues in staff training. Research in Developmental Disabili-ties, 19, 73-87.

Kazdin, A. E. (1998). Research in clinical psychology. Englewood Cliffs, NJ: Prentice Hall.

Larzelere, R. E., & Kuhn, B. R. (2005). Enhancing behavioral parent training with an extended discipline ladder. Behavior Therapist, 28, 105-108.

Luiselli, J. K. (2008). Effects of a performance management intervention on frequency of behavioral supervision at a specialized school for students with developmental disabilities. Journal of Developmental and Physical Disabilities, 20, 53-61.

Maag, J. W. (2001). Rewarded by punishment: Reflections on the disuse of positive reinforcement in schools. Exceptional Children, 67, 173-186.

Marchant, M., Young, K., & West, R. P. (2004). The effects of parental teaching on compliance behavior of children. Psychology in the Schools, 41, 337-350.



Burke et al. 703

Martens, B. K., Hiralall, A. S., & Bradley, T. A. (1997). A note to teacher: Improving student behavior through goal setting and feedback. School Psychology Quar-terly, 12, 33-41.

Mortenson, B. P., & Witt, J. C. (1998). The use of weekly performance feedback to increase teacher implementation of a prereferral academic intervention. School Psychology Review, 27, 613-627.

Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77-88.

Noell, G. H., Witt, J. C., LaFleur, L. H., Mortenson, B. P., Ranier, D. D., & LeVelle, J. (2000). Increasing intervention implementation in general education following consultation: A comparison of two follow-up strategies. Journal of Applied Behav-ior Analysis, 33, 271-284.

Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). Antisocial boys. Eugene, OR: Castalia Publishing Company.

Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psy-chotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829-841.

Reinke, W. M., Lewis-Palmer, T., & Martin, E. (2007). The effect of visual perfor-mance feedback on teacher use of behavior-specific praise. Behavior Modification, 31, 247-263.

Reinke, W. M., Lewis-Palmer, T., & Merrell, K. (2008). The classroom check-up: A classwide teacher consultation model for increasing praise and decreasing disrup-tive behavior. School Psychology Review, 37, 315-332.

Stormont, M. A., Smith, S. C., & Lewis, T. J. (2007). Teacher implementation of pre-correction and praise statements in Head Start classrooms as a component of a pro-gram-wide system of positive behavior support. Journal of Behavioral Education, 16, 280-290.

Sutherland, K. S., Wehby, J., & Copeland, S. (2000). Effects on varying rates of behavior-specific praise on the on-task behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 8, 2-8.

Sutherland, K. S., Wehby, J. H., & Yoder, P. J. (2002). Examination of the relationship between teacher praise and opportunities for students with EDB to respond to aca-demic requests. Journal of Emotional and Behavioral Disorders, 10, 5-13.

Walker, H. M., Ramsey, E., & Gresham, F. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Belmont, CA: Wadsworth.

Witt, J. C., Noell, G. H., LaFleur, L. H., & Mortenson, B. P. (1997). Teacher use of inter-ventions in general education settings: Measurement and analysis of the independent variable. Journal of Applied Behavior Analysis, 30, 693-696.




Wolf, M. M., Giles, D. K., & Hall, R. V. (1968). Experiments with token reinforcement in a remedial classroom. Behaviour Research and Therapy, 6, 51-64.

Bios

Raymond V. Burke, PhD, is responsible for program research and evaluation at Behaven Kids and an adjunct faculty member in the Department of Education and Human Sciences at the University of Nebraska, Lincoln with research interests in behavior disorders, treatment fidelity, and treatment for children and adults with autism spectrum disorders.

Monica R. Howard, Board Certified Behavior Analyst, is an intern in the Center for Autism Spectrum Disorders department at the Munroe-Meyer Institute at the University of Nebraska Medical Center. Her current research interests include staff development, vocational training for adults with autism spectrum disorders, and treat-ment for children with autism spectrum disorders.

Jane L. Peterson, Nebraska Licensed Mental Health Practitioner, co-founder of Behaven Kids (a day treatment and family therapy clinic for young children with behavior disorders), co-author of The Sleep Fairy, Mini Methods or Madness: A Behaven Kids Parent Guide and Building Skills in High Risk Families.

Roger W. Peterson, is co-founder of Behaven Kids, a Licensed Mental Health Practitioner, co-author of children’s and parenting books, and has worked with chil-dren and families for over 30 years. He is actively involved in legislation and prac-tices to improve access to mental health services for Nebraska’s children and families.

Keith D. Allen, PhD, is professor of pediatrics and psychology at the Munroe-Meyer Institute at the University of Nebraska Medical Center. His current interests include behavioral approaches to health in children, behavioral parent training, and treatment of autism spectrum disorders.



visual performance feedback: effects on targeted and nontargeted staff

Documents