EFFECTS OF IMMEDIATE PERFORMANCE FEEDBACK ONIMPLEMENTATION OF BEHAVIOR SUPPORT PLANS

ROBIN S. CODDING

CITY UNIVERSITY OF NEW YORK GRADUATE CENTER

AND

ADAM B. FEINBERG, ERIN K. DUNN, AND GARY M. PACE

MAY INSTITUTE

Research has focused on increasing the treatment integrity of school-based interventions byutilizing performance feedback. The purpose of this study was to extend this literature byincreasing special education teachers’ treatment integrity for implementing antecedent andconsequence procedures in an ongoing behavior support plan. A multiple baseline acrossteacher–student dyads (for two classrooms) design was used to evaluate the effects ofperformance feedback on the percentage of antecedent and consequence componentsimplemented correctly during 1-hr observation sessions. Performance feedback was providedevery other week for 8 to 22 weeks after a stable or decreasing trend in the percentage ofantecedent or consequence components implemented correctly. Results suggested thatperformance feedback increased the treatment integrity of antecedent components for 4 of 5teachers and consequence components for all 5 teachers. These results were maintained followingfeedback for all teachers across antecedent and consequence components. Teachers ratedperformance feedback favorably with respect to the purpose, procedures, and outcome, asindicated by a social validity rating measure.

DESCRIPTORS: treatment integrity, performance feedback, behavior support plans,special education

_______________________________________________________________________________

Treatment integrity, the implementation ofan intervention as intended, is a topic of interestamong researchers and change agents (Gresham,Gansle, & Noell, 1993; Watson, Sterling, &McDade, 1997). This interest in treatmentintegrity is accompanied by a recent focus onaccountability by school systems with theenactment of No Child Left Behind (U.S.Department of Education, 2001). Con-sequently, researchers have attempted to iden-tify parsimonious, effective, and time-efficientstrategies to ensure the integrity of school-based interventions (Cossiart, Hall, & Hop-kins, 1973; Gillat & Sulzer-Azaroff, 1994;Mortenson & Witt, 1998; Noell, Duhon,Gatti, & Connell, 2002; Noell, Witt, Gil-

bertson, Ranier, & Freeland, 1997; Witt, Noell,LaFleur, & Mortenson, 1997). One successfulintervention strategy for enhancing treatmentintegrity is performance feedback (Mortenson& Witt, 1998; Noell et al., 1997, 2000,2002; Witt et al., 1997). In the literature,performance feedback has encompassed severalcomponents including (a) review of data, (b)praise for correct implementation, (c) correctivefeedback, and (d) addressing questions orcomments.

Performance feedback has been used inregular education settings to improve theimplementation of academic interventions(Mortenson & Witt, 1998; Noell et al., 2000;Witt et al., 1997), peer tutoring (Noell et al.,2000), the use of contingent praise (Jones,Wickstrom, & Friman, 1997; Martens,Hiralall, & Bradley, 1997), and implementa-tion of behavior-management interventions(Noell et al., 2002). Noell et al. (2002)

Address correspondence to Robin Codding, Educa-tional Psychology, CUNY Graduate Center, 365 FifthAvenue, New York, New York 10016 (e-mail: rcodding@gc.cuny.edu).

doi: 10.1901/jaba.2005.98-04

JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2005, 38, 205–219 NUMBER 2 (SUMMER 2005)

205

examined the integrity with which 4 generaleducation teachers employed interventions todecrease out-of-seat behavior and talking out in8 elementary school students. Integrity wasmeasured by permanent products that consistedof data collected either by students’ self-reportor by teachers. This study represents anexample of the effects of performance feedbackon one aspect of behavior interventions: datacollection. In other situations, it may to beimportant to investigate integrity of specificintervention components for complex individ-ualized behavior support plans.

Two other aspects of plan implementationthat deserve consideration are (a) what ante-cedent interventions are implemented to de-crease the likelihood of problem behavior and

(b) the teacher’s response to students’ targetbehaviors. Teachers may not accurately imple-ment both types of procedures with equal inte-

grity because implementation of antecedentand consequence procedures may requiredifferent skills. Implementation of antecedent

procedures requires teachers to plan and pre-vent the occurrence of behavior (Luiselli &Cameron, 1998), whereas consequence proce-dures require teachers to impose contingencies

after the occurrence of problem behaviors(Kern, Choutka, & Skol, 2002).

The frequency and structure of performancefeedback have been a focus of several studies.For example, research has demonstrated thatweekly feedback (Mortenson & Witt, 1998)leads to increases in treatment integrity andmay be more practical for supervisors, clin-icians, and consultants than daily performancefeedback. The schedule and immediacy ofperformance feedback may also influence themaintenance of treatment integrity. For exam-ple, in many studies performance feedbackhas consisted of reviewing data from theprevious day (Noell et al., 1997, 2002; Wittet al., 1997) or week (Mortenson & Witt,1998), rather than providing feedback on thesame day.

The purpose of the present investigation wasto extend the current research on treatmentintegrity by (a) directly examining the effects ofperformance feedback on special educationteachers’ implementation of antecedent andconsequence procedures of ongoing individual-ized behavior support plans, (b) administeringperformance feedback every other week andduring the same day as the observation, and(c) assessing short-term maintenance effects ofperformance feedback.

METHOD

Participants and Setting

The present study was conducted in a privateschool for students with acquired brain injurywho exhibited significant behavior problems.Students were all male and ranged in agefrom 10 to 19 years. Three of the studentshad nontraumatic acquired brain injuries, and 2students had been diagnosed with a traumaticbrain injury. The sample of students observedcomprised approximately 10% of the totalpopulation of the school.

Observational data were collected in twospecial education classrooms on a total of 5teacher–student dyads. Two dyads (i.e., Mr.Canton and Seth; Mr. Martin and Philip) wereidentified from Classroom 1, and 3 dyads (i.e.,Ms. Lowell and Brian; Ms. Malden and Jason;Mr. Mack and Darrin) were identified fromClassroom 2. All teachers participating in thestudy had earned a bachelor’s degree and wereenrolled in a master’s level program in specialeducation. Teachers’ experience working in thisenvironment ranged between 6 and 30 months.

Materials

Behavior support plans. Individualized be-havior support plans were previously created foreach student and were ongoing at the time ofthe investigation. The plans had been in placefor an average of 4 months at the time of thestudy. These plans consisted of individualizedmulticomponent interventions that prescribed

206 ROBIN S. CODDING et al.

both antecedent and consequence proceduresfor students’ targeted problem behaviors.Teachers were expected to employ these be-havior support plans under specific conditions(e.g., activities, times of day, and contingent onstudent behaviors). Six behaviors were targetedfor Seth (tantrums, perseverative speech, in-appropriate speech, inappropriate social behav-ior, teasing, and noncompliance), Philip (majoraggression, minor aggression, inappropriatespeech, inappropriate touching, public expo-sure, and instigation), and Brian (aggression,destruction, peer instigation, inappropriatespeech, inappropriate sexual behavior, andnoncompliance). Four behaviors were targetedfor Jason (tantrums, inappropriate speech, in-appropriate social behavior, and invasion ofspace), and six behaviors were targeted forDarrin (physical aggression, property destruc-tion, wandering, inappropriate verbalizations,mimicking, and noncompliance).

Integrity data sheet. The integrity data sheetwas three pages long and included (a) the typeof procedure (i.e., antecedent or consequence)that was prescribed for that particular student,(b) an operational definition of each compo-nent of the intervention (copied directly fromeach student’s behavior support plan), (c)observer ratings of the teacher implementationof each component (see below), and (d) a spacefor the observer to record comments orexamples. Each component consisted of specificbehaviors that teachers were instructed toengage in, either as a result of the activity astudent was required to perform or in responseto students’ behavior. For example, when askedto change from a preferred to a nonpreferredactivity, Ms. Malden was instructed to provideJason with three transition warnings (i.e., at2 min, 1 min, and 30 s). Another example ofan antecedent component specified that Mr.Mack explain all expectations to Darrin usingfirst-then statements for every activity that heengaged in. Consequences included reinforce-ment such as receiving social attention (e.g., pat

on shoulder, praise, conversation) contingenton appropriate social behavior (e.g., sharing,complimenting peers) or planned ignoringfollowing the occurrence of inappropriatespeech. Table 1 lists all the antecedentand consequence components that were pre-scribed in various students’ behavior supportplans.

The integrity data sheet divided each planinto 10 to 13 individualized components (seethe Appendix for Brian’s integrity data sheet;other participants’ data sheets are available fromthe authors on request). One of three levels ofimplementation integrity was scored under theimplementation rating section of the integritydata sheet: (a) implemented as written (i.e., theentire component was implemented every timethe target behavior occurred or when thesituation required an antecedent component),(b) not implemented as written (i.e., sometimesimplemented the entire component as written,implemented part of the component as written,or did not employ the component as written),and (c) no opportunity to observe (i.e., the targetbehavior did not occur, or the antecedent wasnot present). For example, Ms. Malden receiveda rating of implemented as written if sheprovided all three transition warnings everytime Jason was required to change froma preferred to a nonpreferred activity duringthe observation period. However, she receiveda rating of not implemented as written if sheprovided the three warnings for only two ofthree opportunities that Jason had to changefrom a preferred to a nonpreferred activity orif she provided only two transition warnings.Mr. Mack earned a rating of implemented aswritten if he blocked Darrin from leaving theclassroom, redirected him to the scheduledactivity, and provided a warning that Darrinwould lose a penny for every time that heattempted to elope from his classroom. If Mr.Mack did not carry out all parts of thiscomponent every time that Darrin attemptedto elope from his classroom, he received a rating

TREATMENT INTEGRITY 207

of not implemented as written. In the commentsand examples section of the integrity data sheet,consultants could provide an example of howthe teacher implemented the component.

Percentage of correct implementation wasused as the measure of treatment integrity, andwas calculated by dividing the the number ofplan components that were implemented aswritten divided by the total number of plancomponents teachers had the opportunity toexhibit. The average percentage of the plancomponents that teachers had the opportunityto exhibit was 61% (range, 50% to 90%) forClassroom 1 and 57% (range, 40% to 77%) forClassroom 2 across teacher–student dyads.There were no opportunities for teachers toexhibit all the components in the behaviorsupport plan in any single observation period.It should be noted that consultants took noteson specific deviations from the written pro-cedure.

Social validity. A 10-item questionnaire thatexamined the acceptability of the performancefeedback and corresponding integrity observa-tions was administered to each teacher afterperformance feedback was terminated. Threeitems addressed the purpose of performancefeedback, four items inquired about the

procedures of the observation and feedbacksessions, and three items asked teachers to ratethe outcomes of the feedback sessions (e.g., wasit helpful to increase integrity of plan imple-mentation) (Schwartz & Baer, 1991). Respon-dents were required to rate each item on a Likertscale ranging from 1 (strongly disagree) to 5(strongly agree). Teachers were instructed not toplace their names on the questionnaires, anda box was placed in a common area for teachersto return the forms.

Training

All teachers had received both general andstudent-specific formal training in implement-ing behavior support plans prior to thisinvestigation. After being hired at the school,teachers received 4 hr of training in basicprinciples of applied behavior analysis.Teachers also received ongoing in-service train-ing on general behavior principles four timesannually. One in-service session was conductedduring the course of this project, and the topic(i.e., task analysis) was unrelated to thesestudents’ behavior support plans. Teachersreceived 2 weeks of specific training on individ-ual students’ plans at the time they weredeveloped. This training included reviewing

Table 1

Behavior Support Plan Components Classified as Antecedents and Consequences

Classroom Antecedents Consequences

1 Transition warnings Self-monitoringExplicit instructions DRAModification of presentation of demands Time-outBreaks incorporated into work periods Planned ignoringChoices Ignore and redirectScheduled activities (limit down time) Guided complianceModeling Contingent protective hold

Contingent escort2 Schedule of preferred and nonpreferred tasks DRO

Transition warnings DRAExplicit instructions Planned ignoringModification of presentation of demands Ignore and redirectChoice Block and redirectPreteaching Guided complianceBreaks incorporated into work periods Response costOne-to-one attention Contingent protective holdSeating with few distracters Contingent escort

Note. DRO 5 differential reinforcement of other behavior; DRA 5 differential reinforcement of alternative behavior.

208 ROBIN S. CODDING et al.

the written plan with a consultant (i.e., one ofthe authors), modeling by the consultant in theclassroom with the target student, promptingthe teacher-in-training to employ the compo-nents as written, and immediate performancefeedback given by the consultant. Teachers werenot trained to a standardized criterion; howev-er, training continued until they could verballyreport each component of the plan andreported that he or she could implement theplan. In addition, consultants were present inthe classrooms on a daily basis and providedinformal feedback to teachers.

Procedure

Observation sessions. Observations were con-ducted an average of every 2.1 weeks for eachdyad (range, 1 to 3 weeks). Variations in thetime between observation sessions reflectedteachers’ earned leave time and school breaks.Each observation was planned to be 60 minlong. There were five occasions on whichobservations were less than 60 min (range, 45to 55 min). The length of time for eachobservation session and time between observa-tion sessions remained consistent throughoutthe baseline, intervention, and maintenancephases of data collection. As recommended byGresham et al. (1993), we attempted to reducereactivity by conducting observations on a vari-able-time schedule and used consultants whoroutinely worked in the classrooms as primaryobservers. The consultant assigned to Class-room 1 spent 50% of his total time conductingobservations. Nine percent of the consultant’stime in Classroom 1 was devoted to thisproject. The consultant assigned to Classroom2 devoted 33% of her time to conductingobservations. Six percent of the consultant’stime in Classroom 2 was devoted to thisproject.

Baseline. Baseline consisted of observingeach student–teacher dyad and completing theintegrity data sheet without the teachers’knowledge of the observation. Feedback wasprovided following the observation. Teachers

were not told the purpose of the observationduring baseline. This was not unusual, becauseobservations were a consistent part of the dutiesof experimenters.

Intervention. Performance feedback was im-plemented after stable or decreasing perfor-mance in baseline was demonstrated by eitherthe percentage of antecedent components orconsequence components implemented as writ-ten. The first feedback session was provided toeach teacher after the last baseline session toaddress inadequate plan implementation assoon as possible for the benefit of the student.On the same day as each observation, theexperimenter spent an average of 12 min withthe target teacher outside the classroom. At thistime, components in the behavior support planwere reviewed, and feedback was provided onall the components that were observed.Feedback included providing praise for compo-nents followed as written and constructivefeedback for those components that werefollowed sometimes or not at all. Constructivefeedback consisted of reviewing the specificcomponents observed and explaining how thecomponent should have been implemented.

Performance feedback was terminated afterimproved performance had stabilized. Twomaintenance sessions were conducted for Mr.Canton and Mr. Martin, three maintenancesessions were conducted for Ms. Lowell andMs. Malden, and one maintenance session wasconducted for Mr. Mack. Maintenance sessionswere identical to baseline sessions. The firstmaintenance session occurred 5 weeks after thelast feedback session. Each subsequent follow-up session occurred at 5-week intervals acrossdyads.

Experimental design. A concurrent multiplebaseline across teacher–student dyads designwas selected for each classroom to evaluate theefficacy of performance feedback. Maintenancesessions were employed to examine the short-term effects of the intervention when feedbackwas no longer provided.

TREATMENT INTEGRITY 209

Interobserver Agreement

Interobserver agreement was collected across20% of the sessions. A second independentobserver was present and used the sametreatment integrity form to observe the teachersdirectly. On a component-by-component basis,comparisons were conducted between the

second observer and the primary observer’sresponses. Agreement was calculated by di-viding the number of agreements per compo-nent by the number of agreements plusdisagreements per component and multiplyingby 100%. Mean agreement across dyads was95% (range, 91% to 100%).

Figure 1. Percentage of antecedent and consequence components implemented as written across teacher–studentdyads for Classroom 1.

210 ROBIN S. CODDING et al.

RESULTS

Teacher Implementation of the BehaviorSupport Plan

Classroom 1. Figure 1 presents the percent-age of antecedent and consequence compo-nents implemented as written (correctlyimplemented) for 2 student–teacher dyads inClassroom 1. During baseline, a decreasingtrend was evident for the percentage of con-sequence components that Mr. Canton imple-mented as written (M 5 31%). Performance ofantecedent components implemented correctlywas stable (M 5 69%) during baseline.Following performance feedback there was animmediate increase in the percentage ofconsequence components correctly implemen-ted (M 5 92%) and a gradual increase in thepercentage of antecedent components correctlyimplemented (M 5 92%).

During baseline, Mr. Martin’s performancewas variable for the correct implementation ofantecedent and consequence components. Mr.Martin correctly implemented 100% (eightsessions), 50% (three sessions), or none (twosessions) of the consequence components dur-ing baseline, which makes these data difficult tointerpret. The mean percentage of componentscorrectly implemented during baseline was85% for antecedents and 73% for conse-quences. Following performance feedback,Mr. Martin demonstrated 100% correct imple-mentation across consequence components forthree consecutive sessions (M 5 100%). Therewas little change in Mr. Martin’s correctimplementation of antecedent components(M 5 89%).

Classroom 2. Figure 2 presents the percent-age of antecedent and consequence componentsimplemented as written for 3 student–teacherdyads in Classroom 2. During baseline, a de-creasing trend was evident for the percentage ofantecedent components correctly implementedby Ms. Lowell (M 5 40%). Ms. Lowell didnot correctly implement any consequence com-ponents during baseline (M 5 0%). Following

performance feedback, substantial improve-ments were demonstrated in the percentage ofcorrectly implemented antecedent (M 5 78%)and consequence components (M 5 95%).

During baseline, Ms. Malden’s correctimplementation of antecedent componentswas stable (M 5 47%). Correct implementa-

tion of consequence components appeared tobe increasing in baseline (M 5 27%). Everyattempt was made to have stable or decreasing

trends in baseline prior to implementation ofperformance feedback. However, a clinical de-

cision was made to employ performancefeedback when the implementation of anteced-ent components was stable. Following perfor-

mance feedback, correct implementation ofantecedent (M 5 95%) and consequence(M 5 95%) components improved.

The sequence of baseline and performancefeedback was varied for Mr. Mack and Darrin.During the initial baseline phase, Mr. Mackdisplayed stable performance for the imple-mentation of antecedent components as writ-ten. Mr. Mack did not correctly implement anyconsequence components. Following the initialbaseline, an increase in the frequency and inten-sity of Darrin’s physical aggression requiredseveral revisions in his behavior support plan.Following Revision 1, there was an increase inthe correct implementation of consequencecomponents, and antecedent performanceremained at baseline levels. Revision 2 encom-passed both baseline and performance feedback.Performance feedback resulted in an increase inthe percentage of antecedent and consequencecomponents correctly implemented. Improve-ments in performance were maintained duringRevision 3 across antecedent and consequencecomponents.

Maintenance observations. Treatment integ-rity was maintained at high rates during thefollow-up phase across all teachers (Figures 1and 2). The percentages of antecedent (M 5

83%) and consequence (M 5 100%) compo-nents correctly implemented were maintained

TREATMENT INTEGRITY 211

5 and 10 weeks following the termination ofperformance feedback for Mr. Canton. For Mr.Martin, high levels of correct implementationwere maintained across antecedent (M 5 83%)

and consequence (M 5 100%) components 5and 10 weeks after feedback. For Ms. Lowell,high percentages of correctly implementedantecedent (M 5 89%) and consequence

Figure 2. Percentage of antecedent and consequence components implemented as written across teacher–studentdyads for Classroom 2.

212 ROBIN S. CODDING et al.

(M 5 92%) components were maintained at 5,10, and 15 weeks without performance feed-back. Ms. Malden also implemented highpercentages of antecedent (M 5 92%) andconsequence (M 5 100%) components aswritten during three follow-up observations(i.e., 5, 10, and 15 weeks after intervention).Five weeks after performance feedback wasterminated, Mr. Mack correctly implemented100% of antecedent and consequence com-ponents.

Social Validity

Average ratings across all teachers rangedfrom 4.5 to 5.0 for each item. For the purposeof feedback sessions, intervention procedures,and intervention outcomes, mean ratings of4.8, 4.7, and 4.9, respectively, were obtained,indicating that teachers strongly agreed withitems related to the purpose and procedures aswell as the benefits of the intervention out-comes on their skills and the subsequent impacton their students.

DISCUSSION

The results revealed that accurate implemen-tation of ongoing behavior support plansimproved across 5 student–teacher dyads fol-lowing performance feedback. This researchextends the literature by demonstrating theefficacy of performance feedback on thetreatment integrity of individualized multicom-ponent behavior support plans in a specialeducation setting. This study also showedthat the results of performance feedback weremaintained for up to 15 weeks. In contrast toother studies (Mortenson & Witt, 1998;Noell et al., 1997, 2000; Witt et al., 1997),treatment integrity was assessed using directobservation, and performance feedback wasprovided every other week rather than daily(Noell et al., 1997, 2000; Witt et al., 1997)or weekly (Mortenson & Witt, 1998) andon the same day that the observationoccurred.

Performance feedback resulted in greaterpercentages of both antecedent and conse-quence components correctly implemented for4 of 5 teachers. For the 5th teacher (Mr.Martin), performance feedback resulted inincreases in correct implementation of conse-quence components. These data also illustratethat during baseline, 4 of the 5 teacherscorrectly implemented more antecedent thanconsequence components, although implemen-tation was variable in some cases and at lowerrates than desirable. In the absence of perfor-mance feedback, teachers may have moresuccessfully implemented antecedent compo-nents because these interventions were similaracross students. For example, in Classroom 2, 3students had daily scheduling with preferredfollowing nonpreferred activities and choicesbuilt into all tasks, 2 students needed demandsto be presented in individualized ways, and 2students needed transition warnings. As a result,these antecedent procedures may have operatedas part of the daily routine. In addition, thecomplexity (e.g., various schedules and types ofreinforcement) and number of consequenceprocedures may account for lower treatmentintegrity of consequence components in base-line. Which consequence components weremore likely to be correctly employed (e.g.,positive vs. negative reinforcement) was notexamined; however, research has previouslydemonstrated that teachers tend to focus theirattention on students’ inappropriate behavior(Cooper, Thomson, & Baer, 1970). Future res-earch might consider assessing teachers’ imple-mentation of specific consequence components.

Differences in implementation of antecedentand consequence procedures evident in thisstudy may have interesting implications for thetypes of interventions used to increase teachers’treatment integrity. These findings suggest thatfuture studies should continue to examine theintegrity of individual treatment components(Gresham et al., 1993). Analysis of whethertreatment integrity is a function of teacher

TREATMENT INTEGRITY 213

skill or motivational deficits is also a worthytopic for investigation. For example, exploringwhether treatment integrity increases as a func-tion of performance feedback, teachers’ role inintervention planning, contingent reinforce-ment, training, or a combination of these maybegin to address whether treatment integrity isa result of a skill or motivational deficit.

Despite its documented success, the potencyof performance feedback has been criticized for

several reasons, including the following: (a)

Results have not been maintained at high rates

(Mortenson & Witt, 1998; Witt et al., 1997).(b) It may be time consuming (Mortenson &

Witt, 1998). (c) There may be a lack of

acceptability by change agents (e.g., teachers)

(Mortenson & Witt, 1998) as well as (d)reactivity to observers (Mortenson & Witt,

1998; Witt et al., 1997). To increase the

sophistication of treatment-integrity interven-

tions and specifically performance feedback,attempts to address these concerns were

included in the current study.Improvements in the integrity of behavior

support plan implementation were maintainedin the current study. This may be due tothe length of time performance feedbackwas employed and the latency between ses-sions. Previous research implemented perfor-mance feedback for approximately 6 weeks(Mortenson & Witt, 1998; Noell et al., 2000)compared to 8 to 22 weeks in our study.Whereas other studies utilized daily (Noellet al., 1997, 2002; Witt et al., 1997) or weekly(Mortenson & Witt, 1998) performance feed-back, we provided feedback every other week.Important to note is that performance feedbackwas provided during the same days that directobservations occurred. Consequently, recentexamples of teachers’ behavior could bereviewed during feedback sessions. This sched-ule of performance feedback may lead to themaintenance of treatment effects in the shortterm and is time efficient and practical forclinicians to implement. However, it is unlikely

that these results would be maintained over thelong term. We suspect that periodic collectionof treatment integrity data and subsequentperformance feedback are necessary for highrates of intervention integrity to persist.

The importance of investigating the socialvalidity of performance feedback has evolvedfrom suggestions (Mortenson & Witt, 1998) thatfeedback may serve as a negatively reinforc-ing event for some teachers. All of the teachers inthe current study rated performance feedbackfavorably with respect to the purpose, procedures,and outcome; this is consistent with the findingsof Noell et al. (2002). Assessing the acceptabilityof performance feedback provides a prelim-inary attempt to identify how teachers per-ceive treatment integrity interventions. Futureresearchers may consider experimentally analyz-ing the function that performance feedbackserves for teachers, rather than relying on verbalreport.

The current investigation has several limita-tions. First, it is possible that treatment effectswere confounded by reactivity to beingobserved. Consistent with recommendationsby Gresham et al. (1993), we attempted toreduce reactivity by conducting observations ona variable-time schedule and used consultantswho had been assigned to these classrooms asprimary observers. Therefore, it was commonfor teachers to see these consultants in theclassroom conducting observations. Althoughwe attempted to make these observations un-obtrusive, teachers were aware during theintervention phase that they were beingobserved. A second limitation of the study isthat implementation of performance feedbackoccurred when either antecedent or conse-quence components were stable or decreasing.As a result, experimental control may not beas robust for Ms. Malden, whose correctimplementation of consequence componentsappeared to be improving in baseline. Everyattempt was made to have stable or decreasingtrends in baseline prior to feedback, but

214 ROBIN S. CODDING et al.

a clinical decision was made to employperformance feedback when the implementa-tion of antecedent components was stable.Third, interobserver agreement was based onconsistency between consultants’ ratings oneach plan component, but a more conservativeassessment of agreement would have been toassess whether each consultant recorded thesame type and number of deviations from theplan component in proportion to the numberof opportunities the teacher had to implementthe component. Fourth, it should be noted thatverbal report was used to assess the socialvalidity of performance feedback; this may besubject to rater bias given the frequency withwhich teachers worked with the consultants.Fifth, the functions of individual componentsincluded in performance feedback were notanalyzed. For example, praise and correctivefeedback may be differentially effective. Sixth,we categorized interventions as antecedent andconsequence procedures. Although an attemptwas made to use the literature available to makethese determinations, some procedures (e.g.,providing breaks from work and self-monitor-ing) could be categorized as either antecedent orconsequence procedures. It may be fruitful toexplore the classification of interventions con-sidered to be antecedents and consequences.Finally, the impact of increased treatmentintegrity on students’ maladaptive behaviorwas not examined. Ultimately, the interest indeveloping procedures to increase treatmentintegrity has resulted from its impact ontreatment efficacy. Therefore future studiesshould make an effort to provide correspondinginformation on students’ behavior.

In summary, the results of this study suggestthat performance feedback is an effective andacceptable intervention for increasing thetreatment integrity with which special educa-tion teachers administer antecedent and conse-quence components in the context of anongoing behavior support plan. Although thelength of time this intervention was employed

extends that of previous performance feedbackstudies, feedback was provided for an average of12 min every other week, which is bothpractical and time efficient. This study alsointroduced a method for using direct observa-tion procedures instead of permanent productsto assess treatment integrity.

REFERENCES

Cooper, M. L., Thomson, C. L., & Baer, D. M. (1970).The experimental modification of teacher attendingbehavior. Journal of Applied Behavior Analysis, 3,153–157.

Cossiart, A., Hall, R. V., & Hopkins, B. L. (1973). Theeffects of experimenter’s instructions, feedback andpraise on teacher praise and student attendingbehavior. Journal of Applied Behavior Analysis, 6,89–100.

Gillat, A., & Sulzer-Azaroff, B. (1994). Promotingprincipals’ managerial involvement in instructionalimprovement. Journal of Applied Behavior Analysis,27, 115–129.

Gresham, F. M., Gansle, K. A., & Noell, G. H. (1993).Treatment integrity in applied behavior analysis withchildren. Journal of Applied Behavior Analysis, 26,257–263.

Jones, K. M., Wickstrom, K. F., & Friman, P. C. (1997).The effects of observational feedback on treatmentintegrity in school-based behavioral consultation.School Psychology Quarterly, 12, 316–326.

Kern, L., Choutka, C. M., & Skol, N. G. (2002).Assessment-based antecedent interventions used innatural settings to reduce challenging behavior: Ananalysis of the literature. Education & Treatment ofChildren, 25, 113–130.

Luiselli, J. K., & Cameron, M. J. (Eds.). (1998).Antecedent control: Innovative approaches to behavioralsupport. Baltimore: Paul H. Brookes.

Martens, B. K., Hiralall, A. S., & Bradley, T. A. (1997). Anote to teacher: Improving student behavior throughgoal setting and feedback. School Psychology Quarterly,12, 33–41.

Mortenson, B. P., & Witt, J. C. (1998). The use of weeklyperformance feedback to increase teacher implemen-tation of a pre-referral academic intervention. SchoolPsychology Review, 27, 613–627.

Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E.(2002). Consultation, follow-up, and imple-mentation of behavior management interventions ingeneral education. School Psychology Review, 31,217–234.

Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D.D., & Freeland, J. T. (1997). Increasing teacherintervention implementation in general educationsettings through consultation and performance feed-back. School Psychology Quarterly, 12, 77–78.

TREATMENT INTEGRITY 215

Noell, G. H., Witt, J. C., LaFleur, L. H., Mortenson, B.P., Ranier, D. D., & LeVelle, J. (2000). Increasingintervention implementation in general educationfollowing consultation: A comparison of two follow-up strategies. Journal of Applied Behavior Analysis, 33,271–284.

Schwartz, I. S., & Baer, D. M. (1991). Social vali-dity assessments: Is current practice state of theart? Journal of Applied Behavior Analysis, 24,189–204.

U.S. Department of Education. (2001). The no child leftbehind act. Washington, DC: Author.

Witt, J. C., Noell, G. H., LaFleur, L. H., & Mortenson,B. P. (1997). Teacher use of interventions in generaleducation settings: Measurement and analysis of theindependent variable. Journal of Applied BehaviorAnalysis, 30, 693–696.

Watson, T. S., Sterling, H. M., & McDade, A. (1997).Demystifying behavioral consultation. SchoolPsychology Review, 26, 467–474.

Received July 15, 2004Final acceptance January 18, 2005Action Editor, Louis Hagopian

APPENDIX

216 ROBIN S. CODDING et al.

TREATMENT INTEGRITY 217

Total components in the behavior support plan: 11Total plan components teachers had the opportunity to exhibit: __________Antecedent plan components teachers had the opportunity to exhibit: __________

Antecedent plan components teachers did not have the opportunity to exhibit: __________Antecedent plan components teachers had the opportunity to exhibit:

where implementation was as written: _____ %_________

where implementation was not as written: _____ %_________Consequence plan components teachers had the opportunity to exhibit: __________Consequence plan components teachers did not have the opportunity to exhibit: __________

Consequence plan components teachers had the opportunity to exhibit:where implementation was as written: _____ %_________where implementation was not as written: _____ %_________

218 ROBIN S. CODDING et al.

STUDY QUESTIONS

1. What are the typical components of a performance feedback system?

2. How did the authors attempt to minimize observer reactivity?

3. Briefly summarize the effects observed during the baseline and feedback conditions.

4. Given that performance feedback contains a number of elements, what types of behavioral processes

may have accounted for the observed results?

5. To what did the authors attribute higher levels of treatment integrity for antecedent versus

consequence procedures during baseline?

6. What type of data, not collected in the study, might have strengthened the authors’ conclusions

about improvements in teacher performance?

7. The authors discussed treatment integrity failures as a function of deficits related to skills versusmotivation. Why might feedback as an intervention be unable to distinguish between the two typesof deficits?

8. What are some potential advantages and disadvantages of biweekly observation and performancefeedback?

Questions prepared by Natalie Rolider and Sarah E. Bloom, University of Florida

TREATMENT INTEGRITY 219