Journal of School Psychology 51 (2013) 669–682

Contents lists available at ScienceDirect

Journal of School Psychology

j ourna l homepage: www.e lsev ie r .com/ locate / j schpsyc

Children's intervention strategies in situations of victimization by bullying:Social cognitions of outsiders versus defenders☆

Jeroen Pronk a,b,⁎, Frits A. Goossens a,b, Tjeert Olthof c, Langha De Mey d, Agnes M. Willemen e

a Department of Educational Neuroscience, Faculty of Psychology and Education, VU University Amsterdam, The Netherlandsb LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, VU University Amsterdam, The Netherlandsc Department of Developmental Psychology, Faculty of Psychology and Education, VU University Amsterdam, The Netherlandsd Department of Theory and Research in Education, Faculty of Psychology and Education, VU University Amsterdam, The Netherlandse Department of Child and Family Studies, Faculty of Psychology and Education, VU University Amsterdam, The Netherlands

a r t i c l e i n f o

☆ This research was funded by a grant (431-09-032Vermande.⁎ Corresponding author at: Faculty of Psychology an

Tel.: +31 20 5981713.E-mail address: j.pronk@vu.nl (J. Pronk).ACTION EDITOR: Michelle Demaray.

0022-4405/$ – see front matter © 2013 Society for thehttp://dx.doi.org/10.1016/j.jsp.2013.09.002

a b s t r a c t

Article history:Received 9 December 2011Received in revised form 4 September 2013Accepted 9 September 2013

This study examined the social cognitions of outsiders and defenders about intervening insituations of victimization by bullying. Do outsiders and defenders behave differently invictimization situations because of differences in competence beliefs, or because of aselectivity effect in intervening? These issues were examined in a sample of 102 outsidersand 107 defenders who were classified into these bullying roles through a peer-nominationprocedure out of a total sample of 761 10- to 14-year-old Dutch children. These children werepresented with imaginary victimization events. They answered questions about theircognitions and self-efficacy beliefs about intervening in victimization situations and abouthandling such situations. Outsiders, compared to defenders, claimed to intervene indirectly invictimization situations rather than directly. Defenders, compared to outsiders, claimed tointervene directly in victimization situations rather than indirectly. Both outsiders anddefenders claimed to be more likely to intervene when a friend was being victimized thanwhen a neutral classmate was being victimized. Outsiders and defenders did not differ in theirself-efficacy for indirect intervention, but only defenders claimed a high self-efficacy for directintervention. Both outsiders and defenders claimed to benefit from direct help when theythemselves are victimized, but only outsiders also reported to need indirect help. The resultssuggest that outsiders and defenders behave differently in victimization situations because ofdifferences in competence beliefs rather than because of a selectivity effect. More generally,the results suggest that not only defenders but also outsiders have the intention to helpchildren who are being bullied. However, outsiders' anti-bullying attempts are likely to beindirect and less firm than those of defenders.© 2013 Society for the Study of School Psychology. Published by Elsevier Ltd. All rights reserved.

Keywords:Victimization by bullyingOutsiderDefenderSocial cognitionIntervention strategySelf-efficacy

1. Introduction

Victimization by bullying is generally defined as the repeated exposure to aggressive actions by one or more individuals overtime (Greene, 2006; Olweus, 1993; Salmivalli & Peets, 2009). During those actions some children (one or more bullies)intentionally attempt to harm another individual (the victim). Moreover, an imbalance in perceived or actual power exists

) from the Netherlands Organization for Scientific Research (NWO) awarded to F. A. Goossens and M. M.

d Education, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands.

Study of School Psychology. Published by Elsevier Ltd. All rights reserved.

670 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

between them. Bullying can be directly or indirectly aimed at a victim and can take the form of physical acts (e.g., hitting),possession-directed acts (e.g., taking belongings), verbal acts (e.g., name-calling) and social/relational acts (e.g., ostracizing;Espelage & Swearer, 2003; O'Connell, Pepler, & Craig, 1999; Rigby, 2008). Bullying can be seen as a coercive strategy aimed atreaching and maintaining high ranks in the social hierarchy of the group (Olthof, Goossens, Vermande, Aleva, & Van der Meulen,2011; Reijntjes et al., 2013).

Self-reports have been used to differentiate children as bullies, victims, bully–victims, and a group generally known as“bystanders” (Olweus, 1993, 2010). However, observational studies have demonstrated that the term bystander covers a widerange of different behaviors (Atlas & Pepler, 1998; Hawkins, Pepler, & Craig, 2001; O'Connell et al., 1999). Bystanders include notonly children who are uninvolved in the bullying process but also children who either side with the bully (assistants orreinforcers) or with the victim (defenders). Finally, there is a group of children who tend to shy away from the bullying; they turntheir backs on the bullying. These children are referred to as the outsiders. Peer-reports such as those designed by Salmivalli,Lagerspetz, Björkqvist, Österman, and Kaukiainen (1996) enable the distinction between these participant roles.

Meta-analytic studies suggest that anti-bullying interventions are more effective when they: (a) treat bullying as a groupprocess (Polanin, Espelage, & Pigott, 2012); (b) focus on promoting prosocial bystander behavior (Polanin et al., 2012); (c) have amore intensive program (Ttofi & Farrington, 2011); or (d) target bullying from multiple points of view (e.g., targeting teachersand parents; Ttofi & Farrington, 2011). An example of a school-based anti-bullying intervention that incorporates these aspects inits program is KiVa (Salmivalli, Kärnä, & Poskiparta, 2010). One of KiVa's main aims is reducing bullying by influencing thedifferent bystander-roles. Evaluation of KiVa has proven that it is effective in reducing bullying and victimization behaviors withinclassrooms (Kärnä et al., 2011). Although KiVa effectively reduces bullying and victimization, it is more effective in influencingteacher behavior than in promoting prosocial behavior in bystanders. Specifically with regards to pro-victim attitudes, includinghelping and intervening on behalf of the victim, the effect sizes of KiVa were small (Cohen's d's ≤ .08; Kärnä et al., 2011).Therefore, more knowledge of the different bystander roles is still needed to achieve further gains in the effectiveness of wholegroup anti-bullying interventions. The present study therefore aimed at improving the knowledge of children's social cognitionsabout intervening in victimization situations. The focus was on the bystander groups of outsiders (i.e., those who shy away fromthe bullying) and defenders (i.e., those who actively intervene on behalf of the victim).

There are three reasons for focusing on outsiders and defenders. Firstly, observational studies indicate that peers are almostalways present (more than 85% of the time) when bullying takes place (Atlas & Pepler, 1998; Hawkins et al., 2001; O'Connell et al.,1999). Only in about a quarter of all victimization situations, peers actually discourage the bullying or act to alleviate the victimsuffering caused by it. When peers decide to intervene on behalf of the victim, they do so effectively more than two-thirds of thetime (Hawkins et al., 2001).

The second reason for focusing on outsiders and defenders is that both have been implied to have an anti-bullying attitude.Previous findings have indicated that both outsiders and defenders are attitudinally against bullying and dislike children whobully others (Olthof & Goossens, 2008; Salmivalli & Voeten, 2004). Defenders also behave in a way that matches theiranti-bullying attitude; outsiders on the other hand do not behave according to their anti-bullying attitude—or at least notconsistently enough to be qualified by their peers as defenders (Goossens, Olthof, & Dekker, 2006; Salmivalli et al., 1996; Sutton &Smith, 1999).

The third reason for focusing on outsiders and defenders is their number. The group of outsiders has been found to be as largeas one third of the classroom (Olthof et al., 2011; Salmivalli, Lappalainen, & Lagerspetz, 1998). Another one-fifth of the classroomconsists of defenders. Together they can make up more than 50% of the children in the classroom. Therefore, the sheer number ofchildren, combined with their anti-bullying attitude makes them an attractive focus for interventions promoting defendingbehavior. A genuine reduction in bullying might only be achieved by urging children with an anti-bullying attitude to activelyhelp victims and to stand up for their views (Gini, Pozzoli, Borghi, & Franzoni, 2008; Orpinas & Horne, 2010; Polanin et al., 2012;Salmivalli, 1999; Twemlow et al., 2010).

Previous research has suggested that outsiders and defenders are quite similar to each other. Both are low in reactive andproactive aggression and are equipped with the ability to avoid being victimized themselves (Camodeca & Goossens, 2005). Nextto that, both outsiders and defenders are relatively well liked by their peers (Goossens et al., 2006; Salmivalli et al., 1996), whichis an important motive for hanging out together and for endorsing similar prosocial behaviors (Duffy & Nesdale, 2009; Witvliet etal., 2010). Moreover, outsiders and defenders have been found to be similar in empathy (Gini, Albiero, Benelli, & Altoè, 2008), animportant prerequisite for prosocial behavior (Penner, Dovidio, Piliavin, & Schroeder, 2005). Research has indicated that childrenwho are prosocial, are more likely to be friends with, and to be friendly to, victims. Conversely, having friends can be important forvictims too, as friends protect each other when they are being victimized (Hodges, Boivin, Vitaro, & Bukowski, 1999; Oh & Hazler,2009).

1.1. Competence hypothesis

Still, there are behavioral differences between outsiders and defenders. These differences seem to be at least in part related tosocial self-efficacy (Gini, Albiero et al., 2008). Self-efficacy embodies the belief that one has the right tools to handle oneselfeffectively (Bandura, 1997). Only defender-behavior, but not outsider-behavior was found to be related to competent andassertive conduct in social situations (Gini, Albiero et al., 2008). Similarly, Pozzoli and Gini (2010) found that in victimizationsituations specifically, outsider-behavior was related to avoiding problems, whereas defender-behavior was related to solvingproblems. Outsiders' tendency to avoid problems may explain why they remain passive and do not intervene on behalf of victims.

671J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

Notwithstanding these behavioral differences between outsiders and defenders, previous research has also shown thatoutsiders do occasionally intervene, although not often enough to be seen as defenders by their peers (Goossens et al., 2006;Salmivalli et al., 1996; Sutton & Smith, 1999). Possibly the nature of the intervention is important here. When someone decides tohelp a victim, there is the choice between helping the victim either directly (i.e., confronting the bully) or indirectly (i.e., warningthe teacher or consoling the victim). It may be that outsiders do occasionally intervene in victimization situations but prefer to doso in indirect and less visible ways.

When taken together, the above findings imply that competence differences underlie the behavioral differences betweenoutsiders and defenders. If defenders feel more competent about themselves in victimization situations, they might behave moreassertively and with more confidence in such situations. As a result, defenders will be likely to consider direct interventionstrategies (e.g., confronting the bullies) and to feel efficacious about behaving according to these cognitions too. If outsiders feelless competent about themselves in victimization situations, they will think more about limiting their personal risks in thesesituations (e.g., becoming victimized themselves) and they will resort to more indirect intervention strategies (e.g., consoling thevictim).

Competence differences between outsiders and defenders may also surface when focusing on another issue that has not yetbeen studied before. How do outsiders and defenders wish to be helped when they are being victimized themselves? Informationregarding the specific intervention wishes of outsiders and defenders can help in explaining their cognitions on intervening forsomeone else who is being victimized. Children may only intervene on behalf of victims in ways that correspond with how theywish to be helped themselves when they are being victimized.

1.2. Selectivity hypothesis

Notwithstanding the plausibility of the account described above, it is also conceivable that outsiders and defenders behavedifferently because of differences that are unrelated to their competence. Specifically, outsiders and defenders may also differ inusing their competence and skills in victimization situations because of a selectivity effect. Outsiders may, for example, decide tointervene only when others expect them to do so or when they can identify themselves with the victim. Children have been foundto be more likely to intervene in victimization situations when they expect that their friends would want them to do so (Rigby &Johnson, 2006). Also, several studies have indicated that people are less likely to passively stand by when their friends areinvolved (either as victim or perpetrator) in emergencies (Levine, Cassidy, Brazier, & Reicher, 2002) or in aggressive situations(Tisak & Tisak, 1996). Oh and Hazler (2009) reported that children are more likely to intervene in victimization situations whenthe victim is a friend. Therefore, it is also possible that the occasional defending behaviors of outsiders are due to the fact thatfriends are being victimized. Defenders, on the other hand, may act irrespective of their relationship with the victim. If this is true,there will be no differences in the intervention strategies outsiders and defenders prefer to use in victimization situations.Outsiders and defenders will only differ in their intervention preferences as a result of victim-type.

1.3. Present study

The behavioral differences between outsiders and defenders in intervening in victimization situations may be caused by eithercompetency differences, selectivity differences, or both. Accordingly, different anti-bullying interventions promoting defender-behaviormay be needed. If outsiders lack competence, assertive defender-behavior can be promoted by teaching them social, conflict mediation,and coping skills. A selectivity effect in outsiders based on victim-type calls for a different strategy. Defender-behavior may thenbe promoted by raising awareness of the consequences of being victimized and increasing the identification with victims andtheir needs.

The main focus of this study was the examination of differences in the cognitions of outsiders and defenders in victimizationsituations. However, gender may moderate these cognitions as it is differentially related to bullying and helping behavior (Card,Stucky, Sawalani, & Little, 2008; Trach, Hymel, Waterhouse, & Neale, 2010). Girls have been found to be higher in empathy thanboys (Gini, Albiero et al., 2008; Hoffman, 1977). What is more, classroom peers reported more defending behaviors and attitudesfor girls than for boys (Goossens et al., 2006; Olthof et al., 2011; Salmivalli et al., 1996). In contrast, classroom peers reported morepro-bullying behaviors and attitudes for boys than for girls. When girls act as bullies, they do so by indirect (relational) meansrather than by direct (physical and verbal) means, whereas boys are more general in their bullying behavior (Card et al., 2008;Salmivalli & Peets, 2009; Trach et al., 2010). Experiencing more direct bullying than girls (Card et al., 2008; Trach et al., 2010),boys may think in terms of direct intervention strategies to combat the bullying. Experiencing similar amounts of indirect bullying(Card et al., 2008; Trach et al., 2010), boys and girls may not think differently in terms of indirect intervention strategies when itcomes to intervening in victimization situations.

In the present study, a peer-nomination procedure was used to classify children in the behavioral roles of outsider anddefender in the bullying process. Peer-nominations have been used frequently to assess the bullying participant roles (Goossenset al., 2006; Olthof et al., 2011; Salmivalli et al., 1996). The advantage of peer-nominations is that they are based on informationprovided by those who witness the group process (Pellegrini, 2002). Collecting data from multiple respondents and compilingthem, results in more reliable behavioral assessments of individuals.

To measure the cognitions of outsiders and defenders about intervening on behalf of victims, they were presented withimaginary victimization events and subsequently asked how they thought they would behave in these events. The use of

672 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

imaginary events made it possible to study children's thinking about handling and intervening in victimization events underdifferent—carefully selected—conditions.

2. Method

2.1. Participants

The data (N = 761) were collected during the spring of 2010 in 35 classrooms (13 fifth grade, 16 sixth grade, and 6 combinedfifth–sixth grade) at 17 primary schools throughout the Netherlands. In the Netherlands schools are mandated to haveanti-bullying protocols. The Dutch National School Inspectorate checks the presence and quality of these protocols during yearlyaudits. This mandate assures a similar anti-bullying policy in all schools in the sample. None of the schools in the sample wasusing other anti-bullying interventions at the moment of testing. The cooperation of the schools and classroom teachers wasobtained first. In accordance with a procedure preferred by schools and endorsed by the ethical committee of the faculty, theschools were provided with a letter for the parents (N = 818) about the aim and procedures of the study. Parents could respondby returning a preprinted objection note (n = 19). A small sample of the children who had permission to participate in the studywas either not present at the time of testing (n = 22) or did not want to fill in the questionnaires (n = 16). The study subsampleconsisted of the 209 children out of the total sample of 761 children that could be classified into the bullying roles of Outsider(n = 102) or Defender (n = 107) based on a procedure that will be explained in the Measures section. Participant demographics(age, gender and ethnicity) are summarized in Table 1, with separate columns for the total sample, the study subsample,Defenders, and Outsiders.

2.2. Measures

2.2.1. Involvement in bullyingAn Internet version of Olthof et al.'s (2011) Bullying Role Nomination Procedure (BRNP) was used to classify children as

Outsider or Defender. This peer-nomination procedure is an adapted version of Salmivalli et al.'s (1996) Participant Role Scales.Peer-nomination procedures are based on information provided by all of those involved in the group process. The BRNP is used tocollect and compile the data of all of the observing respondents in the bullying process. As a result, there is the assurance that thebehavioral assessment of each individual within a classroom is obtained reliably (Pellegrini, 2002). The BRNP makes it possible todistinguish the bullying roles of Ringleader Bully, Assistant, Reinforcer, Outsider, Defender, Victim, Bully–Victim and a category ofchildren not involved in bullying. For present study purposes, it was necessary to differentiate pure Outsiders and Defenders.Therefore, a hybrid category of Outsider–Defenders was also identified.

The BRNP has been previously used by Bouman et al. (2012) and by Olthof et al. (2011). With regards to its validity, Bouman etal. (2012) found moderately strong correlations between BRNP-reported and self-reported measures for victimization (r = .44)and bullying (r = .41) behaviors. With regards to the validity of the constructs of Outsider and Defender, Olthof et al. (2011)reported that BRNP-identified defenders were prosocial children with more resource control and more perceived popularity thanother non-bullying children. Also, BRNP-identified outsiders were found to refrain from using dominance-oriented socialstrategies and to be relatively low in resource control and perceived popularity. Moreover, a general finding withpeer-nomination procedures of bullying behaviors is that outsiders and defenders are more frequently girls than boys. Onaverage two-thirds of the children nominated as outsider or defender are girls (Goossens et al., 2006; Salmivalli et al., 1996).Olthof et al. (2011) confirmed this finding with the BRNP. In this study, 64.8% of the children nominated as outsider and 72.8% ofthe children nominated as defender were girls.

Because the present study focused on victimization by bullying in general, an overall measure of bullying and victimizationthat incorporated both boys' and girls' preferred forms of bullying was necessary. Therefore, items about the following five formsof bullying were included in the BRNP: physical (e.g., pushing), possession-directed (e.g., taking one's belongings), verbal (e.g.,

Table 1Demographic sample information for the Total Sample, the Study Sample and separately for Outsiders and Defenders.

Total Sample Study Sample Outsiders Defenders

Nsample 761 209 102 107Age range 10–14 10–14 10–13 10–14M (SD) 11.8 (8.6) 11.7 (8.3) 11.8 (7.7) 11.7 (8.9)

Boys 51.1% 38.8% 51.0% 27.1%Dutch ethnicity children 94.5% 94.3% 98.0% 90.7%

Ngrade5 222 57 24 33Boys 50.5% 43.9% 58.3% 33.3%

Ngrade5/6 142 32 18 14Boys 52.1% 25.0% 38.9% 7.1%

Ngrade6 397 120 60 60Boys 51.1% 40.0% 51.7% 28.3%

Note. All reported age ranges and means (M) are in years. All reported standard deviations (SD) are in months.

673J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

name-calling), direct-relational (e.g., ostracizing), and indirect-relational (e.g., gossiping) bullying. Because cellular and cyberbullying can be considered specific examples of verbal and relational bullying, it was pointed out to the children that these formsof bullying could also be done via Internet or by telephone. Items focusing on cellular and cyber bullying were included to enablechildren to answer the questions about bullying and victimization behaviors as widely as possible. Children needed to answerquestions about bullying and victimization to enable the final classification of children into the roles of Outsider and Defender.

The BRNP started by providing children with a general definition of bullying that incorporated the three aspects of theuniversal definition of bullying (repetition, intentionality, and power imbalance). Specifically, this definition was (a) “Bullyingmeans that one or several children intentionally annoy and humiliate another child again and again to hurt that child to make himor her sad;” (b) “Bullying is different from a quarrel between two children who are about equally strong and from playing jokeson each other;” and (c) “Bullying can be done by hitting, kicking, or pinching a victim; by taking a victim's belongings to destroythem or making them disappear otherwise; by ridiculing or insulting victims or by calling them names; by excluding a victimfrom games or activities; and by gossiping or making other children think bad of a victim otherwise.” Subsequently, a descriptionof a specific form of bullying (physical, possession-directed, verbal, direct-relational, or indirect-relational) was given,accompanied by a list of the various behavioral manifestations belonging to it. For each form of bullying, children were askedto nominate classmates as Victims and as Bullies: (a) “Do you know anyone in your classroomwho is being bullied in this specificway? If so, can you give their name(s)?” and (b) “Do you know anyone in your classroom who carries out that specific form ofbullying? If so, can you give their name(s)?” To be able to distinguish between Ringleader Bullies and Assistants, a third questionwas asked: “You have just mentioned X. Is X someone who starts with bullying in this way or is X someone who joins in afterothers have started bullying in this way?”

After this sequence of 15 items, three further items were used to elicit nominations for the roles of Outsider, Defender, andReinforcer (one item each). Again, children were given an elaborate description of the behavior and asked to nominate childrenfrom their classroom who behaved in that particular way. For the Outsider role the item was “Some classmates do not want tohave anything to do with bullying. They stay away from the bullying, pretend not to see what is going on, and do not take sideswith either the bullies or the victim. Can you give the name(s) of children in your classroom who behave like this?” For theDefender role the item was “Some classmates try to help the victim. They tell them not to feel bad about the bullying, try toconsole them, are nice to them during recess, or contact the teacher to talk about the bullying. Can you give the name(s) ofchildren in your classroomwho behave like this?” Children could not nominate themselves and they could nominate a maximumof five classmates by indicating their names in one of each of the five drop-down lists that included the names of all classmates.The nomination limit was set to five, because children needed to answer 18 peer-nomination items in total (18 times 5 possiblenominations results in 90 nomination possibilities), and they needed to stay motivated during the entire testing procedure.Considering the age of the children under investigation, presenting themwith computer screens with as many drop-down lists asthere were classmates was assumed to be overwhelming and intimidating.

To correct for unequal numbers of children across classrooms, continuous scores were computed per class for each type ofnomination by dividing the number of received nominations by the number of nominators in the classroom (i.e., the number ofparticipating children in the classroomminus the nominee). Although the means of the resulting scores were all between .03 and.10, the scores could range from 0 to 1, with the latter score indicating that all participating classmates nominated the particularchild. Children's overall Ringleader Bully, Assistant of the Bully, and Victim scores were computed from the five items thatmeasured each of these behaviors by following the averaging procedures used by Olthof et al. (2011) and byWitvliet et al. (2010).This computation was not necessary for children's Outsider, Defender, and Reinforcer scores as these were based on one itemonly.

Finally, children were classified into the roles of Outsider or Defender. In the study of Olthof et al. (2011) in which an interviewversion of the BRNP was used, children were nominated on average by 6.5% of their classmates. In the present study, an Internetversion of the BRNP was used, and children were nominated on average by 4.0% of their classmates. The use of an Internet versionof the BRNP instead of an interview version seemed to influence the number of nominations children made. Consequently, usingthe 15% criterion to classify children into their bullying role like Olthof et al. did in their interview version of the BRNP resulted in52.3% of the total sample not meeting the criteria for a participant role, compared to 28.7% of the total sample in Olthof et al.Therefore, consistent with the procedure and recommendations of Goossens et al. (2006), a 10% criterion was used to classifychildren into their bullying role. This 10% criterion resulted in 34.5% of the total sample not meeting the criterion for a participantrole, which was a closer match with the 28.7% in the Olthof et al. study.

Previous reports have indicated that peer-nominated outsiders sometimes also receive nominations for the role of defender(Goossens et al., 2006; Salmivalli et al., 1996; Sutton & Smith, 1999), suggesting that outsiders sometimes act as defenders.Therefore, children with the hybrid role of outsider–defender can be expected. Because the study's hypotheses focused ondifferences between pure outsiders and defenders, the classification procedure differed in one more respect from the procedureused by Olthof et al. (2011). Like in the Olthof et al. study, children were considered to be Defender or Outsider if they scored atleast equal to the criterion value on their primary role (i.e., a score of .10 for Defender or Outsider) and this score exceeded allother continuous peer-nominated bullying behavior scores with a value of at least .01 (i.e., a maximum score of no more than .09for another role). However, in this study, children also needed to fulfill the criterion of a score that was at least twice as high fortheir primary role (e.g., Outsider or Defender) compared with their secondary role (e.g., Defender or Outsider). Children whoscored at least the criterion value on both roles but who did not fulfill this latter criterion, were classified as Hybrid Outsider–Defender (n = 73; nboys = 14). This procedure resulted in the classification of 107 Defenders (nboys = 29) and 102 Outsiders(nboys = 52).

674 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

2.2.2. Imaginary victimization events (IVEs)Three different forms of victimization were manipulated in the IVE: physical, verbal, and relational. The specific events

can be found in Appendix A. These forms of victimization are known to occur most often (Nansel et al., 2001; Rivers &Smith, 1994). Also, boys are more often involved in physical and verbal bullying and victimization, whereas girls are moreoften involved in verbal and relational bullying and victimization (Card et al., 2008; Salmivalli & Peets, 2009; Trach et al.,2010). In all three forms of IVE, three victim-type manipulations (Neutral Victim, Friend as Victim, and Self as Victim) werepresented to the participants (see Appendix A). As a result the children were presented with nine different IVEs, i.e., thethree different forms of victimization crossed with the three different victim-type manipulations. In all IVEs, childrenneeded to answer four items pertaining to different intervention strategies. All items used in the different IVEs can befound in Appendix A.

Similar to procedures used by Rigby and Johnson (2006) and Pozzoli and Gini (2010), children were first presented with anIVE. Subsequently, they were presented with four Intervention Strategy items and asked to rate how likely they were to employeach strategy in a similar situation on a 4-point scale ranging from 1 (never) to 4 (always). Following a study by Kanetsuna, Smith,and Morita (2006), intervention strategies were offered in the form of four items: (a) Confronting the Bully Alone item: “I willconfront the bully or bullies by myself and intervene by telling them: this is enough, stop with the bullying;” (b) Confronting theBully with a Friend item: “I will call a friend and together we will confront the bully or bullies and intervene by telling them: thisis enough, stop with the bullying;” (c) Consoling the Victim item: “I will wait until the bullying has stopped and then I will helpand console the victim;” or (d) Warning the Teacher item: “I will go to the teacher immediately to ask for help to stop thebullying.”

The Victim-type manipulations and the Intervention Strategy items were firstly presented in the context of the physical IVE,followed by the verbal IVE and by the relational IVE respectively. In the first IVE for each form of victimization, the victim wasdescribed as “a kid from your school” to manipulate a Neutral Victim IVE. Only in this Neutral Victim IVE, children were also askedto rate their intervention efficacy for all four intervention strategy items (“How easy is it for you to employ that strategy?”) on a4-point scale ranging from 1 (very difficult) to 4 (very easy). In the second IVE for each form of victimization, the victim wasdescribed as “your best friend” to manipulate a Friend as Victim IVE. In the third IVE for each form of victimization, children wereasked to imagine being the victim of bullying themselves to manipulate a Self as Victim IVE. The Intervention Strategy items wereformulated slightly different to match the latter format (“I wish someone would…” instead of “I will…”).

The Intervention Strategy items presented to the children in the Neutral Victim IVE and Friend as Victim IVE made it possibleto investigate children's Intervention Preparedness and to enable the comparison between Intervention Preparedness for aNeutral Victim and Intervention Preparedness for a Friend as Victim. As children were also presented with items about theirIntervention Efficacy for each intervention strategy in the Neutral Victim IVE, it was also possible to investigate children'sIntervention Efficacy. Finally, the Intervention Strategy items presented to the children in the Self as Victim IVEmade it possible toinvestigate children's Intervention Wish.

To confirm the existence of the hypothesized two-factor structure representing Direct Intervention and Indirect Interventionin the data, confirmatory factor analyses (CFA) were executed in LISREL 8.80. Separate CFA were performed for InterventionPreparedness for a Neutral Victim, Intervention Preparedness for a Friend as Victim, Intervention Efficacy, and Intervention Wish.All CFA were performed on the four Intervention Strategy items of all three manipulated forms of victimization (12 items in total).In each CFA, a Direct Intervention factor and an Indirect Intervention factor were modeled. The Direct Intervention factorconsisted of the Confronting the Bully Alone items and the Confronting the Bully with a Friend items (six items). The IndirectIntervention factor consisted of the Consoling the Victim items and the Warning the Teacher items (six items). As the contents ofthe Intervention Strategy items were the same over the different IVEs, their error variances were allowed to covary in the CFA(e.g., the Confronting the Bully Alone item in the physical, verbal and relational IVEs were allowed to covary). Also, the two latentfactors representing Direct Intervention and Indirect Intervention were allowed to correlate in the CFA.

The CFA demonstrated a close fit for the two-factor structure in the Intervention Preparedness for a Neutral Victim analysis,χ2(41) = 130.14, root mean square error of approximation (RMSEA) = .05 [90% confidence interval = .04–.06], confirmatory fitindex (CFI) = .98, standardized root mean square residual (SRMR) = .04, Tucker–Lewis index (TLI) = .97. The CFA demonstratedan acceptable fit for the two-factor solutions of the Intervention Preparedness for a Friend as Victim analysis, χ2(41) = 228.76,RMSEA = .08 [.07–.09], CFI = .97, SRMR = .03, TLI = .95; the Intervention Efficacy analysis, χ2(41) = 136.34, RMSEA = .06[.05–.07], CFI = .98, SRMR = .04, TLI = .97; and the Intervention Wish analysis, χ2(41) = 196.94, RMSEA = .07 [.06–.08], CFI =.98, SRMR = .03, TLI = .96.

The outcomes of the CFA were used to create composite Intervention Strategy variables representing Direct Interventionand Indirect Intervention. The Direct Intervention variables were mean scores of the Confronting the Bully Alone items andthe Confronting the Bully with a Friend items of the physical, verbal and relational IVEs (six items in total). The IndirectIntervention variables were mean scores of the Consoling the Victim items and the Warning the teacher items of thephysical, verbal and relational IVEs (six items in total). As a result, eight variables could be created that reflected thechildren's social cognitions about intervening in and handling victimization events: (a) Direct Intervention Preparednessfor a Neutral Victim and Indirect Intervention Preparedness for a Neutral Victim (coefficient alpha of .77 and .80,respectively); (b) Direct Intervention Preparedness for a Friend and Indirect Intervention Preparedness for a Friend(coefficient alpha of .82 and .82, respectively); (c) Direct Intervention Efficacy and Indirect Intervention Efficacy(coefficient alpha of .86 and .76, respectively); (d) Direct Intervention Wish and Indirect Intervention Wish (coefficientalpha of .84 and .82, respectively).

675J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

2.3. Procedure

The BRNP and IVEswere administered on a computer thatwas connected to the Internet as part of a larger testing session that alsoincluded othermeasures not relevant for the present study. Computerized testing via Internet has the advantage that it enables one touse a forced response procedure. As a result, it was impossible to encounter missing data in the questionnaires. Each child received alogin name and password to enter thewebsite, thus ensuring correct response recording. The childrenwere tested in small groups ina separate and quiet room in their school. Seating arrangements were made so it was impossible for children to communicate or seethe responses of others. Before the testing session started, the childrenwere informed about the confidentiality of their responses andoffered the possibility to cease participation whenever they wanted to, but none did. The children were urged not to talk with theirclassmates about the questions or their answers. The testing sessions were always supervised by at least two research assistants whowere unfamiliar with the subjects. The research assistants were trained and followed a written research protocol, assuring consistentand correct administration of the questionnaires. The children always started the procedure with the BRNP and ended with the IVE.The different IVEs were presented in a fixed order. The physical IVEs were always presented first, followed by the verbal IVE and therelational IVE respectively. Within each form of IVE, the Neutral Victim IVE was always presented first, followed by the Friend asVictim IVE and the Self as Victim IVE respectively. The total procedure took 30 to 45 min. If children finished before the testing sessionwas over, they were given the opportunity to silently play a game.

2.4. Statistical analyses

The data of the present study had a three-level hierarchical structure: the children (level 1, N = 209) were nested withinclassrooms (level 2, n = 35), and both were nested within schools (level 3, n = 17). Therefore, the assumption ofnonindependence of observations was untenable, and the data needed to be approached with multilevel modeling (Snijders &Bosker, 2004). To this end, the mixed model procedure of SPSS 17.0 with maximum likelihood (ML) estimation was used. Fullinformation maximum likelihood (FIML) estimation in data with small sample sizes (n b 50) at the higher levels can bias therandom effects estimates as it might underestimate the standard errors of the regression coefficients; therefore, such data shouldbe approached with restricted maximum likelihood (RML) estimation (Hox, 2010; Maas & Hox, 2005). FIML estimation on theother hand is preferred over RML estimation when comparing the fit indices (−2 log likelihood) of twomodels that differ in theirfixed effects parameters (Kreft & De Leeuw, 1998). In this study, the sample sizes at the classroom and school levels wererelatively small. However, the conditional models that were compared differed in their fixed effects parameters. Therefore, bothestimation methods were used to analyze the data and the results of both estimations methods were compared.

The Direct and Indirect Intervention variables that were created in the present study were all mean scores based on 6 itemsand were treated as continuous measures with a range of 1 to 4. In the Intervention Preparedness analyses, the variablesIntervention Strategy (Direct versus Indirect) and Victim-type (Neutral versus Friend) were manipulated within-subjects.Similarly, in the Intervention Efficacy analyses and in the Intervention Wish analyses, the variable Intervention Strategy (Directversus Indirect) was manipulated within-subjects. In order to prepare the dataset for multilevel modeling with these repeatedmeasures, a dependent variable needed to be created for Intervention Preparedness, for Intervention Efficacy, and for InterventionWish. The within-subjects manipulated variables of Intervention Strategy, and in the Intervention Preparedness analyses also thewithin-subjects manipulated variable of Victim-type, were indexed on these dependent variables as dichotomous independentvariables (values: 0 and 1). The between-subjects manipulated independent variables of Role and Gender were also modeled asdichotomous variables (values: 0 and 1).

The multilevel models were built according to the procedures of Snijders and Bosker (2004). All analyses started withunconditional models in which the mean levels of the dependent variables were estimated, while taking the variances for thelevels of child, classroom, and school, into account. The unconditional models were used to test the multilevel structure of thedata. Subsequently, prediction models were built that included all predictors and their interaction terms. Finally, nonsignificantpredictors were removed from the models, and the fit indices of the final models were compared with those of the unconditionalmodels to test for model improvement. Specifically, a deviance statistic (−2 log likelihood) was calculated to investigate whethermodel fit improved. The deviance statistic has a large-sample chi-square distribution, with degrees of freedom equal to thebetween-model difference in the number of parameters estimated. The significance of the improvement in model fit was testedwith a χ2 difference test.

All results were tested with an alpha of .05. Because all independent variables were dichotomous, statistically significantinteraction effects were followed up by multilevel models in which the dichotomous variables contributing to the interactionwere investigated separately by means of SPSS's split file procedure. This follow-up modeling was done solely to make theinteractions interpretable. To further increase the interpretability of significant effects in the final multilevel models, descriptivestatistics (means and standard errors of the mean) are reported for significant main effects and interaction effects.

3. Results

3.1. Preliminary analyses

In Table 2 the results of the unconditional Intervention Preparedness model, the unconditional Intervention Efficacy model,and the unconditional InterventionWish model are represented. As can be seen from Table 2, the total variance in the dependent

Table 2Unconditional multilevel models for Intervention Preparedness, Intervention Efficacy, and Intervention Wish, of Outsiders versus Defenders.

Intervention Preparedness Intervention Efficacy Intervention Wish

F (df) Estimate F (df) Estimate F (df) Estimate

Fixed effects parametersIntercept 5635.55⁎ (1, 13.63) 2.87⁎ 2697.31⁎ (1, 15.04) 3.00⁎ 3605.56⁎ (1, 17.20) 2.90⁎

Random effects parametersChild level variance 0.22⁎ 0.07⁎ 0.22⁎

Classroom level variance 0.00 0.00 0.00School level variance 0.00 0.03 0.01

Deviance (−2 log likelihood) 1590.49 751.98 882.66

Note.⁎ p b .05.

676 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

variables was partitioned into components for the nested levels of child, classroom, and school. Partitioning the total variance atall levels, by calculating the intraclass correlations (ICC), indicated that the between-level variance in the outcomes was largelyattributable to variability at the child level (Intervention Preparedness = 44.4%; Intervention Efficacy = 18.0%; InterventionWish = 41.4%). Only small amounts of the variance in the outcomes could be attributed to the classroom or school level(Intervention Preparedness = 0.2% and 0.1%; Intervention Efficacy = 0.7% and 8.9%; Intervention Wish = 0% and 1.3% for theclassroom level and school level, respectively).

In addition to the ICC, design effects were calculated for the nested levels as another indication of the need to take dataclustering into account in model estimation. A design effect larger than 2 indicates that multilevel modeling is needed (Peugh,2010). In the current study, the design effects for the classroom and school levels (Intervention Preparedness = 1.01 and 1.01;Intervention Efficacy = 1.03 and 2.00; InterventionWish = 1.00 and 1.15) did not exceed this criterion. To acknowledge that thedata had a nested structure, the classroom and school levels were specified in the analyses. The classroom and school levelvariances were small and nonsignificant (see also Table 2), and therefore, predictor effects were not modeled to vary acrossclassrooms or schools.1

To test for the potentially confounding influences of Age (in months) and Ethnicity (Dutch, non-Dutch) on the dependentvariables, these variables were added into each unconditional model. Neither had a statistically significant association with thedependent variables (p's N .05). The conditional models for every research question initially also included Age and Ethnicity ascontrol covariates for effects on the independent variables. As these two control predictors did not influence the results, they willnot be reported.

3.2. Intervention Preparedness

The examination of differences in the cognitions of Outsiders and Defenders regarding their Intervention Preparedness invictimization situations and the influence of friendship with the victim on these cognitions started with the unconditionalIntervention Preparedness model. As can be seen from Table 2, the unconditional Intervention Preparedness model estimated hada deviance statistic (−2 log likelihood) of 1590.49.

As a second step the independent dichotomous variables of Role, Intervention Strategy, Victim-type, Gender, and allinteraction terms were added into the unconditional Intervention Preparedness model as fixed effects. A total of 15 predictorswere added into the model. This full prediction model had a deviance statistic (−2 log likelihood) of 1482.78, indicating thatmodel fit was significantly worse without these predictors in the model, χ2(15) = 107.71, p b .001. Finally, all nonsignificantinteraction terms were removed from the model to obtain the best fitting Intervention Preparedness model. In this final step,eight predictors were removed from the model, i.e., Role ∗ Gender, Victim-type ∗ Role, Victim-type ∗ Gender, all three-wayinteraction terms, and the four-way interaction term were removed from the model. The results of the final InterventionPreparedness model, including model fit statistics, can be found in Table 3. The unconditional model's fit was significantly worsethan this model, χ2(7) = 102.26, p b .001. This set of predictors explained 6.51% of the total variance in InterventionPreparedness.

There was a statistically significant interaction (see Table 3) between Role and Intervention Strategy (Role ∗ InterventionStrategy), F (1, 627) = 48.35; p b .001. This interaction indicated that Outsiders and Defenders were different in their cognitionsregarding Intervention Preparedness. The descriptive statistics (means and standard errors of the mean) of the variablescontributing to the interaction between Role and Intervention Strategy in Intervention Preparedness can be found in Table 4.Following-up on this interaction, Outsiders preferred Indirect over Direct Intervention, F (1, 306) = 40.19; p b .001, and weremore prepared to intervene by Indirect means than Defenders, F (1, 209) = 5.02; p = .026. Defenders, on the other hand,

1 Even though the ICC and design effects indicated that the variance in the data was mainly attributable to the child level, the effects of the predictor variablescould, in theory, still significantly vary randomly across classrooms or schools. Therefore, the analyses of the final conditional models for every research questionwere also executed by allowing fixed effects predictor variables to vary randomly across units at the higher levels of classroom and school. Only for InterventionPreparedness, the resulting deviance statistics (−2 log likelihood) of these analyses indicated that model fit improved compared to models with only fixed effectspredictor variables. However, fixed effects parameter estimate values remained the same.

Table 3Final multilevel models for Intervention Preparedness, Intervention Efficacy, and Intervention Wish, of Outsiders versus Defenders.

Intervention Preparedness Intervention Efficacy Intervention Wish

F (df) Estimate F (df) Estimate F (df) Estimate

Fixed effects parametersIntercept 5332.25⁎ (1, 14.23) 3.17⁎ 3155.34⁎ (1, 14.86) 3.16⁎ 3461.82⁎ (1, 20.21) 3.15⁎

Role 0.03 (1, 204.19) −0.23⁎ 8.66⁎ (1, 209) 0.08 2.93 (1, 207.09) −0.26⁎

Intervention Strategy 1.25 (1, 627) −0.38⁎ 63.54⁎ (1, 209) −0.61⁎ 1.39 (1, 209) −0.15Gender 1.38 (1, 200.06) −0.30⁎ 3.66 (1, 208.67) −0.01 8.51⁎ (1, 205.69) −0.37Victim-type 26.26⁎ (1, 627) −0.06

Interaction termsRole ∗ Intervention Strategy 48.35⁎ (1, 627) 0.48⁎ 7.384⁎ (1, 209) 0.24⁎ 4.06⁎ (1, 209) 0.22⁎

Role ∗ Gender – – – – – –

Intervention Strategy ∗ Gender 34.69⁎ (1, 627) 0.42⁎ 10.04⁎ (1, 209) 0.29⁎ 3.59 (1, 209) 0.21Victim-type ∗ Intervention Strategy 10.32⁎ (1, 627) −0.21⁎

Victim-type ∗ Role – –

Victim-type ∗ Gender – –

Random effects parameterChild level variance 0.23⁎ 0.11⁎ 0.21⁎

Deviance (−2 log likelihood) 1488.23 665.57 866.63χ2 difference test χ2(7) = 102.26⁎ χ2(5) = 86.41⁎ χ2(5) = 16.03⁎

Note. Gender was coded as 0 (Boys) and 1 (Girls). Role was coded as 0 (Defender) and 1 (Outsider). Intervention Strategy was coded as 0 (Direct) and 1 (Indirect).Victim-type was coded as 0 (Neutral) and 1 (Friend). The models were hierarchically built and represent the best fitting model. Blank cells in the fixed effectsparameters section of the Table indicate parameters that were not present in the model. Dashed cells in the fixed effects parameters section of the Table indicateparameters that were not estimated in the final model due to nonsignificance in previous models.⁎ p b .05.

677J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

preferred Direct over Indirect Intervention, F (1, 321) = 15.96; p b .001, and were more prepared to intervene by Direct meansthan outsiders, F (1, 207.63) = 8.01; p = .005.

Secondly, as can be seen in Table 3, a statistically significantmain effect for the predictor Victim-type, F (1, 627) = 26.26; p b .001,was found as well as a statistically significant interaction with the predictor Intervention Strategy (Victim-type ∗ InterventionStrategy), F (1, 627) = 10.32; p = .001. These findings indicated that children's relationship to the victim influenced their cognitionsregarding Intervention Preparedness. Children were less likely to intervene for a Neutral Victim (M = 2.77, SE = 0.04) than for aFriend as Victim (M = 2.94, SE = 0.04). Thismain effectwas also found for both Direct Intervention,Mneutral = 2.71, SEneutral =0.05;Mfriend = 2.98, SEfriend = 0.05; F (1, 209) = 94.26; p b .001, and Indirect Intervention, Mneutral = 2.83, SEneutral = 0.05; Mfriend =2.91, SEfriend = 0.06; F (1, 209) = 7.64; p = .006. However, when children decided to intervene for a Neutral Victim, theyweremorelikely to choose Indirect Intervention, F (1, 209) = 7.30; p =.007.When a Friendwas victimized both strategies were chosen equallyoften (p N .05).

Finally, a significant interaction between Gender and Intervention Strategy was found (Intervention Strategy ∗ Gender), F (1,627) = 34.69; p b .001. This finding indicated that boys and girls were different in their cognitions regarding InterventionPreparedness. Boys preferred Direct Intervention (M = 2.90, SE = 0.07) over Indirect Intervention, M = 2.78, SE = 0.06; F (1,234) = 11.72; p = .001. The reverse was found for girls, Mdirect = 2.71, SEdirect = 0.08; Mindirect = 3.02, SEindirect = 0.06; F (1,384) = 29; p b .001. Moreover, girls were more likely to perform Indirect Interventions than boys, F (1, 209) = 8.48; p = .004.No significant gender differences were found for Direct Intervention (p N .05). Using the RML estimation method did not alterthese results.

Table 4Means (M) and standard errors of the mean (SE) for the interaction between Role (columns) and Intervention Strategy (rows) in Intervention Preparedness,Intervention Efficacy and Intervention Wish.

Outsider Defender

M SE M SE

Intervention PreparednessDirect Intervention 2.72 0.06 2.97 0.07Indirect Intervention 2.99 0.07 2.74 0.08

Intervention EfficacyDirect Intervention 2.68 0.06 3.01 0.07Indirect Intervention 3.15 0.05 3.27 0.06

Intervention WishDirect Intervention 2.93 0.07 2.92 0.08Indirect Intervention 2.98 0.07 2.72 0.08

678 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

3.3. Intervention Efficacy

The examination of differences in the cognitions of Outsiders and Defenders regarding their Intervention Efficacy invictimization situations, started with the unconditional Intervention Efficacy model. As can be seen from Table 2, theunconditional Intervention Efficacy model estimated had a deviance statistic (−2 log likelihood) of 751.98.

As a second step, the independent dichotomous variables of Role, Intervention Strategy, Gender, and all interaction terms wereadded into the unconditional Intervention Efficacy model as fixed effects. A total of 7 predictors were added into the model. Thisfull prediction model had a deviance statistic (−2 log likelihood) of 664.68, indicating that model fit was significantly worsewithout this set of predictors in the model, χ2(7) = 87.30, p b .001. Finally, all nonsignificant interaction terms were removedfrom the model to obtain the best fitting Intervention Efficacy model. In this final step, two predictors were removed from themodel, i.e., the Role ∗ Gender interaction and the three-way interaction term were removed from the model. The results of thefinal Intervention Efficacy model, including model fit statistics, can be found in Table 3. The fit of the unconditional model wassignificantly worse than that of the final model, χ2(5) = 86.41, p b .001. The set of predictors explained 15.07% of the totalvariance in Intervention Efficacy.

The results in Table 3 first of all show a main effect of the predictor Intervention Strategy, F (1, 209) = 63.54; p b .001.Children reported a significantly higher Indirect Intervention Efficacy (M = 3.21, SE = 0.04) than Direct Intervention Efficacy(M = 2.85, SE = 0.04). Secondly, Table 3 indicates that there was a statistically significant main effect of the predictor Role, F (1,209) = 8.66; p = .004. Outsiders reported a lower Intervention Efficacy (M = 2.92, SE = 0.05) than Defenders (M = 3.14,SE = 0.05). However, a statistically significant interaction was also found between Role and Intervention Strategy, F (1, 209) =7.384; p = .007. This interaction indicated that Outsiders and Defenders were different in their cognitions regarding InterventionEfficacy. The descriptive statistics (means and standard errors of the mean) of the variables contributing to the interactionbetween Role and Intervention Strategy in Intervention Efficacy can be found in Table 4. The lower Intervention Efficacy ofOutsiders was only reflected in a lower Direct Intervention Efficacy, F (1, 208.35) = 15.11; p b .001. Outsiders were similar toDefenders in Indirect Intervention Efficacy (p N .05).

Finally, Table 3 shows that there was a statistically significant interaction between Gender and Intervention Strategy, F (1,209) = 10.04; p = .002. This finding indicated that boys and girls were different in their cognitions regarding InterventionEfficacy. Boys reported a higher Direct Intervention Efficacy (M = 3.01, SE = 0.07) than girls, M = 2.68, SE = 0.05; F (1,208.71) = 10.50; p = .001. Boys and girls were found to be similar in Indirect Intervention Efficacy,Mboys = 3.23, SEboys = 0.06;Mgirls = 3.19, SEgirls = 0.05; p N .05. Still, both boys, F (1, 81) = 8.08; p = .006, and girls, F (1, 128) = 91.23; p b .001, reported asignificantly higher Indirect Intervention Efficacy than Direct Intervention Efficacy. Using RML estimation method did not alterthese results.

3.4. Intervention Wish

The examination of differences in the cognitions of Outsiders and Defenders regarding their wish for help by others when theythemselves were victimized started with the unconditional Intervention Wish model. As can be seen from Table 2, theunconditional Intervention Wish model estimated had a deviance statistic (−2 log likelihood) of 882.66.

As a second step, the independent dichotomous variables of Role, Intervention Strategy, Gender and all interaction terms wereadded into the unconditional Intervention Wish model as fixed effects. A total of 7 predictors were added into the model. This fullprediction model had a deviance statistic (−2 log likelihood) of 865.34, indicating that model fit was significantly worse withoutthis set of predictors in the model, χ2(7) = 17.32, p = .02. Finally, all nonsignificant interaction terms were removed from themodel to obtain the best fitting Intervention Wish model. In this final step, two predictors were removed from the model, i.e.,Role ∗ Gender and the three-way interaction term. The results of the final Intervention Wish model, including model fit statistics,can be found in Table 3. Again, the fit of the unconditional model without predictors was significantly worse, χ2(5) = 16.03, p =.007. The final set of predictors explained 3.99% of the total variance in Intervention Wish. However, when using RML estimation,the two models' fit did not differ significantly and therefore these results are to be interpreted with caution.

The interaction between Role and Intervention Strategy was statistically significant, F (1, 209) = 7.384; p = .045. Thisinteraction indicated that Outsiders and Defenders differed in the way that they wished to be helped when they are beingvictimized. The descriptive statistics (means and standard errors of the mean) of the variables contributing to the interactionbetween Role and Intervention Strategy in InterventionWish can be found in Table 4. Outsiders had a significantly higher wish forIndirect Help than Defenders, F (1, 209) = 5.87; p = .016, whereas Outsiders and Defenders were similar in their wish for DirectHelp (p N .05). Moreover, Defenders had a lower wish for Indirect Help than for Direct Help, F (1, 107) = 5.43; p = .022, whereasOutsiders had a similar high wish for Direct and Indirect Help (p N .05).

Finally, gender differences were also found. Specifically, a main effect was found for the predictor Gender, F (1, 205.69) = 8.51;p = .004. Boys (M = 2.76, SE = 0.07) had a lower Intervention Wish when being victimized than girls (M = 3.01, SE = 0.06).

4. Discussion

The behavioral differences between outsiders and defenders in intervening on behalf of victims of bullying seem to be linkedto their social cognitions. At the cognitive level, outsiders and defenders were found to differ because of competence rather thanselectivity.

679J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

4.1. Selectivity hypothesis

The selectivity hypothesis that the occasional defending of outsiders might be motivated by their relationship with thevictim could not be confirmed. The cognitions about defending of both outsiders and defenders seem to be motivated byfriendship. However, outsiders' higher expressed likelihood to intervene on behalf of victims who are their friends is furtherevidence of their usefulness in systematic attempts to reduce victimization by bullying. Promoting friendship-bonds betweenoutsiders and victims as well as increasing identification with the victim may be worthwhile techniques to incorporate inanti-bullying interventions. Using these strategies may result in a significant increase of pro-victim intervention behavior inoutsiders.

4.2. Competence hypothesis

With regards to competence, the findings firstly show that outsiders did express a preparedness to intervene in victimizationsituations. However, they indicated to intervene primarily by indirect means. They even indicated a higher preparedness to do sothan defenders. Moreover, outsiders were found to be equal to defenders in their self-efficacy beliefs to perform indirectinterventions. Thus, contrary to Pozzoli and Gini's (2010) suggestion, outsiders' cognitions indicate that they do not cope withvictimization situations by avoiding and refraining from them. Indirect interventions correspond more closely to seeking andoffering social support to resolve victimization situations and their consequences. The finding that outsiders think aboutintervening in favor of the victim by seeking and offering social support, offers a plausible explanation for previous findingsregarding their tendency to perform interventions in favor of the victim from time to time (Goossens et al., 2006; Salmivalli et al.,1996; Sutton & Smith, 1999).

Secondly, defenders indicated a preference for direct interventions in victimization situations and to be more likely to do sothan outsiders. They also indicated to have the self-efficacy beliefs to execute direct interventions. Direct interventionscorrespond with actively helping out to put a stop to victimization and coping with victimization situations by solving them, asPozzoli and Gini (2010) have suggested. Thus, in accordance with previous findings, defenders perceived themselves to be morecompetent and efficacious than outsiders. Extending on previous findings, the competency differences are now also obtained invictimization situations, at least at the cognitive level.

Finally, as expected, defenders (compared to outsiders) expressed a better ability to handle the consequences of beingvictimized themselves, as they less often expressed a wish for someone else's indirect help when being victimized. Indirect helpconsists of alleviating the negative social consequences of victimization. This finding can be viewed as consistent with previousfindings of their higher social competence (Gini, Albiero et al., 2008; Pozzoli & Gini, 2010). It may also indicate that defenders feelmore able to cope with their feelings in response to encountering victimization. Still, both outsiders and defenders expressed awish to be helped directly when they are victimized. Being helped directly results in a quick resolution of victimization,something all children may want. In contrast with previous findings (Gini, Albiero et al., 2008; Pozzoli & Gini, 2010), the presentstudy suggests that both outsiders and defenders may still benefit from gains in assertiveness to combat being victimizedthemselves. The stronger expressed wish of outsiders to receive indirect help as well indicates that, in contrast to defenders, theyalso need to be consoled to handle victimization. It may of course be that outsiders become more upset by victimization thandefenders. However, evidence to support this claim is unavailable.

The differences in the social cognitions of outsiders and defenders that were found in the present study reflect their behavioras measured in the group, albeit in a stronger form. Defenders claimed to intervene directly and to have the self-efficacy to do this.Outsiders claimed to intervene indirectly and were found to have a similar self-efficacy as defenders to do this. This latter claim,however, is not reflected in outsiders' peer reputation. The peer-reported outsider was defined as a non-intervener (i.e., someonewho stays away from the bullying and who pretends not to see it and does not take sides). The peer-reported defender on theother hand, was defined as an indirect intervener (i.e., a victim consoler who is nice to victims, helps them, and warns the teacherabout the bullying). This finding suggests that outsiders claim to do more than their peers have observed. Indirect interventionsare more likely to occur outside of the awareness of the peer group and after victimization has ended. Indirect interventions maythus be less conspicuous and may be more often unnoticed by peers. Another possibility is that children may be more strict inreporting other children's behavior (i.e., less likely to see others as defenders), whereas they over-report their own assumedbehavior (i.e., more likely to see themselves as a defender).

One could also argue that what was measured was how outsiders and defenders desire to act in victimization situations. Theseintervention preferences then, once again, stress the importance of both subgroups of children in attempts to curb the bullying.Defenders expressed to be more committed to help victims than outsiders and expressed to feel more confident about helpingvictims. However, outsiders and defenders expressed a similar confidence in their ability to execute indirect strategies. Researchhas shown that the decision to intervene on behalf of a victim is only made after the evaluation of the concomitant costs andbenefits of doing so (Penner et al., 2005; Piliavin, Dovidio, Gaertner, & Clark, 1981). It may be that outsiders think that the costs fordirect intervention (i.e., retaliation by the bullies) are higher or more important than the benefits of intervening (i.e., popularstatus and likeability). Indirect interventions may be safer, because the perpetrators have already left the scene. Outsiders need tobecome more aware of the fact that they can and need to intervene (more often), and that intervening in victimization situationsis possible without negative consequences. A key factor in achieving outsider involvement may thus be to create more awarenessof the different ways in which they can intervene in victimization situations.

680 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

4.3. Gender differences

The results confirm the expectation that girls would express a higher preparedness to intervene in victimization situations thanboys. Previous reports have already indicated that girls are higher in empathy than boys (Gini, Albiero et al., 2008; Hoffman, 1977).Moreover, based on peer reports, girls were also more often defenders in the present study (see also: Goossens et al., 2006; Olthof etal., 2011; Salmivalli et al., 1996). Surprisingly enough, the general cognitive pattern for girls seems to be more congruent to that ofoutsiders (i.e., cognitively indirect interveners), whereas the general cognitive pattern for boys seems to bemore congruent with thatof defenders (i.e., cognitively direct interveners). However, the gender-segregated nature of bullying behaviormay help in explainingthese counterintuitive patterns. Boys are likely to come across both direct and indirect forms of bullying and victimization (Card et al.,2008; Trach et al., 2010). Direct intervention strategies may be more efficient to combat direct forms of bullying. Girls, on the otherhand, are likely to come across more indirect forms of bullying and victimization (Card et al., 2008; Trach et al., 2010). Indirectintervention strategies to counteract the negative consequences of these types of bullying may then be more efficient.

4.4. Strengths, limitations, and conclusion

The present study has several strengths. Firstly, outsiders and defenders were directly comparedwith each other in this study. Assuch, the present study focused on those children who can have a big impact on the group process of bullying (i.e., thebystander-groupswith a pro-victim attitude). Secondly, focusing on the social cognitions of these children in imaginary victimizationevents enabled us to probe into their potential as anti-bullying force in the classroom. Finally, the present study used peer-reports toclassify children as outsiders or defenders. This multi-informant approach yields more reliable assessments of those involved inbullying and victimization than single-informant approaches (Pellegrini, 2002).

The present study also has some limitations. Firstly, the design of the study was purely correlational. Therefore no definiteconclusions can be drawn about the causality and directionality of the results. Future studies using a longitudinal design are requiredto provide more insight on causality and directionality. Secondly, the present study looked at explicit cognitions in the helpingbehavior of outsiders and defenderswith the use of imaginary victimization events. However, the intervention behavior of children invictimization situations may be guided more strongly by implicit cognitive processes. As children's responses were based on explicitcognitions, they may have been influenced by social desirability in responding. Future studies that investigate both the explicit andimplicit cognitive processes concerning intervention strategies and helping behavior are required to provide more insight in thismatter. Thirdly, imaginary victimization events were used in this study in an attempt to mimic real-life situations of victimization.However, the participants were not asked whether the events that were used resembled their real-life experiences. Future studiesshould investigate the correspondence between imaginary victimization events and children's real-life experiences. Finally, theimaginary victimization events that were presented to the children in this study were always presented in the same order. As therewere a total of nine different victimization events, it is possible that fatigue effects have influenced the results. Future studies that useimaginary victimization events should use amore experimental design inwhich children are classified as outsider or defender beforethey are presented with these events. This strategy would make it possible to counterbalance the order of presentation of the eventsin similar ways for both groups of children without introducing undesirable error terms into the data because of group differences initem presentation.

Notwithstanding these limitations, the present findings suggest that it is important for anti-bullying interventions to teachoutsiders social and conflict mediation skills to align their helping behavior with their cognitions about intervening invictimization situations. Outsiders need to become more aware of the possibilities they have to safely and competently executeinterventions in favor of the victim. Outsiders will also benefit from gaining social and physical competence to be better able tohandle the consequences of victimization—both when they are victimized themselves and when they decide to intervene onbehalf of someone else who is being victimized. The results suggest that defenders may benefit from improving their social andphysical competence too, as they were found to not yet be completely competent in their cognitions about handling the directconsequences of being victimized. In addition, the present findings suggest that stimulating friendship bonds with victims ofbullying and increasing victim identification can promote interventions in favor of the victim as well.

In a broader sense this study suggests that not only defenders, but also outsiders have the intention to help children who arebeing bullied. At the cognitive level outsiders are already intervening in favor of victims of bullying, albeit in more indirect andless firm ways than defenders. Outsiders need to be persuaded to act in accordance with their cognitions.

Appendix A. Imaginary victimization events

A.1. Victimization form manipulations

A.1.1. Physical victimization eventIn your classroom there is a child who often bullies others. This child is strong and often uses force when bullying. It is

someone who bullies for example by hitting, kicking and/or pinching others.

A.1.2. Verbal victimization eventIn your classroom there is a child who often bullies others. This child is very good at insulting and calling others bad names. It

is someone who bullies for example by insulting and ridiculing others.

681J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

A.1.3. Relational victimization eventIn your classroom there is a child who often bullies others. This child is a very sly bully. It is someone who bullies by spinning

tales, causing others to look bad. This child lies and cheats and causes others to be excluded from games and activities bygossiping about them.

A.2. Victim-type manipulations

A.2.1. Neutral victimOne day you see a child being badly harassed by this bully. How would you respond in this situation?

A.2.2. Friend as victimOne day you see your best friend being badly harassed by this bully. How would you respond in this situation?

A.2.3. Self as victimOne day you yourself are harassed by this bully and someone else from your classroom sees this. How would you want this

classmate to respond in this situation?

A.3. Intervention strategy items

A.3.1.1. Confronting the bully alone (direct intervention)I will confront the bully or bullies by myself and intervene by telling them: this is enough, stop with the bullying. How likely is

it that you will employ this strategy?

A.3.1.2. Intervention efficacy (only in the neutral victim IVE)How easy is it for you to employ this strategy?

A.3.2.1. Confronting the bully with a friend (direct intervention)I will call a friend and together we will confront the bully or bullies and intervene by telling them: this is enough, stop with the

bullying. How likely is it that you will employ this strategy?

A.3.2.2. Intervention efficacy (only in the neutral victim IVE)How easy is it for you to employ this strategy?

A.3.3.1. Consoling the victim (indirect intervention)I will wait until the bullying has stopped and then I will help and console the victim. How likely is it that you will employ this

strategy?

A.3.3.2. Intervention efficacy (only in the neutral victim IVE)How easy is it for you to employ this strategy?

A.3.4.1. Warning the teacher (indirect intervention)I will go to the teacher immediately to ask for help to stop the bullying. How likely is it that you will employ this strategy?

A.3.4.2. Intervention efficacy item (only in the neutral victim IVE)How easy is it for you to employ this strategy?

References

Atlas, R. S., & Pepler, D. J. (1998). Observations of bullying in the classroom. The Journal of Educational Research, 92, 86–99. http://dx.doi.org/10.1080/00220679809597580.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.Bouman, T., Van der Meulen, M., Goossens, F. A., Olthof, T., Vermande, M. M., & Aleva, E. A. (2012). Peer and self-reports of victimization and bullying: Their

differential association with internalizing problems and social adjustment. Journal of School Psychology, 50, 759–774. http://dx.doi.org/10.1016/j.jsp.2012.08.004.

Camodeca, M., & Goossens, F. A. (2005). Children's opinions on effective strategies to cope with bullying: The importance of bullying role and perspective.Educational Research, 47, 93–105. http://dx.doi.org/10.1080/0013188042000337587.

Card, N. A., Stucky, B. D., Sawalani, G. M., & Little, T. D. (2008). Direct and indirect aggression during childhood and adolescence: A meta-analytic review of genderdifferences, intercorrelations, and relations to maladjustment. Child Development, 79, 1185–1229. http://dx.doi.org/10.1111/j.1467-8624.2008.01184.x.

Duffy, A. L., & Nesdale, D. (2009). Peer groups, social identity, and children's bullying behavior. Social Development, 18, 121–139. http://dx.doi.org/10.1111/j.1467-9507.2008.00484.x.

Espelage, D. L., & Swearer, S. M. (2003). Research on school bullying and victimization: What have we learned and where do we go from here? School PsychologyReview, 32, 365–383 (Retrieved from ERIC database. (EJ823560)).

Gini, G., Albiero, P., Benelli, B., & Altoè, G. (2008). Determinants of adolescents' active defending and passive bystanding behavior in bullying. Journal ofAdolescence, 31, 93–105. http://dx.doi.org/10.1016/j.adolescence.2007.05.002.

682 J. Pronk et al. / Journal of School Psychology 51 (2013) 669–682

Gini, G., Pozzoli, T., Borghi, F., & Franzoni, L. (2008). The role of bystanders in students' perception of bullying and sense of safety. Journal of School Psychology, 46,617–638. http://dx.doi.org/10.1016/j.jsp.2008.02.001.

Goossens, F. A., Olthof, T., & Dekker, P. (2006). The new participant role scales: A comparison between various criteria for assigning roles and indications for theirvalidity. Aggressive Behavior, 32, 343–357. http://dx.doi.org/10.1002/ab.20133.

Greene, M. B. (2006). Bullying in schools: A plea for measure of human rights. Journal of Social Issues, 62, 63–79. http://dx.doi.org/10.1111/j.1540-4560.2006.00439.x.

Hawkins, D. L., Pepler, D. J., & Craig, W. M. (2001). Naturalistic observations of peer interventions in bullying. Social Development, 10, 512–527. http://dx.doi.org/10.1111/1467-9507.00178.

Hodges, E. V. E., Boivin, M., Vitaro, F., & Bukowski, W. M. (1999). The power of friendship: Protection against an escalating cycle of peer victimization.Developmental Psychology, 35, 94–101. http://dx.doi.org/10.1037/0012-1649.35.1.94.

Hoffman, M. L. (1977). Sex differences in empathy and related behaviors. Psychological Bulletin, 84, 712–722. http://dx.doi.org/10.1037/0033-2909.84.4.712.Hox, J. J. (2010). Multilevel modeling: Techniques and applications (2nd ed.)New York, NY: Routledge.Kanetsuna, T., Smith, P. K., & Morita, Y. (2006). Coping with bullying at school: Children's recommended strategies and attitudes to school-based interventions in

England and Japan. Aggressive Behavior, 32, 570–580. http://dx.doi.org/10.1002/ab.20156.Kärnä, A., Voeten, M., Little, T. D., Poskiparta, E., Kaljonen, A., & Salmivalli, C. (2011). A large-scale evaluation of the KiVa antibullying program. Child Development,

82, 311–330. http://dx.doi.org/10.1111/j.1467-8624.2010.01557.x.Kreft, I. G., & De Leeuw, J. (1998). Introducing multilevel modeling. Thousand Oaks, CA: Sage.Levine, M., Cassidy, C., Brazier, G., & Reicher, S. (2002). Self-categorization and bystander non-intervention: Two experimental studies. Journal of Applied Social

Psychology, 32, 1452–1463. http://dx.doi.org/10.1111/j.1559-1816.2002.tb01446.x.Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social

Sciences, 1, 86–92. http://dx.doi.org/10.1027/1614-1881.1.3.86.Nansel, T. R., Overpeck, M. D., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001). Bullying behaviors among US youth: Prevalence and associations

with psychosocial adjustment. Journal of the American Medical Association, 285, 2094–2100. http://dx.doi.org/10.1001/jama.285.16.2094.O'Connell, P., Pepler, D., & Craig, W. (1999). Peer involvement in bullying: Issues and challenges for intervention. Journal of Adolescence, 22, 437–452.

http://dx.doi.org/10.1006/jado.1999.0238.Oh, I., & Hazler, R. J. (2009). Contributions of personal and situational factors to bystanders' reactions to school bullying. School Psychology International, 30,

291–310. http://dx.doi.org/10.1177/0143034309106499.Olthof, T., & Goossens, F. A. (2008). Bullying and the need to belong: Early adolescents' bullying-related behavior and the acceptance they desire and receive from

particular classmates. Social Development, 17, 24–46. http://dx.doi.org/10.1111/j.1467-9507.2007.00413.x.Olthof, T., Goossens, F. A., Vermande, M. M., Aleva, E. A., & Van der Meulen, M. (2011). Bullying as strategic behavior: Relations with desired and acquired

dominance in the peer group. Journal of School Psychology, 49, 339–359. http://dx.doi.org/10.1016/j.jsp.2011.03.003.Olweus, D. (1993). Bullying at school: What we know and what we can do. Oxford, UK: Blackwell.Olweus, D. (2010). Understanding and researching bullying: Some critical issues. In S. R. Jimerson, S. M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying in

schools: An international perspective (pp. 9–33). New York, NY: Routledge.Orpinas, P., & Horne, A. M. (2010). Creating a positive school climate and developing social competence. In S. R. Jimerson, S. M. Swearer, & D. L. Espelage (Eds.),

Handbook of bullying in schools: An international perspective (pp. 49–59). New York, NY: Routledge.Pellegrini, A. D. (2002). Affiliative and aggressive dimensions of dominance and possible functions during early adolescence. Aggression and Violent Behavior, 7,

21–31. http://dx.doi.org/10.1016/S1359-1789(00)00033-1.Penner, L. A., Dovidio, J. F., Piliavin, J. A., & Schroeder, D. A. (2005). Prosocial behavior: Multilevel perspectives. Annual Review of Psychology, 56, 365–392.

http://dx.doi.org/10.1146/annurev.psych.56.091103.070141.Peugh, J. L. (2010). A practical guide to multilevel modeling. Journal of School Psychology, 48, 85–112. http://dx.doi.org/10.1016/j.jsp.2009.09.002.Piliavin, J. A., Dovidio, J. F., Gaertner, S., & Clark, R. D., III (1981). Emergency intervention. New York: Academic Press.Polanin, J. R., Espelage, D. L., & Pigott, T. D. (2012). A meta-analysis of school-based bullying prevention programs' effects on bystander intervention behavior.

School Psychology Review, 41, 47–65 (Retrieved from ERIC database. (EJ977426)).Pozzoli, T., & Gini, G. (2010). Active defending and passive bystanding behavior in bullying: The role of personal characteristics and perceived peer pressure.

Journal of Abnormal Child Psychology, 38, 815–827. http://dx.doi.org/10.1007/s10802-010-939909.Reijntjes, A., Vermande, M. M., Goossens, F. A., Olthof, T., Van der Schoot, R., Aleva, E. A., et al. (2013). Developmental trajectories of bullying and social dominance

in youth. Child Abuse & Neglect, 37, 224–234. http://dx.doi.org/10.1016/j.chiabu.2012.12.004.Rigby, K. (2008). Children and bullying: How parents and educators can reduce bullying at school. Malden, MA: Blackwell Publishing.Rigby, K., & Johnson, B. (2006). Expressed readiness of Australian schoolchildren to act as bystanders in support of children who are being bullied. Educational

Psychology, 26, 425–440. http://dx.doi.org/10.1080/01443410500342047.Rivers, I., & Smith, P. K. (1994). Types of bullying behavior and their correlates. Aggressive Behavior, 20, 359–368. http://dx.doi.org/10.1002/1098-2337(1994).Salmivalli, C. (1999). Participant role approach to school bullying: Implications for interventions. Journal of Adolescence, 22, 453–459. http://dx.doi.org/10.1006/

jado.1999.0239.Salmivalli, C., Kärnä, A., & Poskiparta, E. (2010). From peer putdowns to peer support: A theoretical model and how it translated into a national anti-bullying

program. In S. R. Jimerson, S. M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying in schools: An international perspective (pp. 441–454). New York, NY:Routledge.

Salmivalli, C., Lagerspetz, K. M. J., Björkqvist, K., Österman, K., & Kaukiainen, A. (1996). Bullying as a group process: Participant roles and their relations to socialstatus within the group. Aggressive Behavior, 22, 1–15. http://dx.doi.org/10.1002/(SICI)1098-2337(1996).

Salmivalli, C., Lappalainen, M., & Lagerspetz, K. M. J. (1998). Stability and change of behavior in connection with bullying in schools: A two-year follow-up.Aggressive Behavior, 24, 205–218. http://dx.doi.org/10.1002/(SICI)1098-2337(1998).

Salmivalli, C., & Peets, K. (2009). Bullies, victims, and bully–victim relationships in middle childhood and early adolescence. In K. H. Rubin, W. M. Bukowski, & B.Laursen (Eds.), Handbook of peer interaction, relationships, and groups (pp. 322–340). New York, NY: Guilford.

Salmivalli, C., & Voeten, M. (2004). Connections between attitudes, group norms, and behavior in bullying situations. International Journal of BehavioralDevelopment, 28, 246–258. http://dx.doi.org/10.1080/01650250344000488.

Snijders, T. A. B., & Bosker, R. J. (2004). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London, UK: Sage.Sutton, J., & Smith, P. K. (1999). Bullying as a group process: An adaptation of the participant role approach. Aggressive Behavior, 25, 97–111. http://dx.doi.org/

10.1002/(SICI)1098-2337(1999).Tisak, M. S., & Tisak, J. (1996). Expectations and judgments regarding bystanders' and victims' responses to peer aggression among early adolescents. Journal of

Adolescence, 19, 383–392. http://dx.doi.org/10.1006/jado.1996.0036.Trach, J., Hymel, S., Waterhouse, T., & Neale, K. (2010). Bystander responses to school bullying: A cross-sectional investigation of grade and sex differences.

Canadian Journal of School Psychology, 25, 114–130. http://dx.doi.org/10.1177/0829573509357553.Ttofi, M. M., & Farrington, D. P. (2011). Effectiveness of school-based programs to reduce bullying: A systematic and meta-analytic review. Journal of Experimental

Criminology, 7, 27–56. http://dx.doi.org/10.1007/s11292-010-9109-1.Twemlow, S. M., Vernberg, E., Fonagy, P., Biggs, B. K., Nelson, J. M., Nelson, T. D., et al. (2010). A school climate intervention that reduces bullying by a focus on the

bystander audience rather than the bully and victim: The peaceful schools project of the Menninger Clinic and Baylor College of Medicine. In S. R. Jimerson, S.M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying in schools: An international perspective (pp. 365–375). New York, NY: Routledge.

Witvliet, M., Olthof, T., Hoeksma, J. B., Goossens, F. A., Smits, M. S. I., & Koot, H. M. (2010). Peer group affiliation of children: The role of perceived popularity,likeability, and behavioral similarity in bullying. Social Development, 19, 285–303. http://dx.doi.org/10.1111/j.1467-9507.2009.00544.x.