Journal of Research in Personality 53 (2014) 84–92

Contents lists available at ScienceDirect

Journal of Research in Personality

journal homepage: www.elsevier .com/ locate/ j rp

The Dark Triad of personality in adolescence: Psychometric propertiesof a concise measure and associations with adolescent adjustment froma multi-informant perspective

http://dx.doi.org/10.1016/j.jrp.2014.09.0010092-6566/� 2014 Elsevier Inc. All rights reserved.

⇑ Corresponding author. Address: Department of Developmental Psychology,Tilburg University, Postbus 90153, 5000 LE Tilburg, The Netherlands.

E-mail address: theoklimstra@online.nl (T.A. Klimstra).

1 In our discussion of existing empirical studies, we attempted to limit ourstudies with sample sizes over 150, because there are very few instancessamples smaller than that are justifiable when examining correlations (SchöPerugini, 2013).

Theo A. Klimstra a,⇑, Jelle J. Sijtsema a, Jens Henrichs a, Maaike Cima a,b

a Department of Developmental Psychology, Tilburg University, The Netherlandsb Department of Developmental Psychopathology, Nijmegen University, The Netherlands

a r t i c l e i n f o a b s t r a c t

Article history:Available online 9 September 2014

Keywords:Dark TriadPersonalityAggressionConfirmatory Factor Analysis

Accumulating evidence suggests that the Dark Triad of personality (i.e., Narcissism, Psychopathy, andMachiavellianism) is useful in explaining individual differences in adult adjustment. The present studyis among the first to examine unique effects of these traits on multi-informant ratings of adolescentaggressive behavior. In two samples (Ns = 611 and 302), we first established measurement invariance,internal consistency and validity of the 12-item Dirty Dozen Dark Triad measure. Furthermore, we foundunique associations of each Dark Triad trait with particular forms of aggression, with inconsistent resultsacross informants providing greater insight into these traits. Overall, Dark Triad traits seem applicable toadolescents and useful correlates of adolescent adjustment, especially if their unique effects are studied.

� 2014 Elsevier Inc. All rights reserved.

1. Introduction combinations of Big Five traits and specific facets underlying these

In the last decade, the Dark Triad of personality (Narcissism,Psychopathy, and Machiavellianism) has caught a great deal ofattention in studies on (young) adults. Yet, little is known aboutthe role of these maladaptive personality traits in adolescence. Inthe present study, the utility of the Dark Triad in adolescence willbe tested by examining (a) the psychometric properties of a briefDark Triad measure and (b) its associations with direct and indirectforms of aggression based on self-, teacher-, and peer-reports.

1.1. Dark Triad

Studies on personality in normative samples have typicallyfocused on general personality traits like the Big Five (McCrae &Costa, 1987). Because there is increasing evidence for the absenceof a qualitative breach between healthy and unhealthy personality(Bernstein, Iscan, & Maser, 2007), one could suggest that using theBig Five is sufficient for capturing the unhealthy and dysfunctionalpart of the personality trait spectrum. Previous studies have shownthat Big Five traits are indeed reasonably well-able to capture indi-vidual differences at the dysfunctional end of the personality spec-trum (e.g., Miller, Bagby, Pilkonis, Reynolds, & Lynam, 2005).However, dysfunctional behavioral tendencies are often complex

traits. Therefore, several models have been developed to captureindividual differences in dysfunctional personality traits in a moredirect manner. Because of the increasing evidence for a continuumfrom normal and healthy personality to abnormal and unhealthypersonality, traits from these models are increasingly often exam-ined in samples drawn from the general population.

There are several taxonomies that attempt to capture the dys-functional personality traits. Of these taxonomies the Dark Triadhas perhaps received the most attention in the last ten years, atleast within the social psychology and personality psychology lit-erature (Jonason, Webster, Schmitt, Li, & Crysel, 2012). This modelincludes the traits of Narcissism, Psychopathy, and Machiavellian-ism (Paulhus & Williams, 2002). Narcissism refers to a tendency tohave grandiose (i.e., inflated) self-views supplemented by intrapsy-chic and interpersonal strategies to maintain these self-views(Twenge & Campbell, 2003). Individuals exhibiting high levels ofPsychopathy tend to be impulsive as well as low on empathy(Paulhus & Williams, 2002). Finally, Machiavellianism refers to atendency to exploit others to one’s own advantage by adopting amanipulative interpersonal style (cf. Wilson, Near, & Miller, 1996).

Several studies have shown that Dark Triad traits are stronglyassociated with Big Five traits (Jonason & Webster, 2010; Lee &Ashton, 2005; Paulhus & Williams, 2002). Across these studies,1

selves toin whichnbrodt &

T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92 85

Agreeableness has been shown to be consistently negatively associ-ated with Machiavellianism and Psychopathy. Conscientiousness isconsistently negatively associated with Machiavellianism, and thereis some evidence for a negative association with Psychopathy. Extra-version is consistently positively associated with Narcissism. Someevidence has been found for a negative association of Neuroticismwith Psychopathy (Jonason & Webster, 2010; Paulhus & Williams,2002), but generally this Big Five trait is not consistently associatedwith the Dark Triad. Openness is also not consistently associated withthe Dark Triad. These associations indicate that the Dark Triad as awhole captures the antagonistic (i.e., non-agreeable) side of personal-ity, while each trait has some additional unique associations with BigFive personality traits (Furnham, Richards, & Paulhus, 2013).

The associations with the Big Five highlight that the Dark Triadtraits are conceptually different from each other but also overlap inthe sense that they all capture certain socially undesirable tenden-cies (Jonason & Webster, 2010). Specifically, Dark Triad traitsshare about 50% of their variance (e.g., Jonason, Li, & Buss, 2010).Therefore, when studying Dark Triad traits it is recommended toinclude all three traits to be able to examine which effects aredue to the shared variance between Narcissism, Machiavellianism,and Psychopathy, and which effects are due to their unique vari-ance (Jonason & Webster, 2010).

Previous research including the entire Dark Triad in a singlestudy on adolescents is sparse. This may be due to the fact thatmeasuring these traits with traditional separate questionnaires forPsychopathy, Machiavellianism, and Narcissism means that 91items have to be used (Webster & Jonason, 2013). Recently, a briefalternative for these lengthy measures has been developed: Thetwelve-item Dirty Dozen questionnaire (Jonason & Webster,2010). Compared to using three separate questionnaires, the DirtyDozen has the additional advantage of using the same rating scalefor each of the Dark Triad traits. Furthermore, the psychometricproperties of the subscales of the Dirty Dozen appear to be betterthan those of instruments that have been used to measure thesetraits separately. For example, the Dirty Dozen uses a likert-typescale for Narcissism instead of the dichotomous items that are usedin the Narcissistic Personality Inventory (Raskin & Terry, 1988) thathas traditionally been employed to tap this construct. In addition,the most commonly used Machiavellianism measure (MACH IV;Christie & Geis, 1970) frequently yields relatively low Cronbach’sAlphas, which points to internal consistency issues (Jonason &Webster, 2010).

Obviously, the brevity of the Dirty Dozen comes with somecosts regarding the bandwidth of its subscales (Miller et al.,2012). Nevertheless, it provides a viable and practical alternativeto lengthy measures tapping Dark Triad traits separately (e.g.,Jonason & Luéveno, 2013). Yet, so far, only one study employedthe Dirty Dozen measure in an adolescent sample (Muris,Meesters, & Timmermans, 2013). This study used a relatively smallsample (N = 117) and the Dirty Dozen measure was not formallyvalidated.

1.2. The role of age and gender

It has long been questioned whether self-reports on personalitytraits in adolescents should be used, but in the last decade therehas been increasing evidence regarding the validity and reliabilityof adolescent self-reports on Big Five traits (Soto, John, Gosling, &Potter, 2008) and pathological traits (De Clercq, De Fruyt, VanLeeuwen, & Mervielde, 2006; De Clercq et al., 2014; Tromp &Koot, 2010). Adolescent self-reports on personality already seemreasonably valid and reliable in ten-year olds (Soto et al., 2008).However, the older adolescents get, the more differentiated theirview on personality (i.e., the lower the average correlationsbetween traits) becomes (Allik, Laidra, Realo, & Pullmann, 2004;

Soto et al., 2008). Specifically, traits such as Agreeableness andConscientiousness may be highly correlated in young adolescents,but such correlations tend to drop as adolescents grow older. Thissuggests that older adolescents are better able to distinguish, forexample, prosocial behavior from organized and planful behavioraltendencies. Given that the Dark Triad traits are strongly associatedyet distinguishable entities, one might expect that older adoles-cents provide increasingly more differentiated self-ratings on theDark Triad as they grow older. Information on trait differentiationis important in order to better understand whether the nature ofthe Dark Triad is similar or changes throughout adolescence.

In addition to age effects on the structure of the Dark Triad,there may also be gender effects. In general, it is getting increas-ingly common in psychology to test for measurement invarianceacross groups in order to examine whether results obtained inone group (e.g., men) can be compared to those obtained inanother group (e.g., women). Such tests are crucial in order to besure that, for example, gender differences truly represent differ-ences between men and women, and not just measurement error(cf. van de Schoot, Lugtig, & Hox, 2012). Unfortunately, thereappear to be no studies that have established measurement invari-ance across gender for Dark Triad measures. Still, gender differ-ences in the Dark Triad have been examined in studies assessingthese traits collectively in adults. Although there were some incon-sistencies across studies, men tend to score higher than women onall three Dark Triad traits (Furnham et al., 2013). The one studythat examined these traits collectively in adolescents (Muriset al., 2013), and studies solely focusing on adolescent Narcissism(e.g., Ojanen, Findley, & Fuller, 2012), Psychopathy (e.g.,Decuyper, De Bolle, De Fruyt, & De Clercq, 2011), and Machiavel-lianism (e.g., Peters, Cillessen, & Scholte, 2010) had similar results.However, these gender differences were not large. Additionally andas previously mentioned, measurement invariance across genderhad not been established in studies, which makes the validity ofthese gender differences disputable.

1.3. Associations with direct and indirect aggression

Research examining linkages of personality traits with adoles-cent adjustment is flourishing. However, such studies (e.g., DeBolle, Beyers, De Clercq, & De Fruyt, 2012; Klimstra, Luyckx, Hale,& Goossens, 2014) tend to focus on Big Five traits and do notexplicitly examine the complex combinations of these traitsreflected in the Dark Triad.

In studies linking the Dark Triad traits to adjustment, the asso-ciations of these traits with different subtypes of aggression havebeen of great interest. Typically, overt or direct types (i.e., openphysical or verbal forms) of aggression are distinguished from rela-tional or indirect types (i.e., gossiping, spreading rumors) ofaggression (e.g., Björkqvist, Lagerspetz, & Kaukiainen, 1992; Card,Stucky, Sawalani, & Little, 2008; Heilbron & Prinstein, 2008). Thereis a considerable literature supporting the distinction betweenthese subtypes of aggression. The divergent validity has, for exam-ple, been shown in studies on gender differences. A meta-analysis(Archer, 2004) revealed that boys consistently score higher onmeasures of direct aggression. Gender differences in indirectaggression are (depending on the method) either non-existent orsuggest that girls score slightly higher than boys. A recent study(Tackett, Herzhoff, Reardon, De Clercq, & Sharp, 2014) suggeststhat these types of aggression are differentially associated withdysfunctional trait domains of Disagreeableness and EmotionalInstability and their facets. These results point to the potential roleof personality in explaining individual differences in direct andindirect aggression.

Dark Triad traits have also been linked to direct and indirectaggression in previous studies. Psychopathic traits (or conceptually

86 T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92

similar Callous–Unemotional traits) have been shown to bepositively associated with both forms of aggression (Kerig &Stellwagen, 2010; Lau & Marsee, 2013). However, there are at leastsome studies suggesting that the links with physical aggression arestronger and more replicable (Coyne & Thomas, 2008; Stickle,Marini, & Thomas, 2013).

Across studies, Narcissism was often linked to direct forms ofaggression (Bukowski, Schwartzman, Santo, Bagwell, & Adams,2009; Kerig & Stellwagen, 2010; Lau & Marsee, 2013). However,across the board linkages of Narcissism with indirect (relational)forms of aggression appeared to be stronger. It should be noted thatthere is also at least one study (Ojanen et al., 2012) that found no sig-nificant association of Narcissism with direct or indirect aggression.

Overall, a similar pattern of findings emerged for Machiavel-lianism, as this trait was also related to both forms of aggres-sion (Lau & Marsee, 2013), but more strongly to indirectaggression than to direct aggression (Kerig & Stellwagen,2010). The fact that Machiavellianism mediated the associationbetween Narcissism and indirect (relational) aggression pointsout that its crucial to account overlap between Dark Triad traitswhen examining their associations with aggression (Kerig &Stellwagen, 2010). Unfortunately, there are few studies consider-ing all three Dark Triad traits simultaneously in predicting directand indirect aggression.

Another issue is that different studies tend to use differentreporters (e.g., self-reports on aggression in Lau and Marsee(2013), teacher reports on aggression in Kerig and Stellwagen(2010)). This may explain some of the inconsistencies in the resultsof those studies, as due to biases different reporters may providevery different information on aggression (cf., Tackett, Waldman,& Lahey, 2009). However, it may also be that particular informantssimply do not have access to all the contexts in which target ado-lescents may display aggression (e.g., Card et al., 2008). For exam-ple, teachers are unlikely to know how much aggression childrendisplay at home. To get insight into informant biases and to getthe most accurate and complete information regarding aggression,multi-informant data should be employed (e.g., Kraemer et al.,2003).

In a large-scale study, Hawley (2003) employed this strategy byexamining associations of Machiavellianism with self-, teacher-,and peer-reported aggression. Individuals classified as Machiavel-lians (i.e., labeled bi-strategic controllers by Hawley (2003)) hadelevated levels of self- and peer-reported aggression, but averagelevels of teacher-reported aggression. They also were perceivedas popular by their peers. Machiavellians may be high on perceivedpopularity because they adequately combine pro- and antisocialbehavior, as suggested by a study on perceived popularity(Dijkstra, Lindenberg, Verhulst, Ormel, & Veenstra, 2009). That is,adolescents high on both prosocial and antisocial behaviors weremore often perceived as popular compared to those who only dis-played antisocial behavior.

Until now, there appears to be only one study that used multi-informant data on adolescent aggression and that considered allDark Triad traits simultaneously. That study (Muris et al., 2013)only found unique associations of Psychopathy and Machiavellian-ism with aggression. However, a relatively small sample wasemployed, direct and indirect aggression were not distinguished,and only self- and parent-reports of aggression were used. This isa limitation, because adolescent aggression is also common inclassrooms, and peers and teachers are likely to provide moreaccurate information on aggression regarding that context. In thisregard, peer-nomination techniques in which each child in a class-room reports on the aggressive tendencies of every other child inthe classroom may be particularly informative. Thus, further studyof the linkages between adolescent aggression and the Dark Triadof personality is needed.

1.4. The present study

The main goal of the present study is to demonstrate the utilityof the Dark Triad of personality for research on adolescent adjust-ment. To this end, the psychometric properties of a concise DarkTriad measure (i.e., the 12-item Dirty Dozen; Jonason & Webster,2010) will be evaluated by examining its factor structure, testingfor measurement invariance across gender and gender differencesin mean levels, and examining associations with Big Five personal-ity traits in a sample of older adolescents (16.8 years old on aver-age) and a sample young adolescents (12.8 years old on average).These data allow for studying whether the psychometric propertiesof the measure are sound at different ages in adolescence. Based onprevious studies of young adult samples (typically >20 years old),we expect that a three-factor model with separate factors of Nar-cissism, Machiavellianism, and Psychopathy will provide the mostoptimal fit. Based on aforementioned previous studies on adoles-cent Big Five traits, we do expect that the Dark Triad traits willbe stronger associated with one another in the younger samplewhen compared to the older sample. Such a trait-differentiationprocess has not yet been tested for the Dark Triad.

The present study will be the first to use rigorous scalar (i.e.,strict) invariance tests to examine whether this model appliesequally to (i.e., is invariant across) both gender groups. We expectthat this will be the case. In line with previous research, boys areexpected to score higher than girls on all three traits. Regardingassociations with Big Five traits, we expect a negative associationof Agreeableness with all Dark Triad traits, a negative associationof Conscientiousness with Machiavellianism and Psychopathy, apositive association of Extraversion with Narcissism, and a nega-tive association of Neuroticism with Psychopathy.

In addition, the present study will be the first to link multi-informant ratings of both direct and indirect aggression pro-vided by adolescents themselves, their teachers, and their peersto adolescent Dark Triad personality traits while accounting forthe conceptual overlap between these three traits. As all threeDark Triad traits represent a part of the maladaptive trait spaceand reflect antisocial tendencies, they should all be related tosome form of aggressive behavior. However, all three traits rep-resent somewhat different antisocial tendencies, and thereforedifferential associations with direct and indirect aggression canbe expected. Specifically, based on the aforementioned studies,we expect Psychopathy to be associated with both forms ofaggression, but more strongly with direct aggression than withindirect aggression. Both Narcissism and Machiavellianism areexpected to be associated with both forms of aggression, butmore strongly with indirect aggression than with direct aggres-sion. The availability of aggression data from three informantsallows us to gain insight into unique and shared perceptionsregarding aggression of adolescents themselves, their teachers,and their peers.

2. Method

2.1. Participants

Sample 1 consisted of 611 adolescents (54.2% girls;Mage = 16.79 years, SD = 0.87) attending various high schools inthe Dutch-speaking part of Belgium (Flanders). Sample 2 consistedof 302 adolescents (48.3% girls; Mage = 12.79, SD = 0.78) participat-ing in the Study of Personality, Adjustment, Cognition, and Emo-tions (SPACE) conducted at two high schools in the Netherlands.Big Five, Dark Triad, and self-reported aggression data were avail-able in both samples; data on teacher- and peer-reported aggres-sion were only available in Sample 2.

T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92 87

2.2. Procedure

Both studies were conducted in accordance with the guidelinesof the local IRBs. That is, we first obtained permission from schoolprincipals to administer questionnaires during class time. Second,parents were informed via a detailed letter describing the studycontent and goals, and were given the opportunity to object totheir children’s participation. After we received parental permis-sion, students were informed about the study and asked whetherthey wished to participate. Psychology master students visitedthe schools and asked adolescents to fill out the questionnaire. Inaddition, these students were available to answer questions withregard to the meaning of the items that were administered.

2.3. Measures

2.3.1. Dark Triad of personalityDark Triad personality traits were self-reported by adolescents

using a Dutch version of the Dirty Dozen (Jonason & Webster,2010). The Dirty Dozen measures Narcissism (e.g., ‘I tend to seekprestige or status’), Machiavellianism (e.g., ‘I tend to manipulateothers to get my way’), and Psychopathy (e.g., ‘I tend to lackremorse’) with 4 items each rated on a 9-point scale ranging from1 (‘strongly disagree’) to 9 (‘strongly agree’). All 12 items were firsttranslated by two Dutch-speaking researchers with an intimateknowledge of English. The two translations were largely identical.In case of minor discrepancies, the final translation was chosen byconsensus. Similarly, the Dutch version was translated back intoEnglish by two other Dutch-speaking researchers with an intimateknowledge of English. The back-translated items closely resembledthe original English-language items.

2.3.2. Big Five of personalityBig Five personality dimensions were self-rated by adolescents

with the Dutch version (Denissen, Geenen, van Aken, Gosling, &Potter, 2008) of the Big Five Inventory (John, Donahue, & Kentle,1991). The Big Five dimensions of Neuroticism (8 items; e.g., ‘Isee myself as someone who worries a lot’), Extraversion (8 items;e.g., ‘I see myself as someone who is talkative’), Openness (10items; e.g., ‘I see myself as someone who is original, comes withnew ideas’), Agreeableness (9 items; e.g., ‘I see myself as someonewho has a forgiving nature’), and Conscientiousness (9 items; e.g.,‘I see myself as someone who does a thorough job’) were assessedon a 5-point scale ranging from 1 (strongly disagree) to 5 (stronglyagree). Internal consistency was acceptable, as Cronbach’s Alphasacross all Big Five scales ranged from .72 to .82 in Sample 1 andfrom .68 to .79 in Sample 2.

2.3.3. Self-reported aggressionSelf-reported aggression was measured with the Dutch version

(Hale, VanderValk, Akse, & Meeus, 2008) of the Direct and indirectaggression scale (Björkqvist et al., 1992). In this study we used twosubscales of this questionnaire: (a) direct aggression (5 items; e.g.,‘When I’m mad at a classmate, I will kick or strike him/her’) and(b) indirect aggression (12 items; e.g., ‘When I’m mad at a class-mate, I will spread vicious rumors as revenge’). Items were scoredon a 4-point scale, ranging from ‘never’ to ‘very often’. Cronbach’sAlphas for the direct and indirect aggression subscales were .83and .80, respectively, in Sample 1 and .87 and .83 in Sample 2.

2.3.4. Teacher-reported aggressionTeachers reported on adolescent direct and indirect aggression

in Sample 2 only. They were asked to fill out the Children’s SocialBehavior Checklist-Teacher Form (CSBC-T; Crick, 1996). In thisstudy, we used direct aggression (4 items; e.g., ‘This child hits,shoves, or pushes peers’) and indirect aggression (7 items; e.g.,

‘This child spreads rumors or gossips about some peers’) subscales.Items were scored on a 5-point scale, ranging from ‘do not agree atall’ to ‘very much agree’. Cronbach’s Alphas for direct and indirectaggression subscales were .94 and .91, respectively.

2.3.5. Peer-reported aggressionIn Sample 2, peers reported on direct and indirect aggression by

nominating classmates ‘who quarrel or fight a lot with others’ and‘who often gossip or exclude others’, respectively. Adolescentsreceived a roster with the names of all classmates listed. Amongstothers, adolescents could nominate same- and cross-sex class-mates who behaved aggressively. Standardized measures for directand indirect aggression were constructed by dividing the numberof nominations by the number of classmates.

3. Results

3.1. Internal consistency, factor structure, measurement invariance,and gender differences

First, we examined the internal consistency of the Dirty Dozenamong adolescents. As Cronbach’s Alphas were high for subscalesof Narcissism (.85 and .84 in Samples 1 and 2, respectively), Machi-avellianism (.79 and .74 in Samples 1 and 2, respectively), and Psy-chopathy (.74 in both samples), internal consistency wassatisfactory.

Second, we examined the factor structure of the Dutch-lan-guage Dirty Dozen among adolescents. For this purpose, we usedConfirmatory Factor Analyses (CFAs) in Mplus 7 (Muthén &Muthén, 2012) using Maximum Likelihood Robust (MLR) estima-tion. First, we conducted model comparisons to examine whetherour data favored the proposed three-factor structure over a moreparsimonious one-factor structure. For such comparisons, the useof multiple criteria has been advocated (e.g., Vandenberg &Lance, 2000), as different criteria provide information on differentsources of model misspecification. We relied on three commonlyused criteria to compare nested models: a scaled v2-difference test(Satorra & Bentler, 2001), which should be non-significant, and thedelta (D) Comparative Fit Index (CFI) and delta (D) Root MeansSquare Error of Approximation (RMSEA), which should be <.010and <.015, respectively (Chen, 2007). Only if two of these criteriawere satisfied, we would favor the less parsimonious three-factormodels over the more parsimonious one-factor models. Absolutefit indices of the various models were also taken into account, withCFIs of .90 and larger and RMSEAs of .08 and smaller consideredsatisfactory (Kline, 2005).

Table 1 reveals that in both samples three-factor models fittedthe data significantly better than one-factor models (ps scaledv2-difference tests <.05, DCFIs >.010, and DRMSEAs >.015). Thefinal three-factor models also had satisfactory absolute fits to thedata, although we had to allow error correlations between twoitems (i.e., ‘I tend to want others to admire me’ and ‘I tend to wantothers to pay attention to me’) to get the CFI across the .90 bench-mark in both samples. Despite that three-factor models clearly fit-ted the data better than one-factor models, Table 2 indicates thatcorrelations among the three latent factors were high in both sam-ples. Subsequent comparisons based on Fisher’s r-to-z transforma-tion showed that all correlations between the three latent factorswere higher in Sample 2 than in Sample 1 (ps < .001). Factor load-ings of the resulting three-factor models for both samples are alsoshown in Table 2.

In the next step, we examined measurement invariance acrossgender groups. Several levels of measurement invariance can bedistinguished (Vandenberg & Lance, 2000). For valid mean-levelcomparisons between groups one needs to establish configural

Table 1Confirmatory Factor Analyses Dirty Dozen.

Sample 1 (N = 611) Sample 2 (N = 302)

v2 df CFI RMSEA (90% CI) v2 df CFI RMSEA (90% CI)

1 Factor 572.812*** 53 .745 .127 (.117–.136) 231.215*** 53 .816 .106 (.092–.120)3 Factors 207.571*** 50 .923 .072 (.062–.082) 135.662*** 50 .912 .075 (.060–.091)

Configural invariance3 Factors boys 132.065*** 50 .923 .077 (.061–.093) 97.987*** 50 .917 .078 (.054–.100)3 factors girls 142.488*** 50 .909 .075 (.061–.089) 93.499*** 50 .896 .077 (.052–.101)

Metric and scalar invariance3 factor multigroup no constraints 275.088*** 100 .916 .076 (.065–.086) 191.109*** 100 .908 .077 (.060–.094)Metric invariance 285.444*** 112 .917 .071 (.061–.081) 209.641*** 112 .901 .075 (.060–.091)Scalar invariance 306.511*** 124 .913 .069 (.060–.079) 235.632*** 124 .887 .077 (.062–.092)Partial scalar invariance 224.068*** 122 .897 .074 (.059–.089)

Note: v2 = Chi-Square; df = degrees of freedom; CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation and 90% Confidence Interval.*** p < .001.

88 T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92

invariance (i.e., the number of factors and the pattern of factorloadings roughly equivalent across different groups), metric invari-ance (i.e., factor loadings statistically equivalent across groups, alsocommonly referred to as strong invariance), and scalar invariance(i.e., intercepts of items statistically equivalent across groups, alsocommonly referred to as strict invariance).

Table 1 indicates that a three-factor model had an acceptable fitfor boys and girls in both samples, although the CFI of the three-factor solution for girls in Sample 2 was slightly below the .90benchmark for acceptable fit. Nevertheless, it can be concludedthat the number of latent factors and the pattern of factor loadingsis roughly equivalent for boys and girls. These results indicate sat-isfactory configural invariance of the Dutch-language Dirty Dozenmeasure.

Next, we tested for metric (i.e., strong) and scalar (i.e., strict)invariance by running a series of multigroup CFAs with boys andgirls as groups. In order to establish metric invariance, we com-pared models in which factor loadings were allowed to differ forboys and girls (i.e., the 3 factors multigroup unconstrained modelin Table 1) to models in which we constrained factor loadings tobe equal for boys and girls (i.e., the metric invariance model inTable 1). Metric invariance models fitted the data just as well asmodels in which factor loadings were not constrained to be equalfor boys and girls (ps scaled v2-difference tests > .05, DCFIs < .010,and DRMSEAs < .015 in both samples). As such, our findings dem-onstrate metric, or strong, invariance.

Table 2Standardized factor loadings of the final three-factor Confirmatory Factor Analysis solutio

Item Sample 1

M

I tend to manipulate others to get my way .724I have used deceit or lied to get my way .740I have used flattery to get my way .644I tend to exploit others toward my own end .695I tend to lack remorseI tend to be unconcerned with the morality of my actionsI tend to be callous or insensitiveI tend to be cynicalI tend to want others to admire meI tend to want others to pay attention to meI tend to seek prestige or statusI tend to expect special favors from others

IntercorrelationsMachiavellianismPsychopathyNarcissism

Note: M = Machiavellianism, P = Psychopathy, N = Narcissism. All factor loadings and int

To test for scalar invariance, we compared metric invariancemodels (in which item intercepts were still allowed to vary acrossgender) to models in which we additionally constrained item inter-cepts to be equal for boys and girls (i.e., scalar invariance models).In Sample 1, we found full scalar (i.e., strict) invariance, asmetric invariance and scalar invariance models fitted equally well(p scaled v2-difference tests > .05, DCFIs < .010, andDRMSEAs < .015). In Sample 2, we were only able to establish par-tial scalar invariance, as intercepts of item 4 (‘I tend to exploit oth-ers toward my own end’; Machiavellianism) and item 11 (‘I tend toseek prestige or status’; Narcissism) were significantly different forboys and girls.

For observed scale scores, mean comparisons are no longervalid if only partial measurement invariance can be established(Steinmetz, 2013). However, latent means can still be comparedas long as at least two of the intercepts and factor loadings ofone latent factor can be constrained to be equal across groups(Steenkamp & Baumgartner, 1998). As this was the case, subse-quent mean comparisons across gender will be based on latentrather than observed means.

With latent mean comparisons, the latent mean of the referencegroup (i.e., boys) is zero. In this case, if the latent mean for girls issignificantly different from zero (p < .05), there is a significant dif-ference between boys and girls in latent means of Dark Triad traits.In Sample 1, latent means of girls were significantly lower onMachiavellianism (M = �.241, SE = .092; p = .009) and Psychopathy

ns and correlations among latent factors in Sample 1 and Sample 2.

Sample 2

P N M P N

.665

.666

.644

.686.702 .660.637 .563.758 .789.562 .701

.728 .750

.735 .636

.754 .774

.743 .804

.684 .627 .824 .835.320 .584

ercorrelations were significant at p < .001.

T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92 89

(M = �.204, SE = .093; p = .029) when compared to boys. In Sample2, we found a statistical trend for Machiavellianism suggesting thatgirls again had lower mean levels when compared to boys(M = �.243, SE = .139; p = .079). The mean difference in Psychopa-thy that we found in Sample 1 was fully replicated in Sample 2,as girls again had lower mean levels than boys (M = �.331;SE = .134; p = .014). In addition, we found that girls had signifi-cantly lower levels of Narcissism in Sample 2 (M = �.312;SE = .138; p = .023).

3.2. Associations with Big Five personality traits

We first examined bivariate correlations between Dark Triadtraits and Big Five personality traits in both samples. However,because Dark Triad traits are known to be substantially associatedwith one another, we also ran a series of structural equation mod-els (SEM) in Mplus 7 (Muthén & Muthén, 2012) in both samples toexamine unique associations between Big Five personality traitsand Dark Triad traits. In these models, Big Five traits served asindependent variables and Dark Triad traits served as dependentvariables. Associations between independent variables, betweendependent variables, and predictive paths from independent todependent variables were all estimated. Therefore, the model hadzero degrees of freedom and was a so-called fully saturated model.Such models yield no meaningful absolute fit statistics.

Table 3 shows that Agreeableness was negatively associatedwith all Dark Triad traits in both samples. Conscientiousness wasnegatively associated with Machiavellianism and Psychopathy inboth samples. Extraversion was positively associated with Machia-vellianism and Narcissism in both samples, and positively associ-ated with Psychopathy in Sample 2 only. Neuroticism wasnegatively associated with Psychopathy in both samples, whereasOpenness was positively associated with Machiavellianism in Sam-ple 1 only.

3.3. Associations with aggression

We also used SEM models to test the unique associations of theDark Triad traits with aggression reported by adolescents them-selves, their teachers, and their peers. In these models, Dark Triadtraits were the independent variables whereas aggression mea-sures served as dependent variables.

In a first pair of models, self-reported direct and indirect aggres-sion served as dependent variables. Table 3 shows that indirectaggression was positively associated with Machiavellianism andNarcissism in both samples. Direct aggression was positively asso-ciated with Psychopathy in both samples. In Sample 2, Machiavel-lianism was also positively associated with direct aggression.

The next models were only run in Sample 2, because the vari-ables in these models were not examined in Sample 1. In the firstof these models, teacher-reported direct and indirect aggressionserved as dependent variables. Table 3 shows that direct aggres-sion was uniquely and positively associated with Psychopathy,whereas no unique correlates were found for indirect aggression.

In the next model, peer-reported direct and indirect aggressionserved as dependent variables. No unique correlates of peer-reported direct aggression were found, but Psychopathy was aunique positive correlate of indirect aggression (see Table 3).

4. Discussion

The main purpose of the present study was to demonstrate theutility of the Dark Triad for understanding adolescent develop-ment. Our findings suggest that the Dark Triad can be measuredin a reliable and valid manner among adolescents, and that its

traits contribute to a better understanding of individual differencesin adolescent adjustment.

4.1. Measurement and gender differences of adolescent Dark Triadtraits

Acceptable Cronbach’s Alphas underscored the internal consis-tency of the three subscales representing Machiavellianism, Psy-chopathy, and Narcissism. Furthermore, a series of ConfirmatoryFactor Analyses (CFAs) clearly suggested that, despite substantialinter-correlations between Dark Triad traits, the three traits areseparable entities in adolescents like they are in adults (e.g.,Jonason & Webster, 2010). Interestingly, we found that theintercorrelations between Dark Triad traits were higher in ouryounger sample when compared to the older sample. This is in linewith previous studies on the Big Five (Allik et al., 2004; Soto et al.,2008) that showed that intercorrelations between traits tend todrop as adolescents get older, suggesting that adolescentsgradually get a more differentiated view of personality. Ourstudy extends this line of research by showing that a similartrait differentiation process appears to be applicable to the DarkTriad.

Of particular importance is that measurement invariance testsshowed that the factor structure was similar for boys and girls interms of the pattern of factor loadings (i.e., metric, or strong,invariance) and the pattern of means across items (i.e., scalar, orstrict, invariance). Such rigorous measurement invariance testshad not been applied previously to the concise Dirty Dozen DarkTriad measure (Jonason & Webster, 2010), also not in studies onadults. This is surprising, as establishing measurement invarianceis a crucial step that should to be taken before scores on a measureor associations with external measures (e.g., aggression) can becompared across different groups such as gender groups (e.g.,Vandenberg & Lance, 2000). Our findings thus suggest that gendercomparisons in previous studies likely yielded valid conclusionsthat are not just due to measurement error. Thereby, our study fillsa crucial gap in the accumulating evidence regarding the soundpsychometric qualities of the Dirty Dozen (Jonason & Luéveno,2013; Webster & Jonason, 2013) and adds credibility to claimsregarding gender differences made in previous studies using thesubscales of this measure.

We found gender differences in Dark Triad traits that were inline with previous studies (Decuyper et al., 2011; Jonason &Webster, 2010; Muris et al., 2013). That is, boys tended to scorehigher than girls in most instances, but especially on Psychopathy.There was somewhat less convincing evidence for gender differ-ences in Machiavellianism and Narcissism, as boys scored higheron these traits in one sample, but not in the other sample. This sug-gests that gender differences in Dark Triad traits are in need of fur-ther study. Such research should take a developmental approach,as previous studies on ‘normal’ personality traits (e.g., the Big Five)have shown that gender differences in personality might be pres-ent at some ages in adolescence but not at other ages (e.g., Soto,John, Gosling, & Potter, 2011).

Associations of Dark Triad traits with Big Five traits obtained inadult samples (Jonason & Webster, 2010; Lee & Ashton, 2005;Paulhus & Williams, 2002) were also largely replicated in our sam-ples. Furthermore, we also found a positive association of Extraver-sion with Machiavellianism in both samples. This may be becauseboth Extraversion (e.g., Denissen & Penke, 2008) and Machiavel-lianism (Wilson et al., 1996) have been linked to tendencies tomove oneself into the center of attention and take up leadershiproles. The associations of Dark Triad traits with Big Five traits inthe present study once more underscore the validity of the DirtyDozen and suggest that it is a reliable and valid questionnaire suit-able for measuring Dark Triad traits in adolescents.

Table 3Correlations and SEM regression coefficients.

Sample 1 Sample 2

Machiavellianism Psychopathy Narcissism Machiavellianism Psychopathy Narcissism

r b r b r b r b r b r b

Big FiveE .155⁄⁄⁄ .148⁄⁄⁄ .049 .031 .217⁄⁄⁄ .192⁄⁄⁄ .007 .147⁄ .033 .150⁄⁄ .047 .130⁄

N �.010 �.036 �.088⁄ �.142⁄⁄⁄ .023 .052 .147⁄ .094 .003 �.113⁄ .101 .083O .111⁄⁄ .103⁄⁄ .020 .035 .231⁄⁄⁄ .212⁄⁄⁄ �.133⁄ �.043 �.203⁄⁄⁄ �.081 �.020 .036C �.255⁄⁄⁄ �.209⁄⁄⁄ �.161⁄⁄⁄ �.103⁄⁄ �.045 �.019 �.308⁄⁄⁄ �.198⁄⁄ �.310⁄⁄⁄ �.128⁄ �.190⁄⁄ �.148⁄⁄

A �.357⁄⁄⁄ �.338⁄⁄⁄ �.438⁄⁄⁄ �.441⁄⁄⁄ �.149⁄⁄⁄ �.160⁄⁄⁄ �.346⁄⁄⁄ �.277⁄⁄⁄ �.528⁄⁄⁄ �.540⁄⁄⁄ �.183⁄⁄ �.145⁄⁄

AggressionDirect: self .246⁄⁄⁄ .112 .318⁄⁄⁄ .252⁄⁄⁄ .151⁄⁄ .021 .390⁄⁄⁄ .173⁄ .448⁄⁄⁄ .345⁄⁄⁄ .264⁄⁄⁄ .020Indirect: self .426⁄⁄⁄ .266⁄⁄⁄ .317⁄⁄⁄ .076 .392⁄⁄⁄ .254⁄⁄⁄ .523⁄⁄⁄ .350⁄⁄⁄ .338⁄⁄⁄ .083 .447⁄⁄⁄ .197⁄⁄

Direct: teacher .017 .107 .123⁄ .172⁄ �.034 .042Indirect: teacher .184⁄⁄ �.056 .214⁄⁄⁄ .169 .153⁄⁄ �.046Direct: peer .131⁄ .153 .132⁄ .100 .011 �.123Indirect: peer .057 .059 .144⁄ .164⁄ �.045 �.149

Note: E = Extraversion, N = Neuroticism, O = Openness, C = Conscientiousness, A = Agreeableness. The Big Five traits served as predictors in path models where Dark Triadtraits served as dependent variables, whereas the aggression measures served as dependent variables in path models where Dark Triad traits served as predictors. There wereonly gender differences in the associations of Neuroticism with the Dark Triad in Sample 1. Specifically, Neuroticism was negatively associated with Machiavellianism(b = �.179; p = .002) in boys and positively associated with Machiavellianism in girls (b = .116; p = .016). In addition, Neuroticism was only significantly associated withPsychopathy (b = �.250; p < .001) in boys and not in girls, and only significantly associated with Narcissism in girls (b = .145; p = .003) and not in boys. No further genderdifferences in these associations were found. Associations between the Dark Triad and self-reported aggression in Sample 1 are based on n = 421, because only 421 of the 611adolescents were asked to complete measures of direct and indirect aggression. There were no significant differences in mean levels of the Dark Triad between these 421adolescents and the 190 adolescents that were not asked to complete these measures.

* p < .05.** p < .01.

*** p < .001.

90 T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92

4.2. The Dark Triad and adolescent direct and indirect aggression

Our findings regarding associations of Dark Triad traits withmulti-informant reports on aggression highlight the potential util-ity of these traits in providing greater insight in adolescent adjust-ment. The fact that all Dark Triad traits, which all reflect offensiveand antisocial behavioral tendencies (Paulhus & Williams, 2002),were related to at least some form of aggressive behavior can beinterpreted as evidence for the convergent validity of the DirtyDozen in adolescents. However, these traits were also differentiallyrelated to direct and indirect aggression, which testifies the diver-gent validity of both these forms of aggressive behavior and thethree Dark Triad traits.

Replicable unique associations of Psychopathy with self- andteacher-reported direct aggression were perfectly in line with ourhypotheses. Psychopathy tends to be associated with a lack ofempathic concern (e.g., Paulhus & Williams, 2002), which may leadto egocentric behavior and little concern as well as little insightinto possible consequences of such behavior (cf. Malterer, Glass,& Newmann, 2008). Hence, when trying to get their way, theseindividuals are likely unconcerned about possible adverse effectsof direct aggressive behavior.

Psychopathy was also positively associated with indirectaggression as reported by peers and teachers, but not withself-reported indirect aggression. Using indirect aggression canbe a conscious and efficient coercive strategy, because if it goesundetected, it may save the perpetrator from the adverse socialconsequences of openly exhibited aggression. However, ourresults suggest that individuals with high levels of Psychopathymay be unaware of employing indirect aggressive acts (e.g., gos-siping), whereas peers and teachers do attribute such acts tothese individuals. Possibly, individuals with high levels of Psy-chopathy have problems with understanding others’ perspectiveand affective states (e.g., Pardini, Lochman, & Frick, 2003) andare not even aware of the fact that they hurt others’ feelings.Research on adult psychopathic offenders (Cima, Tonnaer, &Hauser, 2010) complements this interpretation, as they foundthat both non-psychopathic offenders and healthy controls have

the knowledge of moral standards of right and wrong, but psy-chopaths fail moral emotions, which normally guide appropriatebehavior. Therefore, it could be that individuals with higher lev-els of Psychopathy know that they are exhibiting indirectaggression and know that this is wrong, but underestimate theseverity of their behavior because they misjudge its impact.However, more research is needed before the role of moral cog-nitions in the association between Psychopathy and aggressivebehavior can be fully understood.

Our findings further suggest that individuals with high levels ofMachiavellianism, in contrast to individuals with high levels of Psy-chopathy, may deliberately use indirect aggression to get their way.That is, self-reported data indicates that individuals with high levelsof Machiavellianism reported using indirect aggression. However,they appear to exert this aggression in a covert manner, as theirpeers and teachers hardly seem to detect it. This pattern of findingscorresponds to some extent with Hawley’s (2003) definition ofMachiavellians as bistrategic controllers balancing coercive andprosocial behavior. Our study fits this definition, by suggesting thatindividuals with high levels of Machiavellianism implement thecoercive part of their behavior in an indirect manner which is hardto detect by others. Direct aggression was, unlike in previous stud-ies, not consistently associated with Machiavellianism. This is likelydue to the fact that previous studies did not examine all Dark Triadtraits simultaneously (Kerig & Stellwagen, 2010) or did not distin-guish between direct and indirect aggression (Muris et al., 2013).

Surprisingly, Narcissism was only uniquely associated with self-reported indirect aggression, and unlike in previous studies, notwith direct aggression. Again, this may be because previous studiestypically did not control for the overlap between the Dark Triadtraits and hence did not examine their unique effects. Given thatNarcissism comes with strategies to maintain inflated self-views(e.g., Twenge & Campbell, 2003), it is not surprising that Narcissismis associated with indirect rather than direct aggression. Directaggression is likely to damage one’s social network, whereas indi-rect aggressive acts like gossiping have even been found to be pos-itively associated with social status (Cillessen & Mayeux, 2004).Thus, employing indirect rather than direct aggression may allow

T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92 91

narcissists to use coercive strategies while maintaining the highsocial status and accompanying positive social reinforcement theyneed to maintain their inflated self-views.

Importantly, our findings show that it is crucial to study all DarkTriad traits collectively and account for the shared variancebetween these traits (see also Kerig & Stellwagen, 2010). For exam-ple, Psychopathy is the only Dark Triad trait that is not uniquelyassociated with self-reported indirect aggression, but also the onlytrait with replicable unique associations with self- and teacher-reported direct aggression. This pattern of unique associations isin line with theoretical predictions, but would not have emergedif only zero-order correlations would have been taken into account.Therefore, like previous findings obtained among adults (Jonasonet al., 2010), our findings underscore that it is crucial to study theseDark triad traits together to uncover their unique features.

In addition, the overall picture regarding our findings on genderdifferences in the Dark Triad traits and the associations of the DarkTriad traits with direct and indirect aggression can also informresearch on gender differences in direct and indirect aggression.Typically, such studies report higher levels of direct aggression inboys and negligible gender differences in indirect aggression(Archer, 2004). Our findings converge with these findings, as theonly replicable gender difference we found applied to the key cor-relate of direct aggression (i.e., Psychopathy). Similarly, gender dif-ferences in the most important correlates of indirect aggression(i.e., Narcissism and Machiavellianism) were not replicable.Although the associations of Dark Triad traits with aggression sub-types were dependent on the rater (i.e., the adolescents them-selves, their peers, or their teachers), this pattern of findings isquite striking and points to the potential role of personality traitsin gender differences in direct and indirect aggression.

4.3. Strengths and limitations

A first strength of the present study concerns its use of rigoroustests to establish the psychometric properties in general, and mea-surement invariance in particular, of the Dirty Dozen Dark Triadmeasure. Second, we used multi-informant data on both directand indirect aggression. Despite these strengths, several limita-tions need to be acknowledged.

First, the use of self-reported data on Dark Triad traits mayappear questionable given, for example, the manipulative tenden-cies of individuals with high levels of Machiavellianism. Thus, onecould expect self-representation biases to play a role in these self-reports. However, previous research (e.g., Hawley, 2003; Jonason &Webster, 2010; Jonason et al., 2010) and findings obtained in thepresent study suggest that self-reports on the Dark Triad can proveaccurate. Similarly, it remains unclear who is the best rater ofaggression. Different raters provide different perspectives, partlybecause their observation of aggression is limited to a specific con-text (Card et al., 2008). For example, teachers can assess aggressionin the classroom, but have less knowledge on aggressive acts out-side the classroom. Therefore, using multi-informant data andexamine whether results replicate across raters is probably thebest way to proceed.

Second, the length of the questionnaire we used is quite mini-mal and does not cover the full breadth of tendencies that are sub-sumed in the Dark Triad traits (Miller et al., 2012). As a result,associations of the Dirty Dozen Dark Triad traits with external cor-relates tend to be conservative estimates (Jonason & Luéveno,2013). In addition, specific facets within Dark Triad traits (e.g.,admiration and rivalry within Narcissism; Back et al., 2013) cannotbe distinguished.

Third, the present study was cross-sectional. In most studies,Dark Triad traits are described as predictors of subsequentproblematic behavior, but engaging in problematic behavior may

also have an effect on personality. In fact, previous longitudinal stud-ies focusing on general personality traits (i.e., Big Five) and problembehavior symptoms already showed that the linkages between per-sonality and problem behavior are often bidirectional, and that cor-related change of personality with problem behavior is also possible(e.g., De Bolle et al., 2012; Klimstra et al., 2014). Therefore, longitu-dinal studies are needed to uncover the direction of effects betweenDark Triad traits and problem behavior symptoms.

Fourth, we only measured subclinical levels of Dark Triad traits.Thus, our findings cannot be generalized to clinical narcissists andpsychopaths. Because there is no evidence for a breach betweenhealthy personality and personality pathology (Bernstein et al.,2007), there is no theoretical reason to expect different associa-tions between Dark Triad traits and problem behavior in clinicalsamples when compared to normative samples. Still, it should betested whether or not these theoretical predictions hold by alsoexamining the associations between Dark Triad traits and problembehavior in clinical samples.

5. Conclusion

Despite the aforementioned limitations, our study clearlyshows that adolescent Dark Triad personality traits can be mea-sured in a psychometrically sound manner. Moreover, our findingsinvolving multi-informant ratings of aggressive behavior suggestthat these traits are important correlates of adolescent psycholog-ical adjustment. Our findings underline that it is crucial for futurestudies to consider all three Dark Triad traits simultaneously tocapture their unique effects. Using such designs, the Dark Triadof personality should be studied more extensively in adolescents,as these traits may not only explain individual differences in ado-lescent aggression, but also provide greater insight into otheraspects of adolescent functioning.

References

Allik, J., Laidra, K., Realo, A., & Pullmann, H. (2004). Personality development from12 to 18 years of age: Changes in mean-levels and structure of traits. EuropeanJournal of Personality, 18, 445–462.

Archer, J. (2004). Sex differences in aggression in real-world settings: A meta-analytic review. Review of General Psychology, 8, 291–322.

Back, M. D., Küfner, A. C. P., Dufner, M., Gerlach, T. M., Rauthmann, J. F., & Denissen, J.J. A. (2013). Narcissistic admiration and rivalry: Disentangling the bright anddark sides of narcissism. Journal of Personality and Social Psychology, 105,1013–1037.

Bernstein, D. P., Iscan, C., & Maser, J. (2007). Opinions of personality disorder expertsregarding the DSM-IV personality disorders classification system. Journal ofPersonality Disorders, 21, 536–551.

Björkqvist, K., Lagerspetz, K. M. J., & Kaukiainen, A. (1992). Do girls manipulate andboys fight?: Developmental trends in regard to direct and indirect aggression.Aggressive Behavior, 18, 117–127.

Bukowski, W. M., Schwartzman, A., Santo, J., Bagwell, C., & Adams, R. (2009).Reactivity and distortions of the self: Narcissism, types of aggression, and thefunctioning of the hypothalamic–pituitary–adrenal axis during earlyadolescence. Developmental Psychopathology, 21, 1249–1262.

Card, N. A., Stucky, B. D., Sawalani, G. M., & Little, T. D. (2008). Direct and indirectaggression during childhood and adolescence: A meta-analytic review of genderdifferences, intercorrelations, and relations to maladjustment. ChildDevelopment, 79, 1185–1229.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurementinvariance. Structural Equation Modeling, 14, 464–504.

Christie, R., & Geis, F. L. (1970). Studies in machiavellianism. New York, NY: AcademicPress.

Cillessen, A. H. N. C., & Mayeux, L. (2004). From censure to reinforcement:Developmental changes in the association between aggression and social status.Child Development, 75, 147–163.

Cima, M., Tonnaer, F., & Hauser, M. (2010). Psychopaths know right from wrong,they just don’t care about it. Social Cognitive and Affective Neuroscience, 5, 59–67.

Coyne, S. M., & Thomas, T. J. (2008). Psychopathy, aggression, and cheatingbehavior: A test of the cheater-hawk hypothesis. Personality and IndividualDifferences, 44, 1105–1115.

Crick, N. R. (1996). The role of overt aggression, relational aggression, and prosocialbehavior in the prediction of children’s future social adjustment. ChildDevelopment, 67, 2317–2327.

92 T.A. Klimstra et al. / Journal of Research in Personality 53 (2014) 84–92

De Bolle, M., Beyers, W., De Clercq, B., & De Fruyt, F. (2012). General personality andpsychopathology in referred and nonreferred children and adolescents: Aninvestigation of continuity, pathoplasty, and complication models. Journal ofAbnormal Psychology, 121, 958–970.

De Clercq, B., De Fruyt, F., De Bolle, M., Van Hiel, A., Markon, K. E., & Krueger, R. F.(2014). The hierarchical structure and construct validity of the PID-5 traitmeasure in adolescence. Journal of Personality, 82, 158–169.

De Clercq, B., De Fruyt, F., Van Leeuwen, K., & Mervielde, I. (2006). The structure ofmaladaptive personality traits in childhood: A step toward an integrativedevelopmental perspective for DSM-V. Journal of Abnormal Psychology, 115,639–657.

Decuyper, M., De Bolle, M., De Fruyt, F., & De Clercq, B. (2011). General andmaladaptive personality dimensions and the assessment of callous–unemotional traits in adolescence. Journal of Personality Disorders, 25, 681–701.

Denissen, J., Geenen, R., van Aken, M., Gosling, S., & Potter, J. (2008). Developmentand validation of a Dutch translation of the Big Five Inventory (BFI). Journal ofPersonality Assessment, 90, 152–157.

Denissen, J. J. A., & Penke, L. (2008). Motivation individual reaction normsunderlying the five-factor model of personality: First steps towards a theory-based conceptual framework. Journal of Research in Personality, 42, 1285–1302.

Dijkstra, J. K., Lindenberg, S., Verhulst, F. C., Ormel, J., & Veenstra, R. (2009). Therelation between popularity and aggressive, destructive, and norm-breakingbehaviors: Moderating effects of athletic abilities, physical attractiveness, andprosociality. Journal of Research on Adolescence, 19, 401–413.

Furnham, A., Richards, S. C., & Paulhus, D. L. (2013). The dark triad of personality: A10 year review. Social and Personality Psychology Compass, 7, 199–216.

Hale, W. W., VanderValk, I., Akse, J., & Meeus, W. (2008). The interplay of earlyadolescents’ depressive symptoms, aggression and perceived parental rejection:A four-year community study. Journal of Youth and Adolescence, 37, 928–940.

Hawley, P. H. (2003). Prosocial and coercive configurations of resource control inearly adolescence: A case for the well-adapted machiavellian. Merrill-PalmerQuarterly, 49, 279–309.

Heilbron, N., & Prinstein, M. (2008). A review and reconceptualization of socialaggression: Adaptive and maladaptive correlates. Clinical Child and FamilyPsychology Review, 11, 176–217.

John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five Inventory-Versions 4aand 54. Berkeley, CA: University of California, Berkeley, Institute of Personalityand Social Research.

Jonason, P. K., Li, N. P., & Buss, D. M. (2010). The costs and benefits of the dark triad:Implications for mate poaching and mate retention tactics. Personality andIndividual Differences, 48, 373–378.

Jonason, P. K., & Luéveno, V. X. (2013). Walking the thin line between efficiency andaccuracy: Validity and structural properties of the dirty dozen. Personality andIndividual Differences, 55, 76–81.

Jonason, P. K., & Webster, G. D. (2010). The dirty dozen: A concise measure of theDark Triad. Psychological Assessment, 22, 420–432.

Jonason, P. K., Webster, G. D., Schmitt, D. P., Li, N. P., & Crysel, L. (2012). The antiheroin popular culture: Life history theory and the dark triad personality traits.Review of General Psychology, 16, 192–199.

Kerig, P. K., & Stellwagen, K. K. (2010). Roles of callous–unemotional traits,narcissism, and machiavellianism in childhood aggression. Journal ofPsychopathology and Behavioral Assessment, 32, 343–352.

Klimstra, T. A., Luyckx, K., Hale, W. W., & Goossens, L. (2014). Personality andexternalizing behavior in the transition to young adulthood: The additive valueof personality facets. Social Psychiatry and Psychiatric Epidemiology, 49,1319–1333.

Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.).New York: Guilford Press.

Kraemer, H. C., Measelle, J. R., Ablow, J. C., Essex, M. J., Boyce, W. T., & Kupfer, D. J.(2003). A new approach to integrating data from multiple informants inpsychiatric assessment and research: Mixing and matching contexts andperspectives. American Journal of Psychiatry, 160, 1566–1577.

Lau, K. S. L., & Marsee, M. A. (2013). Exploring narcissism, psychopathy, andmachiavellianism in youth: Examination of associations with antisocialbehavior and aggression. Journal of Child and Family Studies, 22, 355–367.

Lee, K., & Ashton, M. C. (2005). Psychopathy, machiavellianism, and narcissism inthe five-factor model and the HEXACO model of personality structure.Personality and Individual Differences, 38, 1571–1582.

Malterer, M. B., Glass, S. J., & Newmann, J. P. (2008). Psychopathy and traitemotional intelligence. Personality and Individual Differences, 44, 735–745.

McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personalityacross instruments and observers. Journal of Personality and Social Psychology,52, 81–90.

Miller, J. D., Bagby, R. M., Pilkonis, P. A., Reynolds, S. K., & Lynam, D. R. (2005). Asimplified technique for scoring DSM-IV personality disorders with the five-factor model. Assessment, 12, 404–415.

Miller, J. D., Few, L. R., Seibert, A., Watts, A., Zeichner, A., & Lynam, D. R. (2012). Anexamination of the dirty dozen measure of psychopathy: A cautionary taleabout the costs of brief measures. Psychological Assessment, 24, 1048–1053.

Muris, P., Meesters, C., & Timmermans, A. (2013). Some youths have a gloomy side:Correlates of the dark triad personality traits in non-clinical adolescents. ChildPsychiatry & Human Development, 44, 658–665.

Muthén, L. K., & Muthén, B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA:Muthén & Muthén.

Ojanen, T., Findley, D., & Fuller, S. (2012). Physical and relational aggression in earlyadolescence: Associations with narcissism, temperament, and social goals.Aggressive Behavior, 38, 99–107.

Pardini, D. A., Lochman, J. E., & Frick, P. J. (2003). Callous/unemotional traits andsocial-cognitive processes in adjudicated youths. Journal of the AmericanAcademy of Child & Adolescent Psychiatry, 42, 364–371.

Paulhus, D. L., & Williams, K. M. (2002). The dark triad of personality: Narcissism,machiavellianism, and psychopathy. Journal of Research in Personality, 36,556–563.

Peters, M., Cillessen, A. H. N., & Scholte, R. H. J. (2010). Clueless of powerful?Identifying subtypes of bullies in adolescence. Journal of Youth and Adolescence,39, 1041–1052.

Raskin, R. N., & Terry, H. (1988). A principal components analysis of the narcissisticpersonality inventory and further evidence of its construct validity. Journal ofPersonality and Social Psychology, 54, 890–902.

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic formoment structure analysis. Psychometrika, 66, 507–514.

Schönbrodt, F. D., & Perugini, M. (2013). At what sample size do correlationsstabilize? Journal of Research in Personality, 47, 609–612.

Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The developmentalpsychometrics of big five self-reports: Acquiescence, factor structure,coherence, and differentiation from ages 10 to 20. Journal of Personality andSocial Psychology, 94, 718–737.

Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personalitytraits from 10 to 65: Big-Five domains and facets in a large cross-sectionalsample. Journal of Personality and Social Psychology, 100, 330–348.

Steenkamp, J. M., & Baumgartner, H. (1998). Assessing measurement invariance incross-national consumer research. Journal of Consumer Research, 25, 78–90.

Steinmetz, H. (2013). Analyzing observed composite differences across groups: Ispartial measurement invariance enough? Methodology, 9, 1–12.

Stickle, T. R., Marini, V. A., & Thomas, J. N. (2013). Gender differences inpsychopathic traits, types, and correlates of aggression among adjudicatedyouth. Journal of Abnormal Child Psychology, 40, 513–525.

Tackett, J. L., Herzhoff, K., Reardon, K. W., De Clercq, B., & Sharp, C. (2014). Theexternalizing spectrum in youth: Incorporating personality pathology. Journal ofAdolescence, 37, 659–668.

Tackett, J. L., Waldman, I. D., & Lahey, B. B. (2009). Etiology and measurement ofrelational aggression: A multi-informant behavior genetic investigation. Journalof Abnormal Psychology, 118, 722–733.

Tromp, N. B., & Koot, H. M. (2010). Dimensions of normal and abnormal personality:Elucidating DSM-IV personality disorder symptoms in adolescents. Journal ofPersonality, 78, 839–864.

Twenge, J. M., & Campbell, K. W. (2003). ‘‘Isn’t it fun to get the respect that we’regoing to deserve?’’ Narcissism, social rejection, and aggression. Personality andSocial Psychology Bulletin, 29, 261–272.

van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurementinvariance. European Journal of Developmental Psychology, 9, 486–492.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurementinvariance literature: Suggestions, practices, and recommendations fororganizational research. Organizational Research Methods, 3, 4–70.

Webster, G. D., & Jonason, P. K. (2013). Putting the ‘‘IRT’’ in ‘‘dirty’’: Item responsetheory analyses of the dark triad dirty dozen – An efficient measure ofnarcissism, psychopathy, and machiavellianism. Personality and IndividualDifferences, 54, 302–306.

Wilson, D. S., Near, D., & Miller, R. R. (1996). Machiavellianism: A synthesis of theevolutionary and psychological literatures. Psychological Bulletin, 119, 285–299.