Abstract

Background: The psychometric properties of the Revised Child Anxiety and Depression Scale

(RCADS) have been established cross-culturally, yet psychometric evidence is lacking for

English speaking European population. Aim: The current research sought to further cross-

validate the measure in a non-clinical Irish adolescent sample, and to test for gender and

age-based differential item functioning in depression and anxiety. Method: Participants

were Irish post-primary school students (N=345; 164 male; 12-18 years, M=14.97, SD=1.44).

Confirmatory factor analysis for categorical data (confirmatory item factor analysis), and

Multiple-Indicator Multiple-Cause (MIMIC) modeling to identify items displaying possible

metric invariance, were conducted. Results: A six-factor model fit the data well in both

gender samples, and both school cycle, as a proxy for age, samples. Gender-based metric

invariance for 5 of 47 items, and age-based metric invariance for 3 items, were identified.

However, the magnitudes were small. Internal consistency and validity were also

established. Conclusions: While a number of items demonstrated minor metric invariance,

there was no evidence that they influenced overall scores meaningfully. The RCADS can

reasonably be used without adjustment in male and female, younger and older, adolescent

samples. Findings have implications for the use of the RCADS in an English speaking

European population. Declaration of Interest: The authors have no competing interests to

declare.

1

Introduction

The prevalence of internalising disorders, primarily mood and anxiety, among

adolescents is well evidenced globally (Farbstein et al., 2010; Frigerio et al., 2009; Kessler et

al., 2012; Merikangas et al., 2010; UN 2013), including in Ireland (e.g., Author, 2012; Cannon

et al., 2013). These disorders originate both independently and co-morbidly in youth (Jones,

2013; Pine et al., 1998), and are so-called due to the tendency for distress to be expressed

inwards (Cosgrove et al., 2011). There is a need for valid assessment measures that identify

emerging difficulties and subsequently enhance early intervention and prevention of mental

health difficulties in adolescence. It is also important that measures of internalising

disorders recognise the comorbidity of anxiety and depression (Mash & Hunsley, 2007).

Given the subjective nature of internalising disorders (Bernstein et al., 1996), self-report

measures are recommended as the first step in assessment (Law & Wolpert, 2013), and the

availability of multiple informant versions e.g. parent-report, in addition to self-report,

contributes to clinicians’ broader understanding of these difficulties (Southam-Gerow &

Chorpita, 2007).

Revised Child Anxiety and Depression Scale (RCADS)

The Revised Child Anxiety and Depression Scale (RCADS; Chorpita, 1998), for which

there are both self- and parent-report versions (Chorpita, 2003), was named the briefest

child and adolescent assessment measure aligned with the Diagnostic and Statistical Manual

(DSM) in a review by Southam-Gerow & Chorpita (2007), and has been deemed most

sensitive to clinical change in a review of four prominent measures (Wolpert, Cheng, &

Deighton, 2015). The clinical and research value of the RCADS is the measure’s ability to

differentiate between comorbid and independent aspects of internalising disorders (Mash &

Hunsley, 2007), and balance evidence-based assessment with real-world research demands

2

(Ebesutani et al., 2012). As such, the RCADS has been recommended for use as a routine

outcome assessment measure in clinical settings in the UK (CORC, 2014).

The 47-item freely-available measure simultaneously assesses five subcategories of

anxiety: Generalised Anxiety Disorder (GAD), Separation Anxiety Disorder (SAD), Social

Phobia (SP), Panic Disorder (PD), Obsessive-Compulsive Disorder (OCD); as well as Major

Depressive Disorder (MDD). The measure generates total scores for each subscale, a Total

Anxiety score (GAD, SAD, SP, PD, OCD) and a Total Internalising score of all six subscales. Of

note, OCD has since been re-categorised in the DSM-V (APA, 2013) and excluded from the

anxiety disorders category. However, the most recent manual continues to recognise the

high comorbidity between OCD and anxiety disorders. Both share treatment approaches,

and it is therefore clinically useful to continue to assess OCD in line with anxiety disorders

(Ebesutani, Korathu-Larson, Nakamura, Higa-McMillan, & Chorpita, 2016).

A six-factor structure of the five anxiety and one depression subscales has been

supported with children and adolescents in Hawaiian school and clinical samples (Chorpita,

Yim, Moffitt, Umemoto, & Francis, 2000; Chorpita, Moffitt, & Gray, 2005), school samples in

Australia (deRoss, Gullone, & Chorpita, 2002), Denmark (Esbjørn, Somhovd, Turnstedt, &

Reinholdt-Dunne, 2012), and France (Bouvard, Denis, & Roulin, 2015), Southern American

Caucasian youth (Trent et al., 2013), and African-American youth (Brown et al., 2013; Trent

et al., 2013). The six-factor structure has also been supported for the RCADS-P in American

clinical (Ebesutani, Bernstein, Nakamura, Chorpita, & Weisz, 2010) and school samples

(Ebesutani et al., 2011), a Spanish cohort (Park, Ebesutani, Bose, & Chorpita, 2016), and

good fit was also observed in a Dutch sample for the five anxiety disorders (Mathyssek et al.,

2013).

3

In contrast, Muris, Meesters, & Schouten (2002) did not support the six-factor

structure and thus developed a 25-item version of five five-item subscales excluding OCD.

However, Ebesutani et al. (2012) noted that the reliability and reduced content diversity of

the Total Anxiety subscale may have been jeopardised. As an alternative, Ebesutani et al.

(2012) retained the OCD subscale, reduced the anxiety subscales to five items each and

maintained the full MDD subscale. A two-factor anxiety/depression structure was

subsequently supported (Chorpita et al., 2005; Ebesutani et al., 2012; Ebesutani et al.,

2016). A five-factor model combining GAD and MDD (Ebesutani et al., 2010; 2011), a one-

factor Internalising Disorders model (Chorpita et al., 2005; deRoss et al., 2002), and

hierarchical models (Brown et al., 2013) have also been examined in previous research.

However, the six-factor model has been the most widely supported.

In terms of abbreviated versions, Park et al. (2016) examined the 25-item structure,

proposed by Ebesutani et al. (2012), with the Spanish RCADS-P but did not find consistent

support for its validity compared with the 47-item Spanish RCADS-P. Sandin, Chorot,

Valiente, & Chorpita (2010) provided support for a 30-item version in a Spanish population,

and Stevanovic et al. (2016) has demonstrated validity and a six-factor structure of a 37-

item version of the measure across ten ethnically and culturally diverse countries. The

measure did not perform well in an eleventh Philippines sample however, highlighting the

importance of examining the validity of assessment measures across cultures prior to

implementation. The authors note that there is a distinct lack of evidence for the RCADS in

an English speaking European population, yet nonetheless the RCADS has been

recommended for use in the UK (CORC, 2014).

In an Irish context, neither the RCADS nor RCADS-P have been extensively considered

as viable assessment measures. One study utilised the RCADS-25 in an Irish Child and

4

Adolescent Mental Health Service (CAMHS) observing satisfactory reliability (Wynne, Doyle,

Kenny, Brosnan, & Sharry, 2016). However, the psychometric properties were not

investigated. The current research begins to address this gap in evidence by investigating

the psychometric properties of the original 47-item RCADS in a non-clinical Irish adolescent

sample.

It is also important to establish the measurement invariance of a scale to allow for

meaningful comparisons between various groups. For the RCADS, this involves assessing the

degree to which the measurement of depression and anxiety disorders is equivalent or

invariant across different subpopulations and groups. In order to make valid comparisons of

scores between groups, it is important that measurement equivalence is established. For

example, if the RCADS is measurement invariant across groups, individuals with similar

levels of the construct being measured (e.g., anxiety symptoms) will score similarly on each

item of the measure, and scoring metrics can therefore be considered equivalent across the

groups (Reise, Widaman & Pugh, 1993). Metric invariance can be assessed by examining

whether there are differences in scores of individual items across groups after controlling

for levels of the latent construct being measured.

The aim of this study was to conduct a metric invariance analysis to determine

whether RCADS scores, and subscale scores were equivalent between males and females,

and between those in Junior and Senior cycle. The first aim of the study was to examine the

factor structure of the RCADS using confirmatory item factor analysis (CFA for categorical

data) to establish a baseline model. The second aim was to test metric invariance in RCADS

items across both males and females, and across Junior and Senior school cycle (lower and

upper secondary school). Finally, the third aim was to examine reliability and validity of the

RCADS in a sample of Irish adolescents.

5

Methods

Participants

Participants were 350 second-level students (186 female) aged 12-18 years

(M=14.97, SD=1.44). In Irish second-level schools, years 1-3 are known as Junior Cycle

(approximate age range 12-15 years) and Years 4-6 are known as Senior Cycle (approximate

age range 16-18 years). Junior and Senior Cycle were used as proxies for age. Students were

sampled from Junior Cycle years (Total=169; 1st=26; 2nd=133; 3rd=10) and Senior Cycle years

(Total=181; 4th/Transition Year=64; 5th=117) across four schools (one urban mixed sex, one

urban single sex, two rural mixed sex). Schools did not allow 6 th Year students to participate

due to final exams. The majority of students identified themselves as White (91.4%), 86% of

which identified as Irish (5.4% White not Irish, 3.4% Asian, 2.9% Other/half Irish, 1.4%

Black, .6% Eastern European, .3% Irish Traveller).

Measures

Revised Child Anxiety and Depression Scale (RCADS)

The RCADS items are scored 0-3 (never – almost always) for GAD e.g. ‘I worry about

things’, SAD e.g. ‘I feel scared if I have to sleep on my own’, SP e.g. ‘I worry I might look

foolish’, PD e.g. ‘I suddenly become dizzy or faint when there is no reason for this’, OCD e.g. ‘I

have to do something in just the right way to stop bad things from happening to me ’, and

MDD e.g. ‘I feel worthless’. A scoring sheet available from Child First

(www.childfirst.ucla.edu) generates raw and T scores for each subscale, a Total Anxiety

Scale and a Total Internalising Scale. Standardised T scores are generated using gender and

US school grade. Grades were adapted to equivalent Irish second-level years (7 th Grade=1st

Year; 8th=2nd; 9th=3rd; 10th=4th; 11th=5th; 12th=6th). Higher scores indicate greater levels of

difficulty. The RCADS has previously demonstrated high reliability, and good internal

6

consistency and convergent validity (Bouvard et al., 2015; Ebesutani et al., 2010; Esbjørn et

al., 2012).

Depression, Anxiety and Stress Scale Short Version (DASS-21)

The shortened version of the Depression, Anxiety and Stress Scale (Lovibond &

Lovibond, 1995) consists of 21 statements with three subscales relating to depression e.g. ‘I

felt that I had nothing to look forward to’, anxiety e.g. ‘I was worried about situations in

which I might panic and make a fool of myself’, and stress e.g. ‘I felt that I was rather

touchy’. Responses range from never to almost always (0-3). Higher scores indicate greater

distress. Previous research has illustrated the validity of the DASS-21 with adolescents

(Miller et al., 2015; Willemsen, Markey, Declercq, & Vanheule, 2011), including a national

study of Irish adolescents (Author, 2012). The DASS-21 Depression and Anxiety subscales

were used in the current study.

Procedure

Ethical approval was granted through the university with which the authors were

affiliated. Students completed questionnaires comprising questions relating to demographic

information, the RCADS self-report version, and the DASS-21 self-report version, in class or

year groups. Parent/guardian consent was sought, and parents/guardians were asked to

consider whether their child had experienced or was experiencing any mental health

difficulties and whether they would be able to participate in the research as a result of this

or any other factors e.g. learning disability. Students also provided consent prior to

participation.

Statistical Analyses

Five respondents were omitted for missing data greater than 20% (Ebesutani et al.,

2010). For remaining participants (N=345), the amount of missing data was low and ranged

7

from .47%-3.76% (RCADS) and .21%-.84% (DASS-21). Missing data were treated as pairwise

missing which is the default when the weighted least squares (WLSMV) estimator is used

(Asparouhov & Muthén, 2010). Skewness and kurtosis were within the required ranges to

indicate data were normally distributed.

The latent structure of the 47-item RCADS was tested using confirmatory factor

analysis (CFA) for categorical data. Five alternative models identified from the literature

were specified and tested. Overall the aim of testing alternative models was to determine if

the internalising disorders assessed by the RCADS were better presented as six correlated

dimensions rather than five, two, or one dimensions, and if there was a hierarchical

structure (second-order factors) that explained the associations between the first-order

Depression and Anxiety dimensions. Model 1 is a correlated six factor model, of the six

RCADS subscales. Model 2 is a correlated five factor model, where GAD and MDD are

combined. Model 3 is a correlated two-factor model of Anxiety and MDD. Model 4 is a one-

factor model of Internalising Disorders. Model 5 tested that there was a hierarchical

structure for the RCADS items, whereby the six RCADS subscales of GAD, SAD, SP, PD, OCD

and MDD loaded onto an overall latent factor of internalising disorders.

Each model was specified and estimated using Mplus 7.1 (Muthén & Muthén, 2013)

using the robust weighted least squares estimator (WLSMV) based on the polychoric

correlation matrix of latent continuous response variables. The WLSMV estimator is the

most appropriate estimator of ordinal indicators in a CFA for categorical data (Brown, 2006).

The WLSMV estimator has been shown to produce correct parameter estimates, standard

errors and test statistics (Flora & Curran, 2004). The error variances were uncorrelated for

all models. Goodness of fit for each model was assessed with a range of fit indices including

the relative Chi-square (chi-square/df) (relative χ2; Tabachnick & Fidell, 2007), the

8

Comparative Fit Index (CFI; Bentler, 1990), the Tucker-Lewis Index (TLI; Tucker & Lewis,

1973), the Root Mean Square Error of Approximation (RMSEA; Browne & Cudek, 1993) and

RMSEA 90% Confidence Intervals, and Weighted Root Mean Square Residual (WRMR;

Muthén & Muthén, 2013). Values indicating acceptable model fit are .90 or above for CFI

and TLI, while values up to .08 for RMSEA indicate reasonable errors of approximation, with

values up to .05 indicate close model fit (Jöreskog & Sörbom, 1993). Chi-square tests are

sensitive to larger sample sizes (Hooper, Coughlan, & Mullen, 2008), therefore the relative

chi-square (chi-square/df) was used as an index of fit, with values less than 2 indicating a

good model fir (Ullman, 2001). It is therefore recommended that Chi-square tests should

not be taken as the sole model indicator of model fit and that the various fit indices should

also be examined (Schermelleh-Engel, Moosbrugger, & Müller, 2003), as per the current

study.

To assess differences, if any, for gender and age, Multiple Indicators Multiple Causes

(MIMIC) models were conducted to determine gender-based or age-based metric invariance

(Holland & Warner, 1993), i.e. differences in item scores across groups while controlling for

an overall latent variable, was present. To investigate this, MIMIC models test 1) the

relationships between items and factors (i.e. the baseline measurement model), 2) factors

and the group variable (i.e. structural regression coefficients), and 3) items and the group

variable (i.e. the direct effect). If different scores on the observed variables emerge between

groups who share the same underlying latent variable, metric invariance is established

(Haroz, Ybarra, & Eaton, 2014). After establishing the baseline model, latent variables are

regressed on variables representing the groups under investigation, with significant

regression coefficients indicating significant mean differences between groups (i.e. Male/

Female, Junior/Senior cycle) at the latent variable level. Modification Indices (MI) for the

9

direct paths from the group variable to the RCADS measure are inspected to determine the

presence of DIF. Modification indices (MI) identify paths that if added to the model would

significantly improve model fit by reducing the chi-square by 5 or more. In a sequential

manner, the path with the largest MI was included in the model and the model was re-

estimated. This process continued until there were no MIs greater than 5. It is important to

rule out DIF and establish measurement invariance, as it can lead to incorrectly detected

group differences and prevent the use of assessment measures across certain populations

(Borsboom, 2006; Teresi & Fleishman, 2007).

Next, convergent validity was examined using Pearson’s r correlations between

RCADS and DASS-21 Depression and Anxiety subscales. To examine divergent validity,

significant differences between the correlations were examined using Fisher’s z-tests, as it

was expected that the correlations would be moderate given the comorbidity of anxiety and

depression. To assess reliability, internal consistency was established using Cronbach’s

alpha. Item-total correlations were also examined, with .30 taken as the cut-off for

adequate item-total correlation values (Nunnally & Bernstein, 1994).

Results

Confirmatory Factor Analysis for Categorical Data

As shown in Table 1, the original six-factor model yielded good model fit in the

current sample, χ2(1019)=1563.95, p<.01, demonstrating excellent CFI=.96, TLI=.96,

RMSEA=.034 (90% CI .035-.043) and adequate WRMR=1.09 and a relative chi-square value

of 1.534 indicating a good model. Figure 1 displays the path diagram with standardised

factor loadings, and standard error variances for each item in brackets. The second-order

model originally investigated by Brown et al. (2013) was also supported in the current

sample, χ2(1028)=1566.47, demonstrating similarly good fit to the original model with equal

10

CFI, TLI, RMSEA and 90% CI values, and similarly adequate WRMR=1.10, and a relative chi-

square value of 1.524 indicating a good model. Good fit for the second order model is

unsurprising given the high correlations between the RCADS subscales, confirming the

presence of a latent factor of internalizing disorders. Nonetheless, the original six-factor

model was considered to be a more parsimonious solution. Poorer model fit was observed

for the five, two, and one-factor models when compared with the six-factor and second-

order models (Table 1).

MIMIC

To investigate metric invariance, MIMIC models were employed where the latent

variables were regressed firstly on the gender variable (Male/Female) and secondly on the

school cycle variable (Junior/Senior Cycle) to examine the direct effects. For gender, this

model made little difference to the overall fit, χ2(1060)=1641.51, CFI=.96, TLI=.95,

RMSEA=.04 (90% CI=.036; .044), WRMR=1.13, and chi-square/df ratio =1.55. However,

inspection of the modification indices revealed that direct effects from the gender variable

to five items would improve the model fit if freely estimated. Each direct effect was added

sequentially to determine if there were significant differences, while controlling for any

difference in the overall level of the latent factor between males and females. The final

model, with direct effects from the gender variable to five items, was χ2(1055)=1592.21,

CFI=.96, TLI=.96, RMSEA=.04 (90% CI=.035; .042), WRMR=1.10, chi-square/df ratio =1.51 and

provided a better fit. The factor loadings for the baseline and metric invariance corrected

models are presented in Table 2. The regression coefficients for the structural effects of

gender on the latent factors and the five non-invariant items are presented in Table 3. These

findings indicate that females reported significantly higher levels of GAD, SP, SAD, OCD, PD,

and MDD, than males. The largest effect was for MDD, with females having a mean

11

score .69 of a standard deviation higher than males, followed by GAD and SAD with a mean

score of .55 of a standard deviation above the mean. The direct effects from the gender

variable to all five items were statistically significant, while statistically controlling for the

latent variables in the model. This indicated that females scored significantly higher on

feelings of worry when someone may be angry with them (RCADS8; SP), feeling scared to

take a test (RCADS7; SP), and fear of being at home alone (RCADS5; SAD). Males were

significantly more likely to endorse feeling nervous going to school (RCADS18; SAD) and

feeling that nothing is much fun anymore (RCADS6; MDD). The effect sizes were small, and

indicated that the mean difference between males and females on the items ranged

between .48 to .67 of a standard deviation.

The latent variables were also regressed on a variable representing school cycle

(Junior/Senior Cycle) as a proxy for age to examine the direct effects. This model made little

difference to overall fit, χ2(1060)=1631.87, CFI=.96, TLI=.96, RMSEA=.04 (90% CI=.036; .04),

WRMR=1.09, chi-square/df ratio= 1.54. However, inspection of the modification indices

revealed that direct effects from the school cycle variable to three items would improve the

model fit if freely estimated. Similar to gender, each direct effect was added sequentially to

determine if there was significant metric invariance, while controlling for any difference in

the overall level of the latent factor between groups. The fit of these models are presented

in Table 1. The final DIF corrected model was χ2(1057)=1604.01, CFI=.96, TLI=.96,

RMSEA=.039 (90% CI=.035; .04), WRMR=1.08, chi-square/df ratio =1.52, which provided a

better fit. The factor loadings for the baseline and DIF corrected models are presented in

Table 2.

The regression coefficients for the structural effects of school cycle on the latent

factors and the three items identified as demonstrating DIF; feeling scared away from home

12

(RCADS46), worrying at night (RCADS45), and responding to stressors with feelings in the

stomach (RCADS3) are presented in Table 3. These findings indicate that Senior Cycle

students reported significantly higher levels of GAD, SP, OCD, and MDD, while those in the

Junior Cycle reported significantly higher levels of SAD and PD. The largest effect was for

SAD and PD, with Junior Cycle students having a mean score of .19 of a standard deviation

higher than those in Senior Cycle. The other effects were small, with estimates ranging

between .02 to .13. Further, the direct effects from the school cycle variable to items

RCADS46, RCADS45, and RCADS3 were statistically significant. This indicated that Junior

Cycle students scored significantly higher on feeling scared away from home, while Senior

Cycle students were more likely to endorse higher rates of stomach sensations when

experiencing problems while statistically controlling for the latent variables in the model.

The effect sizes were small, ranging from .35 to .63.

Validity

Correlations, all of which were significant, between the RCADS subscales, and the

RCADS and DASS-21 Depression and Anxiety are displayed in Table 4. All RCADS subscale

correlations were significant, indicating the high level of comorbidity of anxiety and

depression and the possibility of a latent internalizing disorders factor. To establish validity,

correlations between the RCADS and DASS-21 were examined. The RCADS MDD subscale

(r=.79) and Total Internalising factor (r=.73) were most strongly correlated with the DASS-21

Depression subscale compared to the DASS-21 Anxiety subscale. The RCADS PD subscale

(r=.72) and Total Anxiety subscale (r=.71) were most strongly correlated with the DASS

Anxiety subscale. The Total Internalising overall scale was also strongly correlated with the

DASS-21 Anxiety subscale (r=.72). The strong correlation reported between the Panic

Disorder subscale and the DASS-21 Anxiety subscale is understandable considering that

13

many of the DASS-21 anxiety items reflect the construct of panic e.g. ’’I felt I was close to

panic’’. These results provide evidence of convergent validity for the RCADS, and further

highlight the comorbidity of anxiety and depression as indicated by the strong correlations

for the RCADS Total Internalising scores with both DASS-21 subscales.

Significant correlations between the RCADS and the DASS-21 subscales were also

shown by gender, and by Junior versus Senior Cycle as a proxy for age. The correlations

between the six RCADS subscales and the DASS Depression and Anxiety subscales were

higher for the Junior Cycle than for the Senior Cycle, landing primarily between r=.59 and

r=.81 for Junior Cycle, and between r=.39 and r=.76 for the Senior cycle. Correlations were

also higher for females than males, landing primarily between r=.49 and r=.83 for females,

and r=.32 and r=.63 for males.

Furthermore, divergent validity was partially supported for the total sample (Table

4). Of the RCADS anxiety scales, the highest correlations with the DASS-21 Anxiety subscale

were for PD (r=.72), and Total Anxiety (r=.71), with all three correlating lower with the DASS-

21 Depression subscale. The difference for the PD subscale was significant suggesting

divergent validity. SAD and OCD correlations were moderately higher for the DASS-21

Anxiety subscale than the DASS-21 Depression subscale, while the GAD and OCD

correlations were moderately higher with the DASS-21 Depression subscale. The highest

correlations for the MDD (r=.79) and Total Internalising (r=.73) were with the DASS-21

Depression subscale. Both correlated relatively highly with the DASS-21 Anxiety subscale,

however the difference for the MDD subscale was significant thus indicating divergent

validity. As previously noted, the strong correlations across the anxiety and depression

constructs are unsurprising given the comorbidity of both.

Internal Consistency

14

Internal consistency for the RCADS subscales ranged from adequate to excellent,

α=.60 to .96 (Table 5). When examined by gender and age, differences in alpha-levels were

not substantial. Of note, alpha for the SAD subscale was higher for the Junior than the

Senior cycle. Separation anxiety is more prevalent in children and younger adolescents

(Copeland, Angold, Shanahan, & Costello, 2014), hence this may explain why the subscale

may have performed better with the younger adolescents in the current study. The PD

subscale showed a lower reliability for males than females, which may suggest that these

items are more salient for females than males and support existing findings on gender

differences in panic disorders, which are more prevalent amongst females (McLean,

Asnaani, Litz, & Hofmann, 2011). Corrected item-total correlations are displayed in Table 6,

with no items emerging below the cut-off of .30 for each subscale.

Discussion

With the aim of establishing the psychometric properties of scores from the 47-item

RCADS in a previously unexamined sample of Irish adolescents, the current study provided

further cross-cultural validation of the six-factor structure of the measure in an English

speaking European sample. The second-order model also demonstrated fit indices similar to

the six-factor model thus indicating the presence of a hierarchical latent ‘internalising

disorders’ factor, which explains the comorbidity and subsequent high correlations between

anxiety and depression as assessed by the RCADS. However, the RCADS is utilised in clinical

settings such as Child and Adolescent Mental Health Services (CAMHS) in the UK (CORC,

2014), for the purposes of diagnosis and formulation of specific anxiety and depressive

disorders among adolescents, with a significant value being the ability to detect six disorders

independently using a single measure (Mash & Hunsley, 2007). As such, the original six-

factor structure was deemed to be the most parsimonious and clinically useful model in the

15

current study, which is the first to support this model in an English speaking, European

population. Reliability and convergent validity were also established, and there was

evidence for divergent validity for the MDD and Panic Disorder subscales.

In order to determine whether the RCADS was invariant across gender and age,

whereby school cycle was used as a proxy, metric invariance was investigated. For gender,

five of 47 items showed DIF between males and females. Females reported being more

likely to worry when someone was angry (RCADS8), to be scared to take a test (RCADS7),

and to be afraid to be at home alone (RCADS5), while males reported being more likely to

have trouble going to school in the morning as a result of being nervous or afraid

(RCADS18), and that nothing was much fun anymore (RCADS6). While females overall tend

to report greater levels of anxiety and depression, more nuanced gender-based differences

have been observed within disorders (McLean & Anderson, 2009). In terms of anxiety,

females have greater tendencies to worry (McLean & Anderson, 2009) while Bennett,

Ambrosini, Kudes, Metz, & Rabinovich (2005) observed within-disorder differences in levels

of depression among adolescent males and females. The authors reported that females

experienced greater levels of guilt, concentration problems, difficulty working, etc., while

males experienced higher levels of clinical anhedonia, i.e. inability to feel pleasure or fun in

day-to-day activities. Males also tend to exhibit behavioural reactions to anxiety and

depression (Brownhill, Willhelm, Barclay, & Schmied, 2005), which may contribute to the

greater levels of trouble going to school and higher rates of apathy towards enjoyment of

activities as reported by males in the current study.

Differences between Junior and Senior school cycles were also observed. Younger

adolescents in the Junior Cycle reported being more likely to feel afraid staying away from

home overnight (RCADS46), and older adolescents in the Senior Cycle reported being more

16

likely to worry at night (RCADS45) and to experience feelings in the stomach in response to

stressors (RCADS3). Trends in anxiety disorders differ across age groups with Separation

Anxiety Disorder being most prevalent amongst children and younger adolescents, and

subsequently decreasing in mid-adolescence while rates of other disorders including Panic

Disorder increase (Copeland et al., 2014). These studies may explain these specific age

differences in anxiety symptoms and why fear of staying away from home was most

prevalent in Junior Cycle, and the Panic Disorder item of experiencing feelings in the

stomach in response to stressors was more reported in Senior Cycle.

The divergent validity of the RCADS was only preliminarily supported in the current

study. All anxiety and depression subscales for both the RCADS and DASS-21 were at least

moderately correlated, with high correlations observed between anxiety and depression

subscale due to the comorbidity of anxiety and depression. The correlations between MDD

and the DASS-21 Depression subscale, and PD and the DASS-21 Anxiety subscale, were

determined to be significantly higher than MDD and the DASS-21 Anxiety subscale, and PD

and the DASS-21 Depression subscale, using Fisher z-tests for investigating differences in

correlations. These findings indicate that the MDD and PD subscales in the current study

successfully detected their individual latent construct best. The performance of the MDD

subscale is unexpected. The PD subscale may have performed best in the current study

given that the DASS-21 was used as the comparative measure as the anxiety items of the

DASS-21 are more aligned with the construct of panic, e.g. ‘’I was aware of the action of my

heart in the absence of physical exertion (e.g., sense of heart rate increase, heart missing a

beat).’’ These findings further emphasise the clinical value of the RCADS as a measure able

to detect individual anxiety disorders (Mash & Hunsley, 2007).

17

A significant strength of the current research is the novel contribution the findings

make to the existing literature and clinical use of the RCADS, given that the measure is

recommended for use in clinical settings in the UK despite the lack of evidence regarding the

psychometric properties of the RCADS in an English speaking European population. In

addition, the study employed novel statistical methods for investigating the RCADS as a

four-point response scale using WLSMV to compare models, and MIMIC to investigate

differential item functioning to determine metric invariance. A limited number of previous

RCADS studies (e.g. Ebesutani et al., 2011) have done so, with none focusing specifically on

the youth-report RCADS across gender and age. The data handling techniques employed in

the current were an added strength given that alternative methods, such as maximum

likelihood estimation, can produce incorrect standard errors, reduce the strength of the

relationship between variables and result in potential pseudo-factors (Brown, 2006).

It would be of value for the RCADS subscales to be examined against anxiety-only

and depression-only measures e.g. SCARED (Hale, Raajimakers, Muris, & Meeus, 2005), BDI-

II (Beck, Steer, & Brown, 1996), to explore the divergent validity of the measure in terms of

these comorbid constructs further, given the inability of the DASS-21 to detect differences

between anxiety disorders. In addition, examining the RCADS with unrelated constructs e.g.

externalising difficulties (Chorpita et al., 2005) would also be beneficial. To further

investigate the clinical value of the RCADS, future research should examine the criterion-

related validity of the measure in terms of predictive and concurrent validity for variables

such as wellbeing, suicidality, substance use, etc., as this is an under-examined area of the

RCADS. In addition, the timeframe of the current study did not allow for test-retest

reliability to be examined with this population, and this should be examined in further

18

research given that the RCADS is utilised as an outcome tracking measure in the UK National

Health Service and elsewhere.

Sampling issues were a further limitation of the current research. The sample lacked

6th Year students due to exam restrictions. Although school type was varied, national

representativeness was not established, schools varied in terms of socio-economic

catchment areas, and differences for students from varying ethnic and cultural backgrounds

were not considered. The RCADS has previously demonstrated validity and reliability across

ethnic samples (Kösters, Chinapaw, Zwaanswijk, van der Wal, & Koot, 2015; Stevanovic et

al., 2016; Trent et al., 2013). Given that minority ethnic and cultural groups often experience

increased risk of mental health difficulties and disparities in healthcare (Alegria, Green,

McLaughlin, & Loder, 2015; O’Neill & Lowry, 2014), it is important to identify suitable

measures to detect distress in such populations. The current research provides a basis from

which further studies in an Irish context can be conducted e.g. with multi-ethnic samples,

with clinical samples, and validation of the RCADS-P. Shortened versions of the RCADS (e.g.

Ebesuanti et al., 2012; Sandin et al., 2010) should also be considered, given the need for

brief assessment measures in mental health care.

Ultimately, the validity of the RCADS as an assessment measure of internalising

disorders had not been established in an English speaking population prior to the current

study, despite recommendations for use of the RCADS in the UK (Law & Wolpert, 2013). The

current study was the first to establish the psychometric properties of the 47-item, six-

factor structure of the RCADS in a non-clinical Irish adolescent sample, providing evidence to

support the clinical utility of the measure in an English speaking, European population.

Despite the current findings revealing differences in some items on the RCADS, it should be

noted that these effects were small. Future research is warranted to establish whether

19

gender and age differences in certain items are consistent across a range of adolescent

samples.

References

Alegria, M., Green, J. G., McLaughlin, K. A., & Loder, S. (2015). Disparities in child and

adolescent mental health and mental health services in the U.S. New York: William T.

Grant Foundation.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental

disorders (5th ed.)

Asparouhov, T. & Muthén, B. (2010). Computing the strictly positive Satorra-Bentler chi-

square test in Mplus. Retrieved from

http://www.statmodel.com/examples/webnote.shtml

Author (2012).

Beck, A.T., Steer, R.A., & Brown, G.K. (1996). Manual for Beck Depression Inventory-II. San

Antonio, TX: Psychological Corporation.

Bennett, D. S., Ambrosini, P. J., Kudes, D., Metz, C., & Rabinovich, H. (200%). Gender

differences in adolescent depression: Do symptoms differ for boys and girls? Journal

of Affective Disorders, 89(1-3), 35-44.

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin,

107(2), 238-246.

Bernstein, G. A., Borchardt, C. M., & Perwien, A. R. (1996). Anxiety disorders in children and

adolescents: A review of the past 10 years. Journal of the American Academy of Child

& Adolescent Psychiatry, 35, 1110-1119.

Borsboom, D. (2006). When does measurement invariance matter? Commentary. Medical

Care, 44(11), S176-S181.

20

Bouvard, M., Denis, A., & Roulin, J. (2015). The French version of the Revised Child Anxiety

and Depression Scale (RCADS) in a nonclinical sample. Swiss Journal of Psychology,

74(3), 119-127.

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford.

Brown, R. C., Yaroslavsky, I., Quinoy, A. M., Friedman, A. D., Brookman, R. R., & Southam-

Gerow, M. A. (2013). Factor structure of measures of anxiety and depression

symptoms in African American youth. Child Psychiatry & Human Development, 44(4),

525-536.

Browne, M. W., & Cudek, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, &

J. S. Long (Eds.). Testing structural equation models. Newburg Park, California: Sage

Publications.

Brownhill, S., Wilhelm, K., Barclay, L., & Schmied, V. (2005). ‘Big build’: Hidden depression in

men. Australia & New Zealand Journal of Psychiatry, 39(10), 921-931.

CAMHS Outcomes Research Consortium (CORC). (2014). CYP IAPT and CORC: How do they

link? How are they different? Retrieved from

http://www.corc.uk.net/resources/downloads/

Cannon, M., Coughlan, H., Clarke, M., Harley, M., & Kelleher, I. (2013). The Mental Health of

Young People in Ireland: A report of the Psychiatric Epidemiology Research across the

Lifespan (PERL) Group. Dublin: Royal College of Surgeons in Ireland.

Chorpita, B. F. (1998). Revised Children’s Anxiety & Depression Scale (RCADS). Retrieved

from www.childfirst.ucla.edu/Resources.html

Chorpita, B. F. (2003). Revised Children’s Anxiety & Depression Scale – Parent Version

(RCADS-P). Retrieved from www.childfirst.ucla.edu/Resources.html

21

Chorpita, B. F., Moffitt, C. E., & Gray, J. (2005). Psychometric properties of the Revised Child

Anxiety and Depression Scale in a clinical sample. Behaviour Research and Therapy,

43, 309-322.

Chorpita, B. F., Yim, L., Moffitt, C., Umemoto, L. A., & Francis, S. E. (2000). Assessment of

symptoms of DSM-IV anxiety and depression in children: A revised child anxiety and

depression scale. Behaviour Research and Therapy, 38, 835-855.

Copeland, W. E., Angold, A., Shanahan, L., Costello, E. J. (2014). Longitudinal patterns of

anxiety from childhood to adulthood: The Great Smoky Mountains Study. Journal of

the American Academy of Child and Adolescent Psychiatry, 53, 21–33.

Cosgrove, V. E., Rhee, S. H., Gelhorn, H. L., Boeldt, D., Corley, R. C., Ehringer, M. A., Young, S.

E., & Hewitt, J. K. (2011). Structure and etiology of co-occurring internalizing and

externalizing disorders in adolescents. Journal of Abnormal Child Psychology, 39(1),

109-123.

deRoss, R. L., Gullone, E., & Chorpita, B. F. (2002). The Revised Child Anxiety and Depression

Scale: A psychometric investigation with Australian Youth. Behaviour Change, 19(2),

90-101.

Ebesutani, C., Bernstein, A., Nakamura, B. J., Chorpita, B. F., & Weisz, J. R. (2010). A

psychometric analysis of the Revised Child Anxiety and Depression Scale – Parent

Version in a Clinical Sample. Journal of Abnormal Child Psychology, 38, 249-260.

Ebesutani, C., Chorpita, B. F., Higa-McMillan, C. K., Nakamura, B. J., Regan, J., & Lynch, R. E.

(2011). A psychometric analysis of the Revised Child Anxiety and Depression Scales –

Parent Version in a school sample. Journal of Abnormal Child Psychology, 39, 173-

185.

22

Ebesutani, C., Korathu-Larson, P., Nakamura, B. J., Higa-McMillan, C., & Chorpita, B. (2016).

The Revised Child Anxiety and Depression Scale 25-Parent Version: Scale

development and validation in a school-based and clinical sample. Assessment

(Published online before print).

Ebesutani, C., Reise, S. P., Chorpita, B. F., Ale, C., Regan, J., Young, J., ... Weisz, J. R. (2012).

The Revised Child Anxiety and Depression Scale Short Version: Scale reduction via

exploratory bifactor modeling of the broad anxiety factor. Psychological Assessment,

24(4), 833-845.

Esbjørn, B. H., Somhovd, M. K., Turnstedt, C., & Reinholdt-Dunne, M. L. (2012). Assessing the

Revised Child Anxiety and Depression Scale (RCADS) in a national sample of Danish

youth aged 8-16 years. PLoS ONE, 7(5), e37339.

Farbstein, I., Mansbach-Kleinfeld, I., Levinson, D., Goodman, R., Levav, I.,Vograft, I., ... Apter,

A. (2010). Prevalence and correlates of mental disorders in Israeli adolescents:

Results from a national mental health survey. Journal of Child Psychology and

Psychiatry, 51(5), 630-639.

Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of

estimation for confirmatory factor analysis with ordinal data. Psychological Methods,

9(4), 406-491.

Frigerio, A., Rucci, P., Goodman, R., Ammaniti, M., Carlet, O., Cavolina, P., ... Molteni, M.

(2009). Prevalence and correlates of mental disorders among adolescents in Italy: The

PrISMA study. European Child & Adolescent Psychiatry, 18(4), 217-226.

Hale III, W. W., Raaijmakers, Q. A. W., Muris, P., & Meeus, W. H. J. (2005). Psychometric

properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED) in

23

the general adolescent population. Journal of the American Academy of Child &

Adolescent Psychiatry, 44, 283–290.

Haroz, E. E., Ybarra, M. L., & Eaton, W. W. (2014). Psychometric evaluation of a self-report

scale to measure adolescent depression: The CESDR-10 in two national adolescent

samples in the United States. Journal of Affective Disorders, 158, 154-160.

Holland, P. W., & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ: Erlbaum.

Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modeling: Guidelines for

determining model fit. Journal of Business Research Methods, 6(1), 53-60.

Jones, P. B. (2013). Adult mental health disorders and their age at onset. British Journal of

Psychiatry, 202, s5-s10.

Jöreskog, K., & Sörbom, D. (1993). LISREL 8: Structural Equation Modeling with the SIMPLIS

Command Language. Chicago, IL: Scientific Software International Inc.

Kessler, R. C., Avenevoli, S., Costello, J., Georgiades, K., Greif Green, J., Gruber, M. J., ...

Merikangas, K. R. (2012). Prevalence, persistence, and sociodemographic correlates

of DSM-IV disorders in the National Comorbidity Survey Replication Adolescent

Supplement. Archive of General Psychiatry, 69(4), 372-380.

Kösters, M. P., Chinapaw, M. J. M., Zwaanswijk, M., van der Wal, M. F., & Koot, H. M. (2015).

Structure, reliability, and validity of the revised child anxiety and depression scale

(RCADS) in a multi-ethnic urban sample of Dutch children. BMC Psychiatry, 15(132).

Law, D. & Wolpert, M. (2013). Guide to Using Outcomes and Feedback Tools with Children,

Young People and Families. London, UK: CAMHS Outcomes Research Consortium,

NHS IAPT & UCL Anna Freud Centre CAMHS Evidence Based Practice Unit.

Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the Depression, Anxiety and Stress

Scales (2nd Ed.). Sydney: Psychology Foundation.

24

Mash, E. J., & Hunsley, J. (2007). Assessment of child and family disturbance: A

development-systems approach. In E. J. Mash & R. A. Barkley (Ed.). Assessment of

Childhood Disorders Fourth Edition. (pp.3-52). New York: The Guilford Press.

Mathyssek, C. M., Olino, T. M., Hartman, C. A., Ormel, J., Verhulst, F. C., & van Oort, F. V. A.

(2013). Does the Revised Child Anxiety and Depression Scales (RCADS) measure

anxiety symptoms consistently across adolescence? The TRAILS study. International

Journal of Methods in Psychiatric Research, 22(1), 27-35.

McLean, C. P., Asnaani, A., Litz, B. T., Hofmann, S. G. (2011). Gender differences in anxiety

disorders: Prevalence, course of illness, comorbidity and burden of illness. Journal of

Psychiatric Research, 45(8), 1027-1035.

McLean, C. P., & Anderson E.R. (2009). Brave men and timid women? A review of the gender

differenes in fear and anxiety. Clinical Psychology Review, 29(6), 496-505.

Merikangas, K. R., He, J., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., ..., Swendson, J.

(2010). Lifetime prevalence of mental disorders in US adolescents: Results from the

National Comorbidity Study – Adolescent Supplement (NCS-A). Journal of the

American Academy of Child & Adolescent Psychiatry, 49(10), 980-989.

Miller, D., Vinet, E.V., Xu, X., Bt Mamat, N. H., Richardson, B., & Román, F. (2015). Factorial

invariance of the DASS-21 among adolescents in four countries. European Journal of

Psychological Assessment, 31(2), 138-142.

Muris, P., Meesters, C., & Schouten, E. (2002). A brief questionnaire of DSM-IV defined

anxiety and depression symptoms among children. Clinical Psychology and

Psychotherapy, 9, 430-442.

Muthén, L. K., & Muthén B. O. (1998-2013). MPlus user’s guide, 7th edn. Los Angeles, CA:

Muthén & Muthén.

25

Nunnally, J., & Bernstein, I. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-

Hill.

O’Neill, A., & Lowry, E. (2014). Health Inequalities and Young People in Ireland: A review of

the literature. Ireland: National Youth Council of Ireland (NYCI).

Park, A. L., Ebesutani, C. K., Bose, D., & Chorpita, B. F. (2016). Psychometric properties of a

Spanish translation of the Revised Child Anxiety and Depression Scale – Parent

Version. Journal of Psychopathology and Behavioral Assessment, 28(307).

Pine, D. S., Cohen, P., Gurley, D., Brook, J., & Ma, Y. (1998). The risk for early adulthood

anxiety and depressive disorders in adolescents with anxiety and depressive

disorders. Archive of General Psychiatry, 55, 56-64.

Reise, S. P., Widaman, K. F., Pugh, R. H. (1993). Confirmatory factor analysis and item

response theory: Two approaches for exploring measurement invariance.

Psychological Bulletin, 114(3), 552-566.

Sandin, B., Chorot, P., Valiente, R. M., & Chorpita, B. F. (2010). Development of a 30-item

version of the Revised Child Anxiety and Depression Scale. Revista de Psicopatologia

y Psichologia Clinica, 15(3), 165-178.

Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the Fit of Structural

Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures.

Methods of Psychological Research, 8(2), 23-74.

Southam-Gerow, M. A., & Chorpita, B. F. (2007). Anxiety in children and adolescents. In E. J.

Mash & R. A. Barkley (Ed.). Assessment of Childhood Disorders Fourth Edition. (pp.

347-397). New York: The Guilford Press.

Stevanovic, D., Bagheri, Z., Atilola, O., Vostanis, P., Stupar, D., Moreira, P. …, Ribas, R. (2016).

Cross-cultural measurement invariance of the Revised Child Anxiety and Depression

26

Scale across 11 world-wide societies. Epidemiology and Psychiatric Sciences.

Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics (5th Ed.). Boston:

Pearson/Allyn & Bacon.

Teresi J.A., & Fleishman J.A. Differential item functioning and health assessment. Quality of

Life Research, 16(Suppl 1), 33–42.

Trent, L. R., Buchanan, E., Ebesutani, C., Ale, C. M., Heiden, L., Hight, T. L., … Young, J. (2013).

A measurement invariance examination of the Revised Child Anxiety and Depression

Scale in a Southern Sample: Differential item functioning between African American

and Caucasian Youth. Assessment, 20(2), 175-187.

Tucker, L.R & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor

analysis. Psychometrika, 38, 1–10.

Ullman, J.B. (2001). Structural equation modeling. In B.G. Tabachnick & L.S. Fidell (2001).

Using Multivariate Statistics (4th ed, pp.653-771). Needman Heights, MA: Allyn &

Bacon.

United Nations. (2013). Social Inclusion of Youth with Mental Health Conditions. New York:

UN.

Willemsen, J., Markey, S., Declercq, F., & Vanheule, S. (2011). Negative emotionality in a

large community sample of adolescents: The factor structure and measurement

invariance of the short version of the Depression Anxiety Stress Scales (DASS-21).

Stress and Health, 27, e120-e128.

Wolpert, M., Cheng, H., Deighton, J. (2015). Measurement issues: Review of four patient

reported outcome measures: SDQ, RCADS, C/ORS and GBO – their strengths and

limitations for clinical use and service evaluation. Child and Adolescent Mental

Health, 20(1), 63-70.

27

Wynne, C., Doyle, C., Kenny, R., Brosnan, E., & Sharry, J. (2016). A first-level evaluation of a

family intervention for adolescent social, emotional and behavioural difficulties in

Child and Adolescent Mental Health Services. Journal of Child & Adolescent Mental

Health, 28(1), 33-46.

28