the diagnostic interview schedule for children (disc2.1) in spanish: reliability in a hispanic...

Pergamon

0021-9630(95)00083-6

/ ChiU Psychoi. Psychiat. Vol. 37. No. 2. pp. 195-204. 1996Eisevier Science Ltd

O 1996 Association for Child Psychology and PsychiatryPrinted in Great Britain. All rights reserved

0021-9630/96 $15.00 + 0.00

The Diagnostic Interview Schedule for Children (DISC-2.1) in Spanish:Reliability in a Hispanic Population

Julio C. Ribera, Glorisa Canino, Maritza Rubio-Stipec, Milagros Bravo, Jose J. Bauermeister,Margarita Alegria, Michel Woodbury, Sara Huertas and Luz M. Guevara

Hector R. Bird

Columbia University

Daniel Freeman

University of Texas

University of Puerto Rico

Patrick E. Shrout

New York University

The reliability across time, informants and interviewers of the Spanish translation of theDISC-2.1 was tested on a Puerto Rican Hispanic sample using a test-retest design. Levelsof reliability between clinic and community samples and between younger and olderchildren were compared to explore the sources of low reliability for certain pyschiatricdisorders. Parents' reports tended to be more reliable than those of their children, althoughthe difference was less obvious with older children. Reliability was generally higher forthe externalizing disorders and when the second interviewer was a psychiatrist rather thana lay interviewer.

Keywords: DISC, reliability, stability, Hispanic

Abbreviations: Re 93-175: The Diagnostic Interview Schedule for Children, DISC-CDiagnostic Interview Schedule for Children—Child version, DISC-P Diagnostic Inter-view Schedule for Children—Parent version, MECA Methodologic Epidemiologic Studyfor Children and Adolescents, SES socioeconomic status

The use of structured interviews that cati beadministered by lay interviewers to arrive at computergenerated diagnoses is rapidly becoming the standard inthe field of child psychiatric epidemiology. This hasreduced the high costs entailed in using clinicians tomake diagnostic assessments in epidemiological surveys,and has increased the reliability of child psychiatricdiagnosis (Gutterman, O'Brien & Young, 1987; Caninoet al., 1987a; Helzer & Coryell, 1983). This developmentparallels the use of structured psychodiagnostic inter-views for clinical and epidemiologic research in adults(Anthony et al., 1985; Burnam, Karno, Hough, Escobar& Forsythe, 1983; Canino et al., 1987b; Helzer et al.,1977b, 1985; Karno, Burnam, Escobar, Hough & Eaton,1983; Robins, Helzer, Croughan & Ratcliff, 1981;Robins, Helzer, Ratcliff & Seyfried, 1982; Robins,1985).

Several psychiatric interviews have been developedfor use with children; among them, the DiagnosticInterview Schedule for Children (DISC). The DISC is aninstrument which has undergone a process of systematic

Requests for reprints to: Dr Julio C. Ribera, VA MedicalCenter, Psychology Service (116B), One Veterans Plaza, SanJuan, PR 00927-5800, Puerto Rico.

revisions that has been ongoing during the past decade.The instrument allows nonclinician interviewers to askquestions in a highly standardized and structuredfashion. Response options are fixed and can be computeranalyzed to generate diagnoses according to theDiagnostic Statistical Manual (DSM) criteria (AmericanPsychiatric Association, 1987). The first testing of thepsychometric properties of the DISC was done on asample of psychiatric inpatients and outpatients (Cost-ello, Edelbrock, Kalas, Kessler & Klaric, 1982; Costello,Edelbrock, Duncan, Kalas & Klaric, 1984; Edelbrock etal., 1985,1986). This version was translated into Spanishby H. Bird and his colleagues and used in a majorpsychiatric epidemiological survey of the child andadolescent population of Puerto Rico (Bird, Canino,Rubio-Stipec & Ribera, 1987a; Bird et al., 1987b, 1988;Bird, Gould & Staghezza, 1992). Further work with theEnglish version was conducted by D. Shaffer and hiscolleagues which resulted in the development of theDISC-R (Piacentini et al., 1991, 1993; Schwab-Stone etal., 1993; Shaffer et al., 1988, 1993). Subsequent fieldtesting of the DISC-R led to further revisions thatresulted in one of the most recent versions, the DISC-2.1. Work with the DISC-2.1 has continued as part of acollaborative agreement between the NIMH and theUniversities of Puerto Rico, Emory, Columbia, and Yale.

195

196 J. C. RIBERA et al.

During the first 2 years of this collaborative agreement(better known as the Methodologic Epidemiologic Studyfor Children and Adolescents or, MECA) the feasibilityof administering the DISC-2.1 was tested in community,clinic, and convenience samples, and the psychometricproperties of the instrument were examined at all foursites.

In Puerto Rico the testing of the DISC-2.1 involved aprocess of translation, adaptation and testing of itsreliability and validity. The translation of the instrumentinto Spanish and its adaptation to the culture involved acomprehensive model (Gaviria et al., 1985; Flaherty,1987) which focused on cross-cultural equivalency infive dimensions: semantic, content, technical, criterion,and conceptual. This model had been successfully usedin the translation of other epidemiological instruments inPuerto Rico (Bravo, Canino & Bird, 1987; Bravo,Canino, Rubio-Stipec & Woodbury-Farina, 1991). Thisprocess of translating and adapting the DISC-2.1 hadbeen described in detail elsewhere (Bravo, Woodbury,Canino & Rubio-Stipec, 1993). In this paper we presentthe reliability findings of the Spanish translation of theDISC-2.1. The validity of the instrument will bereported in subsequent papers.

In studying the reliability of a diagnostic instrumentfor children and adolescents such as the DISC-2.1, thereis a need to take into account data obtained frommultiple informants. Although children's ability toreport on their feelings and behaviors has long beenrecognized (Rutter & Graham, 1968), most cliniciansand researchers agree that reports from other sourcessuch as the child's parents or teachers are necessary inorder to achieve a comprehensive picture of the child'spsychiatric state. The DISC, in all its versions, has atleast two forms; one to be administered to the child(DISC-C) and another to be administered to a parent oradult caretaker (DISC-P). Thus, considerable attentionhas been given to comparing the parent and child reportson the DISC (Bird, Gould & Staghezza, 1992;Edelbrock, Costello, Dulcan, Conover & Kalas, 1986;Piacentini et al., 1993; Rubio-Stipec et al., 1994;Schwab-Stone et al., 1993) as well as on other structuredpsychiatric interviews (Herjanic & Reich, 1982;Kashani, Orvaschel, Burk & Reid, 1985; Reich,Herjanic, Welner & Gandhy 1982; Weissman et al.,1987). Poor to modest agreement between parent andchild reports has been found in most of these studies.

Generally, most studies have found that parents tendto be more reliable informants than children (Edelbrocket al., 1986; Gutterman et al., 1987; Schwab-Stone et al.,1993). In reviewing the test-retest reliability of the DISCand the DISC-R, Schwab-Stone et al. (1993) found thatthe reliability of parent reports on both instrumentstended to be better than that of children's reports. Instudying the reliability of the DISC-2.1, Piacentini et al.,(1993) concluded that parents tend to be more reliableinformants for Attention Deficit Hyperactivity andOppositional Defiant Disorder while children tend tobe more reliable for Major Depression and ConductDisorder.

The reliability of diagnostic instruments for childrenis not only influenced by the type of informant used butby the age of the child evaluated. Edelbrock et al. (1985)

explored the impact of age differences on DISCsymptoms using 6-18-year-old children referred forinpatient and outpatient mental health services. Theauthors found that younger children tend to be lessreliable and show a greater decline in total symptomscores in second administration of questionnaires thanolder children. Parental reports on the DISC were morereliable for younger children (Edelbrock et al., 1985).The age effect on reliability has not been explored forthe more recent versions of the DISC (DISC-2.1).

Another factor affecting reliability of diagnosticinstruments is the type of sample used when testingthe instrument. Previous reliability studies of the DISC(and of most other structured or semi-structureddiagnostic instruments for children and adolescents)have been conducted with clinical rather than com-munity samples. The DISC was developed primarily forthe assessment of psychopathology in the generalpopulation, so that limiting the testing of the instrumentto clinical samples is not altogether appropriate. There isevidence from adult epidemiological surveys thatpsychiatric cases drawn from the community aregenerally less impaired, have fewer psychiatric symp-toms and seldom seek treatment, as compared with thosedrawn from clinical samples (Helzer et al., 1977a). Morerecently evidence from the MECA study has shown thatchildren from the community who meet the diagnosticcriteria have fewer symptoms than children drawn fromclinical settings (Jensen, Salzberg, Richters & Watanabe,1993). One would therefore expect that reliability ismore difficult to obtain in the community, where thepositive cases might be at or very close to, the diagnosticthreshold. Prior evidence has shown that reliability in thediagnosis of children and adolescents is easier to achievein referred and severe cases and more difficult to achieveon cases near the diagnostic threshold (Canino et al.,1987a). Additionally, it has been argued that individualswho receive mental health treatment are more familiarwith the psychiatric interview process and its languageand are thus more likely to reaffirm the presence ofsymptoms in a test-retest paradigm (Jensen et al., 1993).

In this paper we first report the test-retest reliabilityof the Spanish version of the DISC-2.1 in a probabilitysample of a Puerto Rican Community. The study adds tothe existing pool of data on the instrument by examiningits reliability across time, across two informants (parent/child), and across different interviewers (lay/psychia-trist) in a Hispanic community. A comparison of thelevels of reliability in clinical vs community samples andin younger vs older children is made in order to explorethe possible sources of disagreement for some specificpsychiatric disorders in children.

Subjects and Methods

Sample Description

A probability community sample (N = 248) of children aged9-17 years from the San Juan Metropolitan Area was selectedusing the 1990 U.S. Census. Census Enumeration Districts orBlocks groups served as Primary Sampling Units. A multi-stage sampling procedure was used to select householdsegments. Sampling blocks were distributed at random intotwo community sub-samples of 124 subjects each (subsample

DISC-2.1 IN SPANISH 197

Table 1Age, Sex, and SES Distribution for the Community and Clinic Subsamples of Children

Age/Sex

9-11 yearsFM

12-17 yearsFM

SES*LowMiddleHigh

Subsample AN

2724

3439

7149

4

Community

AT =124%

52.947.1

46.653.4

57.339.5

3.2

Subsample BN

1927

3840

595411

N=124%

41.358.7

48.751.3

47.643.5

8.9

Subsample AN

31

618

2341

N=28%

75.025.0

25.075.0

82.114.33.6

Clinic

Subsample BN

51

1123

4150

N = A6%

41.758.3

32.467.6

89.110.90.0

*SES refers to HoUingshead's socioeconomic index.

A and subsample B) to carry out two separate studies designedto measure the psychometric properties of the DISC-2.1. Twoclinical convenience samples (A and B) were also selected.These consisted of 74 children from mental health, drug abuse,and primary health care clinics. Cases from the various settingsused to recruit the clinical subsamples were thought to be at agreater risk for the targeted disorders. More specifically, 28clinical cases were added to subsample A, (18 outpatientclinical cases and 10 cases from a substance abuse treatmentcenter) and 46 cases were added to subsample B (whichincluded 16 psychiatric outpatient cases, 10 cases from ashelter for flood victims identified through a public pediatricoutpatient clinic, and 20 participants in a substance abusetreatment center). The second clinical convenience sample wasdesigned to test the validity of the instrument and thus a largernumber of cases was included to compensate for the tendencyof clinicians to identify lower numbers of positive cases withstructured interviews. In both subsamples the first interviewwas conducted by a lay-interviewer; in subsample A thesecond interview was also conducted by a lay-interviewerwhereas in subsample B, the second interview was conductedby a child psychiatrist.

The community subsamples resemble the 1990 Censuspopulation of children in Puerto Rico aged 9-17 years. Theyare evenly distributed by gender and most of them are of lowsocioeconomic status (SES). As expected in clinical samplesof the public sector in Puerto Rico, children in the clinicalsubsamples are predominantly male proceeding from the lowsocioeconomic class (Annual Report of the Administration ofthe Mental Health and Anti-addiction Services of Puerto Rico,1993).

The response rate for the first and second interviews afterenumeration on subsample A was 91.2% and 95.2%,respectively for a total response rate of 86.8%. The responserate for subsample B was 90.0% for time one and 90.3% fortime two for a total response rate of 81.2%.

Design and Procedures

Two variations of a test-retest reliability design were usedto test the reliability of the instruments across time and acrossinterviewers. The first administration of the DISC-2.1 wasconducted simultaneously with the parent and the child at theirhouseholds by two separate interviewers. These Time 1interviews were identical for both subsamples (N = 248).Although all interviewers at the second administration of theinstrument were unaware of the results of the first interview.

the second interview differed in the two subsamples. Eachparent-child dyad in subsample A was interviewed in theirhousehold on two separate occasions within a 2-week periodby a different lay interviewer each time. In subsample B, thesecond administration of the DISC-2.1 to each parent-childdyad was done by the same child psychiatrist at our ResearchInstitute. Participants were aware of the level of training ofwho was conducting the interview each time (lay intervieweror child psychiatrist). Conducting the second interview ofsubsample B at our Research Institute may have stressed thefact that the corresponding interviewer was a child psychia-trist. The order in which the respondents in subsample B(whether parent or child first) were interviewed was randomlyassigned. The second interview for subsample B involvedvarious phases. The data presented in this report is limited tothe first of these, in which the psychiatrists administered theDISC-2.1.1 adhering strictly to its structured format in asimilar fashion to the lay interviewer administration. This firststep of the time 2 interview with subsample B constituted areliability test of the DISC-2.1 (lay-psychiatrist administra-tion). It involved the administration of the instrument to thesame subjects under basically the same circumstances as thosepresent with subsample A but varying the level of training ofthe second interviewer. The use of psychiatric interviewers atthe second administration of a structured interview has beenconsidered as a special case of reliability in which the trainingof the interviewer is varied (Magaldy, Rogler & Tryon, 1992).The use of two different interviewers at the second adminis-tration of the instrument introduces maximum variability ofconditions to the test of reliability and thus submits theinstrument to a more demanding test.

The lay-administered interviews in both subsamples A andB were conducted by 21 lay interviewers who had completed a2-week training on the administration of the DISC-2.1 usingprocedures developed at Columbia University. All layinterviewers had at least a bachelor degree, while some hada master degree in various disciplines within the socialsciences. The clinician-administered time 2 interviews done onsubsample B were conducted by 10 qualified child psychia-trists who received a training on the administration of theDISC equivalent to that of lay interviewers. All interviewswere audiotaped, and 20% of the audiotapes were randomlyreviewed for quality assurance purposes. Quality" controlprocedures also included review of questionnaires, as well astelephone calls, letters or personal contacts with 20% ofrandomly selected interviewees to confirm that the interviewshad been properly conducted.


Table 2Reliability of the Spanish DISC-2.1, Combined Reports for the Community and Clinic Samples, Lay-Interviewer to Lay-Interviewer

Diagnostic clusterSpecific diagnosis

Anxiety DisordersSocial phobiaAgoraphobiaOveranxiousPanicSeparation anxietyGeneralized anxietySimple phobia

Depressive DisordersMajor depressionDysthymia

Disruptive DisordersConduct disorderOppositionalAttention deficit

Any DISC Disorder

N

205330407330

17557

38

+ / -N

3142

16077

11111069256

29

Community sample(N =

-/+N

6312011122140244

124)*

- / -N

61893735998047709398

10187

100978047

k

.32t

.55

.63

.13t-t.46

-.03t.48t.27t.29

-.02.66.82.56.53.46t

kSE

.081

.151

.197

.110—

.157

.029

.123

.139

.145

.014

.088

.122

.149

.131

.073

+/+N

102140663

101039852

14

+ / -N

4213012621220324

Clinic(N =

-/+N

2020010112031122

sample28)*

- / -N

81468

171010679

14101114144

A:

.50

.611

.21t

.59+

.77

.77

.18

.69

.73

.69

.58

.90

.59

.38t

.40

kSE

.174

.240

.328

.194—

.156

.150

.194

.162

.146

.198

.165

.099

.179

.254

.200

*N values among specific diagnoses due to missing values.tStatistically significant attenuation.|Less than five positive cases at time 1.Note: Columns designated (+/+) signify both interviewers found the cluster or specific diagnosis to be present, ( - / - ) both found itto be absent, and (+/-) (-/+) disagreement as to its presence.

Statistical Analyses

Diagnoses were generated by computer algorithms follow-ing strict DSM-III-R criteria based upon the report of parents(usually mothers) and children separately, as well as for theircombined DISC-2.1 report. The algorithm for the combinedreport allowed for diagnostic criteria to be met using an "or"rule, that is, the symptom is considered positive if reported aspresent by either the parent or the child. Reliability levels forcommunity and clinical samples were estimated separately forall analyses except those pertaining to age in which bothsamples were combined in order to obtain enough positivecases. The algorithm classifies as "missing" any subject forwhom the missing values are sufficient to prevent classifi-cation as either present or absent. Such missing subjects areexcluded from the reliability analyses for the diagnosis underconsideration.

Reliability indices were calculated on specific diagnosesand on four supraordinate diagnostic clusters that aggregatediagnostic categories into higher rank domains. Four suchdomains were examined: Anxiety Disorders, DepressiveDisorders, Disruptive Behavior Disorders, and Any DISCDisorder. The anxiety disorder cluster includes, social phobia,simple phobia, agoraphobia, overanxious disorder, separationanxiety, and generalized anxiety disorder. Included in thesupraordinate grouping of depressive disorders are thediagnoses of dysthymia and major depression. The disruptivebehavior supraordinate grouping includes the diagnoses ofattention-deficit disorder hyperactivity, conduct disorder, andoppositional defiant disorder. The "Any DISC Disorder"category refers to the presence of any of these previouslymentioned DISC disorders.

The kappa statistic was used to measure reliability. Kappa isa standard measure of concordance which controls for chanceagreement. Fleiss (1981) considers kappa levels below .40 asrepresenting poor agreement beyond chance, between .40 and

.75 as representing fair (acceptable .40-.59; good .60-.74)agreement, and above .75 as excellent. The Kappa statisticpools all disagreements and may be affected by the frequencyof cases which qualify for a syndrome or diagnosis. In general,unstable kappas are obtained for low base rate diagnoses.

The attenuation phenomenon (decrease in positive symp-toms reported upon readministration of instruments) has beenobserved to be a source of decreased reliability in test-retestdesigns (Jensen et al., 1993). Chi squares were generated toidentify significant reductions in the number of positive casesfrom first to second administration of the instrument to assessthose instances in which low reliability could be attracted toattenuation.

Our analytical approach focused on the community sample,since the DISC was developed for the assessment ofpsychopathology in the general population. Results from theclinical sample are mainly presented to confirm the tendenciesobserved in the community.

Results

The findings suggest adequate reliability for mostdiagnoses of the Spanish DISC-2.1.

Reliability Across Time

There was concordance across time for most of thepsychiatric disorders under consideration as measured bythe DISC-2.1. In the community sample, the test-retestreliability of the instrument when administered by twolay interviewers was generally acceptable (k > .40) foranxiety disorders. Every instance in which kappas were<.4O there was evidence of significant attenuation. Thesignificant reduction in the number of cases in thesecond administration of the instrument tended to lower


Table 3Reliability of the Spanish DISC-2.1, Combined Reports for the Community and Clinic Samples, Lay-Interviewer to Psychiatrist





Any DISC Disorder

+/+N

2174502393317015

27

+/-N

194181676663

14083

29

Community sample(N =

- /+N

6120050011011003

124)*

- / -N

65704450866253609696

10188

102987352

k

.47

.70

.70

.50t

.19

.42

.71

.43

.43

.39J

.42t

.oot.19t

.75

.43t

kSE

.087

.124

.165

.143

.161

.169

.110

.174

.174

.275

.166—

.161

.137

.076

+/+N

205290459

11105

197

111031

+ / -N

7418143355342135

Clinic(N =

- /+N

2102073222330322

sample46)*

- / -N

1423101526182720212228162726195

k

.58

.57

.761

.41

- t.20.53.67.62.61.53.66.84.78.68.49

itSE

.121

.166

.228

.146—

.177

.170

.134

.128

.132

.169

.115

.109

.106

.130

.163

*N values among specific diagnoses due to missing values.tStatistically significant attenuation.JLess than five positive cases at time 1.Note: Columns designated (+/+) signify both interviewers found the cluster or specific diagnosis to be present, ( - / - ) both found itto be absent, and (+/-) (-/+) disagreement as to its presence.

the kappas. However, in the presence of significantattenuation, this study cannot disentangle whether thelow kappas observed can be explained by the studydesign (test-retest) or by a true unreliability of theinstrument. In the clinical sample, although concordancefor some specific anxiety disorders was poor (e.g. simplephobia), most kappas ranged from acceptable (over-anxious disorder) to excellent (generalized anxiety). Thekappas for social phobia, agoraphobia, and panicdisorder, which ranged from poor to good are consideredunstable due to the small number of cases identified atthe first interview.

Contrary to the clinical sample, where the reliabilityof depressive disorders was good, the test-retestreliability of depression in the community was poor.This might be explained by the fact that at thesupraordinate level depression in the community sub-jects also showed significant attenuation.

Agreement levels ranging from acceptable to ex-cellent were obtained for the disruptive behaviordisorders in the community sample. In the clinic sample,agreement levels for the disruptive behavior disordersranged from acceptable to excellent for ConductDisorder. The kappa for Attention Deficit Disorder waspoor. However, this finding is also based on a smallnumber of cases and thus is considered to be unstable.

Summarizing, diagnostic agreement tended to behigher in the clinical sample than in the communitysample for most combined specific and supraordinatediagnostic categories as expected. The exception was"simple phobia" in which concordance was somewhathigher for the community respondents. In contrast to theclinical samples, significant attenuation was evident in

various supraordinate and specific categories of dis-orders in the community.

Reliability Across Interviewers

In the community sample, concordance was usuallyhigher when the second interviewer was a psychiatrist(see Table 3) rather than a lay interviewer (see Table 2).More diagnoses in the lay-lay comparison showedsignificant attenuation than in the lay-psychiatristcomparison. In the lay-lay comparison significantattenuation occurred with anxiety and depressivedisorders, whereas in the lay-psychiatrist comparisongreater attenuation occurred with the disruptive behaviordisorders. These patterns were not observed in theclinical sample since similar levels of reliability wereobtained regardless of who conducted the secondinterview.

Reliability Across Informants

Parents' report tended to be more reliable than thoseof their children (see Tables 4 and 5). However, thisdifference was less noticeable when age was taken intoconsideration (see Table 6). The reliability of children'sreports increased with age but as a general rule, parentstended to be more reliable than children.

This tendency was particularly noticeable in thecommunity sample when the second interviewer was apsychiatrist in which all concordance levels were greaterfor parents (see Table 5). Significant attenuation levelswere identified in the children but not in the parents'reports. As with the combined parent-child reports, a


Table 4Reliability of the Spanish DISC-2.1, Parent and Child Reports for the Community and Clinic Samples,Lay-Interviewer to Lay-Interviewer





Any DISC Disorder

Community sample

Parentk

.42

.65

.80t

.18

- t-.Oit

.oot

.38-.01-.Oit-.Oit

.60

.oot

.47

.55

.44

(AT =124)

Childk

.38*

.49t

.35*- .03

- t.65.09t.46.22*.26*.00.52.71.39t

-.02t.38

Clinic sample(AT =28)

Parentk

.41

.46

.23

.59—.65.64.40.51.52.32.52.56.65.33.33

Childk

.35-.05tl.OOt

.oot- t.oot.64t.00.44.50.64t.56.86

-.07t.45t.58

*Statistically significant attenuation.tLess than five positive cases at time 1.

higher number of disorders showed significant attenu-ation in the lay-lay administrations, than in the lay-psychiatrists' administrations. Acceptable to excellentconcordance levels for parent reports were observed formost diagnoses in the community for either the lay-layor lay-psychiatrist comparisons (i.e. overanxious,generalized anxiety, and all the depressive disorders)

or both (i.e. all the phobias, oppositional, and attentiondeficit).

In the clinical sample, parents report was generallymore reliable than that of their children with theexception of conduct disorder. No significant attenuationwas detected for either the parent or child report in thissample. When age was taken into consideration, older

Table 5Reliability of the Spanish DISC-2.1, Parent and Child Reports for the Community and Clinic Samples,Lay-Interviewer to Psychiatrist

Diagnostic dusterSpecific diagnosis




Any DISC Disorder

Community sample(N=124)

Parentk

.61

.75•42t.65

.j .

.33t

.74

.55

.65t

.65t

.49t

.52

.oot

.39t

.71

.61

Childk

.31*

.48

.16- .27

- t-.03t

.00

.39

.32

.32

.oot

.18

.oot

.oot

.39t

.35*

Clinic sample(N=46)

Parentk

.61

.68

.64t

.54

- t.31.38t.69.56.64.66.47.36t.68.65.34

Childk

.50

.oot

.oot

.06

- t.08t.66t.47.55.69.48t.52.72t

-.04t.87.53

* Statistically significant attenuation.tLess than five positive cases at time 1.


Table 6Reliability of the Spanish DISC-2.1, Parent, Child and Combined Report by Age Group in the Community and Clinic SamplesCombined, Lay to Lay-Interviewer and Lay-Interviewer to Psychiatrist

Lay interviewers to lay-interviewerAge group 7-11

(N=55)12-17

iN=97)

DiagnosticCluster

Parentk

Childk

Combinedk

Lay-interviewer to psychiatrists(iV=58)

jLess than five positive cases at time 1.

Parentk

Childk

(N=n2)

Combinedk

Anxiety disordersDepressive disordersDisruptive disordersAny DISC disorder

.68

.48t

.73

.65

.16

.48t-.03t

.18

.39

.73

.81

.42

.34

.41

.49

.34

.46

.24

.61

.53

.35

.48

.59

.49

Anxiety disordersDepressive disordersDisruptive disordersAny DISC disorder

.67

.65t

.52

.69

.30-.03t

.651

.38

.58

.65t

.38

.54

.62

.64

.60

.55

.46

.64

.36

.50

.51

.59

.73

.49

children tended to be more reliable than youngerchildren.

CommentsThis paper reports on the first reliability study of the

Spanish translation of the DISC-2.1 in a Hispaniccommunity sample. The findings are promising andsuggest that the DISC-2.1 in Spanish is an acceptablyreliable instrument that can be used in epidemiologicalresearch with Hispanic children and their parents.Reliability of the instrument is higher for the externaliz-ing disorders and for parent reports. DSM-IIIR classifi-cation based on the DISC-2.1 was generally reliableacross time, informants, and interviewers for manypsychiatric disorders.

Acceptable to excellent reliability levels were ob-tained in the community for most anxiety and disruptivebehavior disorders. Poor reliability was observed fordepressive disorders, overanxious and generalizedanxiety disorders. However, acceptable reliability forthese disorders was obtained only when the psychiatristwas the second interviewer. Attention deficit hyper-activity disorder and all of the phobias, in particular,showed greater reliability when interviewed by apsychiatrist at the retest.

The findings from this study are consistent with thosepreviously reported in other research using both theDISC (Edelbrock et al., 1985) and other instrumentssuch as the DICA (Herjanic & Reich, 1982); reliability isgenerally greater with the externalizing disorders (Dis-ruptive Behavior Disorders) than it is with the inter-nalizing disorders (anxiety and depression). Theseauthors have concluded that the disruptive behaviordisorders are characterized by observable, concretebehaviors that are more easily recalled and more difficultto conceal. By contrast, internalizing disorders arecharacterized by subjective, inner states. The differencein reliability may also be related to the fact that

externalizing disorders tend to be more pervasive andof greater chronicity, whereas internalizing symptoma-tology may be more transient and situational andtherefore less likely to be recalled, particularly whenthe symptomatology has subsided.

There are some methodological issues associated withthe design of our study that can explain the variability ofthe findings across interviewers and informants. Onelimitation of the design for the lay-psychiatrist com-parison is the fact that both parent and child wereinterviewed by the same psychiatrist. Although the orderof the interviews was determined at random, it isimpossible to assess the impact that having the sameinterviewer may have had on the positive responsesgiven by the second interviewee. While the psychiatristsadhered to the structure of the DISC-2.1, it isconceivable that an affirmative response in the firstinterview could have led the psychiatrist to inadvertentlycommunicate certain cues to the second respondent thatwould also prompt an affirmative response. If thishappened, it could have had some bearing on the higherlevels of agreement obtained in the lay-psychiatristcomparison, as well as on the apparently lower levels ofattenuation. Our data suggest (data not shown) thatagreement between lay and clinician administeredDISC-2.1 on parental report was equally good regard-less of the order in which respondents were interviewedfor the Any DISC Disorder Category. The child reportreached a similar level of agreement when the parentwas interviewed first. A poor kappa was obtained for thechild report when the parent was interviewed second.However, the latter finding is based on a very smallnumber of cases and thus it is considered to be toounstable to draw any conclusion.

The fact that respondents in subsample B were awarethat the second interview was conducted by a childpsychiatrist may also explain the higher reliabilityindexes obtained in this subsample; particularly whenthe parent was interviewed first. Informants may have


perceived the re-interview with a psychiatrist in aresearch setting as more meaningful than that with alay interviewer at their household, and thus could havelead participants to report symptoms previously en-dorsed. However, the design of the lay-psychiatristcomparison does not allow us to disentangle the possibleeffect of the interviewers' prior training, the interviewsetting, having the same psychiatrist administer theDISC to both parent and child, and the awareness of therespondents of who was performing the interview uponthe reliability of the instrument.

In those instances in which there was poor reliabilitythe phenomenon of attenuation or a significant reductionin the report of symptoms at the second administration,was uniformly observed. This was true in the lay-laycomparison and to a lesser extent, in the lay-psychiatristcomparison. Lower reliability was invariably associatedwith a systematic reduction of the symptomatologyreported in the second administration (attenuation) ratherthan to a random change in response. Therefore, whenpoor reliability was observed, it was associated toattenuation, which could be attributable more to thetest-retest design than to the instability of the instrumentitself. However, as previously stated, with the presentdesign it is not possible to disentangle poor reliabilitydue to attenuation from poor reliability of the instrumentper se.

Questions have been raised regarding the costeffectiveness of interviewing both the parent and thechild in epidemiological research. In the present study,higher reliability was obtained for approximately half ofthe disorders when the parent and child information wascombined than when the parents' or the child's reportswere considered alone. Parents consistently showed atleast acceptable reliability for most of the disordersunder consideration and parental report was usuallymore reliable than that of their children except forseparation anxiety and conduct disorders, for whichchildren showed better reliability than parents, thispattern is consistent with that observed using earlierversions of the DISC in English (Schwab-Stone et al.,1993), as well as the English version of the DISC-2.1(Piacentini et al., 1993; Jensen et al., 1993). The betterreliability exhibited by adult informants is probably theresult of cognitive factors that improve the adult's abilityto deal with abstract concepts and understand themeaning of questions, as well as facilitate a betterachoring of temporal sequences related to onset andduration. This hypothesis is consistent with results fromprevious research (Edelbrock et al., 1985), and issubstantiated by the results of the present study in whicholder children were more reliable than younger children.Young children were particularly unreliable in theirreports in the lay-lay interviews. This poses somequestions regarding the use of younger children asinformants in DISC interviews and the results suggestthat while it may be desirable to include adolescents asinformants of their symptomatology, the same may notbe true for young children.

The psychometric properties of most diagnosticinstruments have been tested in clinical samples. Aspreviously noted, clinical cases tend to produce morestable reports than community respondents, probably

because their symptoms are more severe and becausethey are more familiar with the psychiatric interviewprocess (Helzer et al., 1985; Jensen et al., 1993).Whatever the virtues of testing instruments with clinicalsamples (e.g. they are more likely to provide greatervariance along the psychopathological parameters ofinterest), the psychometric properties of an instrument tobe used in the general population also need to be testedin a community sample. Through the present study, theDISC-2.1 in Spanish has been subjected to a stringenttest of its reliability since it was tested in the communitywith two types of interviewers. As expected, betterconcordance was observed in the clinical than in thecommunity samples. Our results generally show accep-table to excellent reliability indices for the DISC-2.1 inthe community particularly for most anxiety andexternalizing disorders. Caution should nevertheless beexerted when using the DISC in community samples formeasuring depressive as well as overanxious andgeneralized anxiety disorders with lay interviewers. Atthis point, although in clinic samples the reliability ofthese disorders is excellent, in the community it is poor.

Attenuation of child reported symptoms in secondadministrations was a major source of unreliability.These findings have implications for longitudinal de-signs. In as much as significant attenuation is observedover periods of time longer than 2 weeks, the stability ofsymptoms over time would be affected by thismethodological artifact. The development of creativeways of diminishing attenuation is therefore advisedwhen using instruments such as the DISC-2 in long-itudinal research.

The reliability of results in the clinical sample of thisstudy fall along the lines of those obtained by Schwab-Stone et al. (1993) and those obtained by Jensen et al.(1993) using the English versions of the DISC withsimilar samples. However, results in our communitysample tended to be lower than those obtained in clinicalsamples. The similarities between our results and thoseof researchers using both the earlier versions of theEnglish DISC (such as the DISC-R, Schwab-Stone et al.,1993; Edelbrock et al., 1985) and the same version(DISC-2, Jensen et al., 1993) are encouraging. Theversions of the DISC that have been used differ not onlyin language but also in their format, content, anddiagnostic algorithms. The validity of the Spanishversion of the DISC-2.1 is also being examined as partof the MECA study and will be reported in the nearfuture. The DISC, either in its English or Spanishversions, is still in a developing stage. Based upon theresults of the first phase of its testing, the instrument hasbeen thoroughly revised (DISC-2.3) in an attempt toimprove its reliability and validity.

Summary

The reliability across time, informants, and inter-viewers of the Spanish translation of the DISC-2.1 wastested on a Puerto Rican Hispanic sample using a test—retest design. The DISC-2.1 is a highly structuredinstrument for children and adolescents that can beadministered by lay interviewers. It is used in epide-miological research for ascertaining the presence of


psychiatric disorders in children and adolescents, basedon the (DSM-III-R) diagnostic system. Levels ofreliability between clinic and community samples andbetween younger and older children were compared toexplore the sources of low reliability for certainpsychiatric disorders. Parents' reports tended to be morereliable than those of their children, although thedifference was less obvious with older children.Reliability was generally higher for the externalizingdisorders and when the second interviewer was apsychiatrist rather than a lay interviewer. Findingssuggest that the DISC-2 in Spanish is a reliableinstrument useful for epidemiological research withSpanish-speaking children and their parents.

Acknowledgements—The authors acknowledge the signifi-cant contributions of Elizabeth Costello, Ph.D. and PeterJensen, M.D. in their review of this article. This research wassupported by the Epidemiology Section of the NationalInstitute of Mental Health (NIMH) as part of a collaborativemethodologic epidemiologic study of children and adolescents(MECA) between the Universities of Columbia, Emory, PuertoRico (MH46732), Yale and the NIMH.

ReferencesAmerican Psychiatric Association (1987). Committee on

Nomenclature and Statistics. Diagnostic and statisticalmanual of mental disorders, third edition, revised.Washington, DC: American Psychiatric Association.

Annual report of the Administration of Mental Health andAnti-addiction Services of Puerto Rico (1993). G.P.O. Box61, San Juan, Puerto Rico 00936.

Anthony, J. C , Folstein, M., Romanoski, A. J., Von Korff, M.R., Nestadt, G. R., Chahal, R., Merchant, A., Brown, H.,Shapiro, S., Kramer, M. & Gruenberg, E. M. (1985).Comparison of the lay Diagnostic Interview Schedule and astandardized psychiatric diagnosis. Archives of GeneralPsychiatry, 42, 667-675.

Bird, H. R., Canino, G. J., Rubio-Stipec, M. & Ribera, J. C.(1987a). Further measures of the psychometric properties ofthe Children's Global Assessment Scale (C-GAS). Archivesof General Psychiatry, 44, 821-824.

Bird, H. R., Canino, G. J., Gould, M., Ribera, H. C , Rubio-Stipec, M., Woodbury, M., Huertas, S. & Sesman, M.(1987b). The use of the Child Behavior Checklist as ascreening instrument for epidemiological research: Resultsof a pilot study. Journal of the American Academy of Childand Adolescent Psychiatry, 26, 207-213.

Bird, H. R. Canino, G. J., Rubio-Stipec, M., Gould, M., Ribera,J. C , Sesman, M., Woodbury, M., Huertas-Goldman, S.,Pagan, A., Sanchez-Lacay, A. & Moscoso, M. (1988).Estimates of the prevalence of childhood maladjustment in acommunity survey in Puerto Rico. Archives of GeneralPsychiatry, 45, 1120-1126.

Bird, H. R., Gould, M. S. & Staghezza, B. (1992). Aggregatingdata from multiple informants in child psychiatry epide-miology research. Journal of the American Academy ofChild and Adolescent Psychiatry, 31, 7&-85.

Bravo, M., Canino, G. J. & Bird, H. (1987). La traduccion yadaptacidn del esquema de entrevista diagnostica DIS enPuerto Rico. Acta Psiquidtrica y Psicologica de Latino-america, 33, 27-42.

Bravo, M., Canino, G. J., Rubio-Stipec, M. & Woodbury-Farina, M. A. (1991). A cross-cultural adaptation of apsychiatric epidemiologic instrument: The DIS adaptationin Puerto Rico. Culture, Medicine and Psychiatry, 15,1-18.

Bravo, M., Woodbury, M., Canino, G. & Rubio-Stipec, M.(1993). The Spanish translation and cultural adaptation ofthe Diagnostic Interview Schedule for Children (DISC) inPuerto Rico. Culture, Medicine and Psychiatry, 17, 329-344.

Burnham, A., Karno, M., Hough, R. L., Escobar, J. L. &Forsythe, A. B. (1983). The Spanish Diagnostic InterviewSchedule: Reliability and comparison with clinicaldiagnosis. ArcAi'vei of General Psychiatry, 40, 1189-1196.

Canino, G. J., Bird, H. R., Rubio-Stipec, M., Woodbury, M.,Ribera, J. C , Huertas, S. E. & Sesman, M. J. (1987a).Reliability of child diagnosis in a Hispanic sample. Journalof the American Academy of Child and AdolescentPsychiatry, 26, 560-565.

Canino, G. J., Bird, H. R., Shrout, P. E., Rubio-Stipec, M.,Bravo, M., Martinez, R., Sesman, M., Guzman, A.,Guevara, L. M. & Costas, H. (1987b). The SpanishDiagnostic Interview Schedule: Reliability and concordancewith clinical diagnosis in Puerto Rico. Archives of GeneralPsychiatry, 44, 720-726.

Costello, A. J., Edelbrock, C. S., Dulcan, M. K., Kalas, R. &Klaric, S. H. (1984). Report of the NIMH DiagnosticInterview Schedule for Children (DISC).

Costello, A. J., Edelbrock, C. S., Kalas, R., Kessler, M. K. &Klaric, S. H. (1982). NIMH Diagnostic Interview Schedulefor Children. NIMH, Bethesda, MD.

Edelbrock, C , Costello, A. J., Dulcan, M. K., Conover, N. C.& Kalas, R. (1986). Parent-child agreement on childpsychiatric symptoms assessed via structured interview.Journal of Child Psychology and Psychiatry, 27, 181-190.

Edelbrock, C , Costello, A. J., Dulcan, M. K., Kalas, R. &Conover, N. C. (1985). Age differences in the reliability ofthe psychiatric interview of the child. Child Development,56, 265-275.

Flaherty, J. A. (1987). Appropriate and inappropriate researchmethodologies for Hispanic mental health. In M. Gaviria(Ed.), Health and behavior: research agenda for Hispanics(pp. 177-186). Chicago: University of Illinois Press.

Fleiss, J. L. (1981). Statistical methods for rates andproportions. New York: Wiley.

Gaviria, M., Pathak, D. S., Mitchell, T., Flaherty, J. A.,Winthrop, R. M., Martinez, H., Garcia Pacheco, C ,Richman, J. & Birz, S. (1985). Developing instruments forcross-cultural research. Paper presented at the AmericanPsychiatric Convention, Dallas, Texas.

Gutterman, E. M., O'Brien, J. D. & Young, G. (1987).Structured diagnostic interviews for Children and adoles-cents: current status and future directions. Journal of theAmerican Academy of Child Psychiatry, 23, 621-630.

Helzer, J. E., Clayton, P. J., Pambakian, R., Reich, T.,Woodruff, R. A. & Reveley, M.A. (1977a). Reliability ofpsychiatric diagnosis: II. The test/retest reliability ofdiagnostic classification. Archives of General Psychiatry,43, 136-141.

Helzer, J. E. & Coryell, W. (1983). Editorial: More on DSM-III: How consistent are precise criteria? Biological Psy-chiatry, 18, 1201-1203.

Helzer, J. E., Robins, L. N., McEvoy, L. T., Spitznagel, E. L.,Stolzman, R. K., Farmer, A. & Brockington, I. F. (1985). Acomparison of clinical and diagnostic interview schedulediagnoses. Archives of General Psychiatry, 42, 657-666.

Helzer, J. E., Robins, L. N., Taibleson, M., Woodruff, R. A.,Reich, T. & Wish, E. D. (1977b). Reliability of clinicaldiagnosis: I. A methodological review. Archives of GeneralPsychiatry, 34, 129-133.


Herjanic, B. & Reich, W. (1982). Development of a structurepsychiatric interview for children: Agreement between childand parent on individual symptoms. Journal of AbnormalChild Psychology, 10, 307-324.

Jensen, P., Salzberg, A. D., Richters, J. E. & Watanabe, H. K.(1993). Scales, diagnoses and child psychopathology: I.CBCL and DISC relationships. Journal of AmericanAcademy Child Adolescent Psychiatry, 32, 397-406.

Karno, M., Burnam, A., Escobar, J. L., Hough, R. L. & Eaton,W. W. (1983). Development of the Spanish-languageversion of the National Institute of Mental HealthDiagnostic Interview Schedule. Archives of General Psy-chiatry, 40, 1183-1188.

Kashani, J. H., Orvaschei, H., Burk, J. P. & Reid, J. C. (1985).Informant variance: The issue of parent-<hild disagreement.Journal of American Academy of Child Psychiatry, 24, 437-441.

Magaldy, R. G., Rogler, L. H. & Tryon, W. W. (1992). Issuesof validity in the Diagnostic Interview Schedule. Journal ofPsychiatric Research, 26, 59-67.

Piacentini, J., Jensen, P., Shaffer, D., Richters, J., Roper, M.,Bourdon, K., Davies, M. & Chen, J. (1991). Attenuation of'child psychiatric diagnosis: A detailed analysis. Paperpresented at the Annual Meeting of the American Academyof Child and Adolescent Psychiatry, San Franscico,California.

Piacentini, J., Shaffer, D., Fisher, P., Schwab-Stone, M.,Davies, M. & Gioia, P. (1993). The Diagnostic InterviewSchedule for Children-Revised Version (DISC-R). III.Concurrent criterion validity. Journal of the AmericanAcademy of Child and Adolescent Psychiatry, 32, 658-665.

Reich, W., Herjanic, B., Welner, Z. & Gandhy, P. R. (1982).Development of a structure psychiatric interview forchildren: Agreement on diagnosis comparing child andparent interviews. Journal of Abnormal thild Psychology,10, 325-336.

Robins, L. N., Helzer, J. E., Croughan, J. & Ratcliff, K. S.(1981). The NIMH Diagnostic Interview Schedule: Itshistory, characteristics, and validity. Archives of GeneralPsychiatry, 38, 381-389.

Robins, L. N. (1985). Epidemiology: Reflections on testing thevalidity of psychiatric interviews. Archives of GeneralPsychiatry, 42, 918-924.

Robins, L. N., Helzer, J. E., Ratcliff, K. S. & Seyfried, W.(1982). Validity of the Diagnostic Interview Schedule,version II: DSM-III diagnosis. Psychological Medicine, 12,855-877.

Rubio-Stipec, M., Canino, G., Shrout, P., Dulcan, M., Free-man, D. & Bravo, M. (1994). Psychometric properties ofparents and children as informants in child psychiatryepidemiology with the Spanish Diagnostic InterviewSchedule for Children (DISC.2). Journal of Abnormal ChildPsychology, 22, 703-720.

Rutter, M. & Graham, P. (1968). The reliability and validity ofthe psychiatric assessment of the child: I. Interview with thechild. British Journal of Psychiatry, 114, 563-579.

Schwab-Stone, M., Fisher, P., Piacentini, J. C, Shaffer, D.,Gioia, P. & Davis, M. (1993). The Diagnostic InterviewSchedule for Children-Revised Version (DISC-R): II. Test-retest reliability. Journal of the American Academy of Childand Adolescent Psychiatry, 32, 651-657.

Shaffer, D., Schwab-Stone, M., Fisher, P., Davies, M.,Piacentini, J. & Gioia, P. (1988). A revised version of theDiagnostic Interview Schedule for Children (DISC-R):Results of a field trial and proposal for a new instrument(DISC-R). Technical Report.

Shaffer, D., Schwab-Stone, M., Fisher, P., Cohen, P.,Piacentini, J., Davies, M., Edelbrock, C. & Regier, D.(1993). The Diagnostic Interview Schedule for Children-Revised Version (DISC-R): I. Preparation, field testing,interrater reliability, and acceptability. Journal of theAmerican Academy of Child and Adolescent Psychiatry,32, 643-650.

Weissman, M. M., Wickramaratne, P., Wamer, V., John, K.,Prusoff, B. A., Merikangas, K. R. & Gammon, G. D. (1987).Assessing psychiatric disorders in children: Discrepanciesbetween mother's and children's reports. Archives ofGeneral Psychiatry, 44, 747-753.

Accepted manuscript received 8 March 1995

the diagnostic interview schedule for children (disc2.1) in spanish: reliability in a hispanic...

Documents