(2005).D. P. Flanagan& P. L. Harrison(Eds.).

Contemporary intellectual assessment:Theories, tests, and issues (2nd ed.; pp. 251-268).

New York: Guilford.

12

Issues In Subtest Profile Analysis

MARLEY W. WATKINS JOSEPH]. GLUTTING

ERIC A. YOUNGSTROM

M ore than 250 million standardized tests arc ,Hlrninisrcrcd to puhlic school students each year in the United States (Sal via & Yssddykc, 1998). Although mll ch $chool­based testing is accomplis hed in groups, a substantial nu mber of standardized tt:StS are also employed in individual evaluations. For eX:lmple~ millions of children served in spe­cial education programs have particip:m:d ;n individ ual ps),choedl1cation;:l\ evalu<lt ions (U.S. Department of Educarion, 100'1). Indi­vidual evaluations to determi ne special edu­c::t rion ehgibllity often include a srand.udized measure of intellectual function ill~. Of rhe ava ilable mdividua l intelligence rests, the \'{fcchslcr $t:a!cs arc the most frequently used (Ka ufman & lichtenberger, 2000). Among sdltlol-age t:hildrell, the \Vechsler Im elli­gence Scale for Children (WISC; Wechs­ler, 1949), Wechsler Intelligence Scale for ChilJr~n-R(;viscd (WrSC-R; Wechsler, 1974), and Wechsler Intelligence Scale for Childrcrl- Thi rd Edi tion (WISC-I1I; Wechs­ler, 1991) have been the most pop ular (Kamphaus, Petoskey, & Rowe, 2000; Oak· land & HIl, 1992) fo r decades.

Typically, interpreta tion of individual 111-

tdl igence tests is based on a hierarchical, top-down model that first considers the globa l IQ scores. Next, to extract more in­formation from lhe lest, distinct p,u [(; rns or profiles of subtest scores are .malyzed.

251

Finally, mdividua l subtests are considered In

iso lation. This practice of interpreting the pattern of subtest scores attained by child(en on individual measures of mtelligence is knuwn as S1Ibtest fJYo file. analysis. Based on these principles, elaborate subrest interpreta­lion systems (Ka ufman, 1994; Sattler, 200 .1 ) have achieved wide popularity in school psy­chology traini ng and practice (Alfonso, Oak­land, LaRocca, & Spanakos, 1000; Grorh­\;lamar, 1997; Kaufman, 1994; Pieiifer, Reddy, Klelzel , SchIIlclzcr~ & Boycr, 2000). For example, approximately 74% of school psychulogy tra ining programs place moder­ate to gl'eat emphasis on lhe use of su btcst scores in rh e: ir individual cognitive assess­ment courses (Alfonso et aL, 2000). Among a sample of school psychologists, Pfeiffer and coll eagues (2000) fou nd that almost 70% reported profile analysis LO be a useful feature of the WISe-III, and 29% reported that they derived spt:cifie va lue from indi vid­lIal WISC-I1I subtests.

PURPOSES

Profile analysis has typically been applied fo1' twO major purposes: (I ) diagnostically dis­criminating average and exceptional chil­dren, and (2) identifyinB specific cognitive strengths and weaknesses. Wechsler (19S8 )

252 INTERPRETIVE APPROACHES

himself may have encouraged the process of diagnostic., lIy iIHcrprering child ren's sllbtcst profiles when he ;Jdvanced the hypothesis that child hood schizophrenia could ~ iden ­tified by high scores on the Pictu re Comple­lion and Obiect Assembly subrests and low scores on the Picture Amm gemcnt ami Digit Syrllho] subtests . Eventually, more than 75 patterns of subrest variation were identified fo r rhe Wechsler series alone (McDermotl, Fantuz.zo, & Glutting, 1990). Currently, ma ny differen t 5ubu.st profiles arc purpo n ­cd ly diaguostic of learning and emotional problems (e.g. , Kaufm:ln, 1994).

To identify cognitive strengths and weak­nesses, specific patterns of su brest scores are presumed (0 5ubsranria liy invalida te global inrdl igem:e indi(;cs (G roth-Ma rnat, 1997), so th at subtesr patterr:s, rather than IQ compositt:s, bt:eome the foc us of interpreta­tion. Subtests that are significo ntly higher or lower than a child's own average arc cousid ­ered relative strengths or weaknesses, re­spectively. Next, hypotheses concerning the lI nderlying causes of significant subtest vari­ations ~lfe identified by locating abilities thought to be shared by tWCl or more sub­tests. Extensive lists of the abilities pre­sumed co underlie each sll bresc are provided by Kaufman (1994), Sauler (200 1) , and Kaufnlllll and Lichtenberger (2000). Fina lly, th ese hypotheses arc used ro generate educa­t ional and psychological interventions and (I.'medial suggestions (Z l.' idner, 2001).

BRIEF HISTORICAL REVIEW OF PROFILE ANALYSIS RESEARCH

Scatter

Attempts to analyze IQ subrest variations d:u e back to the inceptioll of slandard ized iutelligence testing (Zachary, 1990). f or ex­ample, Binet suggested rhat t he passes and failures of psychotic or a lcoholic examinees would exhibit more sCtlrter across age levels on his sca le than would other examinees (see Matarazzo, 1985). £ a.riy researchers hyporh­esized that subtest searter wou ld predil:t ~(,. hula~tk VOlelllial o r mem~rshj p in exccp­tiona I groups (Hartis & Shakow, 1937). Un­even subtest scores were assumed to be signs of pathology or of greater potentia l than in­dicated by averaged lQ com posites. H un­dreds of studies we re conducted to test these

hypotheses. Based on their aT1a lys~<; o f de­cades of IQ subtest scatter research, Ktamer, Henning-Stour, Ullman , and Schellenbl:rg (1987) fo und no evidence tilat subtest scatter uniquely identified any diagnostic group, and they opi ned that "we rega rd scatter analysis as inefficient and inappropri:'lte" (p. 45). A narrative review of 70 years of research on suhtest scatter also arrived at pessimistic conclusions concerning its diagnosric accuracy (Z immerman & Woo­Sam, 1985 ). Although isolated studies found abnorma l scatter within cl inical groups, dif­f:.:rences tcnded to disappear when adeq uate comparison samples were u~d. Zimmerman :md Woo-Sam (1985) observe.:d thaL "exten­sive scatlcr proved to be both typical and 'normal,' and thus of limited lise as a diag­nost ic fcalure" (p. ~78) .

Sinlliar negari"e results were reportc~d in a longitudi nal srudy of t hl: WISC-R. Moffitt and Silva (19~7) concluded that per ina lol l, neurological, and hea lth problems did !lot

cause extrcme Verbal IQ-Performance lQ (VH ... H'H ... U d iscrepancies, and found that nei­[her behavior problems nor motor problems were significandy related to VIQ-PIQ scores. Furrhermore, VIQ-PTQ score.: dis(.;repancies wcrc unrelia ble across time. T hat is, the ma­jori ty of children with extreme VIQ-PIQ score discrepancie.:s did not rrI<l intain such a large difference when tested with the wrsc­R 2 years later. T hus "VIQ-PIQ d iscrep­ancies ate of doubtful diagnostic value" (M offitt & 5ih·a, 1987, p. 773).

T he quantitative combination of results from 94 s[Udl l.'s (N = 9,372) also demon­strated that subtest scatter and scatter be t""een VIQ and PfQ fa iled to uniquely iden­tify children wi rh lea rning disabiliries (LOs) (K1V:l1c & Furtlcss, 1984 ). For example, the average VIQ-PIQ difference for child ren WiTh LDs was only 3.5 poims-a difference found in 7~% of the normal population. In sum, subrest seaner was determi ned to be of "Iitrle value in LD diagnosis'" (Kavale & Forness, 1984, p. 139).

Suble$l Profiles

Given the popularity of the \'(Iechsler scales, \,(fechsie r subtest profiles have bet:n thc source of much resea rch . For ex ample, the va lidity of usi ng wise and WISC-R subrests for diagnosi ne LDs wa s the focLis of a mela-

analysis by Kavale and Forness (1984). This qua ntitative summary of 94 studies revealed that " the differential diagnosis of LD using the wrsc, .1h_hollgh intuitivcl y appealing, appearS to be unw;lrramed" because "'re­ga rdless of the m:mner in which WISe sub­tests were grouped aod regrouped, no recat­egorization, profile, pattern, or factor cluster emcrgcd as a 'dinically' significant indicator of LD '" (Kavale & forness , 1984, p. 150).

Taking another approach, Mueller, Dell­ois, and Short (1986) statistically clustered the WISC-R subtest data of 1·19 samples of Ilonexcep tiona l and exceptional children (N = 13,746) to detenmoe whether profiles would erneree that wert: diagnostically char­a..:teriscic of various disabilities. Results indi­cated that WISGR subtest profiles wcre typ­ically marked by gene_ral intellectual level but could not reli.lbly distinguish among di­agnostic groups. Like Kavale and Fomess (1984 ), Mueller and colleagues concluded rh:lt \V .. ch"ler <;\lh rf':~r Jlrnfi1 ... ~ were not help­ful in differentiating among children with emotional and learnine impairments, and they recummended that IQ tests be used only to estimate global intellectual functioning.

The poor diagnostic accu racy of slJ bresr pro files has generaljzed across tests and cul­mres. For example, R ispens and colkaglll:s (1997) analyzed the ability of subtest profiles from the Dutch version of th e WISC-R to dis ti ngllish among .5 11 childrell with COIl­

duct d isorder, mood disorder, anxiety disor­der, a ttention deficit disorder, and other psy­cliialr!e d!surdt:rs. Ris lJt"J1~ <llId colleagues fo und thar subtest patterns did not sig­nificandy diffcr across the various groups and concluded that " WISC profiles ... can­not comribnte to differentia l di<lgnosis" (p. 1593).

Snbte-st profiles hav .. <1 1so bf"f:n f(l ll nri ro be unexceptional markers when empirically generated subrest profiles serve as a norma­tive sta ndard against which suhtest profiles obtained from clinical groups are compared. If subtest profiles are markers of disability, then unique profiks should be fo und in lhe sampl e wifh disa bilities. On the other hand , if subtcsr profiles arc not distinctive of dis­ability, profiles from the sample without disa bilities shol1 1d replic<lte for the sample: with disabilities. To tes\" this hypothesis, Watkins and Kush (1. 994) a ppli~d norma­tive \XfISC-R ~ublesl profiles (McDerlllott,

253

Glutting:, ]olles, \'(latkins, & Kush, 1989) 1"0

1,222 stlldents with LDs, emotional handi­caps, and memal retard<lfion. T hey found that 96% of the children with disabilities dis­played subrest profiles that were sim ilar TO

those of the WISC-R slandardization sa m­ple. No statistical or logical patterns could be detected in the sllbtest scores o f the 4% of students w ith disabilities who exhibited pro­fi les dissimi lar to [hose Df the srandardiza­lion sam ple.

Another normative comparison applied [he Wechsler Adult Intelligence Seak--Rcvised (WAIS-R) standardization sample core pro­fi les to 161 adults with brain dam<1.ge and fu und that 82 % exhibited typical Or uornlal subtest profiles (Ryan & Bobac_, 1994 ). Pa­tients with ul1 ique profiles did no t differ on the basis of age, education, or organic etiol­ogy, so the a typica l p rofiles did not contrib­ule any diagnostic information. The WAlS -R core profiles were also applied to 194 college sfllden ts with LDs (Maller & McDermott, 1997). Almost 94% of these students were fo und to ha ve norm:n i\·dy typical subtest profiles. Unique profiles we re disparate and not indicative of subtypes of LDs .

Hi sloo';cal Review: Summary

Subtest scatter had been determined to be clinically ineffectual as eady as 1937 (Harris & Shakow, 1937). Cautions concerning the accuracy of subtest profiles were frequent (McNemar, 1964; Simer sen & Sll therbnd, 1974). By 198J, Frank was able to say tha t "in spite of the fact that the Wt:chs1cr looked like it would be ideal for a comparat ive study of the intellecruallcognitive beholvior of vari ­OllS psychopathological types, 40 years of re­search has fai led to support that idea" (p. 79). A cn mnhrive hocly of fe.<>e:l fch eviele.nee has shown that neither subtest scatter nor subtest profiles demonstrate acceprable ac­curacy ill discriminali ng among d iagnostic groups.

METHODOLOGICAL, STATISTICAL, AND PSYCHOMETRIC PROBLEMS WITH SUBTEST PROFILE ANALYSIS

Fundamental methodological, statistical, and psychometric problems cause subtest aualysis n:suils 10 ue lUore ilJu50!Y tlJau (eal

2~4 INTERPRETIVE A PPROACHES

and to rcpresem more of a sll'lred pro­fessional myth (Fa list, 1990) than clinical­ly astme detective work (Kaufman, 199/~ ) . Although a large numbl-.r of pro blems have been i d~ntifien (Glutting, \"'('atkins, & You ngstrom, 2003; \X:.1tk ins, 1003), this chapter conCentr:HCS on four major issues: reliability, i!Jsarillc measuremen.t, group mean di{{ereuces, and inverse probabiljties.

Reliability

The wea k reliabilit), of Sll bn::st scores has beel! repeatedly demonstrated. For examp1t::, none of the WISe·I11 subrests reached (he internal-consistency teliabi lit>· criter ion of <:= .90 recommended by Salvia and Ysseldyke (1998) fo r making decisions about individu­a ls. Similar resu lts were found on the Wechs­ler Adult Intelligence $c;lle-Third Edition (\vATS-lIT; Wechsler, 1997), where the me­dian internal cons istency of the sllbtests was .85, and (he \,\/echsler Inrell igence Scale fo r Childrell-Fourth Frli rinn {WISC-IV; Wechs­ler, 2003}, where the median interna l consis­tency of the subtests ", .. s .86.

T he stabi lity of subcest scores across time has a lso ucen inadeqll;'lre for indi vidual deci­sions. After ;'Idministcring (he Durch version of the WISC·R to a group of ch ildren three times at 6-monrh imcrva(s, .:-.Jeyens and Aldenkamp ( 1996 ) concluded t bat, "subtest scores show only f"ir to good, or even pOOt stahili!}', wh ich suggesLS that cha nges in sub­test scores, as stated :'y many researchers, sh ol1ld not /be taken] into account when evaluating the cU~llili \'(' development of chil elren o( nocmol intelligence" {po 1681. Mocl: recently, Smith , Smith, Bramlett, an d H icks (1999) tested a smaJi sa mple of ru ra l school ch ildren with the WISC-1lI and agnin 3 years later. T he merlian stahi lity coefficient for the su brests IVai .59, comp:1red to .83 for the com posi te IQ scores. Us ing a large national sampl(' of stu dents twice tested for special ed uca tion eligibility (N = 667), Canivez and Watkins (19981 fou nd a med ian sllbtest sta­bility coe({icient of .68 and a median stabil­it~· coefficient of .87 fo l' the IQ composites. E\'en short -term stability coeffi clents have been i.nadequate for individual decision;;. For example, the median short-term stabil ity of the Stanford- Binet Intelligence Scales, Fifth Edirion (Raid, 20031 subtests \vas .825, in conrrast to .90 for the Full Scale lQ {FSIQ }

score, and the median W ISGN short-term stabili ty coefficient wa s .78 for subtests, comp:lred to .89 for the FSIQ.

Howe\'er, t hese p ublished reliability coeffi ­cients are overestimates of the typica l reli­abili ty of subtesc scores, lor at least two rea­sons. Firsr, they were calculated by tesf companies or researchers who paid dose at­tention to standardizati on procedure" and double-checked scoring accuracy (Feld t & Brennan, 1993; T horndike, 1997) . In clinica l practice, :u lmiui:'lfaci0l1> clerical, and scor ing errors are ubiquitous (Belk, LoBello, Ray, & Zacha~ 2002). Seconrl, il1lernal­consistency and stabi lity reliability codfi­cienrs do nOt mke inro account all tyP{$ of measurement error. For example, Schmidt, Le, and !lies (2003 ) fo und thar the reli<lbili ty of a measure of genei'al menta l ability was o .... erestima ted by ;1bout 7% when tt;'lnsien r measurement error was ignored.

Even worse, subtesr profile analysis in~ .... n lvcs the comparison of multiple differe nce scores. T he. rel iabi lity of the difference be­tween twO scores is lower t ha n the reliability of either scnre a lone (Feldt & Brennan, 1993). Thc increased error generated by the usc of difference scores makes even the best subtest-to·subtest compa rison unstable (e.g., the reliability of the \'<fISC-1ll Block Design and Vocabu lary :.uute:;> ls is .B7, bnr the reli ability of their difftrence is .76).

Ipsative MealureOlent

Ma ny suhresr interpretative systems move 11\\':1 )' from no rmatiZIl' ml";'I.l:urement and Ln­stead rely on i[ISalille measurement princi­ples (Cattell, 1944). That i .~, subtest scores a T!: subtracted from mean compos it~ scores for an indi\"irlual. T hereby, the scores al'e transformed into person-rdlltive metries a nd away from their original popularion­rehrive melric {McDermott et aI. , 1~901 . Tps<ltive measurement is concerned with how a chil d 's s ubtes[ scores rel ate to his or her personalized , ayer:l ~c ped ormance, "nrl discounts the infl uence of global intell i· gence (jensen, 2002 ). For example, n\'o hypothetical students' Itormative (popu la· tion·relative) an d ipsa rive (J>(:tson·relati\'e) scores are displayed in Table 12.1 . These two srudenrs h.we identical ipsa ri ve scores, bur [heir normative scores arc very differ­enr.

Subk:;1 Profi le A nalysis 255

TABLE 12. 1. Normative (Population-Relative) and Ipsative (Person-Relativl!) Scorcs for Two Hypothetical Examinees

SlllJCll l A Sludenl n Norm. Ipsa[J\·c Korm. Ipsati\·c

Sulm:st $cor~ scon: $(0"';- $Core

A 3 -4 La _4 B 7 0 14 LJ C " .4 lS .4 "-·i .. "" 7 0 14 0

The ipsat ive perspective holds llltuitive ap pea l, because it seems to isolate and amplify aspt:cls uf cogll ili\'!: aui li ty Nt:vt:fllll:less transformation of the sco~~ metric fro~ normative to ips:nivc i!'; ps)'chomctrically problematic. For example, McDermott and colleagues (1990) demonstrated tha t the ip­satil.:l tiOIl of WISC-R scores produced a loss of almost 60'% of that tl""st·s r l""liable va ri­ance. McDermott and Glutt ing (1997) rcpli ­cated those results with the Different ial Abil­ity Scales (DAS; Elliott, 1990) and \VJSC-TTl. Thcy found tha t, on average, ipsati,'e scores lost nvo-thirds to th ree-fourths of the re­liable informarion providcd by normative scores.

This information loss has hcen concretely demonstrated by analyzll1g the stability of ipsatlve suhrest profiles. The remporal stabil­iry of subtcst profiles among 303 ra l1 dolTl­h· selected children tested twice as part of \XTISC-R val idation studies, and alflong 18~ children h .... ice tested for special educa ­tion eligibility, W;:IS compmed (.McDermott, Famuzw, Gl uttillg, \'('atkins, & Baggale~', 1992). Classificatorr stability of relative cog­nitive strengths and weaknesses idcnri fied by suutesl elevations and depfession~ was neal: cha nce levels for both groups of ch ildrcn. Liviugston, Jellnings, Re ynolds, and Gray (2003 ) also found that the stability of \'(lJSC­R su btest profil es across a 3-y,~ar tcst- retest interval \vas tOO low for clinical use.

Similar tempora l inswbiliry of subtest pat­ter ns was fo und fo r 579 students who were tWICe tested for special edllco.tion digibility with rhe WISC-III acros~ a 2 .S-year ime[val (Watkins & Canivez, l004 ). Based on 66 subtest composites, suhtest-based strengths and weaknesses n::pJ icatcJ across ttst- retesl occasions at chance levels (med ian bl.ppa :::

.02). BeC:l.use subtest-based cognitive strengths alld weakntsses are unrel iable, recommenda ­tions based on them wil! also be unreliable.

Roth pr3 ctic:1 I :1 r rl tht'ClTetic:1 1 ;lna lyses suggest that the mathematical propenies of ipsative methods are profonnd ly different from lhose of familiar norma tive methods (I licks, 1970; !v1cDermott et 0.1., 1990, .J 992). T hus ipsativc ,ubres[ scores cannOt be interpreted as if they possessed the psycho­metric properties of normative scores. Based on this evidence, Jensen (2001 ) concluded th:l.t " the ips:l.tive use of test batte(ies like the \Xi'echs ler scales, the Kaufman scales, and the Stanford-Binet IV is nearly worthless" (p. 11).

Group Mean Differences

Identification of IQ 5ubtest profiles has gen­f'. ra ll" been based upon statistically signifi ­cam group differences . That is, a group of du ldre'n wiTh;} pMricl1b r rl isorcir-r i~ icit'nri ­fied, and their mean subleSt score is com­pared to the mean su~test score of a group of chil dn:n \virhollt thi: di sorder. Statistically Sign ificant su btest score diffel-ences between the nvo groups are subsequently interprered as evideuce that lht profile is diagnoslically accurate fo r individuals. However, differ ­ences in group mea n scores rnay not support individua l interpret<ltion. ;-Jor does statisti­ca l significance of gronp diffcn:ncl:S cq uate to individual discrimination. As nOled by El­wood (19 93), " SIgnificance alone does /lot n:net:l dle size of lhe gru ulJ differcIKt:s uur does it Imply the test can discnmmate sub­jects v.rith slIfficient .1~Cllracy for cl inica 1 lise " (p. 409; original emphasis).

This situation illustrates reliance on cbssi­cal validity methods instead of the more ap­propriate clini.:a l util ity approach (Wiggins, 19R5). Averas(; grollP sl.Ihtest SCDrc differ­ences rn dicate that Rroups can be discnmi mred. This classical validiry approach can ­not be uncritiea llv extcnded to conclude that mean group differences are distinctive enouch to differentiate amonc i'ldizJiduais. Figures 12.1 and 12.2 ill ustrate this di­lemma. They display hypothetical score dis­tributions of ehildrel} from nondisabled and disabled populations. Group mean differ­ences 3re d ca rly diSCI:rnible in both, bur the overlap berween distributions til Figure 12.2 m.akes it difficult to determine group memo

256 INTErU'[lG I"lVC A PPHOACI I[S

fiGURE. 12.1. Jlyporhelic31 1tsr score dimibu­tions from nondisabll!d (le(t) and disabled (r ight) pDpulauons that overlar (sh;'ided region ) only at th~ extremes.

bersh ip accurately for rhose individ uals in rhe sha ded region. Alth.ough real score distri­bu l iou~ arc morc similar to Figure 12.2, many resea rchers ~nd ciiOlcians act as if Fig­ure 12.1 dcscribc~ the n:lat iv(: score d isr.ribu ~

tions. There ,lee fo ur possible OutComes when

one is using lest scores to d iagnose a disabi l­ity: true positive, true TlcRative . false positille. an d false negative. Two outcomes are correct (trllc positive and erue negative), and twO are inco rrect (fa lse positive and false negative). True posithres a re children with disabil iric:. .. who are corre-etl}' identifie-d as such by the lest. False positives arc ch ildren idemified by [he test as ha\'ing :1 disability who do not ac­tually have one. In conrcast , false negaTives are children wi th disabililies who arc lIot identified b)' the test as having disabilities. A leSt wilb a low false-negative ra te has high sensitivi ty, and a test with a low false­positive rate ha~ high specifici ty.

FIGURE 12.2. H}"poth~tica l (est score disui· burions from nondisabled (left) and disabled (nght) populations dUll iho lY considerable over­la p (shaded te~on).

The relative proportion of correct :'Ind in­correct diagnostic decisio ns depeuds 011 t he cut score used. f or exa mple-, a cut score at the mea n of the no rma l distribut ion in Figure 12.2 produces borh a hiSh true-positive and a high false -positive ra rc. T har is. ir coerectly identifies those who are d isabled. but it makes many mistakes for those who are not disabled . In cnntrast, a ell[ score at the mean of the disabled distr ibution makes few false­positive errors but m:lny fa lse-negative ec­rors. Figure 12.2 graphically disp lays the tradeoffs between sensi ti vity ilnd specificity tha t are alwa~' s encountered when test seort:s a re used to different iate groups (Zarin & Earls , 1993).

Bevond cut scores, lhe accuracy of diag­nosti~ decis ions is dependent on the- base ra re or prevalence of the p artic ular disability ill rhe population being assessed . Very rare d is­abilitics a r<: difficu lt for a rest to identify ac­cura tely (Meehl & Rosen, 1955). This issue is relevant for psychologica l p r."lct ice and re­search, because tlIany disabilities :lfe by defi­nition unusual or rare.

Altho ugh sensit ivity alld specificity a rc both desira ble- a ttributes of a diagnostic test, t hey are dependent on the cut sco re and prevalence rate. T hus neither provides a unique measure of diagnost ic accnrac), (Mc­.Fa ll & Treat, 1999). In coulrast, by system­atically using all possible cur scores of a diag­nosric test and gr.1phing crue-posirive aga inst fa lse positive decision rates, one can deler­mine the full range of that test 's diagnos­tic ut ili ty. Designated the receilJer operating characteristic (ROC). th is procedure was origimdly applied more than 50 yea rs ago to determine how well an electronics receiver was able to distinguish signal from noise (Dawson-Saunders & Trapp, 1990). Bc(,;ausc they are not con founded by cut scores or prevalence rates, ROC methods were sub­seq uently w idely adopted in the physical (Swcts, '1988). med ica l (Dawson-Saunders & Trapp, 1 ~90 ) . and psychological (Swets, 1996) sciences. More recently, ROC meth­ods wcre strongly endorsed fo r eva luating the accuracy of psycholo~ica l assessments (McFall & Treat, 1999; Swets, Dawes, & ~'lona han, 2000).

As illustrated in Figure 12.3, the main di­agonal of a ROC curve represents chance d i­agnostic accu racy. T he more accurate the

Subtest Profile An~IYS1~ 257

1.0 y---------::::::::::::?1

.8

* .6 ~

• , .' • 0

0 m

.4 2 c

, .'

.o ~------__ --------__ --------__ ------__ ------~ o .2 .4 .6 .8 1.0

False Positiv" Rat"

FIGURE 12.3. ROC curve fot the WISC-Ill LO] fo( 445 students with LDs :Jno 2,200 stucl~nts wi r.hout disabilities (AUe = .64 ). Data from Watkms, KliSh. , and Sch;lefer (2002); s~t': this article for a fil ii rePOrt of this study.

tcst, the further the ROe curve will deviate toward the upper left of the graph. This can be Clu:m ti fid hy calcl1 lating the area 1t1Ider the CIIrt'e (AVC), which provides an over" all accuracy index of the [eST (Henderson, 1993). AVe valut::> ca ll [aug!;"! [lUlII.5 \0 1.0. An AVe value of .5 signifies that the test has chana: discrimination. In contrast, an AVe va lue of 1.0 denotes perfect discrimination. AUe values of.5 to.7 indicare low test accII­rac), • . 7 to.9 indicate moderate test accuracy, and _9 to 1 _0 indi("Me niSh ttst ;1C_i.'"lmJCY (Swets, 1988). More practically, the AVe can be interpreted in terms of twO smdents: one drawn at filfldom from rhe non disab1cd pop ulation, and one randomly selected fr0 111 the disa bled population. The AVe is the prohah ili ty rhil t the teSt will correctly iden­t ify the student from the disabled popula­tion.

Clinical utility statistics (e .g., sensitivity, specificity, ROC) must be considered when one is eval uating the accuracy of test scores used to differentiate individuals . Grou p sep-

aration is necessary, but nOt sufficient, for accur.lre decisions abom individuals.

(nverse Probabilities

Anolher fatal flaw Hl much subtest profile re­sea rch is the use of inverse proh:lhilitics. Al­though related to the group-individual prob­lem, it is particularly perniciolls in dinical p ractice. Specifically, ir is genera ll y nor UIl ­

derstood that the probability of a particu lar scorf' o n :l cl i:l e;no,,,t ic rest, eiven mem hership in a diagnostic group, is different from the probabili ty of membership in a d iagnOSTic group, given a particular score Oil a di­agnostic ttst (.McFall & Treat, 1999). For example, the probability of beine a chronic smoker, gi ven a diagnosis of lung cancer, is about .99, but the probability of h:lving lung cancer, given chronic smoking, is only around .10 (Gambrill, 1990). That is, 99% of patienrs with lung ca ncer arc chronic smokers, but only 10% of smokers develop lung cancer.

INT ERPRET IV E APPROACH ES

This quandary can be illustrated with a hypothetical subtcst analysis example. A small group of children with LDs are lo­cated, <l nd their \'{'lSC Sl1btest scores are ana ­lyzed. h is fo und Ihat many of these ~hil dren exhibit ;1 profile matked by relati\'ely de­pressed scores on four speci fi c Su btesls. Thus the probability of this subtest profile is high, given th:H a ch ild h.1s an LO. H owever, clin i­cal use of subrest profiles is predicated on a different probabiliry-mmely, determining the probability that a referred child has an LD, given the subtest profile. Inverse proba· bili tics systcmaticall y on:n:stimare prmpee­tive accuracy (Dawes, 1:.'93).

RECENT PROFILE ANALYSIS RESEARCH

Diagnosis

Scatter

Tables of subrest seIttec have been included in the WISC-Ill, WAlS-IIl, and WISelV manuals. Accom panyi ng rhese tab les is the comment tha t subtesr scatter has "ft'equendy been considered diagnostically significant" (Wechsler, 2003, p. 46). Th us cl inical inte rest in subtest scatter has remained hi~h .

It is often <lsf;erted that (he presence of marked subtest variabilJt )' reduces the pre­dictive validiry of the IQs (KlUfman, 1994). When lhis hypothesis was tcstcJ with the WAlS·III, it was found to be incorrect fo r predicring memory in dices (Ryan, Kreiner, & Burton, 2002) . However, f iordlo, H ale, McGrath, Ryan, and Quin n (2002) app lied regression commonaliry ana lysis to WISC-III factor scores used to predict achievement. They concluded that a gener::!1 intell igence factor (g) did not exist for about 80'70 of th eir s:l. mple, .1nd they sllt;gested rh:j t fdctor varia bility justified profile analysis. The sci­enrific status of g is tOO complex to explicate for this chapter, but the resul ts of Fiorello and colleagues rWl CQunter to massive amOl1nrs of existing research (Jensen, 1998). When mult iple regression is app lied, there is no universally accepted definition of predic­tor imporLance (A:a:n & Budescu, 2003) and there is no consensus on which method is best employed ro explain the relati ve impor­tance o f multiple correla ted predictors

(Whittaker, FOl1ladi, & Williams, 2002). More critically, Pr::d hazur ( [ 997) reponed that communality analysis is not "a valid ap­proach for ascermining relative imporrance of variables" ("' _ 275 ).

To test rhe predicti ve validity and diagnos­lic utility oI\Xi1SC-HI subtest scattcr, twO sam­ples o f students were analyzed: the 1,118 stu­dents in rhe WISC-IlIf\Xlechsler Individ ual Achievement Test (WIAT; Wechsler, 1992) linking sample, and the J ,302 stud ents with LOs from 46 slaLeS from Smith and Watkins (2004 ). For predictive validity, the FSIQ of the WTSC-IlII\VTAT s::!mplr. was enr~rcd as a pre­dictor in multiple regression, followed by the absolute amount of subtest scatter among the 10 lIIandatory suhte"t". The FSIQ was sllh­sranrially predictive of WI AT reading achieve­m ent (R =0 .70}, hut t he additioll of subtest scatter did not improve prediction (R = .70). In terms of diagnosric utility, subtest scatter of the WISC-IlII\'V1AT samp le was compared to the subtestscatter of the sample with LDs. Fig­ure 11.4 shows thaT the discrim ination was at chance levels (AUC = .51 ).

T he \'al\Jt: of W ISC-HI su btest scattf:r as a diagnosric indicator was also analyzed by Daley and Nagle (1 996 ) among 308 children with I.D~. They found that "Sllbtest scatter and Verbal- Performance discrepancies do not appea r to hold any special utili ry in [he diagnosis of Jeanung disab ilities" (p. 331 ). T hese negative results we re replicared in sev­eral othcr stud ies (Durnont & Will is, 1995; Greenway & Milne, 1999; Iverson, Turner, & Green, 1999). More definitive research was conducted by Watkins (1999), using the \'7TSc·m standardization sample as a nor­mative comparison group. Subtest varia bili lY as quantified by range and varia nce exhib ­ited no diagnostic utility in rlist inguishing 68-1 children with LDs from the 2,200 chil­dr~n of rhe WISe-lIT stal1 d:m:l i:r.atio l1 samp le. Likewise, lhe number of Subtesls dev iating from examinees ' VIQ, PIQ, and FSIQ by ±3 points exhibitcd no diagnDstic uti lity in dis­ti nguishing the children of theWlSC-lU Stall­dard izarion sample from 684 children wirh LDs (W'<ltkills & Worcel l, 2000). B<lscd upon these resulrs, Watk ins and Worrell concluded that " using subtest variability as a ll indicator of learning disabilities would constitute a case of acting in opposition to scientific evi­dence" (2000, p. 30g).

Subte.t Profile Aualys;~ 259 1.0 y-________________________ ...",

.8

~ .6

.2

.o ~------__ --------__ ------__ --------__ ------~ o .2 .4 .8

Fcll"tl Pnsitilltl Ra te

FIGURE 12.4. ROC :malysis of 10-subleSt scatter betweeo 1,118 students in rli(' WlAT norm sample and J ,302 studeIlt~ with LDs {AUC " .51 ).

S ubtest Profi les

ACID Profile

The ACID subre~t profil e is probably the most venerable. Based on the Arithmetic, Coding, Information, and Digit """'Spall sub­teStS, it has been applied to rile WISe, WISC¥ R, and WISe-III. Most recently, Prifitera and Dcrsh (1993) compated percentages of children with WIse-III ACID profiles from samples with LDs and ::tttenrion-ddicitlhy­peraclivity disorder (ADHO ) to percentages showing the ACID profi le in r.h{: WTSe-ili standardiza tion sample. Their findings un­covered a greater incidence of ACID profiles in clin ical s:lmples, wi th approxi mately 5% of the children with LOs and 12% of those with ADHO showing the ACID profile, while such a configurarion occurred in only 1 % of the cases from the WlSC-III stan­dardiz;uion sample. Based upOn these data, Prifitera and Dersh (1993) concluded that ACID profiles "are useful for diagnosric pur­poses" because "the presence of a pattern or

patterns would suggest strongly that the dis­order is present" (pp. 50-51).

Ward, Wa rd, H:ut, Young, :lnd Mollner (1995) investigated the prevalence of the WISC-llI ACID profile among children with LD:; (N '" 382) am! fuum.l a pn::lla!t"'uct"' ralt"' of 4.7% (vs. the expected rate of 1 %). Ob­tai ning si milar ACID resul ts from a sample of children with LDs (N = 165), Daley and Nagle (1996) suggested practitioners "inves­tigate the possibility of a learning disability" {p o 330) when confronted by an ACID pro­file.

The studies described above relied on the group mea n difference approach to incor­rectly infer diagnostic uti Li lY. Watkins, Kush, and Glutting (1997a) evaluated the diagnos­tic utility of the \'(olSC-IIT ACID profile among children with LDs. As in previous re­search, ACID profiles were more preva lent among children with LDs (N == 612) than among nondisabled children (N = 2,158). H owever, when ACID profi les were Llsed ttl

classify students into disabled and nondis-

Z6<J INTEl tPl~ETJVE APPRQACI1ES

:lhkd groups, they opera ted with consider­able error. At best, only 51 % of the children idenrified b>, a positive ACID profile wefe prc\·jously diagnosed as having LOs. These data indicuted thot 0. randomly $E']ected child widl a ll LO had a lIlore severe ACTO profile than a (nndom lr sdected child without an LD about 60% of the rime (AUe = .60). AJ­though margiu:J.lly better than chance, rhe de_gree of accuracy was quite' low (cf. classifi­catory criteria presenred by Swets, 1988).

SCAD Profile.

Whereas the ACI,D profile has a long history, the SCAD profile (based on lhe Svmbol Search, fading, ~rithmetic, and QigTr Span su btcst.~) seems co h.w e bc(!" first srudiecl by Prifitera and Dersh in 1993. Kaufman (1.9~4) opined rh ,n Arithmetic, Codi ng, and Digit Spall have "been qui te effective at iden­tifying exceptionnl groups fro m normal ones, and ... are like a land mine that ex­~lodes on a divers ity oi abnorma l popula­tiOns bllt leaves most normal samples lln­scathed" (p. 213 ). Kaufman concluded that the SCAD profile is «an important piece of t:vidcnce for diagnosing a possible abnormal­ity" (p. 221 ); it "won't identi fy lhe type of exceptiona lity, but rthe profile isl likely to be valua ble fo r making a presence-absence de­cision and helping to pin point specific areas of deficiency" (p. 214).

Prifitera and Dersh (1993) found the SCAD profile to be more common with in a samplc of chi ldrcll with LD.lt (n = 99) ~nd ~n ­othe.r sample of child ren wnh AU HU (11 = 65 ) than withill the \XfISC-IlI stalltlardi:t.a­tion sample. Using this imbalance of preva­lence cates as guidance, Prifi teC3 ::Ind Dersh suggested that the subtest cOllfiguratioIl would be " usefu l in the diagnosis of LD and AD HD" Ip. 53).

These claims were tested by Watkins, Kush, :tnd (;llltf ing ('1997b) w ith chilrl ren who were enrolled in learning and emotional dis::lbiliry programs (N = 365). When these children were compared to tilt: WISc·nr sta~d~rdization sample via diagnostic utility statiStiCS, an Aue of .59 was generated. Thus, contrary to Kaufman 's (1994) asser­tion, SCAD subtest scores were not found 10 be importilnt e \'idence for diagnosing exceptionaliries.

Looming Disability Index

The Learning Disability Index (LDt; Lawson & Inglis, 1984) is of particulnr interest, be­ca lise it was h}'pothesized to rel:ue to specific neuropsychological deficits of students with LOs. Lawson and Jn~lts (1984) conjectured that \'X' ISC-R subleSlS are sensilive 10 the presence of LDs in direct proportion to their verbal satura tion , which is in turn n:lared to

left-hemisp here dysfunction. '1 his theoreri­~al. link lx:rwecn resc and bram funcnonine IS ImpOrtant, because cO ll temporary defini­tions of LOs specify ::In endogenous etiology rcl:uccl ro "central nervous system dysfunc­tion" (Hammill, 1990, p. 82).

Comparisons of groups of students with and widlOut LOs iound significulltI)· higher mea n LOI scores among students with LDs than among sLuderH~ in regular educa­tion (Bellemare, Inglis , & Lawson, 1986; Clampit & Silver, 1990 ). Srarist ic:~l1 )' siBn.ifi­can t LD I differences between groups were subsequently interpreted as evidence that the LDi is d iagnostica ll y effe('live. For example, Kaufma n (1990) concluded thar the LDl taps a scquenria l-simuha nc()lls processing dimen­SIon and IS "'qUIte .... a luable fOf distingu ishing learning-disabled children from normal chil­dren" (p. 354).

However, mean differences between groups <tre insufficient fo r individual diag­nostic decisions. Consequently, more appro priate dia(7losric urility st::lristics were ap­plied to the WISC-III LDl scores of 2,053 students previously diagnosed with LDs and 2,200 students without LDs. SubsampJcs of students with. specific reading (11 = 445) and math (1l = I h8) disabililies permirted more refined assessment o f rhe efficacy of rhe LDI. ROC analyses revealed th ar the LD1 resulted in a correct diagnOSTic deci sion on ly 55- 64% of the time (Watkins, Kush, &: Schaefer, 2002). See Figure 12.3 for an iIIustrilfive ROC curve for srudents with speciiic reading disabili ties (n;:; 445 ) compared to the 2,200 students ill l he nonnati ve samp le. Thc.ltc re­sults demonstrated thar the Lor is an invalid diagnost.ic indic:aor of LDs.

Diagnosis: Summary

\'O:lhen properl}' analyzed, data have consis­temly failed to find stlbtest scaner or subtest

profiles to he diagnosticallr accura[c. Based on th eir review of IQ subtest scatter research, Kline, Snyder, Gnilmene, and Castellauos (1992 ) suggested that psycholo­gists " have pursued scan er anal}'sis .. _ wi th liule success. It is tiJlle to lrLOVe on" (I" J 1). That suggestion was reiterated by McGrew and Knopik (1996), who remarked that "considering the years of scudy attributed to rhe concept of scatter nod the lack of an em­pifical fo un dation, it is recommended that fut ure research efforts be directed else­where" (po 362). Clearly, there ill no scient ific support for the use of subtest scattef to lf1

form diagnosis or prediction. Likcwillc, rl~sults have been cOtl llisten t in

indicating tha r subtesr profiles offer little di­agnostic advantage. Li mited ~pacc prohibits reviews of research regarding other sub­test profiles, bur diagnostic milit}' results have been unfailingly negati ve (Gl uLting, McDermott, \X1atki ns, Kush, & Konold, 1997; Glutting, McGrath, Kampha us, & McDermott, 1992; Cussin & Javorsky, 1995; Lowm:11l, Schw:mz, & Kampha us, 1996; Oh, Glutting, & McDermott, 1999; Piedmont, Sokolove, & Heming, 1989; Smith & W/atkins, 2004; TCl;rer & Korducki, 1998; Watkins, 1996). Saub' (2001 ) also concluded th:!.t su btcst profiles '"c;mn nt be used for classification purposes or to arrive at a diagnostic label" (p_ 299). Simibr cau­tions regarding the use of subtcsts for di­agnostic decisions have been offered by K'aufman and Lich tenberger (2000) and Kamphaus (2001) . Unmistakably, abundant scientific evidence and expert consensus rec­ommend against the use of subtesL profi les for the diagnosis of childhood [eartllng and b(:havinr disorders.

Specific Cognitive Strengths and Weaknesses

Although there is general agrecment that TQ subtest-based diagnosis should be avoided, the use of subtesr profiles to identify spe­cific eo~niti\'e slrenl?,ths and weakncsses is freq uen tl? recommended. As articulated by Kau fman and Lichtenberger (1998), the ex­aminer "must generate hypotheses about :In indi,·idllal's assets :mcl deficits" (p . 192). :--Jexc, the exami ner must "confirm or deny these hypotheses by exploring multiple sources

261

of evidence" (I" 192). Finall y, "well-va li­dated hypotheses must then be translated into meaningflll, practical recommend., ­tiOLlS" (p. 192} concerning interventions, ill­strucrional strategies, and remediaTion activ­ities (Croth-Marnat, 1997).

Subtest interpret,nion systems provide hun dreds of hypotheses to consider when lQ subtest patterns are obtained (Kallfman, 1994; Sattler, 2001 ). The enormous number of hypotheses makes it impossible to review the entire scientific li terature. However, sub­teST pr{)filc~ arc often :lllsumed to be related to a wide variery of learmng and behavioral variables (Kaufmnn, 1994; Sattler, 2001).

Academic Achievement

One way to test the utility and .'a lid lty of subtest scores is to decompose profiles inca their elemental cOlllponents. The ullique, ill­cremental predictive validiry of t!;lch compo­nent can then be au::t lyzed sepa rately to de­termine what aspect or aspects, if an}', of the subtesr profile are related ro academic per­formance. To this end, Cronbach and GIeser (1953} cautioned d:af su brest profiles con­tain only three types of information: elella­tion, SUlttcr, and shape. Elevation iJlforma­rion is represented by a person's aggregate perfor mance (i .e. , mean normative score) across subtests. Profile scarfer is defined by how widd r scores in a profi le divcrge fro~l its mean_ Scatter is typically operationalized by the standard deviation of the subtest scores in a profile. FinaJly, shape information reflects where "ups and downs" occur in a profile. Evt:n iftwo profiles have the same d­evation and scatter, their high and low points Ill :ly br. difff'rr.nt.

WatkiJls and Glutting (2000) tested the in­crt'ment<ll validity ot W ISC-III sllbtest proult' level, scatter; and shape in forecaSling aea­dt"mic performance. W ISC-III subre-st pro­filc..~ were decomposed into th<:i r thn:t: funda­men tal elements and sequentially regressed OntO reading and mathematics achievement scores fo r Ilonexceptional (n = 1 ,J 1 8) and exception"I (II = 538) children_ Profile eleva­tion was stati stically and practica ll y sign ifi ­cant for both nonexceptiona l (j{ = _ 72 to _ 75) aocl excepTional (R = .36 to .61 ) children. Profi le scaLter did not aid in the prediction of ach ievement. Profile shape accounted for an

262 INT ERPRETIVE APPROACHES

additional 5-8% of the variance in achieve­ment measures: One pattern of relatively high verha! scores positively prcdic.:tcd bOTh reading and mathematics achievement, and a p:J.[rern of rebtively low scores on t he \X1TSC­III Arithmetic subtest was negatively related to mathem atics. Beyond these rwo somewha t Hltuitive pattem s, profile shape information had inconsequential incumental validity for borh nonexceptional and exceptional chil­dren. In olher words, it was the averaged, norm-referenced information (i.e., eleva­tion) contained i ll sl..Ibrest profi les that vest predicted achievement. This Information is essmtially redund;mt to that conveyed by global IntellIgence scores.

Similar results have be~n obtain~d bv otber researcilers with a variety of intelli­gence tests (e.g. , the \VISC-R, Stanford­Bi net, \'(looJcock-juhosoo, eu:.) (Glutting, Youngstrom, Ward, Ward, & Hale, 1997; H:lle & Raymond, 1981; Hale & Saxe, 1983; Kahana, Yo ungstrom, & Glutting, 2002; Kline et a l. , 1992; McDermott & Glutting, 1997; McGrew & Knopik, 1996; Ree & Carretta, 1997; Youngstrom, Kogos, & Ghming, 1999 ). loterestingly, research has consistently demonstrated that genera l mental ability also accounts for the vast m:l ­jority of the predictable vari::!.oce in job learning among adults (Ree & Carretta, 2002; Schmidr., 2002). G;ve.n these results, Kline and colleagues (1992) concluded tha t " the most useful information from IQ-rype tests is the overall elevation of the c..: hild's profile. Profi le shape information adds rela­tively little wlique information, and there­fore examiners should not overinterpret pa rticular parte..rns of scores" (p . 431). T horndike (1986) similarly conduded that 80-90% of the predictable variance in scho­lastic perform ance is accounted for by gen~ era! ability, with only 10 20% accoun ted for by a ll other scores in IQ tests. From tbese findi ngs, it has been concluded that subtesl scatter and shape offer minimal assistance for gencr::!.ting h ypothes;;:~ about children's academic performance.

Learning Behaviors

Teacher Hni ngs of child learning behaviors, as operationali:ted by the Learning Behaviors Scale (LBS; McDermorr, Green, Francis, &

StOlt, 1996), reflect four relatively indepen­dent subareas: competence motivation, attiu lIlc..: [(Jward learni ng, am.::nrion/pcrsis­tence, and strategy/flexibility. The DAS and LRS were conomled wi t.h:l n:ltionally repre­sentative sample of 1,250 children. When DAS and LBS scores were compared, DAS glDbal abi lity accounted for f! .2 '% of learning behavior, and DAS subtests only increased rhe explained va riance by ·1.7%.

Te~t Session Behaviors

It is widely assumed that astute test session observ;:n ion and clinical insight allow psy­chologists to draw va lid mferences regarding an examinee's propensities an d behaviors outside tbe testing situation (Sparrow & Da­vis, 2000). T hat is, a quiet child during rest­ing is :Jssnmed to bl: re ti ri ng in other sit­uations, an active child is inferred to be energetic in rhe classroom, and so on . How­ever, a syn lhesis of research 0 11 test session behaviors fo und that the average correlation hct\vecII test session behaviors and conduct 10 other environments was only .18 (Glut­ting, Youngstrom, Oakland, & \X7:nk ins, 1996).

Among the normativ~ sample of the Guide to t.he Assc..;smmt of Test Session Behavior (Glutting & Oakland, 1993), a standardized test session obsr. rv:nion scale, there was lirrle differential vanability across WISe III sub­resrs (Oakland, Broom, & Glurring, 2000). In fact, global ability accounted for 9.2% of rest session behaviors, and the addition of WIse-III ~uhtcsts only explained another 3 .2% (McDermott & Glutting, 1997).

Dellavioral Adjustment

Subtest p rofiles are commonly assu med to reflect d ispositions that allow IIlfel"ences about school behavior and adjustment. To test this assumption, a nationa lly representa­tive sample of 1,200 children was adminis­tered the DAS, and their teachers inde­penden tly provided standardized ratings of school and classroom beh:JViors on the Ad ­justment Scales for Children and Adolescents (ASCA; McDermott, Mamon, & Start, 1991). The ASCA provides measures of six core syndromes: attention -deficit hyperactiv­iry, solitary aggressive (provocative ), soli tary

aggressive (impulsive). oppositional defiant, diffident, and avoidant. Fnllow illg the method of K:u lfman (1994), the 5% of chil­urell with the mOSt unusu:1\ DAS &Il hresr p ro­files were identified alld Lhen matched on the basis of age, race, gender, pafl"nr edllc.ui on levels, and overall lQs to an cqual number of cotn pari ~on groulJ children without unusual su hrest profiles. There were no significant differences between these twO Rroups on the six ASCA behavioral scales. Nor were there Sig nificant differences on :u;adelll ic teses. T hus academic and behavioral prob lems were not related to unllsual DAS subtcst profi les (Glutting, NieDerman. Konold, Snclbaker, & Wa tkins. 1998).

Th iS oegari\'c result was replicated by an analysis of the relationships between IQs from the \VISC-llJ and classroom behaviors me:Jsured by th e ASCA (Glutting er nl. , 1 9~6 ) . Four of the 56 correlarions berween the \VJSC-III and ASCA rcached Slalisticai significanc{;' al p < .05, but this m tio bMdy exceeded the expected ch:::ance ratc (If three signi fi cant correl ations. The average coeffi­cien t was also low (average r '" -.04, wi t.h a 95% confideoce intt:rval o f - .20 to +.13) and in dicated thaL a s much as 99% of the score variation was unique to eac h instrument. These meager results were nOt surpris­ing: " 11 previous investigations showed an :lverage relationsh ip of -. 19 between chil­dren's scores on indi\'idually administered IQ rests and rheir home an d ~chool beha\'­ior" (Glutti ng el a!.. l~%, p. 103).

Spt.ocific Strengths and \Veaknesses: Summary

Must hypotheses genera ted from subtest variation "are either untesu:d bv science or unsupported by ::K;icmific findings" (Knmp­haus, 1998. p. 46). The evidr.nct: rh:n cxists regarding relat ionships between subtest pro­files and socially important aca de_mic and behavlo ral outcomes is, ar beST. weak: Subtl'.:st profile information contributes 2-8'Yo va riance beyond general abiliry to rhe pred iction of achievement, and 2-3 % to the prediction of learning behaviors and reST ses­sion beh,wio(s. Hypo[hcsi:t.ed relationshi ps bct WCCIl subtest profiles and mi!:a sures of psychosocial behavior bave p<:rsisrently fai led to achieve statistical or dinical signifi-

263

cance_ Th us ;'neither subt~st patterns nor profil~s of IQ have bWl systematically fo und to ~ related to personality \'ariables" (Zeidllcr &: M atthews, 2000, p . .'i85). Afre-r' reviewing the rl'::sca reh on su utest analysis. Hale alld Green (995) concluded that "knowledge of a child's sll brest profile docs nor appreciably help the clinician in predict­ing either academic achievem~nr levels o r behaviora l difficulties" (p. 98) . Thus hypoth­eses del'ived from subtest ,'!l alysis "are based on the clinicia n's acu men an d nor un any sonnd n:scan:h base" (Kam pha us, 200 1, p. 5~8 1 .

GENERAL CONCLUSIONS

~1an)' researchers have fo und the popub rity of IQ subtesr p rofile analysis to grea tly out strip ils meager scientific su ppon (Braden , 1997; Bray, Kehle, & H imze, 1998; Grc..~­ham &: Witt, 1997; Kamphaus, 2001 ; Reynolds & Kamphaus, 2003; Watkins, 2000,2003). Although sublest profile analy­sis has not demonstrdred adequa te reliability, diagnostic miliry', or treaTment va lidity, it con tinues to be endorsed by assessment spe­cialists and applied widely in trai ning and pracrice.

Apparent ly, subtest profile interp rera­rion flourishes due to its illtui ll\'e appeal (Bracken, rv1cCa llull1, & Crain, 1993) and clinical tradition (Shaw, Swerdlik, & La urenr, 1993). SUutcst-based incel'pretation systems are often justified by prescicTltifi('; argumenrs w itho ut consideration fo r the professional obligations of psychologisrs (American Ed ucational Rc-"can:h Associa­t iOll, American Psychological Association, & )jational COllncil on Measurement in Ed­ucation, 1999), the lirni t~ t iOlLs of clin ic;, ! judgment (Garb, 2003 ), or the dema n d~ of scientific reasoning (Gihbs, 2003 ). Scientific psychological practice cannot be sustained by dinical conjecmres, personal am:cdorcs, and unveri fbble bel iefs that have consis­tcn Lly fai led empi rical validation. Given (he pauciry of empirical support, subtest profile amd ysis can best be described OIS reliance on clinica l delusions, illusions, myths, or folk­lore. Consequently, psychologists should es­chew interp retation of cognith-e subtesr pro­files.

264 INTERPRETIVE APPROACIICS

REFERENCES

Alfonso, v. c., Oaklund, T. D., LaRoc<:a, R.. & Spanako>, A. (2000\. The COUl"lle on individual cOJ;ni tive aS$e$SmeEll. S£hool Psychology Re"i~w, 29. 52 64.

American Educational Research As~oc i ation, American Psychological AssOCIation, & Natiorul Council on .'VIe3surement in 1!duc:llion. (1999). Stlllidards for educational find psychological /C$fillg. Washington, DC: I\merican Educationa l Research ,\.sociarion.

.. hen, R., & Bucic>cu, D. V. (2003). The dominance analysis approach fo r comparing predictors in multi· p It: r~gr~~~ion. P$y£hologirol Meth ods, 8 , 1 2.9~148 .

Belk, M. S., l.oJl..>,lIo, S. G., Ra)" G. E. , & Z"ch ar, 1". (2002). WISC-III adm inistration, dt. r ic~ll, ;l nri :<>Cor· ing errors made by !tlJcient eX.lmin~rg.. }Ollmal of I'sycho~ducatjonal Asussmellt. 20. 290-300.

Bellemare, 1-: G., Inglis, .J., & Law$on, .I. S. (1986 ]. learnin~ d\sabi lity indices d~nvcd fn,lm J prmcipal c()UEpO"~"t> an~l~s is of the WlS(-R: A study of learni"g disauled ulEd no~m~l bu)'~ . Cmmdill" ]{)lIr­""/ of Beh"Uloml SCIence, 13. 86~91

.Bcacken, B. A., McCallum, R. S., & Crain, R. _\1. (199-' ). wrSC-TII subtest composite reliabil il ie$ and speci ficiTies: [nterprelive aid,. j ounull of PsydJoed,, ­Gariunal Asstument i\llo//(Jgr.rph Series, Adva//us ill Psychologicoi AS.'essmenr: lXiuhsli:r llltelllg~lIct

Scale for Children-Third Editloll, pp. 2.2.- 34 . Braden, J. P. (1997). The practbl impact of intelleCTUal

a5~essmcnt i.sues. School PS)'chology Rn'ie/v, 26 , 2112--2<18.

Br~y, M . A .• K~hle, T. J., & Hinl~e, J. M . (} 998). Profite a'l.1lysis with the \Vechsler 5:.~ 1~.s : Why does ir per. sist? School Psyr.hology Int'mUltionai, 19, 209~220 .

Canivl"z. G. L., & Watkin.~, I'd. W. (1 99R ]. Long-term stabilltr of the Wechsler Intelligence Sca l ~ fOT Children-Third Edition. 1'S:lChologiclll Assessmem, IV, 1 t15~291.

Carte!!, l{, B. (1944). Psychologtcal measurement: Nor­"mti,·t, ipsuti'·e, ill1eru~-tive . PsychuloSiwl R .. liierv. 51, 292~303.

Clampit, M. K., & Silver, S . .J. (1.990). Demograph i~­

char~c teriS l ics and mean profiles of I"aruing disahil iry index SEEbsers (If tI.e SI;,oda ,diwt;on sample of the Wechs ler Inrd1ieenr.e Sc.ale for Children- Revised . journal of 1 .~aTllinR fJ i.«l I,ili!iP.." n , 2t>'~2M.

Cronbach, L. J., & GIeser, G. C. ( 1953). Assessing simi· lari ty brrwecn profiles. 1'5}" hological Hlilleril' , 50, 456-473 .

Daley, C. E., & Nagle, R. J. (1996). Rclcvanccof WISC· TIl indicators for a~:;es~ment of karning dl~ab iliti ~~ . JUllrnal of P$ychoeducatianal A5$CS$mellf, H. 320-33.1.

Dawe~, R. M. (199.1). Prediction o f th" h,tur~ ""rsus an under~tanding of tne past: A \).lSic asymnletrf. Am,rr­iC-<ln }()/ornal of Psychology, 106, 1 ~24.

Dawson-Saunders, B., & Trapp, R. G. {1990). llasi, and clinical biosratiuic.s . Norwalk, CoT: Appleton & Lange.

Durnoll{, R., & Willis, J. O. (1995). In1Ta~ub[cst scarrcr o.n Ihe WISC·UI :Qr var ious clinical samples 1'$ . the sland:lld i>:atioQ sample: An exanlin3ti0E1 of WISC iolklore. } u"T7l41 of PsydroeducaltOtlll1 Assnsm ellt, 13,271 285 .

Elliott, C. D. (1990). Di/fanmlial Ability St;.r/las: l"tro­dllctory and tedmiG-<l1 hm:dli rlOk . San Antonio, TX: rsychological Corporation.

Elwood, R. W. (1993). Psycho!()gical tem and cl inical discriminations; Beginning to address the base rafe problem. Clmical i'$ych%g)' Retllcrjl. 13, 40'-419.

Fausr, D. (1990). Data integra tion in legal c~·all,lations: Can clinicians d ~! i\'''r 0" their p r~mise>? Beh(/jlioml ,Sciences ami rhe u.w, 7, 469 483 .

Feldt, L. S., & Brennan, R. L. (1993). Re-liability.ln R. L Lin" (F..d .), Educalim",1 m""surrmf!1lt (:lrd r.d., pp. 105~ 146). Pboenix, AZ: Oryx Pr~ss .

Fiordlo, C. A., 11a1~, J. It, McGraTb, M., Ryan, K. , & Quinn, S. (2002). IQ inrerpr.::tatinn for children wi th flat and var iable lest profiles. Leammg and hrd1l'id­IIal Differmces. 13, 115- 125.

Frank, C. (1983). The \VechsleT enterprise: An IIS$ess· 11,..,,1 of the dCfJ.-iopment. $t ,uC/uTe, mrd "se of rhe W echsler lesls vf inlelli8e1rce. New York: Peqp rno[l Prt'-SS.

Gambrill. E. (1 990). Critic.-.llhink;',g in diniwl pTac· lia: Impro villg the accumc), of i" dgme'Jts and deci­sions abollt dienll. San PUnCl!C"': Jossey·&.ss.

Garb, H. N. (Z003 ]. Clinicll Judgment and mechanical pnxliction. In 1. B. Weiner ISerics Ed. ) & J. R. Gra­hall} & J. A. ='Iaglicri (Vol. Eds.), H,mdbook of psy­chology: Vol. 10. IlS5assmenr pS)'ch%gy (pp. 27~ 42). New York: Wile}'.

Gibbs, L E. (Z003:. Evide.nce-base.d pmctia for the helping profeS5io .... " A praaiwl guide u,il/' integrated mult;m~di,r. P.~c i fic Grove, CA: Brooks/Colt. .

Glutting, J. J., .\kD~rmol1, P. ;\.., Konold, T. R. , Snclbaker, A. J., & WatkiM, .M. W. ( 19911) . .vlor~ up~ and down.s of ~lEb l e5t analr$i~: Criterion "alidity of the DAS with an ume!ected cob() rt. Scllool I's ydml­ogy R<""ifft~ 27 . .>99- 612.

Gluttinll, J. J., McDcrrnott, P. A., Walkins, M. \'(/. , Kush, J. C., & Kunold, T. R. (19971. Tht: bast: nile pwblelll " ud it~ ~()ns~qu"llces ["r in[erpretinll ehilu rt:n's abil · ity proiiw;. School fsyriJo/Q[Jy Review, 26, 176-18S.

Gluttine, J. J., lo,1.cGrath, E. A., Kamphaus, R. W. , & i\ .. leDt. rmon, P. A. (1992). T" ... onnmy ~no vnl ioit}·of subten profiles on the Kaufman Assessment Battery for Cluldren. j01lr1lO1 of Sp/fda/ Educatioll, 26, 85~ II.>.

Glutting, J. J., & Oakland, T. (1993). Gllid~ 10 III" as­sessment of te~t·stssion hehaliior. San Anwnio, TX: P>)"chological COlpm-:ltion.

Ghltt i,, !,:, J 1 ' \V'>f kins, M \'Ii, IV: Y~"mg~trom , E. A (2003). Mul(if" r.to red M d noss-b~rlelY abil iry ~~­

s~~~ment5 : ;\re rh fy wmth rh" effo rt? In C. R. Reynolds & R. W. K~ mrh~E1~ ([..is.), Handhuok ()f pS)lc/lological and educaricmal a5S~S5mmt of chil­dren: Illtelfig~nce, aptitude, and achieli~me'lr (2nd ed., pp. 343~374 ! . New York: Guilford Press.

GluuiOi\, J. j., Youngstrom, J:::, A" Oakland, T., & W~rkins, 1\ \. W, (19%). 5ituanooal ~pe"ifl(il)' and generahry of teS I bcha,·;u l'S fo r samples o f norm;\! and referred ch ildren. School Psydlology R~vi~w, 25, 94-1 07.

Glmting.J.J., Young~l rom, E. A., Ward, T., Ward, S., & Ha le, R. L. (199i). lnaemerunl eificaey of WlSC-JU fat"!: ... r $CI.m::S in J)rerlicting achi!\,cmcrlt: What do the~ teli us? PS)'c/lo/()gical AS$Cum t llf. 9, 29~-301.

Grl'eOway, P., &. Milne, L (1999). Rd:ltionship bl:tween ~y.:hl'lpathology, learning disabtl iti~, Ut both and WIse-III subtut SCl\m:f 1II adul~ems. Psychology in the Schools, 36. 103-J08.

Gresham, F. M,. &: Wm, j. C. (1997), Utility of intelli. gcm::e tests for tfCatment phllll riug, classificnrinn, and placement dC"ilun i: Rco:ell t empir ir.1I1 find ings and fUlm., d ir.-ctions. Sdlool l'sycb%gy Q ljarterl)" J.l, 2 '19 267.

Grolh·M .. 1rrlnl, G. ( 1997). Handbook of psychvlVK"a{ aS$~ssmlr"t (.lrd ed.) . New York: Wiley.

Gu5~in, B., &. Javorsk y, J. (1995). The utility of the Wlse·1II FlUdom from Disrra,tibiJJty in th~ diagoo· sis of youth with amntion defici t hrP<'rocti"i t), dis.or· der in a psyciliaITIc s~mple. DUJCl1Osrjq,,~, 2 1, 29-40.

Hale, R. l., & Green, E. A. (1995). hm lleaual evalua· tiun. In l. A. H~iden & M. Hersen (Eds.), illtrodllC' tIO" to dimC~11 p£}'d!olog)' (pI" 79- 100). Ntw York: Plenuru Press.

Hal"" R. L., I\{ R..1ymond. M. R. (98 1). Wechsler Intel­ligence Sc~ ... le for Children-KevIsed patterns Q/ ~ tre ngths and we3kncsscs as pro;dkt1lr~ of the intelli­gence achievement rcJauonship. Dia/tnostique, 7, 35-42.

Hale, K. L , & 5a11<:,J. E. (1983). Profile ll n lll ysi~ o f the Wechsle r Intelllgeoc", Sa. le for Chilrlren-Re\·iscd. jOllmal of PsychQCJwcaf;uMI ~ssment, J, 155-162.

HUlllllJ jll , D. D, (1 990). O n deflIling learn ing di.abi ll· lies: All t:mnging consensus. Journal (If Lmrnm/t Oisabilit iu, 23, 74-84.

Harris, A. J., & Shakow, D. (1937). The d ini<-.,,] signifi­c.11lCe o f numeric.tl measurcs of s<''ll (t~r 011 tht Stanford-BlOet . Psychological B"IIl'lin, 14, 134-150.

1 !enderson, A. K. ( 1993). Ass~sillS: I~.t uccuracy and its cl ini~al CUI15tllueuct'S: A prim .. r tnr recei l'<'!r operat· Ing charal1erisric mrve 3n;\ l ysi~. AmUlls of Clinicaf BiomL .... islry, 30, 52 1- 539.

Hicks, L E. (1970). Some prl'lperries o f ipsaove, norma· III'". "nd fOl(erl -choic ~ norm..ui l'e nl:asun:s. PSj'Cho· logical B"lietill, 74, 167- 184.

Iv~roon , G. L, Turner. R. A., & Green, r. (1999). Predic tlve va lidity of WAIS·R VIQ-P1Q s plit~ ill pe[l;(lll~

with major depression. Journa/ flf ainicol Psycho/. o~ 55, 519- 524.

jensen, A. R. (1998 ). TIJ(' gfaClor: The $CI/!l'''e of m~mal ability. Wl~tPUrt, CT: Pra~,·.

jensen, A. R. (2002). SASP ;Iltefview$: Arthur R. JenUl\ . SA$P Ne", •. 2(4), R- 19.

K~ hnn~, S. Y., Young~trl'lm, E. A., & Gluning, 1- j. (2002). Facto r and subresr dlSerepancres un th~ Dif·

2., fer<':!lfi" l Al-lil it)· Sca les: Examinmg prevaleuL'" and "a lidit)' in predIcting lcldenue achie"en r~tlt . Assess· mem, !O/, !I.I.-~;.

Karnphan ... R. W. (1998). Intel ligence !t!$t interpreta­l ion: Actins in the absence of evidence. In A. Prifi lera & D. H. 5.lklofske (Eds.), 'i1?/SC·/lJ dmical ~~ ami illurprtration: Scieutin·pra,trtwnt,r l,errpeCl.ive.s (pp. 39-57). S"ll Diego, CA: AClld"mic Pre$S.

Kamphaus, R. \V. (2001;' Clinic'l! (l.'se$$ment of (hild ond adoicscel,t IIlt,lIigcnce (2nd M.), Bos!on: Allyn &:. Bacon.

Kamphaus, R. \V., PetOlkey, ~. D., & Rowe. E. \V. (7.000). Current trends In psychological It . , int: o f ch ild ren. r n'lfessiollaf P~yc"ology: Research lind Praair.t, 3 1, 155-164.

Ka ufman. A. S. (1990). A.<Sf'S.<in,l'! odo/e5GIIJI and ad,,{f intelliglmr.e. Boston: AUyn & BacoD.

Kaufm~n, A, S. (1994). intelligent us/iJlg with II" WISC-1f1. N~w YOlk: Wi l e~·.

Kaufman, A. 5., & Lichl cnbcrg~1", E. O. (1998), Tntellh':­rual nscssment. In A. S. fklillck & .\11. H enen (Eds.), ComprlhlnSItlf.' dmicJI Pf)·,;},oloIJY: \'01. 4. A.s.s<!Ss­m,n: (pp. 187_' 38). N" .. w Yorl:: Elw:,·ier.

Kaufman, A. S,. Be Lichlenbo:rger, E. O. (2000). e{.U'n­

tials of Wise·lll a"d \VPPSJ R asse~me" t. Kew YOlk: Wiler.

Ka,,~ l<'!, I{ , A., & [<omen, S. R. (l984!. A lUe !a·lilnaly~is

of fhl': v<1lidi1r of Wecltslcr so:a le pro fii t~ IImi recJ.t~goriullions: l'a tterns or parodies? Ltaming Dilabiliry Q uartLTly. 7. 136 156.

Klint:, R, B., Snr der, J., Guilmette,S., & CaSlt ll an~,.\1. (1992) . Reilltlve useful nC$s of elevlti..,!!, vW"i~bili l)·,

and shape info rmation from \VlSC-R, K-ABC, allci Fourth Ed itIon SI:lIlford-Billet pIofiles in p reciicling achieverncm. PSj'cbolQgr(/f1 ASStS;j;me"'. 4. 426-432.

Kramer, J. j., lIenning-5toUl , M .. Ullman, D. P .. I\{

Schellenberg, R. P. (1987). The "j"bililf ur ....:"ttr: , analysis on the WISC·R and rhe SBIS: E.)(;lmining 3

vest ige. jOllrual of P'i",I,h{)rdu<'lI/iuJ1al A5.lessmmt, 5, 37-47.

LawS(ln, J. 5., & Ir.glis, J. (1984). T he psychorru:tric a~· sessment of children WIth learning disabilities: All in­dex derived from a pn ncipal compuneuts ana lysis o f !he wlse·l{. lUI/rna! uf Learning D;,~abr'litie~, 17. 5 17-522.

Livingston, R . H., Jennin~9, E., Re)'nulds, C. R. , I\{ Gray, R. M. (2003). MuiliYarilife analyses uf the profil", srabihty of inteJligcn;e tests: H igh fur IQ9. low to "~ I). low for $ubte$t ana l )·s<:~. Arcbi~'$ of Cliniazi N~,.rops)'(h%S)'. 18. 4 87- 507.

Lowman, M. G ., Schwan~ , K. A .. & Karnph1nlS, R. \V.I. (1996). WISC·HI mud facror: Critical measlJrem",n t issues. C4,wdlall JOII",alu( Schoo/l'sychology, 12, 15-22.

Maller, S. J., & M.:Dmnolt, I'. A. (1997). WA1S·R pIO· fi l", ~nal ysi.1 fOf c() JI ~ge students with Icarnin!: Ji~3bil· ili e~ . School P~)" hology J< tvi~[jI, 26. 575-585.

Ma!a rn7.7.O, J. D. (1985). I'sychologlcal assCismcm of in­rdligencc. In H . I. Kaplan & S. j. Saoo.·k. ( Ed~. l,

L66

Cornprebl.'nsi'Je Ic<lbouk of psychiatry (vu!. L 4(11 ('d., pp. 501-513). Baltimore: William, & Wilkius.

,\kO<:rmctr, P. A .. F3'ltU7,1.0, J. W., & C lunin!,;, J. J. ( \ 990). In_,( ~1r no to Sllhr~~' ~n~lysi~: A c.;-i riq lle on W...:hsler rheoq' :mil pr~ Cli(e_ ,,,,orna/ of P.wchol'Ju­,attOllal A55eS,' mem, S, 290-302.

McDermott, r. A., famuno, J. \~/. , GlUTting, J. J .. Walkin., !vi. WI., & Bagga1cy. R . ,\. (1992). llIu~ion

<;of meaning in the ipsadvc ~5SCSSll1cnt of children's abili{~·. /o1lr"al of Spedal Educatioll, 2;, 504--526.

,%;Dnmult, P. A., & GIIIII ;" ".1- J- (1997). Informing ~ly l jSli( [.,'HIUII\; b<,hu,-;or, ,i;spus;rion. <luJ a<:hi<:~e ­m.,nt Th rough ahi lity .~"hlesr,;-or. m<Hf ilh,s;nns of meaning? &11001 P$}'cholo(:y RIlUir-w, 26, \6.}-\75.

:\ lcDtrmott, 1'. A., Glutting, J. -,-, JOl1tS, j. N., \'\-'atkiM, 1\-1. \'1,'., & Kush, J. (19119). Core proiile trpes in the \VISC-R Il3 l ional sample; Structu re, mcmbcnhip, and appucltions.l'sych%gic<1! Asscssl/I~m, 1,292-299.

lI.kDerllloll, P. A., GUt'Il , L. F., Francis, J M., & Swtr, D. H. ( 1996) . U'aming Beha!iioTS Sca!e. Philadel ­phia; Edll'll.e-tr1:; & Clinical Science.

1I.kDermou, P. A., Marston, N. (;., & SlOtt', D. H. (199J). Adius/In"nl Scale.< fm' Children anti Adol"$­CBllts. Philadelphi.l; f..:lumetric & Clinical Science.

McFall, R . . \1 ., & Tre.lr, T. A. (1999). Q".lnr.i fying the information \'a luc or di01cai asse,sments with signal (,bel.'tion the...,f)'. Jlm-mai Review uf Ps),cho/ogy, 50, 215-1 <1 J.

Jl.kGr(w, K S., & Knupik, S. N. (1996). The rel an ...,l1 -:.hip bel"'",ell imn-coguiti,·e ~~!{er on the Woodcock-John$on Ps},,'ho-&Iucutionu l Ball~ry~

R ~\'i~ .. d ",,<I ~cht>Ol M:I. ipv(Omf' rlt . loum.t11 of Sr.lu)ol l'$Yw%gy, 34, 351-3(,4 .

McNelILlc, Q. (1964 ). Lost; Ouc intdJigcllce? Whr? AmerieJl1 pj),chologist, 19, 871-882.

Meehl, 1'. c., &. ROsell, A. (1955). All te\."edeor pmbabll. 11Y and the effldmcy (If psychometric siglls, pam:rns, ur <.:utting .cor<.:s. P>')'ch%lf,ieal Bllllclm, 52, 194-2 16 .

Moffitt, T E .• & S;lvl , P. A. ( 1987). \'(l ISC-R Verbal a"d Pfrform ance lQ di~re p'"1C)' n ~II Imso:-ll'lC.tt;d coho rt, C l illi(JI.~igll i ficance and longitu<l inal swhilily. Jour-1li11 of Consultins and Clinical P$y"hology.. S5, 768-774.

t.Iudle r, H. H., Dennis, S. S., & Short, R. ! I. {198G}. A ",e l~·e"p l ora l ion of wlse·1{ h"tor '''ore I'lX)file';;I' a fUllctiL>n of diagllO$IS and :ntclkctll:ll kvel. Cana· dIan .Iow",,/I of S,hw! Psychology. 2. 2 t- 43.

Ne~<t"n~, L. G. J., & A1Jenb"'h A. P. (1996). Slabil iqo' of cognitive measu res in ch ild re-u uf a~~f'''!;'' abil ity. (hild Ncurop5yelJolog)\ 2, 16:\-170.

Oakbnd, T., p.rOOI)1., J., & GlllrnnJ3:, J. (2000). Usc of freedum from di~tr~,cibi liTJ' and p [oces~in!; speed '" <'Issess children's tt u-t.lking bchaVIIll&. Journal of School J>syeho!og}', 38. 46:1-475.

O.w.land. T .. & H u, S. 11992). The tOP 10 tests used with children and youth worldwide. BIII/crm of the InternatiONal Tes/ Commissi,,!? 19, 99-120.

Oh, H. J., Glutting.J. J .• & .\kDermutt, P. A. (1999). An q.>idemolul;;ic'ai--<:oitort study of DAS VlOces~i"o; spetd f:actor; How well doe-s it ide.n.tify conc urrenr Khie,,~men t and beha .. iQ' problem.? Journlll of P,y­rhocd''';Jlriona/ A';$eS$m~nt, 17, 362.- 375.

I'edhazllc, £. J. (19!:1i). Multiple regnssion in D~/;m,ioral reH!arch .. E.xplanatlOlI and pr('dlCfioll (Jrd cd. ) . F()rt

Worth, TX, Haccourt Bracc. /'feiffcl, S. I., Reddl', L. A., Klctzci, J. f., Schmelzer, E.

R .. & Bo)·cr. L M. (2000). The pC:Jctinoncc 's '1C"," of IQ t~s ti ng \!nd profil", anal)'si •. Sehuul P>')'chol(J~'

Qunrlt:rly, 15. 376-JS5. Piedm(ml , R_ L , .~lIkol(lw·, R. L , & Fleming, j\·L Z.

(1989). ,\ n ex"mi tlotioJl of some diaStlo$l'ic st ratesie~ im·olvillg the W~hsln intdlig~ncc sc~l~s. fs)'dlO/og. ital ASSeS5I>1elU, 1.1111-1 85.

I'cifit~r3, 1\., & Decsh,J. (1993). Base cates of W'[SC-U I di.:lgnostie sllb!est patterns among normal, ItJ rning­di~ablcd. and ADHI) iampks. jOlm!al of l'sye/lOedl" mliorwl ASSt'$smenl. WfSC-flI MonugTl1/,h. PI'. 43 55.

Ree, M.J., & Carrel lO, T. R. (1997). Who! nmke, ml ap titlJdc fC$f ,'aljd~ rn R. F. Oillol) (F.J.), Handbook 0"

/I'$Iin8 (p ]l. 65-81 ). \'(/p~rpo rl. CT l,reenwoo rl Prr.'8. Ree, M. j., &: C1 rreTt:l, T. R. (2002 ). g2K. l1m" .. n Per·

fnrmana, 1.1" • . 1-2.1. RcrT\old~, C. R .. & Kamphaus, K. W. (2003). Re),nolds

fnteflectual AS5~SiI>lenl 5eal~s and the Reynulds in·

telleetual Screenin~ Test "Torns/Qual manllal. Lllt;,;. fl., P~~chologj<::ll A.se~sme(lt Resources.

Ri $ peIl~, 1-. Swaab. H .. \"all Jen Ourd. E 1- c C., Cohell - K~tl~nis, P. , va" eng,,\and, H., &: ,'a ll YJ.lCr~n, T ( 199i). wIse p r<,fi1,os in chilrl r~}'chi ~lfi" d ;~r."O­

sis; Stm~ or lIons~nse? jOlmw.1 of tlll~ Amlrical) Acadlmy of Child ami At/olesum Psychiatry, 36, 1587-1594 .

Roid, G. H. (100J). Sr.11lford- lJlJlel Intelligence Scales, fifth Edition: Tuhu/C{// nUIJIlIal. itasca, Il; Rlvec· s ide .

RY:I1l, J. 1-. & &,h~c. D_ L (1994). >JeuroJiagnosnc 1m.

pli catiotl~ of uniq llt- profiles of tht- \Y,I«;hs le r Adult l ll lclI ir,encr. Scalr~Revisrn. P.<y,;lJolog;,;a! As.<r..<s· menl, 6, 360-36J.

Ryall,.J. j., Krtinr.r, D. S., & Bunon, D. n. (2002). Does high scatttr affect [he predictive va lidity (l/ \VAIS·!II 1<)8: Applied Ni!urops)'choIOK';>i 9. 173-1 n .

S"IVIA, j., 8< YHd dyko. J. E. (1998 ). Assa.-sm~nt (7 rh cd. ). Boston: HOllgh!o1I 1 .... 1iftlin.

SsttJet, j. \ .1. (200 1). AH('ssmem of "bildre": C08",'ill(' al,p!ictJlio"s ('Il l, etl. ). SaIl Diego, CA: Jerome M. Saliler.

Schntidt, F. L. (2001 ). The role of gellt"ra l cognitive abil­ity :1nd job Jl'C!fo,ma ncc; Why thae can nor be ~ de· bot~ . /l",,,,,,, Pe,f",-",,,,,",,, 15, 187- 2 10.

Schlmdt, t'. L., Le, H., & Il i ~., It (200J). Beyond alpha; I\n empir ica l exa mination of the effecTS of diiferem sources of measurement error on re lj ,\b iJity esnma te, for measure, of indi\"idu~l d jff~rellcc!i CQnStrUCfS.

Prych(Jlogieal Methods. 8, 206-224.

SlIhr!:.';1 Profile Anal~i~ 267

Shaw, S. R" Swcrdlik, 1\1. E_. & lliureru, J. ( 1993). Re· \'Iew of the WISe·IIL In B. Brnckcn &: R. S. )"kCallum (EJq, Ad""mcC$ in prychO('Juwrional a~· $('$$/1/£1'11 (I'll. \.) 1 160). BrnnJon, VT: Clinkal Ps\·· chnlogy.

5Imel1~n, R. j.. &. SLltherbnd, J. (1974). I'iycho!ogical ,U!.l!.um~nr oi hrain d amage: Tht Wechsler 5~~lcs . Acadcmic Thmrp); 10.69-81.

Smith, C. B .. &. \'(.Iatkins, ~l . W. (2(f()4j. Dillgnrmic uril ­iry of the Bannat)'llc WISC-II1 p~m'rn. Learnlllg Dis· llbiUries RlSttlrch and Practice. I~. 49-56.

Smith, T., Smith, B. l., Brallllcn, R. K., & Hick~, N . (1999, Aprill. W?rSC- II I stability I")~r a thrr.lf )'MY "e' riod. Paper prcSt:nreri ~r the a nn" ,,1 mettinll of the Natinnal AS5OCi~ ti"n oi School Ps)'chnlogius, La, Vcr,as, NV.

Sparrow,S. S., & D.wis, 5. Iv!. (2{JOO). Reccnt Jd\'~nccs in the 1lS!.tssmCnt of intcll4:cncc Jnd CO lin ilion. JOIIT' na/ of Child Ps)'cho/(}gy a"d fry·cI!uII,..,~ 41. 11 7 131.

Swcrs, J. A. (1988). )'1caSllr)Jl!l th ~ al't:lItiKy ui di:Igll(ls' tk srstcnls. SO('!ICC'. 240. 1285 1293.

SWtts. J. A. (1996). SiS'U</ dctcction tJICO''Y ,/lid RIC analysi., In p~.vcb<)lug)' (lll</ diIl8I1o<is, 1"lIl'ctl'{l pa· I'r ' ." .\'ln l ,,,·~h, NJ f.dhanm .

Sw~", J. A., D,lWts, R. M., & ."IOildhan, J. t2000j, l'sy· chnlot;ical scIence c,In unpro>'e dilgnOSllC decisions. ['srdm/oglcaf SClma III thc PllbllC In/ntst, 1, 1- 26.

Tu tu, I'. A" /3( Korduckl, K. (I~98 ). t\sst'umcnt of emotionally d isrurbcd children WIth the WISCIII . In .J,. l'rifitc rOi & D. H. Saklofske ([(.1$.1, Wlse·lIE dm; cal USI' (111(/ ",tN",..'''';'''''' S!r:lrr.li<l [mu .. ,itinn' .... (I') "

~pel;tillcs (pp. J 19-138). 5.,n Di~bO, CA: Ac.,demic Prcss.

Thorndike, R. L (1986). The role of general ability In

prediccioll . joumal of Vocatlmud JJth.1l'ior, 2~, 331.-3.19,

Thorndike, R. M. ( 1997). MeJSllUmenr and C1'allfatll)n in psyd,%!rY (md ,duc:1riotl (6.h cd.) Upper S"dd(~ Rivc r, ~j : McrnlL

U.S. Dl:plIr[lIIo:ul (If Education. (2DOI). TI~'cn"~/i,st ,/>1'

mUll repor/II> u.mgress 0>1 tll~ iH1plemenlalion of 'hI' JI'II/i,liJu,1i. wilh Dis.abilities ;:'/',,(;all""l .... cr. JcssuP, MD: A"thot.

Ward, S. It, \'\Iud, T. J., I LlIt, C. V., YOU<Ig, D. L., & Molhle" N. R. (1995). T he incidence and utility of the ACID, ACJ1)S, and SCAU , roilles in II referred popuboon. PS)'CI'0Iog)' In the Schools, J2, 267-276.

W·a tkins. M. W. (1996). Diagno.uc uti1it~, of tho: wise I!1 dcvcJupmcntallIldex as a pr .... J iccor of lellrnlnll tli. abilities. Joun/al of Leorni"g Dl<iOhili/;cs. 19, 305-.H2.

W!lTkm~, M . W. ( 1999). Di:lsnos,tic " ,iii,), o f WISe·1I1 S"hl~1 vJriabilit)· .un(')ng studrau with learmng d is· :lhilities. OJnadial/ /ou", .. / of Samot Psycboiog)" t 5, 11-10.

Watkins, 1\1. W. (2000). Cognitiw p rofile :ln3.lysit: i\ sha red prufeSl iunal myth, S,hool /'s)',ho lo8Y Quar. lert); 15.465-47:1.

Warkins, .... 1. '\XI. (1003). IQ &" brC.<:t analysis: Clinical acum~n nr clin ic;}1 inusion~ Scitntl(i, R~I/jelfl of :\·Im· lal llealllJ Pllwiu, 1, 11 8-1 4 1.

Watkins, .\'1. W., & Cmivr%, G. L. (2004). '(emporll stahtl"y of \VISC·lU !iubt:cs! composite: $tr .. nglh ~ and wC3 kn~n~'S. Psy(hO/OKIwl ASSt'ssmi-",', 16, 133-131).

W:llkin~, M. W .• & G l uui"~, J. J. (2000). locremCIII ~ 1 \'aljdit ~, of W ISC III pwfae dl'~ a!joll, ~ ,atle r, J ,1(i :;jmre infor!ll;lrioLI for preriir.r ing re","ling and I'l.uh achie\'ement. I's\'choIORi(al A~se.>"lm"'1t. 12, 402-408.

Watk in~ .. \1. W .. & Kush, J. c. (1:194). Wcch.kr su blCJt a l1a lysii: The right war, the wrong war, Uf no wa)': School Ps)'chology Keneu, 21. 640-651.

Watkins, ~1. W., KlIsh, j. C., & Gluning, J. j. (1997l). DISCTimmam :rnd pn:ditli\'c v:Uld ily of Ihe W ISC 1II AOD profIle among t hilJn:n with icaming disahili ti~. P5~'cI'olog)' in th<l S.:bm.ls, 34, 309-3 19.

W~tk jJl~, 1\-1. W., Ku~h, J. c., &. GI,ming, J. J. ( 19971'1). Prl'!v31~nc.e nmi d i.1gn(lSt ic ut ility of tht WISe·1II SCAI) prnfile .,mong children with diJaf>i!itJe~ .

School !'~)'chofop")' Quarterly. 12, 235-248. Wark i n~,:V1. \Xi., Kush, J. C., & !lochader, B. A. (2002) .

Diagnostic Uliht)' of The Learn ing Olsabil ir}' Ind~x. Jot/rnal of Lel1r11mg D,sal,ilities. JJ. 9l:1 10.l.

W:ltkms, .J\ J. W. , & Wart~lI, F. C. (l OOO). Dia!;;"ostk ut ility of the numhc: of \vIS(·1Il $ub~n. dC"'iatine frolll mt'3n ~rlOruI3I1C" Alllong snodenrs ,,:ith ll':lrn­ing Ji~h i htit-s. Psychology in th~ Schook 17, JO.'-30.9.

\'(/ed'L~ I ~r. D. (1949). Il'tel!sler Inrdligmu Scale {or Childnn, Ntw York; J's)'chological C'.orporatirln.

\,(Iechsler, D. (1 ~5~), T/;(' meaSIITI:mCllt and appraisal vf adulr inrelligmu (4 th cd. ). Bal! illJlm.:: WiUl~ ln~ & Wilkins.

Wechll lcr, D. (1974 ). Wt{hd~r J .. tl'lligf.'ncc Scale for Chlldr~n-RC'l/i~C'd. Ktw York , Psycl'<..>!O',;ir;rl Corpo-

We<:hsl~ r, D. (199 1). Wl!Chslu fmclli}:c" ce SC'il/, for Oildre"_ Third Ed;·,ioll . San :\ntnnio, TX: r~ych{)­logiC;l1 Corpomrron.

\'i'echs l~r, D. (l 992}. Wee/uler Irtdil'idua/ A,bict'tmc,,/ Test. Sa n Antonio, TX: I 's~'chological CO(pclullon.

\'(Iechskr, D. (1~97). W(chsll!!' Adrift Imell'gcllu' SCIl/c_ Th"d E.dit,on. S3 0 Ant<..> mo, TX, P~ycho lo8i­

cal (vrpora liuu. w<'Chslcr, D. (2003). lY/<'chs/e, r"lel/igelln' ,r;"'/I! fm

Child'l'lI-Fourth £Ai/io'l: Tfldm;,:al Dud i"tI!,pr~.

tive ",a!lual. San Anrnnin, TX: Ps},.;holugic.,1 Curpo' r~ tjQn.

Whil r:-olp.f. T. A., Fnul~di, R. T., & Williams, N. J. (2002) . ~tctm;mng prcdi<::tor jnlport.~nec in multi ­ple regreSSIon urldcr \'~ n~'Ii correla tional and dl!ilri· bU[lon~1 condioons.jollrnai of i"Iudl'm ,'lpplt~d Sla· tl$tical,\ielhods, 1.354-366.

\'i7is.:~i ns , J. S. (1988 ). (','rSO lMI,ly ,md pres/icllDn: PHiO clples vf pI'nlmal,I')' ''''5essme"l. ?vbluhar. FL: Kriegt"r.

268 INTEI{PI{CTWE APPI{UAClIE~

YQungsrro lll, E. A., KOg05, J. L., & Glutting, J. J. ( 999). Im;rcmenta l cffkal:Y of Differential Abiliti' S<:3J~S factor xor~> in p.rnlil:ting individual adlleve· ment cr ij"ria. School Ps)'chology Quarterly, 14,26-39.

Zachary, R. A. {I 990}. Wechsler's intell igence scalts: Theor~ tica l and practica l cOll5icieratiom. Journal of !'s~',hocdu,afiollal Asstl5Smcm, Ii, 276-2~~.

Zarin, D. A., & Earls, E (I 'N3). Diagnostic decision making in psychiatry. AmerlCMI journal of l'sycl'sa' tl)~ 150, 197-206.

Zeidner, M. (2ool ). Invi ted foreword and inuoducrion.

In]. J. W. AlldrCW~, D. H. Saklohkc, & H. l.Janzen ( F.d~ . ), Hrmdbvok of p>yduxJuClltiv1Ul1 a~~{!~~ment,

Ability. a(hi(lIl!menl, and behavior in c!;iIdrl7l. $~ Il

Diego, CA: Ac~delll ic Press. Zeidner, M., & Matthews. G. (2000). Lllelligenco> and

personality. In R. J. Sternberg (Ed.), Handbook of in· lel!igl!llr:e (pp. 5/l 1-61O). New York : Cambridge Uni­"enity Press.

Zimmerman, I. L., & Woo-Sam, J. M. ( 1985) . elmicai applicanons.1n.l$. B. Wolmao (ed.i, Handbook olin­tellignlu: Theories, measurement5, and appllwtions (pp. 873-898). New York: Wiley.