for many veterans, EBPs have allowed forunprecedented improvements—gains forsome not previously achieved over decadesof suffering. These individual stories arepowerful and gripping and should be cele-brated. However, the work is not done. Thequest to refine and continuously improvethe effectiveness of clinical approaches andpromote their use in routine clinical set-tings must continue and expand. Our na-tion’s veterans and others with mental ill-ness deserve the most effective care wehave today and that we can realize fortomorrow.

REFERENCES

Eftekhari, A., Ruzek, J. I., Crowley, J. J.,Rosen, C. S., & Karlin, B. E. (2013). Effec-tiveness of national implementation of Pro-longed Exposure Therapy in Veterans Af-fairs care. Journal of the American MedicalAssociation Psychiatry, 70, 949–955. doi:10.1001/jamapsychiatry.2013.36

Greene, L. R. (2014). Dissemination or dia-logue? American Psychologist, 69, 708–709.doi:10.1037/a0037007

Holt, H., & Beutler, L. E. (2014). Concernsabout the dissemination and implementationof evidence-based psychotherapies in the Vet-erans Affairs health care system. AmericanPsychologist, 69, 705–706. doi:10.1037/a0037008

Karlin, B. E., Brown, G. B., Trockel, M., Cun-ning, D., Zeiss, A. M., & Taylor, C. B. (2012).National dissemination of cognitive behav-ioral therapy for depression in the Departmentof Veterans Affairs health care system: Ther-apist and patient-level outcomes. Journal ofConsulting and Clinical Psychology, 80, 707–718. doi:10.1037/a0029328

Karlin, B. E., & Cross, G. (2014). From thelaboratory to the therapy room: National dis-semination and implementation of evidence-based psychotherapies in the U.S. Departmentof Veterans Affairs health care system. Amer-ican Psychologist, 69, 19–33. doi:10.1037/a0033888

Karlin, B. E., Trockel, M., Brown, G. K., Gor-dienko, M., Yesavage, J., & Taylor, C. B.(2013). Comparison of the effectiveness ofcognitive behavioral therapy for depressionamong older versus younger veterans: Resultsof a national evaluation. Journals of Geron-tology: Series B: Psychological and SocialSciences. Advance online publication. doi:10.1093/geronb/gbt096

Karlin, B. E., Trockel, M., Spira, A., Taylor,C. B., & Manber, R. (2014). National evalu-ation of the effectiveness of Cognitive Behav-ioral Therapy for insomnia among older ver-sus younger Veterans. International Journalof Geriatric Psychiatry. Advance online pub-lication. doi:10.1002/gps.4143

Karlin, B. E., Walser, R. D., Yesavage, J.,Zhang, A., Trockel, M., & Taylor, C. B.(2013). Effectiveness of acceptance and com-mitment therapy for depression: Comparisonamong older and younger veterans. Aging and

Mental Health, 17, 555–563. doi:10.1080/13607863.2013.789002

McHugh, R. K., & Barlow, D. H. (2010). Thedissemination and implementation of evi-dence-based psychological treatments. Amer-ican Psychologist, 65, 73–84. doi:10.1037/a0018121

Steenkamp, M. M., & Litz, B. (2014). One-size-fits-all approach to PTSD in the VA not sup-ported by the evidence. American Psycholo-gist, 69, 706–707. doi:10.1037/a0037360

Stewart, M. O., Raffa, S. D., Steele, J. L., Miller,S. A., Clougherty, K. F., Hinrichsen, G. A., &Karlin, B. E. (in press). National dissemina-tion of Interpersonal Psychotherapy for de-pression in Veterans: Therapist and patient-level outcomes. Journal of Consulting andClinical Psychology.

Trockel, M., Karlin, B. E., Taylor, C. B., &Manber, R. (2014). Cognitive BehavioralTherapy for insomnia with Veterans: Evalua-tion of effectiveness and correlates of treat-ment outcomes. Behaviour Research andTherapy, 53, 41–46. doi:10.1016/j.brat.2013.11.006

The opinions expressed in this comment do notnecessarily represent the official policy positionof the Department of Veterans Affairs (VA).

Correspondence concerning this commentshould be addressed to Bradley E. Karlin, Edu-cation Development Center, Inc., 96 MortonStreet, 7th Floor, New York, NY 10014. E-mail:bkarlin@edc.org

http://dx.doi.org/10.1037/a0037874

Individual Expertise VersusDomain Expertise

James ShanteauKansas State University

David J. WeissCalifornia State University, Los Angeles

It is certainly flattering to have one’s re-search cited 20 years after publication. Thedanger, however, is that views can becomeoutmoded. Tracey, Wampold, Lichtenberg,and Goodyear (April 2014) addressed thequestion of whether “psychotherapy is aprofession without any expertise” (p. 218).They answered affirmatively, citing thesuggested criterion that experts ought toprofit from experience (Shanteau, 1992)and supporting earlier speculations thattherapists do not. Two possible reasons areoffered: (a) lack of access to reliable out-come feedback and (b) use of inappropriateinformation-processing strategies. We donot disagree with either the authors’ assess-ment of the expertise of individual psycho-therapists or their reasoning as to why. Inthe past two decades, however, new in-sights have emerged on expertise in various

domains, including psychotherapy. In par-ticular, we have developed a general, rela-tivistic perspective on expertise that in-vokes performance-based criteria (Weiss &Shanteau, 2003, 2014). In this commen-tary, we wish to highlight three distinctionsthat have emerged from recent research onexpertise.

Diagnosis Versus Treatment

Psychotherapists in particular and medicalpractitioners in general engage in two lev-els of decision making. Diagnosis, a purelyjudgmental task, is challenging becausethere are hundreds of possible conditionsdescribed in the 947 pages of the DSM-5(American Psychiatric Association, 2013).Diagnosis is what is usually examined inexpertise studies. Treatment involves notonly judgment but also the additional skillsneeded for implementation. For treatment,however, there are far fewer options (Shan-teau, Edwards, & Weiss, 2009).

For the patient’s well-being, the keyis to select a therapy that works regardlessof the diagnosis. It is analogous to medicaldoctors telling a feverish patient with anunknown ailment to “take two aspirin andcall me in the morning” because that rem-edy often works. Effectiveness of treatmentis often independent of accuracy of diag-nosis. Expertise is highly task specific; apractitioner could be good at diagnosis andweak on treatment, or vice-versa.

Individual Versus Domain Expertise

Hammond (1996) argued for two types ofcriteria for assessing judgmental compe-tence: Coherence refers to agreement witha theory. Correspondence refers to agree-ment with an external reality. “Modern sci-entific reasoning advocates using both co-herence in the form of rationalism andcorrespondence in the form of empiricism”(Dunwoody, 2009, p. 117).

Applied to expertise, correspondenceis a sufficient condition for establishing thecredibility of individual experts. For manydomains in which experts work, unfortu-nately, correct answers are seldom known(at least in a timely fashion). In particu-lar, for psychotherapy, outcomes are de-layed and often distorted (Tracey et al.,2014). In such situations, we have arguedfor assessment using a coherence crite-rion that is built on two necessary condi-tions for expert judgment (Weiss, Shan-teau, & Harries, 2006).

Most of our recent research on exper-tise uses an index, the CWS (Cochran-Weiss-Shanteau), which incorporates twoabilities: discrimination and consistency(Weiss & Shanteau, 2003). To be effective,

711October 2014 ● American Psychologist

ThisdocumentiscopyrightedbytheAmericanPsychologicalAssociationoroneofitsalliedpublishers.

Thisarticleisintendedsolelyforthepersonaluseoftheindividualuserandisnottobedisseminatedbroadly.

an expert must be able to discriminate be-tween cases in a consistent fashion. Ourresearch reveals that length of experiencerarely predicts expertise except during theearly period of training. Using the CWScriterion, experienced counselors wereable to diagnose depression better thannovices; however, master’s-level stu-dents were even better (Witteman, Weiss,& Metzmacher, 2012). This pattern hasbeen repeatedly observed across domains.Experts do not necessarily keep improvingwith experience. From our current perfor-mance-based perspective, experience andimprovement over time are both irrelevantto determining extent of expertise.

We suggest that coherence criteriacan be applied at the individual level,whereas correspondence criteria can beapplied at the domain level. Thus, indi-vidual therapists may or may not demon-strate expertise in diagnosis and treat-ment. The key questions are (a) Can thetherapist consistently diagnose accordingto recommended guidelines? and (b) Canthe therapist identify and apply the req-uisite treatment?

Whether the diagnoses and treatmentsthat constitute the collective wisdom of thefield produce better health outcomes is adifferent question. The evidence cited byTracey et al. (2014) suggests that psycho-therapy, as a whole, does possess expertise.The expertise of a field progresses as moreeffective procedures are developed, whereeffectiveness is assessed by correspon-dence criteria.

Consider two purported experts, onein medicine and one in astrology. If thedoctor and astrologer both follow conven-tional practice, then they have demon-strated individual expertise; they have cor-rectly followed the standards of their fields.It is a separate question whether followingthose procedures produces useful results;medicine usually does, astrology usuallydoes not.

Outcome Versus Process

Ward Edwards, founder of research on de-cision making, highlighted a vital distinc-tion between decision-making process anddecision outcome. The former refers towhat the decision maker actually does,whereas the latter depends on factors oftenunrelated to the decision, including envi-ronmental influences and chance occur-rences (Vlek, 1984).

Tracey et al. (2014) recognized thisdistinction, although they related process topsychotherapeutic protocol. We concurwith Edwards and colleagues (Vlek, 1984)that process depends on internal weights

and subjective values, along with process-ing rules.

Suppose a therapist processes the in-formation about a patient appropriately buta bad outcome ensues because of externalinfluences on the patient unrelated to ther-apy. We would regard this as positive ev-idence regarding the therapist’s expertiseeven though the outcome was a “failure.”

Conclusions

Research on expertise is providing impor-tant insights into many fields of study, in-cluding psychotherapy. Given the unstableenvironment of a psychotherapeutic en-counter, it should not be surprising to findthat therapists fail to meet the high stan-dards set by experts in other fields such asweather forecasting. That is not the fault ofthe therapists; rather, it reflects the diffi-culty of psychotherapy.

REFERENCES

American Psychiatric Association. (2013). Diag-nostic and statistical manual of mental disor-ders (5th ed.). Arlington, VA: American Psy-chiatric Publishing.

Dunwoody, P. (2009). Theories of truth as as-sessment criteria in judgment and decisionmaking. Judgment and Decision Making, 4,116–125.

Hammond, K. R. (1996). Human judgment andsocial policy: Irreducible uncertainty, inevita-ble error, unavailable injustice. New York,NY: Oxford University Press.

Shanteau, J. (1992). Competence in experts: Therole of task characteristics. OrganizationalBehavior and Human Decision Processes, 53,252–266. doi:10.1016/0749-5978(92)90064-E

Shanteau, J., Edwards, W., & Weiss, D. J.(2009). Diagnosis and treatment. In J. W.Weiss & D. J. Weiss (Eds.), A science ofdecision making: The legacy of Ward Ed-wards (pp. 308–320). New York, NY: OxfordUniversity Press.

Tracey, T. J. G., Wampold, B. E., Lichtenberg,J. W., & Goodyear, R. K. (2014). Expertise inpsychotherapy: An elusive goal? American Psy-chologist, 69, 218–229. doi:10.1037/a0035099

Vlek, C. (1984). What constitutes “a good deci-sion”? A panel discussion among Ward Ed-wards, István Kiss, Giandomenico Majoneand Masanao Toda. Acta Psychologica, 56,5–27. doi:10.1016/0001-6918(84)90004-0

Weiss, D. J., & Shanteau, J. (2003). Empiricalassessment of expertise. Human Factors, 45,104–116. doi:10.1518/hfes.45.1.104.27233

Weiss, D. J., & Shanteau, J. (2014). Who’s thebest? A relativistic view of expertise. AppliedCognitive Psychology, 28, 447–457. doi:10.1002/acp.3015

Weiss, D. J., Shanteau, J., & Harries, P. (2006).People who judge people. Journal of Behav-ioral Decision Making, 19, 441–454. doi:10.1002/bdm.529

Witteman, C. L. M., Weiss, D. J., & Metzm-acher, M. (2012). Assessing diagnostic exper-

tise of counselors using the CWS index. Jour-nal of Counseling & Development, 90, 30–34.doi:10.1111/j.1556-6676.2012.00005.x

Correspondence concerning this commentshould be addressed to James Shanteau, Depart-ment of Psychological Sciences, Bluemont Hall492, Kansas State University, Manhattan, KS66506-5302. E-mail: shanteau@ksu.edu

http://dx.doi.org/10.1037/a0037832

Supervision, a NonelusiveComponent of DeliberatePractice Toward Expertise

Eleanor H. McMahanThe University of Georgia

Tracey, Wampold, Lichtenberg, and Good-year (April 2014) provided a thoughtful dis-cussion regarding the difficulty of achievingexpertise in psychotherapy and offered sug-gestions about approaches toward psycho-therapeutic work that may increase expertise,including deliberate practice. While veryhelpful, these suggestions appear to neglectacknowledgement of one of the most widelyused modes of improving practice and facil-itating specific feedback regarding psycho-therapeutic work: supervision. It could beargued that deliberate practice, and thereforeexpertise, is most effectively and efficientlyadvanced through supervision of psychother-apeutic work.

Deliberate practice, defined by the au-thors as “the explicit setting aside of privatetime to review one’s behavior and outcomefeedback, developing plans for improvement,and then following through on these” (Traceyet al., 2014, p. 225), is precisely what qualitysupervision accomplishes. It may be that in-tegration of components which the authorsemphasized as additionally important for thedevelopment of expertise, such as better qual-ity outcome data, specific feedback on impor-tant components of psychotherapy, feedbackrelative to other professionals, a priori hy-pothesis testing, and disconfirmatory ap-proaches, is only really feasible through theobjectivity and accountability enabled by asupervisory process.

It is perhaps self-evident that withoutobjectivity and accountability these pro-cesses cannot be effective in improvingpractice, and given the inherent bias andsubjectivity of psychotherapy, the indepen-dent engagement of these processes to im-prove expertise is a significant challenge.Additionally, it may be particularly usefulto consider the integration of these ele-ments into supervision processes, as hasbeen previously proposed in the supervi-

712 October 2014 ● American Psychologist

ThisdocumentiscopyrightedbytheAmericanPsychologicalAssociationoroneofitsalliedpublishers.

Thisarticleisintendedsolelyforthepersonaluseoftheindividualuserandisnottobedisseminatedbroadly.