€¦ · the behaviortherapist issn 0278-8403 volume 41, no. 1•january 2018 [continued on p. 3]...

the Behavior TherapistI S S N 0 2 7 8 - 8 4 0 3

VOLUME 41, NO. 1 • JANUARY 2018

[continued on p. 3]

January • 2018 1

ABCT

▲

ASSOCIATION forBEHAVIORAL andCOGNITIVE THERAPIES

SPECIALISSUE PSEUDOSCIENCE

in Mental Health Treatment

Special Issue Editor: R. Trent Codd, III

R. Trent Codd, IIIIntroduction to the Special Issue: Pseudoscience in MentalHealth Treatment: What Remedies Are Available ● 1

David N. Rapp and Amalia M. DonovanThe Challenge of Overcoming Pseudoscientific Ideas ● 4

David TrafimowThe Scientist-Practitioner Gap in Clinical Psychology:A Social Psychology Perspective ● 12

William O’DonohueScience and Epistemic Vice: The Manufacture and Marketingof Problematic Evidence ● 19

William C. FollettePseudoscience Persists Until Clinical ScienceRaises the Bar ● 24

Clara Johnson, Shannon Wiltsey-Stirman, and Heidi La BashDe-implementation of Harmful, Pseudoscientific Practices:An Underutilized Step in Implementation Research ● 32

Stuart VyseWhat’s a Therapist to Do When Clients Have PseudoscientificBeliefs? ● 36

Dean McKayThe Seductive Allure of Pseudoscience in Clinical Practice ● 39

Scott O. Lilienfeld, Steven Jay Lynn, and Stephen C. BowdenWhy Evidence-Based Practice Isn’t Enough:A Call for Science-Based Practice ● 42

Lisa A. NapolitanoPseudotherapies in Clinical Psychology:What Legal Recourse Do We Have? ● 47

Monica PignottiExposing Pseudoscientific Practices: Benefits and Hazards ● 51

[Contents continued on p. 2]

Introduction to the Special Issue

Pseudoscience in MentalHealth Treatment: WhatRemedies Are Available?R. Trent Codd, III, Cognitive-Behav-ioral Therapy Center of WNC, P.A.

MANY MENTAL HEALTH professionals deliverinterventions that are unsupported by science.These interventions range from inert to harm-ful. In addition, many consumers of psycholog-ical services espouse confidence in scientificallyunsound theories and their associated interven-tions. The behavioral consequences of suchconfidence is frequently consumer pursuit ofunhelpful treatment, often to the exclusion oftreatments with empirical support. Clinicianand consumer allegiance to unsubstantiatedtreatments is a major barrier to the optimal careof persons with psychological difficulties.

An example of how pseudoscience has inter-fered in my own clinical practice is instructive.There is widespread agreement in the scientificcommunity that exposure and response preven-tion (ERP), which has been available fordecades, is the gold-standard treatment forobsessive-compulsive disorder (OCD). Yet, it isunclear whether most persons with OCDreceive ERP rather than treatments not indi-cated or even contraindicated in the treatmentof OCD. Many anecdotes illustrative of thisproblem are available for sharing. Also availableare examples of patients involved in ERP whosimultaneously received competing advice thatundermined their treatment and did not com-port with the scientific database pertaining toOCD. One salient anecdote involves a formerpatient of mine with particularly severe OCDsymptoms. During my attempt to deliver ERPto him, this patient was variously advised to

2

the Behavior TherapistPublished by the Association for

Behavioral and Cognitive Therapies305 Seventh Avenue - 16th Floor

New York, NY 10001 | www.abct.org(212) 647-1890 | Fax: (212) 647-1865

Editor: Kate Wolitzky-Taylor

Editorial Assistant: Bita Mesri

Associate EditorsRaeAnn AndersonKatherine BaucomSarah Kate BearmanShannon BlakeyAngela CatheyTrent CoddDavid DiLilloLisa ElwoodClark GoldsteinDavid HansenKatharina KircanskiRichard LeBeauAngela MorelandStephanie Mullins-SweattAmy MurellAlyssa WardTony WellsSteven WhitesideMonnica Williams

ABCT President: Sabine WilhelmExecutive Director: Mary Jane EimerDirector of Communications: David TeislerDirector of Outreach & Partnerships:

Tammy SchulerManaging Editor: Stephanie Schwartz

Copyright © 2018 by the Association for Behavioral andCognitive Therapies. All rights reserved. No part of thispublication may be reproduced or transmitted in anyform, or by any means, electronic or mechanical, includ-ing photocopy, recording, or any information storageand retrieval system, without permission in writing fromthe copyright owner.

Subscription information: tBT is published in 8 issuesper year. It is provided free to ABCT members.Nonmember subscriptions are available at $40.00 peryear (+$32.00 airmail postage outside North America).Change of address: 6 to 8 weeks are required for addresschanges. Send both old and new addresses to the ABCToffice.

ABCT is committed to a policy of equal opportunityin all of its activities, including employment. ABCT doesnot discriminate on the basis of race, color, creed, reli-gion, national or ethnic origin, sex, sexual orientation,gender identity or expression, age, disability, or veteranstatus.

All items published in the Behavior Therapist, includ-ing advertisements, are for the information of our read-ers, and publication does not imply endorsement by theAssociation.

The Association for Behavioral and Cog-nitive Therapies publishes the BehaviorTherapist as a service to its membership.Eight issues are published annually. Thepurpose is to provide a vehicle for therapid dissemination of news, recentadvances, and innovative applications inbehavior therapy.

Feature articles that are approxi-mately 16 double-spaced manuscriptpages may be submitted.

Brief articles, approximately 6 to 12double-spaced manuscript pages, arepreferred.

Feature articles and brief articlesshould be accompanied by a 75- to100-word abstract.

Letters to the Editor may be used torespond to articles published in theBehavior Therapist or to voice a profes-sional opinion. Letters should be lim-ited to approximately 3 double-spacedmanuscript pages.

Submissions must be accompanied by aCopyright Transfer Form (which can bedownloaded on our website: http://www.abct.org/Journals/?m=mJournal&fa=TBT): submissions will not be reviewed with-out a copyright transfer form. Prior topublication authors will be asked tosubmit a final electronic version of theirmanuscript. Authors submitting materi-als to tBT do so with the understandingthat the copyright of the published mate-rials shall be assigned exclusively toABCT. Electronic submissions are pre-ferred and should be directed to theeditor, Kate Wolitzky-Taylor, Ph.D., [email protected]. Pleaseinclude the phrase tBT submission andthe author’s last name (e.g., tBT Submis-sion - Smith et al.) in the subject line ofyour e-mail. Include the correspondingauthor’s e-mail address on the cover pageof the manuscript attachment. Please alsoinclude, as an attachment, the completedcopyright transfer document.

INSTRUCTIONS Ñçê AUTHORS

Contents, continued

At ABCT

Classified ● 54

Call for Nominations for ABCT Officers ● 55

Awards & Recognition Ceremony, 2017 ● 56

Call for Award Nominations, 2018 ● 58

52nd ANNUAL CONVENTIONPreparing to Submit an Abstract ● 60

Understanding the ABCT Convention ● 61

Call for Continuing Education Sessions ● 62

Call for Papers (General Sessions) ● 63

ABCT and Continuing Education ● 64

I N T R O D U C T I O N T O T H E S P E C I A L I S S U E : P S E U D O S C I E N C E

January • 2018 3

seek chiropractic care, neurofeedback, andeven allergy shots! For clarity, those inter-ventions were all recommended specifi-cally for his OCD. Sadly, I was not particu-larly persuasive and, despite myrecommendation not to do so, this individ-ual pursued each of these interventions,one after the other, as each failed in turn.Notably, the patient neglected ERP as heworked through this sequence of treat-ments. Also notable is that this patient’sOCD symptoms were so impairing that hewas unable to maintain employment andthus he struggled financially. He was notable to compensate with monetary assis-tance from his family because they did notpossess robust financial resources. How-ever, his financial obstacles did not impedehis pursuit of the recommended interven-tions. Although all of these interventionswere expensive, the allergy shots were par-ticularly costly because they entailed travelcosts (e.g., airfare, accommodations) as theprovider of this intervention resided out ofstate. This patient never returned to me fortreatment, so his terminal outcome isunknown. However, my belief, based onthe science, is that the odds of treatmentsuccess with ERP at my office were favor-able.

The problem of pseudoscience inmental health treatment is not new, unfor-tunately. Scientifically minded practition-ers have directed their attention to thisproblem. One of the primary approaches toaddressing this problem involves the appli-cation of critical analyses to various pseu-doscientific methods followed by the dis-semination of these analyses to consumersand professionals. The hope, of course, isthat these analyses will impact the behaviorof practitioners and their clientele.Whether this approach is effective is dubi-ous, yet it seems to be the dominant strat-egy pursued historically. For example,when soliciting manuscripts for this specialissue, even a well-known pseudoscientifictreatment debunker had difficulty imagin-ing how he could contribute without“taking down certain approaches” specifi-cally by name. This seems to be thecommon way of approaching this problemamong well-intentioned scientists.

My objection to the debunking model isnot a moral one. Rather, given the abun-dance of pseudoscience, it seems safe toconclude that a debunking model isn’t par-ticularly effective. Even if it were successful,it’s not a practical solution because thereare simply too many pseudoscientific inter-ventions to address one-by-one. If one con-siders the rate at which new pseudoscien-

tific treatments seem to propagate, thedebunking model can only result in anendless game of whack-a-mole. Otherstrategies have been tried too, of course,including various forms of advocacy, edu-cation campaigns, and legislative efforts.Yet, the problem remains.

The primary objective of this specialissue is to explore alternatives to the puredebunking model. The contributors' acad-emic disciplines differ, affording fresh per-spectives stemming from their unique andvaried vantage points. Experimental psy-chologists Rapp and Donovan (this issue)open the issue with a presentation of anexperimental literature that can inform theconstruction of interventions targeted atthe remediation of pseudoscientific beliefs.Next, Trafimow (this issue) provides asocial psychological perspective andaddresses two main areas. First, he suggestsimprovements in the science of clinicalpsychology, an area also emphasized byother contributors to this special issue.Second, he recommends a line of researchfocused on practitioner behavior changeusing the Reasoned Action Approach(Fishbein & Ajzen, 2010).

O’Donohue (this issue) and Follette(this issue) focus on research methods inclinical psychology. More specifically,O’Donohue introduces the concept of epis-temic virtue and suggests that it has notreceived adequate attention in CBTresearch. He then underscores its impor-tance and provides recommendations forimproving its presence in CBT science.Follette (this issue) argues that the histori-cal emphasis on efficacy studies in clinicalpsychology to the exclusion of tests ofmechanisms of change has allowed pseu-doscientific interventions to persist byclaims of effectiveness.

Johnson, Wiltsey-Stirman, and La Bash(this issue), coming from the vantage pointof dissemination and implementationresearchers, discuss de-implementation orthe discontinuation of previously imple-mented practices. They consider the gener-alization of de-implementation models foraddressing the problem of pseudoscientificpractices.

Next, behavior analyst Stuart Vyse (thisissue) addresses the problem of clients whoare committed to non-evidence-basedtherapies. He offers several strategies forreasoning with these types of clients basedon recent research on effective discreditingof misinformation.

This is followed by McKay (this issue),who contemplates why mental health pro-fessionals may be particularly susceptible

to pseudoscientific psychotherapy andoffers some recommendations for remedi-ation. Lilienfeld, Lynn, and Bowden (thisissue) then note that evidence-based prac-tice (EBP) has not been particularly suc-cessful in impeding the spread of pseudo-science in psychotherapy. Consequently,they introduce and argue for science-basedpractice as an alternative to EBP.

Then, Napolitano (this issue), trained inboth clinical psychology and law,approaches the problem from a legal per-spective. She makes the case that profes-sional associations and government agen-cies have been ineffective in protectingconsumers and the mental health profes-sions from the negative impact ofpseudotherapies. Consequently, sheemphasizes the value of exploring legaloptions and suggests a specific legal strat-egy.

Finally, Pignotti (this issue) provides uswith an account of her efforts in exposingharmful practices and the high personaland professional costs of her having doneso. There’s much to be learned, as well asadmired, from a reading of this history. Sheconcludes by providing her reflections ofwhat might be learned from her experi-ence.

The problem of pseudoscience inmental health treatment is significant.Please do not read these articles and thenfall into inaction. Allow these articles tostimulate action: Share them widely, exe-cute the actionable items they suggest,and/or initiate a new line of empirical workbased on their content. Numerous suffer-ing human beings are counting on you.

ReferenceFishbein, M., & Ajzen, I. (2010). Predicting

and changing behavior: The reasonedaction approach. New York, NY: Psy-chology Press (Taylor & Francis).

. . .

The author has no funding or conflicts ofinterest to disclose.Correspondence to R.Trent Codd, III,Ed.S., LPC, BCBA, Cognitive-BehavioralTherapy Center of WNC, P.A., 1085 TunnelRoad, 7A, Asheville, NC 28805;[email protected]

4 the Behavior Therapist

PEOPLE HOLD MANY different kinds ofbeliefs. Some are rooted in direct experi-ences, such as that at the end of the day thesun will set, and in the morning the sun willrise in the sky. Others are derived fromexplanations and evidence communicatedby outside sources, such as learning inschool that the world is round. The hope isthat our direct experiences with the world,and the knowledge provided by others, willconverge and be accurate, such that we canuse what we have learned to make deci-sions and solve problems successfully inthe future. The problem, unfortunately, isthat our direct experiences can encouragebeliefs that are incorrect (diSessa, 1993;Vosniadou & Brewer, 1994), and informa-tion provided by others can be wrong (Gar-rett, Weeks, & Neo, 2016; Rapp & Braasch,2014).

For example, consider standing on theedge of a beach, peering out at the water. Inthe distance you can see the horizon. Thisperceptual experience can suggest that theworld just ends; it isn’t curved, but ratherseems to drop off at some distance faraway. What we are seeing does not accu-rately inform us as to the actual shape of theEarth. Also consider that there are groupsthat subscribe to the incorrect idea of a flatEarth, presenting the view with anecdotesand personal tests intended to raise skepti-cism that we do not live on a sphericalplanet (e.g., the Flat Earth Society). Ourdirect experiences, and the informationsupplied by other people, as exemplified inthis case, can inform inaccurate beliefsabout the world.

This case also provides an illustrativeexample of pseudoscience, which we candefine as a set of claims, beliefs, and prac-tices that invoke notions of scientific inves-tigation but that are actually based on mis-understandings and misapplications(sometimes intended and sometimes not)of the scientific method. Pseudoscientificconjectures lack and often run counter toscientific claims derived from accumulatedand generally accepted evidence (Lobato,Mendoza, Sims, & Chin, 2014). Some pseu-doscientific beliefs have their bases in naïvepreconceptions about biology, physics, andchemistry (e.g., Vosniadou & Brewer,1992). Advocates of pseudoscientific beliefs

also often reject wholesale the need for sci-ence by disregarding consideration ofexperimental controls, the importance ofaccumulated evidence, and the theoreticalsupports underlying empirically basedclaims. In efforts to reject scientific consen-sus and to promote their beliefs as validalternatives, these advocates often contendthat nobody can actually know the truth,that evidence and experiments can bebiased (sometimes invoking conspiratorialstances), and that school-supplied under-standings of the world are derived frombook claims rather than from what experi-ence tells us (Lewandowsky, Gignac, &Oberauer, 2013; Lewandowsky, Oberauer,& Gignac, 2013). Contemporary concernsabout the growth of pseudoscience arebecoming increasingly worrisome, linkedto recent sociopolitical events, the ease ofpublishing information through onlinesources, and concerns about journalisticinvestments and integrity (Kahne &Bowyer, 2017; Lewandowsky, Ecker,Seifert, Schwarz, & Cook, 2012).

Pseudoscientific beliefs can have impor-tant consequences for everyday behaviorsand decisions, including our health andwell-being. Consider one particular pseu-doscientific belief—the notion that mentalillnesses are contagious maladies that youcan catch from another person, similar tothe cold or the flu. We highlight this partic-ular belief for three reasons. First, the topicconnects with the theme of this specialissue as considered in other articles in thisvolume. Second, this belief has receivedextended examination in the psychologicalliterature, as accounts attempt to highlightfactors associated with possessing it, as wellas potential outcomes associated with suchthinking (Marsh & Shanks, 2014). Third,this belief is one of a series of incorrectassertions that we have explicitly tested inour own research focused on the conse-quences of exposure to inaccurate informa-tion. Focusing on this belief helps highlightthe broader consequences of learningabout false information as identified inempirical projects (e.g., Marsh, Meade, &Roediger, 2003; Rapp & Braasch, 2014),and is situated with awareness of andrespect for work on mental health treat-ment. In our analysis, we show how expo-

sure to this pseudoscientific claim can haveproblematic consequences. Our discussionthen focuses on processes of memory andlearning that should, under most circum-stances, support successful comprehen-sion, but that can also result in uptake andreliance on inaccurate information. Articu-lating the contributions of these processesfor comprehension helps identify condi-tions and activities that may help reducereliance on inaccuracies (Rapp, 2016). Weconclude by outlining other factors that, inconcert with these processes, contribute tothe pervasive effects of pseudoscience. Ourwork attempts to identify these contribu-tions so as to inform theoretical accounts ofpseudoscientific thinking, and to supportthe design of interventions intended tocombat the acquisition and persistence ofinaccurate beliefs.

Consequences of Exposure toInaccurate Information

To begin, consider the followingexcerpt from a story in which a conversa-tion between two characters, Dane andBrad, turns to the topic of mental illness:

As quickly as Brad had becomeexcited, he calmed down …"Well," hesaid, "if I'm crazy, it's only becauseyou were crazy first and you keepbreathing on me all the time — Icaught it from you."

Dane laughed and said, "I bet youthink you're being funny."

"Right now, I'm just being brain-dead."

Dane forged ahead: "No, really,there's now evidence that you cancatch some forms of mental illnessfrom your friends and loved ones. . . .I was really amazed when I read thisstuff. . . . They now have shown thatthere are some mental troubles thatare passed through the air."

"Mental troubles?"

"Sure — paranoia, hallucinations,fits. All the good stuff. You neverknow what you'll breath in nowadays.You could catch almost anything justby being breathed on by the wrongperson. It's amazing that more peoplearen't aware that mental illness can behighly contagious."

In a series of experiments (Rapp, Hinze,Kohlhepp, & Ryskin, 2014), participants

The Challenge of Overcoming PseudoscientificIdeasDavid N. Rapp and Amalia M. Donovan, Northwestern University

ISBN

:978

-162

6253

407

|US

$24.

95

newharbingerpub l i ca t ions1-800-748-6273 / newharbinger.com

ISBN

:978

-162

6259

430

|US

$16.

95

Essential New Resourcesfor Your Practice

NewEdition

ISBN

:978

-162

6259

782

|US

$16.

95

ISBN

:978

-162

6259

492

|US

$49.

95

NewEdition


R A P P & D O N O V A N

were asked to read a 19-page story, almost8,000 words in length, that potentially con-tained this excerpt, as well as other conver-sations between characters, none of whichwere integral to the plot. Some of the con-versations contained inaccurate assertionsabout the world, as in this example, whileother conversations offered more validassertions. Two versions of the story wereconstructed, with participants assigned toread one or the other, to counterbalancethe 16 presented assertions for accuracy(i.e., 8 of the assertions in each version werepresented in an inaccurate form, with theremaining 8 presented in an accurateform). After reading one version of thestory, participants completed a distractortask to discourage rehearsal and reflectionon the story contents. Finally, participantswere presented with a series of statementsand asked to indicate whether each state-ment was true or false. This validity judg-ment task included statements that refer-enced ideas offered in the 16 criticalassertions, and was administered as a meanof assessing whether the story contentinfluenced participants’ postreading con-siderations of assertion content. Two ver-sions of the validity judgment task werecreated such that half of the statementswere presented as true and the remaininghalf were presented as false.

With respect to our example, half of theparticipants read the story, including theabove excerpt (as well as 15 other asser-tions), while the other half read a versionthat rejected the notion of mental illness asbeing contagious with similar linguist con-tent (and again, along with 15 other asser-tions). During the judgment task, for halfof the participants one of the test itemsasked them to determine whether the state-ment, “Most forms of mental illness arecontagious” was true or not, while the otherhalf of the participants were asked to judgethe statement, “Most forms of mental ill-ness are not contagious.” Again, this is onlyone of a range of test statements includedfor all participants in the task.

The results indicated that participantswho previously read inaccurate assertionswere more than twice as likely to makeincorrect validity judgments, regardless ofthe kind of test statement they were askedto evaluate (i.e., true or false), as comparedto participants who previously read accu-rate assertions. Specifically, if participantsread the earlier excerpt, they showedgreater difficulty rejecting the claim, “Mostforms of mental illness are contagious,” aswell as greater difficulty accepting the state-ment, “Most forms of mental illness are not

contagious,” as compared to participantswho read a version of the story in which theassertion about mental illness being conta-gious was discussed by the characters asbeing obviously wrong. What is notewor-thy about this finding is that the assertionsused in these experiments had been previ-ously normed with members of the popula-tion from which participants were sampled(i.e., undergraduate psychology students atNorthwestern University), which indicatedthey should have been familiar with andknown which version of the claim wasaccurate. Yet despite their accurate priorknowledge concerning the potential trans-mission of mental illness, participants’decisions were contaminated by what theyread.

These results have been replicated avariety of times and emerge across the dif-ferent assertions used in the texts (e.g., Seatbelts do/do not save lives; Brushing yourteeth can lead to/prevent gum disease; Aer-obic exercise strengthens/weakens yourheart and lungs; e.g., Gerrig & Prentice,1991; Prentice, Gerrig, & Bailis, 1997).Besides assertions, similar problematiceffects emerge when participants are pre-sented with inaccurate declarative state-ments (e.g., The Pilgrims sailed to Americaon the Mayflower/Godspeed; The scientistwho discovered radium was Curie/Pasteur;Abraham Lincoln was assassinated byBooth/Oswald), which can subsequently beused to answer related questions (e.g.,Hinze, Slaten, Horton, Jenkins, & Rapp,2014; Marsh 2004). The accumulatedresults indicate being exposed to inaccurateinformation negatively impacts people’sattempts to make decisions and answerqueries involving that same information,even when they should know better.

Mechanisms That Influence Relianceon Inaccurate Information

Recent work has articulated underlyingcognitive processes associated withmemory, language, and comprehensionthat contribute to people’s reliance onpatently inaccurate information (Marsh,Cantor, & Brashier, 2016; Rapp & Braasch,2014; Rapp & Donovan, in press; Rapp,2016). To be clear, these processes supportthe development of accurate understand-ings, as they facilitate the encoding andretrieval of correct information peoplehave experienced. The challenge is thatthese processes operate generally, withproblematic consequences when people areexposed to inaccurate information. Toexemplify this issue, we discuss here two

features of routine cognition that con-tribute to these effects.

FluencyOur judgments about what we know,

and the degree to which what we know isaccurate or requires additional contempla-tion and consideration, is influenced by ahost of factors. One factor that has receivedsubstantial empirical investigation is theease with which people feel they can accessinformation from memory. This is definedas fluency, and our feelings as to how flu-ently we can retrieve our existing under-standings and recall what we have experi-enced also informs expectations as to howvalid we consider that information (Op-penheimer, 2008). Feelings of fluency areoften useful as information that we aremore familiar with and have thought moreabout is often information we shouldindeed feel confident in accepting andreporting. Information that is easilyretrieved from memory is often consideredto be more true than is information forwhich we have to exert effort and delibera-tively search memory to consult (Fazio,Brashier, Payne, & Marsh, 2015). Accurateinformation should be more easily avail-able than inaccurate, inappropriate infor-mation; it should be the information wecan quickly deliver and apply when weneed it.

The challenge is that a variety of cuescan confer feelings of fluency that inappro-priately invoke such confidence (Reber &Schwarz, 1999; Unkelbach, 2007). As such,information that is retrievable can be mis-takenly believed to also be true, or moremodestly, is less likely to be submitted tocareful evaluation and rejection. Consider,for example, having recently read a textpromoting the claim that mental illness iscontagious. Memory traces for thatrecently encoded information are nowmore available for retrieval than other, lessrecently experienced ideas and events.People can misattribute the phenomeno-logical feeling that the information is easilyavailable as an indicator that the informa-tion is valid. This misattribution process isthus a potential contributor to people’s useof inaccurate information, as well as a rou-tine consequence of the normal operationof memory.

We might expect that fluency effectsbased on the recency with which we haveexperienced information would fade, asencoded information, when unrehearsed,becomes more difficult to retrieve aftergoing unconsidered for some time. Whilethis is a reasonable inference, it would

January • 2018 7

Hogrefe Publishing30 Amberwood ParkwayAshland, OH 44805, USATel. 800 228 3749 (toll-free in North America)Fax 419 281 [email protected]

Earn 5 CE credits for readingbooks in the Advances inPsychotherapy book seriesPsychologists and other healthcare providers may earnfive continuing education credits for reading the booksin the Advances in Psychotherapy series and taking amultiple choice exam. This continuing education programis a partnership of Hogrefe Publishing and the NationalRegister of Health Service Psychologists.

The National Register of Health Service Psychologistsis approved by the American Psychological Associationto sponsor continuing education for psychologists. TheNational Register maintains responsibility for this pro-gram and its content.

Readers who are not members of National Register canpurchase each exam for US $25.00 or access to the entireseries of exams for US $200.00. National Register memberscan take the exams free of charge.

Exams are available for 19 topics / books, with new titlesbeing continually added.

Learn more at https://us.hogrefe.com/cenatreg

2017, viii + 80 pp.US $29.80ISBN 978-0-88937-414-0

2nd edition 2017, viii + 94 pp.US $29.80ISBN 978-0-88937-504-8

2nd edition 2017, viii + 124 pp.US $29.80ISBN 978-0-88937-410-2

Latest releases


necessitate individuals never being exposedto the inaccurate information again so as toallow those earlier acquired memory tracesto decay or at least be less accessible givenless attention. This may never actuallyhappen in the real world, though, despitebeing a condition that could be usefully setup and studied in a lab setting. In the realworld, people are often exposed and reex-posed to inaccurate information. Forexample, imagine reading a post on socialmedia presenting a pseudoscientific argu-ment for mental illness being contagious.Such information is often repeatedlyreposted by others, making it more likelythat it will not just be seen once but severaltimes. Repeated exposure to inaccuraciescan bring with it increased feelings of valid-ity in at least two ways. First, repetitionhelps ensure the information remainsrecently experienced. Second, repeatedexposures can make the information feelmore familiar, with ready familiarity alsoconferring feelings of fluency (McGlone &Tofighbakhsh, 2000). The consequence isthat repeated experience with an inaccu-racy can be misattributed as meaning theinformation is more true, or that it shouldbe relied upon, or that it might be recruitedin future considerations about the sametopic.

To summarize, feelings of fluency canconvey information about the validity ofinformation in ways that are inappropriate.Those feelings can be driven by the recencywith which we have experienced informa-tion, and the degree to which we have beenrepeatedly exposed to that information.Political groups and news agencies oftentake advantage of these feelings, sometimesintentionally and sometimes withoutawareness of the consequences. These cuesare also often explicitly associated withtechniques that advertisers, lawyers, andauthors, among other groups, rely upon toconvince, persuade, and entice their audi-ences (Johar & Roggeveen, 2007; Sundar,Kardes, & Wright, 2015). All of these casescould involve pseudoscientific claims.These cues are often useful for informingfeelings as to whether information shouldbe trusted and whether it might be true, ofcourse. However, in many circumstances,those cues are at best uninformative and atworst misleading.

Source MonitoringAnother issue relevant to people’s expe-

riences with information is that they do notseem to be particularly adept or systematicat tagging information as accurate or inac-curate. In the best of situations, people

should carefully evaluate information,skeptically contemplate what they read,see, or hear, and recruit relevant knowledgeto reject information that is incorrect. Out-side of the issue that people often do nothave the appropriate knowledge to conductsuch evaluations, they also do not routinelyengage in careful appraisals of informationcontent or of the sources providing thatcontent even when they should. And if theydo engage in such activity, the products oftheir evaluations are not guaranteed to leadto careful encodings of credibility or accu-racy (Isberner & Richter, 2014).

For example, when participants are pre-sented with information from a source thatshould not be considered reliable, unlessthey receive instructions, repeatedreminders, and guidance to reflect uponand base decisions on source credibility,their subsequent understandings do notseem to include an acknowledgement of alack of credibility (Sparks & Rapp, 2011).Several studies have shown that readers donot outright reject information from unre-liable sources, unless the credibility of thosesources is explicitly identified and associ-ated with performance concerns orrepeated reminders (Andrews & Rapp,2014). Source monitoring, the process bywhich individuals encode informationabout the person or group providing infor-mation, does not seem to be a routine activ-ity during comprehension (Johnson,Hashtroudi, & Lindsay, 1993). Tagginginformation as credible or unreliablewould be useful for guiding subsequentjudgments that invoke retrieval of thatinformation. But lacking such tags, infor-mation that was encoded as false can still beretrieved for subsequent use.

The seeming negligence to engage insuch tagging can emerge for a variety ofreasons, but one important explanationrelates to the allocation of people’s limitedcognitive resources. In our efforts to com-prehend information, we apply mentalresources to determine meaning, buildinferences, rehearse content, and deriveinterpretations (along with a host of otherprocesses). This leaves fewer resourcesavailable for other processes that are notnecessarily critical to building meaning inthe here-and-now, such as source monitor-ing. As a consequence, informationencoded into memory can be jumbledtogether without an effective indexing ofwhich information is accurate and reliable,and which information is inaccurate andshould be discounted for further use(Schwarz, Sanna, Skurnik, & Yoon, 2007).Engaging in more careful evaluation neces-

sitates overcoming the routine, heuristicprocessing we engage in and that is ofteneffective and efficient for everyday reason-ing (McNeil, personal communication,September 1, 2017).

Added to this issue, information inmemory is, at least initially, reactivatedthrough a process some researchers haveidentified as automatic and unguided(Cook, Halleran, & O’Brien, 1998; O’Brien,1995), meaning without being strategicallyretrieved. When a particular cue provokesretrieval in memory, concepts broadlyassociated with that information becomesactivated, with some of those conceptsrising above threshold to be brought intoconscious awareness. The challenge is thatconcepts broadly associated with a retrievalcue can become activated, including closelyrelated and indirectly related information.Given that activated memories are likelynot effectively tagged, a routine conse-quence of retrieval is that inaccurate infor-mation might become available for use.

In sum, comprehension involvesencoding information into memory forsubsequent retrieval. Because people maynot routinely add tags to those encodingsthat reflect the credibility or validity ofwhat has been experienced, retrieval caninvolve reactivating inappropriate, inaccu-rate concepts. Activated inaccurate infor-mation in memory, including pseudoscien-tific claims, even after they have beendebunked, can thus have effects on subse-quent comprehension and decisionmaking.

Discouraging the Useof Inaccurate Information

Given these processes are routinelyrecruited in the service of comprehension,and when enacted on accurate informationare supportive and necessary for buildingeffective understandings, determiningways of “correcting them” when informa-tion is faulty is both challenging and poten-tially misguided. Much of the informationwe routinely encounter is, after all, worthrelying upon. With this in mind, a varietyof recent experimental findings from ourlab have revealed situations in whichpeople are more effective at rejecting inac-curate information. These findings high-light important features of memory andlanguage processing that delineate theallure and influence of pseudoscientificclaims.

When confronted with informationthat is patently inaccurate, people may nev-ertheless encode the information into


January • 2018 9

O V E R C O M I N G P S E U D O S C I E N T I F I C I D E A S

Celebrat ing Its 46th Anniversary

Steven T. Fishman, Ph.D., ABPP | Barry S. Lubetkin, Ph.D., ABPPDirectors and Founders

Since 1971, our professional staff has treated over 30,000 patients with compassionate, empirically-based CBT.Our specialty programs include: OCD, Social Anxiety Disorder, Panic Disorder, Depression, Phobias, PersonalityDisorders, and ADHD-Linked Disorders, and Child/Adolescent/Parenting Problems.Our externs, interns, post-doctoral fellows and staff are from many of the area’s most prestigious universitiesspecializing in CBT, including: Columbia, Fordham, Hofstra, Rutgers, Stony Brook, St. John’s, and YeshivaUniversities.

Conveniently located in the heart of Manhattan just one block from Rockefeller Center. Fees are affordable,and a range of fees are offered.

New York CityINSTITUTE for BEHAVIOR THERAPY

For referrals and/or information, please call: (212) 692-928820 East 49th St., Second Floor, New York, NY 10017e-mail: [email protected] | web: www.ifbt.com

memory, despite being aware it is wrong.And once that information is encoded, itcan potentially be reactivated later to influ-ence comprehension. To combat this, wehave instructed participants to carefullyedit what they are reading as a text unfolds(Rapp, Hinze, Kohlhepp, et al., 2014). Forexample, when a participant encounterspseudoscience that sounds dubious, theymight note skepticism about that informa-tion, or annotate correct ideas that are notbeing reported. These kinds of edits arelikely effective because they encourage anencoding of the accurate information thatis already known, rehearsing that knowl-edge as participants retrieve it and write itdown as they edit. This helps ensure theaccurate information will be available later,and discourages encoding the inaccurateinformation into memory.

Sometimes text content itself canreduce reliance and enhance evaluation, ashas been shown when participantsencounter false information that is implau-sible (Rapp, Hinze, Slaten & Horton, 2014).For example, if people read an accountcontending that mental illness is conta-gious associated with a particularly out-landish set of claims (e.g., involving patho-

genic spirits and demons), the likelihoodthey might consider that idea later is greatlyreduced, in contrast to when the account ismore plausibly motivated (e.g., otherpeople’s behaviors might inform how weshould behave). Some individuals mightstill endorse even implausible ideas inefforts to support their existing world-views, but implausible information oftencalls attention to explicit inconsistencies,discrepancies, and logical leaps that markinformation as inappropriate. These quali-ties can encourage careful evaluation andtagging of that information as wrong.

In both of the above cases, individualsmust be given the motivation to carefullyconsider the validity and plausibility ofwhat they read, as well as the appropriatetools for engaging in evaluation. These areskills that people differentially possess, andthat they opt to apply in different contextsto varying degrees (Gottlieb & Wineburg,2012). The upshot is that explicit trainingon evaluation and media literacy may bebeneficial in helping people overcome theallure of inaccurate information. Thistraining could, for example, and as relevantto pseudoscience, involve exposure to thescientific methodologies and practices that

are involved in developing and testingideas. Increased awareness and familiaritywith applying a scientifically based per-spective should help readers call into ques-tion the kinds of unsubstantiated claimsand false information commonly presentedin pseudoscientific discourse.

Concluding ThoughtsThe cognitive factors discussed above

that support attention to and reliance oninaccuracies are one set of contributors topeople’s use of pseudoscientific claims suchas a flat earth and mental illness as beingcontagious. But these are far from the onlycontributors to such problematic acquisi-tion and reliance. There are a host of otherconsiderations that, in concert with theroutine operations of human memory andlanguage, can lead to surprising and prob-lematic endorsements. People’s naïve the-ories as to how the world works ofteninvoke simple, intuitively appealing expla-nations that can connect to claims associ-ated with pseudoscience (Vosniadou &Brewer, 1992). A lack of familiarity withscientific investigations, including the toolsand practices of scientists, can lead to dis-



trust and confusion that may make otherseeming explanations appealing and viable(Miller, 2004). The filter bubbles thatpeople routinely place themselves in,exposing themselves to and accessinginformation that aligns with their beliefswithout considering alternative perspec-tives and contradictory evidence, can helpdrive pseudoscientific thinking (Lewan-dowsky, Ecker, & Cook, in press;Lewandowsky & Oberauer, 2016). Theunmoderated content available throughsocial media, blog postings, and Internetarticles can make false ideas and claimsavailable to audiences that may not havethe time, energy, or inclination to evaluatethat content carefully (Del Vicario et al.,2015; Kumar & Geethakumari, 2014).Contemporary concerns about “fake news”make this last notion even more worri-some, as individuals who promote pseudo-scientific claims often like to call into ques-tion whether we can truly know anything,to support arguments that their view, lack-ing evidence, is just as reasonable as anyother (Lazer et al., 2017). Even the routineuse of terms like “theory” and “hypothe-ses,” detached from their more rigorousimplementations to instead be synony-mous with the terms “opinion” and “view-point,” have consequences for how peoplemight opt to think about the claims thatunderlie pseudoscientific conjectures.

Understanding and combatting theinfluence of “fake news,” inaccurate infor-mation, and pseudoscience requires a con-certed, interdisciplinary effort. This willrequire leveraging theoretical understand-ings of cognition and behavior, as derivedfrom the social and medical sciences, withapplied understandings derived from prac-tices including journalism and educationaldesign, as well as from domains studyingtopics such as persuasion, media literacy,and critical evaluation (to name a few rele-vant fields and topic areas). The goal is toencourage more careful evaluation on thepart of readers, which hopefully will bene-ficially lead to a reduction in the promotionof and reliance on pseudoscientific dis-course.

ReferencesAndrews, J., & Rapp, D. N. (2014). Partner

characteristics and social contagion:Does group composition matter? AppliedCognitive Psychology, 28, 505-517.

Cook, A. E., Halleran, J. G. & O’Brien, E. J.(1998). What is readily available duringreading? A memory-based view of textprocessing. Discourse Processes, 26, 109-129.

Del Vicario, M., Bessi, A., Zollo, F.,Petroni, F., Scala, A., Caldarelli, G., Stan-ley, H. E., & Quattrociocchi, W. (2015).The spreading of misinformation online.Proceedings of the National Academy ofSciences, 113, 554-559.

diSessa, A. A. (1993). Toward an episte-mology of physics. Cognition andInstruction, 10, 105-225.

Fazio, L. K., Brashier, N. M., Payne, B. K.,& Marsh, E. J. (2015). Knowledge doesnot protect against illusory truth. Journalof Experimental Psychology: General, 144,993-1002.

Garrett, R. K., Weeks, B. E., & Neo, R. L.(2016). Driving a wedge between evi-dence and beliefs: How online ideologicalnews exposure promotes political mis-perceptions. Journal of Computer-Medi-ated Communication, 21, 331–348.

Gerrig, R. J., & Prentice, D. A. (1991). Therepresentation of fictional information.Psychological Science, 2, 336–340.

Gottlieb, E., & Wineburg, S. (2012).Between “veritas” and “communitas”:Epistemic switching in the reading ofacademic sacred history. Journal of theLearning Sciences, 21, 2012.

Hinze, S. R., Slaten, D. G., Horton, W. S.,Jenkins, R., & Rapp, D. N. (2014). Pil-grims sailing the Titanic: Plausibilityeffects on memory for misinformation.Memory & Cognition, 42, 1-20.

Isberner, M., & Richter, T. (2014). Com-prehension and validation: Separablestages of information processing? A casefor epistemic monitoring in languagecomprehension. In D. N. Rapp & J. L. G.Braasch (Eds.), Processing inaccurateinformation: Theoretical and applied per-spectives from cognitive science and theeducational sciences (pp. 353–379). Cam-bridge, MA: MIT Press.

Johar, G. V., & Roggeveen, A. L. (2007).Changing false beliefs from repeatedadvertising: The role of claim-refutationalignment. Journal of Consumer Psychol-ogy, 17, 118 –127.

Johnson, M.K., Hashtroudi, S., & Lindsay,S.D. (1993). Source monitoring. Psycho-logical Bulletin, 114, 3-28.

Kahne, J., & Bowyer, B. (2017). Educatingfor democracy in a partisan age: Con-fronting the challenges of motivated rea-soning and misinformation. AmericanEducational Research Journal, 54, 3-34.

Kumar, K. P. K., & Geethakumari, G.(2014). Detecting misinformation inonline social networks using cognitivepsychology. Human-centric Computingand Information Sciences, 4, 1-22.

Lazer, D., Baum, M., Grinberg, N., Fried-land, L., Joseph, K., Hobbs, W., & Matts-son, C. (2017). Final report from Combat-ing Fake News: An Agenda for Research

and Action. Cambridge, MA: HarvardKennedy School.

Lewandowksy, S., Ecker, U. K. H., & Cook,J. (in press). Beyond misinformation;Understanding and coping with the post-truth era. Journal of Applied Research inMemory and Cognition.

Lewandowsky, S., Ecker, U. K. H., Seifert,C., Schwarz, N., & Cook, J. (2012). Misin-formation and its correction: Continuedinfluence and successful debiasing. Psy-chological Science in the Public Interest,13, 106–131.

Lewandowsky, S., Gignac, G. E., & Ober-auer, K. (2013). The role of conspiracistideation and worldviews in predictingrejection of science. PLoS One, 8, e75637.

Lewandowsky, S., & Oberauer, K. (2016).Motivated rejection of science. CurrentDirections in Psychological Science, 25,217–222.

Lewandowsky, S., Oberauer, K., & Gignac,G. (2013). NASA faked the moon landing— therefore (climate) science is a hoax:An anatomy of the motivated rejection ofscience. Psychological Science, 24, 622–633.

Lobato, E., Mendoza, J., Sims, V., & Chin,M. (2014). Examining the relationshipbetween conspiracy theories, paranormalbeliefs, and pseudoscience acceptanceamong a university population. AppliedCognitive Psychology, 28, 617-625.

Marsh, E. J. (2004). Story stimuli for creat-ing false beliefs about the world. BehaviorResearch Methods, Instruments, & Com-puters, 36, 650–655.

Marsh, E. J., Cantor, A. D., & Brashier, N.M. (2016). Believing that humans swal-low spiders in their sleep: False beliefs asside effects of the processes that supportaccurate knowledge. Psychology of Learn-ing and Motivation: Advances in Researchand Theory, 64, 93-132.

Marsh, E. J., Meade, M. L., & Roediger, H.L. (2003). Learning facts from fiction.Journal of Memory and Language, 49,519–536.

Marsh, J. K., & Shanks, L. L. (2014).Thinking you can catch mental illness:How beliefs about membership attain-ment and category structure influenceinteractions with mental health categorymembers. Memory & Cognition, 42,1011-1025.

McGlone, M. S., & Tofighbakhsh, J.(2000). Birds of a feather flock conjointly(?): Rhyme as reason in aphorisms. Psy-chological Science, 11, 424–428.

Miller, J. D. (2004). Public understandingof, and attitudes toward, scientificresearch: What we know and what weneed to know. Public Understanding ofScience, 13, 273-294.

January • 2018 11

O’Brien, E. J. (1995). Automatic compo-nent of discourse comprehension. In R.F. Lorch & E. J. Obrien (Eds.), Sources ofcoherence in reading (pp. 159-176). Hills-dale, NJ: Erlbaum.

Oppenheimer, D. M. (2008). The secretlife of fluency. Trends in Cognitive Sci-ence, 12, 237-241.

Prentice, D. A., Gerrig, R. J., & Bailis, D. S.(1997). What readers bring to the pro-cessing of fictional texts. PsychonomicBulletin & Review, 4, 416-420.

Rapp, D. N. (2016). The consequences ofreading inaccurate information. CurrentDirections in Psychological Science, 25,281-285.

Rapp, D. N., & Braasch, J. L. G. (Eds.).(2014). Processing inaccurate informa-tion: Theoretical and applied perspectivesfrom cognitive science and the educationalsciences. Cambridge, MA: MIT Press.

Rapp, D. N., & Donovan, A. M. (in press).Routine processes of cognition result inroutine influences of inaccurate content.Journal of Applied Research in Memoryand Cognition.

Rapp, D. N., Hinze, S. R., Kohlhepp, K., &Ryskin, R. A. (2014). Reducing reliance

on inaccurate information. Memory &Cognition, 42, 11-26.

Rapp, D. N., Hinze, S. R., Slaten, D. G., &Horton, W. S. (2014). Amazing stories:Acquiring and avoiding inaccurate infor-mation from fiction. Discourse Processes,51, 50-74.

Reber, R., & Schwarz, N. (1999). Effects ofperceptual fluency on judgments oftruth. Consciousness and Cognition, 8,338 –342.

Schwarz, N., Sanna, L. J., Skurnik, I., &Yoon, C. (2007). Metacognitive experi-ences and the intricacies of setting peoplestraight: Implications for debiasing andpublic information campaigns. Advancesin Experimental Social Psychology, 39,2007.

Sparks, J. R., & Rapp, D. N. (2011). Read-ers' reliance on source credibility in theservice of comprehension. Journal ofExperimental Psychology: Learning,Memory, & Cognition, 37, 230-247.

Sundar, A., Kardes, F. R., & Wright, S. A.(2015). The influence of repetitive healthmessages and sensitivity to fluency on thetruth effect in advertising. Journal ofAdvertising, 44, 375-387.

Unkelbach, C. (2007). Reversing the trutheffect: Learning the interpretation of pro-cessing fluency in judgments of truth.Journal of Experimental Psychology:Learning, Memory, and Cognition, 33,219 –230.

Vosniadou, S., & Brewer, W. F. (1992).Mental models of the earth: A study ofconceptual change in childhood. Cogni-tive Psychology, 24, 535-585.

Vosniadou, S., & Brewer, W. F. (1994).Mental models of the day/night cycle.Cognitive Science, 18, 123-183.

. . .

The authors have no funding or conflicts ofinterest to disclose.

Correspondence to David N. Rapp, Ph.D.,2120 Campus Drive, Northwestern Univer-sity, Evanston, IL 60208;[email protected]

O V E R C O M I N G P S E U D O S C I E N T I F I C I D E A S


MUCH LITERATURE ATTESTS to the exis-tence of a large gap between the science ofclinical psychology and how it is practiced(Garb & Boyle, 2015; Gaudiano, Dalrym-ple, Weinstock, & Lohr, 2015; Katz, 2001;Lilienfeld, Lynn, & Lohr, 2015; McFall,1991; Nunez, Poole, & Memon, 2003;Poole, Lindsay, Memon, & Bull, 1995;Polusny & Follette, 1996; Tavris, 2015). Togain an idea of some researchers’ percep-tions of the gap, consider that Tavrislikened it to that which separates theIsraelis and the Arabs. Assuming the desir-ability of bridging the gap, we mightinquire as to the reasons for its existence togain clues about what to do about it. Onepossibility is that practitioners believe thescience of clinical psychology is so badlyflawed or irrelevant that there is no point inbasing their clinical practices on it. Alter-natively, practitioners might believe thatthe science is neither badly flawed nor irrel-evant, but that they are not capable oflearning it or applying it to their practices.Of course, there are many other possibili-ties too.

In the present article, I use the literaturecited above as providing two startingpoints. First, there is a large science-practi-tioner gap. Second, it is important to bridgethe gap. These starting points suggest atleast two possibilities: the gap can bebridged by inducing clinical scientists tomove in the direction of practitioners or byinducing practitioners to move in thedirection of clinical scientists. From thepoint of view of evidence-based practice, itis more desirable for practice to move inthe direction of clinical science than forclinical science to move in the direction ofpractice. There doubtless are institutionalchanges that could aid in moving practi-tioners in the direction of clinical science,but these will not be discussed here.Instead, consistent with a social psychologyfocus, I take the goal as that of inducingpractitioners to change their behaviors tobe more in line with clinical science. Animportant step in changing such behaviorsis to diagnose the reasons why practition-ers perform them or fail to perform desir-able behaviors. The most widely researchedsocial psychology program for understand-

ing and predicting behavior is the reasonedaction approach (see Fishbein & Ajzen,2010, for a comprehensive review). As willbe explained at some length, there are dif-ferent routes to behavior, and these differ-ent routes might imply quite differentinterventions to induce practitioners tochange their behaviors.

There are three main sections. The firstsection includes a brief discussion of twoflaws in the actual science. I wish to makeclear that although I support practitionersusing the science of clinical psychology(indeed, this is the point of the present arti-cle), there can be little doubt that there ismuch wrong with the science itself thatresearchers should fix. The argument thatpractitioners should attend to the scienceof clinical psychology would be augmentedby improvements in that science. Thesecond section explains the reasonedaction approach and what it implies aboutpossible reasons for the science-practi-tioner gap in clinical psychology. The thirdsection discusses implications for how todesign research to investigate behaviorsrelevant to reducing the science-practi-tioner gap.

The Science of Clinical PsychologyBecause the science of clinical psychol-

ogy is a subset of the larger field of psychol-ogy, it is plagued with some of the prob-lems that plague psychology moregenerally. For example, clinical psychologydepends on the null hypothesis significancetesting procedure (NHSTP). But the proce-dure has come under much fire for beinglogically invalid (e.g., Bakan, 1966; Carver,1978, 1993; Cohen, 1994; Grice, Cohn,Ramsey, & Chaney, 2015; Kass & Raftery,1995; Kline, 2015; Meehl, 1967, 1978, 1990,1997; Rozeboom, 1969, 1997; Schmidt,1996; Schmidt & Hunter, 1997; Trafimow,2003, 2006 Trafimow & Marks, 2015, 2016;Valentine, Aloe, & Lau 2015). In short, thefact that a finding is unlikely given the nullhypothesis fails to justify an inverse infer-

ence that the null hypothesis is unlikelygiven the finding. The editors of Basic andApplied Social Psychology (Trafimow &Marks, 2015) banned the NHSTP for thesimple reason that low p-values (less than.05) fail to provide a logically defensiblejustification for rejecting null hypothesesand accepting alternative ones. The Amer-ican Statistical Association (Wasserstein &Lazar, 2016), though stopping short of sup-porting the ban, admitted that the NHSTPfails to provide a sufficient reason forrejecting null hypotheses or drawing anyconclusions whatsoever other than the tau-tological one that if p is a low value, theprobability of the finding is low given thenull hypothesis. As the primary “evidence”for the efficacy of treatments comes fromstatistically significant p-values, a skeptichas clear grounds for his or her skepticism.Researchers should consider alternatives(e.g., Trafimow, 2017; Trafimow & Mac-Donald, 2017).

Another problem concerns the failureof researchers to distinguish cleanlybetween theoretical assumptions andassumptions that are auxiliary to the maintheory (hereafter, auxiliary assumptions),but are nevertheless necessary to derivetreatments. The importance of auxiliaryassumptions comes from a more generalconcern in science than clinical psychol-ogy, or even psychology more generally.Philosophers of science long have recog-nized that theories contain nonobserva-tional terms (e.g., Duhem, 1954; Lakatos,1978). In clinical psychology, “anxiety”might be considered a nonobservationalterm. But every science uses nonobserva-tional terms. In fact, the Nobel LaureateLeon Lederman (1993) pointed out thatNewton used mass as a nonobservationalterm that even lacks an independent defin-ition!1 Despite this lack, Newton’s equa-tion, force = mass x acceleration, is possiblythe most important equation in the historyof physics. Because theories containnonobservational terms, there is no way toderive empirical predictions from themexcept by using auxiliary assumptions thatlink the nonobservational terms in theoriesto the observational terms in empiricalhypotheses. Haley used Newton’s theory topredict the reappearance of the comet thatnow bears his name, in conjunction withauxiliary assumptions about the presentposition of the comet, gravitational influ-

The Scientist-Practitioner Gap in ClinicalPsychology: A Social Psychology PerspectiveDavid Trafimow, New Mexico State University

1The nonobservational term “mass” should not be confused with the observational term“weight.” That these are different can be seen easily merely by considering that an object of thesame mass would weigh different amounts on different planets.

January • 2018 13

S C I E N T I S T - P R A C T I T I O N E R G A P

Why? The practice of psychology has become increasingly complex and competitive. Certification by ABPP demonstratesthat psychologists have met the standards and possess the competencies required in their specialty.

Board Certification Is Valuable and is Increasingly Becoming an Expectation in our Profession:• ABPP listing enhances practitioner credibility for clients and patients• Distinguishes you from other psychologists in the job market• Potential salary benefits by the VA, hospitals, and other health care facilities• May replace the APA accredited internship requirement by the VA• Enhances qualifications as an expert witness• Facilitates inter-jurisdictional licensing and practice mobility• Streamlined credentialing process for licensing boards, insurance companies, and medical

Three Steps: (1) Review of education and training; (2) Submission of a practice sample; and(3) Collegial, in-vivo examination. Please note that a consulting service to support the application process is anavailable option and is free of charge for all candidates.

Specific requirements and online application: http://www.abpp.org/i4a/pages/index.cfm?pageID=3299

Early Entry Application: Start Early! Application fee is discounted from $125 to $25 for graduate students,interns, and residents: http://www.abpp.org/i4a/pages/index.cfm?pageID=3299

Senior Option: With 15 years of postdoctoral experience in cognitive and behavioral psychology there is flexibilityin the requirement for a practice sample: http://www.abpp.org/i4a/pages/index.cfm?pageID=3299

When are Exams Conducted? Exams are conducted in different places, but are usually done at the APA (Washington,DC, 1st week in August 2017) and ABCT (San Diego, 3rd week in November 2017) annual conferences. This year examscan also be conducted at the ABPP conference in San Diego in early May 2017. Other exams can be arranged in otherlocations on a case-by-case basis.Note: At ABCT in November, we will have a Q & A session for those who want more information about board certification

Become an ABPP Board CertifiedSpecialist in Behavioral and CognitivePsychology

ences, and so on. In clinical psychology,there is no way to derive treatments fromtheories, except in conjunction with auxil-iary assumptions. A theory may lead to anexcellent treatment when used in conjunc-tion with one set of auxiliary assumptionsand a failing treatment when used in con-junction with another set of auxiliaryassumptions. Thus, the success or failure ofa treatment need not provide a strong casefor the worth or lack of worth of the theory.Unfortunately, researchers in clinical psy-chology have not been careful aboutspelling out the auxiliary assumptions thatlead from theory to treatment. This is amajor strike against the science of clinicalpsychology and researchers should remedyit if they wish practitioners to take the sci-ence more seriously.

A third issue concerns level of measure-ment. Suppose that a proper experiment isconducted that shows that a particulartreatment group does better than the con-trol group with respect to, say, a depression

index. And let us even suppose that theeffect size is reasonably large, there was asufficient sample size, and so on. Can weconclude that practitioners should use thetouted treatment? It depends, in part, onwhether one believes that the depressionindex is at least at the interval level of mea-surement (Stevens, 1946). Without anassumption of at least an interval level ofmeasurement (a ratio level would be evenbetter), the effect size calculation is mean-ingless. In fact, several researchers havequestioned whether typical indexes in psy-chology really are at the interval level orratio level of measurement to justify theusual calculations upon which researchersbase their conclusions (e.g., Barrett, 2003;Michell, 1997, 2000, 2008a, 2008b; Morris,Grice, & Cox, 2017). Unfortunately,although the mathematical basis formaking this determination was worked outin the 1970s (Kranz, Luce, Suppes, & Tver-sky, 1971; also see Luce, Krantz, Suppes, &Tversky, 1990; Roberts, 1979; Suppes,

Kranz, Luce, & Tversky, 1989), researchersin clinical science have not taken the trou-ble to test whether their indexes, such asthat which measures depression, actuallyare at the interval level or ratio level of mea-surement. Ironically, just as clinical scien-tists accuse practitioners of failing to attendto the relevant literature in clinical science,it is possible to accuse clinical scientists offailing to attend to the basic mathematicsunderlying the assumed quantitativenature (or lack thereof) of their indexes.

Despite the foregoing criticisms of thescience of clinical psychology, there is nointent to declare the science to be worth-less. There have been gains, too, and thefact of shortcomings provides a poor justi-fication for practitioners being unaware ofthe science of clinical psychology.Although the present section can be con-sidered a slight indictment against how thescience of clinical psychology has beenconducted, the remainder of this articleassumes that practitioners nonetheless

T R A F I M O W


should attend to it and be influenced in theconduct of their clinical practices.

The Reasoned Action ApproachThe major goal of the reasoned action

approach (Ajzen, 1988, 1991; Ajzen & Fish-bein, 1980; Ajzen & Fishbein, 2005; Fish-bein, 1963; Fishbein, 1967, 1980; Fishbein& Ajzen, 1975; Fishbein & Ajzen, 2010) isto understand and predict behavior. Con-sequently, the easiest way to comprehendthe theory is to work backwards frombehavior to its determinants. The immedi-ate determinant of behavior is behavioralintention; people do what they intend to doand not what they intend not to do. Thereare complications to be discussed later, butthese can be ignored for now.

In turn, behavioral intentions are deter-mined by attitudes and subjective norms.Attitudes are people’s evaluations of thebehavior (how much they like or dislike toperform it) and subjective norms arepeople’s opinions about what most otherswho are important to them think theyshould do or not do. Because a personcannot know for sure what others think,this is the subjective part of subjectivenorms. Any particular behavioral intentionmight be influenced more by attitudes ormore by subjective norms: that is, a behav-ior might be more under attitudinal controlor more under normative control. In addi-tion, Trafimow and Finlay (1996) showedthat people also can be more under attitu-dinal or more under normative control,across a wide range of behaviors.

Suppose that a behavior is more underattitudinal than normative control. Tointervene, it is desirable to know the deter-minants of attitudes, which are behavioralbeliefs and evaluations of those beliefs.Behavioral beliefs are judgments about thelikelihood of the consequences that mightarise from performing a behavior whereasevaluations are judgments about how goodor bad each of the consequences would beif they were to happen. In the reasonedaction tradition, attitudes are a function ofeach behavioral belief-evaluation product,summed across all productsAnalogously, subjective norms are deter-mined by normative beliefs and motiva-tions to comply with normative referents.A normative belief is a judgment about thelikelihood with which a specific normativereferent believes one should or should notperform the behavior, and these are pairedwith how much one is motivated to complywith what that person thinks. Thus, subjec-tive norms are a function of normative

belief-motivation to comply products,summed across all products:(SN=∑_(i=1)^kÿ"n_i m_i !).

There is also a measurement model thataccompanies the substantive theory. Thebasic principle, sometimes called the “prin-ciple of correspondence” or the “principleof compatibility,” is that all behaviors havefour elements and these elements must cor-respond across all reasoned action con-structs. That is, each behavior has a target,action, time, and context. For example, thebehavior of “eating a chocolate bar at 3:00on Friday in my office” has the followingelements: target (chocolate bar), action(eating), time (3:00 on Friday), and context(in my office). To perform well at predict-ing behaviors; measures of behavioralintentions, attitudes, subjective norms,behavioral beliefs, evaluations, normativebeliefs, and motivations to comply; allshould mention the same four elements oftarget, action, time, and context. Researchperformed in the 1970s (e.g., Davidson &Jaccard, 1975, 1979), specifically on themeasurement principle, supports thatexcellent prediction is obtained when it iscomplied with fully, but that a mismatchon even one of the four elements is prob-lematic.

Thus far, we have the received viewfrom the 1970s (e.g., Ajzen & Fishbein,1980; Fishbein, 1980; Fishbein & Ajzen,1975), but Ajzen (1988) added the notionof perceived behavioral control. The origi-nal theory only was meant to apply tobehaviors that people are capable of per-forming, but Ajzen wanted to extend thetheory to behaviors that people might notbe capable of performing. Although thereis no way to measure actual control over abehavior, it is possible to measure people’sperceptions of their degree of control;hence, the notion of perceived behavioralcontrol came into being. Usually the con-cept is measured by having participantsrespond to items referring to how muchcontrol they have over the behavior andhow easy or difficult the behavior would befor them to perform. But Trafimow et al.(2002) argued that “control” and “diffi-culty” are different concepts that should bekept distinct for the sake of precision. Toback up this claim, Trafimow et al. showedthat it is possible to perform manipulationsthat influence perceptions of control with-out influencing perceptions of difficulty,and to perform manipulations that influ-ence perceptions of difficulty withoutinfluencing perceptions of control. Thus,rather than use what has been demon-

strated to be an imprecise concept of per-ceived behavioral control, it is possible tosubstitute the more precise concepts of per-ceived control and perceived difficulty.And to go with perceived control and per-ceived difficulty, there also are beliefs aboutthe factors that render a behavior underone’s control or not (control beliefs) andabout the factors that render a behavioreasy or difficulty to perform (difficultybeliefs).

Finally, Fishbein (1980) argued stronglythat attitude only consists of a cognitiveevaluation, and also criticized the factoranalytic approaches that indicated an affec-tive component too. While agreeing withFishbein’s criticisms of the factor analyticwork up to that time, Trafimow andSheeran (1998) performed a set of experi-ments that demonstrated that affect andcognition nevertheless need to be sepa-rated. They also showed that “affective”beliefs can be distinguished from “cogni-tive” beliefs. Thus, the reasoned action tra-dition is much richer in the 21st centurythan it was in the 1970s. On the negativeside, this increased richness comes at aprice in parsimony.

Defining the BehaviorThe long description of the theory was

necessary so that the reader could appreci-ate some important ambiguities. One ofthese concerns the behavior of interest. Toreiterate, the presenting problem is thatclinical practitioners fail to consider thescientific evidence that is relevant to theirpractices. But it is not clear what we meanby this. Do we mean that practitionersshould read the scientific literature? If so,how often should they read it, when shouldthey read it, and in what context shouldthey read it?

Or do we mean that clinical practition-ers should apply the scientific literature totheir own practices? If so, when shouldthey do it, to what extent should they do it,and in what context should they do it?

What we might mean is that we wish forpractitioners to perform a set of behaviorsthat will result in evidence-based practice.This is fine, but we need to specify the set ofbehaviors we wish to change in a precisemanner.

Multiple Pathways to BehaviorAnother ambiguity pertains to how to

get to behavior. In the original version ofthe theory, there was an attitudinal andnormative pathway. To that, researchershave added perceived control and per-ceived difficulty. We might even consider


January • 2018 15

affect to be a fifth pathway, though somewould argue that it is part of the attitudeconstruct. Before any sort of intervention islikely to work, it is necessary to figure outwhich pathway predominates for mostpractitioners, assuming, of course, that onehas specified a behavior or set of behaviorsof interest. For example, there is no pointin intervening at the normative level if thebehavior is mostly under attitudinal con-trol.

The usual method for determining con-trol is to use multiple regression withbehavior or behavioral intention regressedonto the other variables. In the traditionalversion of the theory, a large attitude andsmall subjective norm beta-weight is takenas indicating that the behavior is moreunder attitudinal control than normativecontrol whereas the reverse pattern of beta-weights is taken as indicating that thebehavior is primarily under normativecontrol. To make use of the more recent lit-erature, it is desirable to measure perceivedcontrol and perceived difficulty too.Although strong beta-weights and correla-tion coefficients do not prove causationfrom a precursor construct to behavior,they support that some precursor con-structs are better candidates than others forintervention.

The Belief LevelWe have seen that the constructs that

are precursors to behavioral intentions andbehaviors have, in turn, their own precur-sors. And these precursors are beliefs ofvarious types, augmented by evaluations ormotivations to comply. But remaining withbeliefs, we have behavioral beliefs, norma-tive beliefs, control beliefs, difficulty beliefs,and affective beliefs. Which of these are rel-evant to the scientist-practitioner gap?Consider some plausible possibilities.

• It could be that practitioners believe thatusing clinical science will not actuallyhave positive consequences (behavioralbelief);

• Practitioners may believe their col-leagues think they should not use clini-cal science (normative belief);

• Practitioners may believe they do nothave the ability to learn the clinical sci-ence (control belief);

• Practitioners may believe that it wouldbe difficult for them to learn the clinicalscience, or might take too much timeand effort (difficulty belief);

• Practitioners may simply have negativeaffective reactions to the clinical sci-ence, possibly because of being

reminded of unpleasant aspects of grad-uate school (affective belief).

To change the behaviors of practition-ers towards reading relevant clinical sci-ence, changing their own practices inaccordance with relevant clinical science,and so on, it is necessary to know which ofthe foregoing beliefs, or other beliefs notmentioned in the bullet list, determine thebehavior or behaviors of interest. Forexample, if the main obstacle for practi-tioners is a belief that their learning the rel-evant clinical science will not result in pos-itive consequences for their patients, thenan intervention designed to educate themto see how relevant clinical science canresult in positive consequences for theirpatients is likely to be effective. However, ifthe problem is at the level of a control ordifficulty belief, such education likely willbe ineffective. And to make the problemmore complex, I stress that the bullet-listedbeliefs compose only a small set of thepotentially relevant ones.

Zeroing in on an InterventionThere are at least three stages to zeroing

in on an intervention. First, there are twopreliminary studies that the researchermust complete. Second, the researchershould use the data to find out whichbeliefs are good candidates for interven-tion, and design the intervention. Third, itis desirable to perform a third study to eval-uate the effectiveness of the intervention.I’ll present more details of the two prelimi-nary studies as clinical psychologists areless likely to know these. And I will say verylittle about evaluating the effectiveness ofthe intervention because readers of thisjournal are likely to know this already.

How to Conduct Two PreliminaryStudies

Although there are many candidates forrelevant beliefs, they fall into four cate-gories.2 These are beliefs about conse-quences that determine attitudes, norma-tive beliefs that determine subjectivenorms, control beliefs that determine per-ceived control, and difficulty beliefs thatdetermine perceived difficulty. I recom-mend that researchers conduct two studiesto zero in on an intervention, but first, as I

mentioned earlier, it is necessary to specifythe behavior or set of behaviors of interest.

Once a behavior of interest is chosen,the researcher can conduct a two-partstudy. In the first part, the researcher canmeasure behavioral intentions (and actualbehaviors, too, if that is feasible), attitudes,subjective norms, perceived control, andperceived difficulty. It is important to keepthe principle of correspondence in mindfor all measures. By determining which ofthe four precursor constructs (attitudes,subjective norms, perceived control, orperceived difficulty) are good predictors ofbehavioral intentions (or better yet, behav-iors), and which precursor constructs arenot, it may be possible to narrow mattersdown substantially. For example, supposethat attitudes do an excellent job of pre-dicting behavioral intentions (or behav-iors) but that subjective norms, perceivedcontrol, and perceived difficulty do not. Inthat case, the researcher would not have todeal with the latter three precursors in thesubsequent study, and also would not haveto deal with normative beliefs (or motiva-tions to comply), control beliefs, or diffi-culty beliefs.

Measurement reliability and validity areextremely important. For well over a cen-tury (Spearman, 1904), it has been knownthat reliability sets an upper limit on valid-ity. If one imagines two variables, X and Y,that have “true scores” according to classi-cal true score or classical test theory (Gul-liksen, 1987; Lord & Novick, 1968; Spear-man, 1904), the following equation showshow the correlation one might expect toobserve (ρXY) is decreased from the truecorrelation (ρTXTY), depending on the relia-bilities of the measures of X(ρXX') and Y(ρYY'):

As an example, suppose that the truecorrelation is .7 and that the reliabilities ofthe two measures are .7 and .7, respectively.In that case, the observed correlation canbe expected to come out at .49 rather thanat the true level of .7.

Fortunately, because all of the reasonedaction variables are very precisely defined,it is possible to capture most of the mean-ing with very precise items. In fact, Trafi-mow and Finlay (1996) showed that—in

2 For the sake of brevity, I am skipping a possible fifth category, pertaining to affective reactionsto learning or using the science of clinical psychology. However, researchers who seriously wantto pursue this issue might wish to consider this as a possibility too, that should be investigated.


T R A F I M O W

violation of a standard rule of scale con-struction—even single item measures dowell if designed with care, both with respectto test-retest reliability and predictivevalidity. Nevertheless, I recommend usingthree to five items to measure each con-struct, remembering, of course, to obey theprinciple of correspondence, withoutwhich there will be a lack of validity. Ajzenand Fishbein (1980, Appendix A) containexample items and demonstrate how tocreate items that obey the principle of cor-respondence.

But it also is necessary to obtain relevantbeliefs, which leads us to the second part ofthe first study, which depends on open-ended questions. Specifically, theresearcher should obtain behavioral beliefsby asking participants to list the advantagesand disadvantages of performing thebehavior. Again, the principle of corre-spondence needs to be obeyed even at thelevel of beliefs. Moving to normativebeliefs, the researcher should ask partici-pants to list the people whose opinions arerelevant to their performing the behavior.The researcher can obtain control beliefsby asking participants to list specific rea-sons why the behavior might be under theircontrol or might not be under their con-trol. Finally, the researcher can obtain diffi-culty beliefs by asking participants to listspecific reasons why the behavior might beeasy or difficulty for them to perform.

The design of the second study dependson the results obtained in the first study. Ifluck is with the researcher, all but one ofthe precursor constructs can be eliminated,which implies that only one category ofbeliefs is relevant. With less luck, theresearcher might find that two or three cat-egories are relevant. Suppose, for example,that only attitudes do a good job of predict-ing behavioral intentions (or behaviors). Inthat case, it is important to find out the rel-evant behavioral beliefs. Happily, these canbe obtained from the open-ended list ofadvantages and disadvantages of thebehavior obtained in Study 1. Assuming areasonable sample size in Study 1, manybehavioral beliefs may be listed, and it maytake some judgment to decide how manypeople need to have listed a particularbehavioral belief for it to deserve to be usedin Study 2. Ajzen and Fishbein (1980) sug-gested a 70% rule (item listed by 70% of theparticipants), but this is arbitrary and maynot fit any particular case at hand. Once theresearcher decides to include a particularbehavioral belief, participants can be askedto respond on a scale ranging from“extremely likely” to “extremely unlikely”

that indicates participants’ perceptions ofthe likelihood of the consequence if theywere to perform the behavior. An evalua-tion item can be paired with it, asking towhat extent it would be “extremely good”to “extremely bad” if the consequence wereto happen. At the risk of sounding like abroken record, I reiterate that the principleof correspondence must be followed evenat this level. For example, if the behavior is“to read an average of three clinical sciencepapers per week for the next year,” and aconsequence is that “I will get bored,” thenthe behavioral belief item might be as fol-lows: “How likely or unlikely would you beto get bored if you read an average of threeclinical science papers per week for the nextyear?”

Although attitudes tend to be the mostimportant construct for predicting mostbehavioral intentions or behaviors, this isnot always so. It may turn out that subjec-tive norms, perceived control, or perceiveddifficulty also are important, and may evenbe more important than attitudes for pre-dicting a particular behavior. In that case,normative beliefs, control beliefs, or diffi-culty beliefs might be important too, andshould be included in Study 2. As always,the principle of correspondence should beobeyed.

It also might be useful to replicate thefirst part of Study 1, concerning behavioralintentions (or behaviors), attitudes, subjec-tive norms, perceived control, and per-ceived difficulty. A benefit of the replica-tion is that the researcher can be morecertain about which precursor constructsmatter and which do not.

In the end, though, interventions will beat the level of beliefs, and so it is importantto find the ones that matter. This can bedone with simple correlations. Remainingwith attitudes as the most important con-struct, for example, how well do each of thebehavioral beliefs correlate with attitudes?As a complication, recall that it is the sumof belief-evaluation pairs that determineattitudes. Consequently, it also might beworth computing each belief-evaluationproduct separately, to investigate whichproduct terms best predict attitudes.3These will be excellent candidates for inter-vention. As a more general check, the sumof the belief-evaluation products also can

be used to predict attitudes. If theresearcher can find a small number (hope-fully one or two) of belief-evaluation prod-ucts that account for almost all of the vari-ance in attitudes that the sum of theproducts accounts for, those are excellentcandidates for intervention, especially ifthey do a good job of predicting behavioralintentions (or behaviors) too. Alterna-tively, it might be that a different constructmatters. My preliminary bet would be onperceived difficulty as an important con-struct. That is, beliefs having to do withtime, effort, and so on devoted to learningrelevant clinical science literature might belikely to perform well as predictors of, say,learning the clinical science literature.

Designing the InterventionDesigning the intervention is the most

difficult part. The foregoing two prelimi-nary studies can be performed in a rather“automatic” way, following the principle ofcorrespondence, and it is practically a cer-tainty that the result will be a few, or sev-eral, beliefs that are good candidates forintervention. Based on both an extensiveliterature (see Fishbein & Ajzen, 2010, for areview) and my own experiences, I can saywith confidence that, up to this point, fail-ure is extremely unlikely provided that theresearcher complies carefully with theprinciple of correspondence. But fromhere, matters are no longer straightfor-ward. The theory does not tell theresearcher how to intervene, only how tofind the beliefs that matter most for thebehavior.

As an example, suppose that as a resultof the two preliminary studies, the behav-ioral belief pertaining to “being bored if Iread an average of three clinical sciencepapers per week for the next year” turns outto be critical. At one level, the solution isobvious: change that belief! But at anotherlevel, it is far from obvious how to inter-vene to change the belief. How do you con-vince someone that an activity they con-sider to be boring is not boring? Or, failingthat, how do you convince someone not toevaluate being bored so poorly? Perhaps asolution might be to introduce a journalwith the goal of filtering and translatingimportant advances in clinical science sothat useful information is provided with aminimum of boring statistical detail (espe-

3According to traditional reasoned action thinking, belief-evaluation products should beused. Arguably, belief measures or evaluation measures are not at a ratio level, in which caseit might be best not to use products after all. This would constitute an argument that beliefsshould be correlated directly with attitudes and intentions, directly.

January • 2018 17


cially t-tests, F-tests, and resulting p-valuesthat are invalid anyhow).

On the other hand, there are other sortsof beliefs that might pose less of a problemfor intervention. For example, suppose thatan important belief is as follows: “Therewould be no benefit to my clients if I wereto read an average of three clinical sciencepapers per week for the next year.” It maybe possible to cite data showing that theirpatients likely would benefit after all, espe-cially if reading the literature results inactual change in practices to more effectiveones.

The problem can be considered moreabstractly. Whenever a theory is applied tomake an empirical prediction or an appli-cation, it is necessary to make auxiliaryassumptions, as I explained in the first sec-tion of the present article. With respect tothe preliminary studies described in theforegoing subsection, the requisite auxil-iary assumptions have been worked out ingreat detail, thereby reducing the creativeload on the researcher. In contrast, when itcomes to interventions, relevant auxiliaryassumptions have not been worked out,and so the researcher is thrown on his orher own ingenuity and creativity.

Evaluating the InterventionBecause readers of this journal are

already knowledgeable about evaluatinginterventions, this section can be kept brief.But it seems useful to make the followingpoints. First, it is important to evaluateintervention effects with means other thanp-values. As I pointed out earlier, not onlyhave these come under much criticism, buteven aficionados of p-values admit thatthey fail to indicate how well an interven-tion works. Most statistical authorities rec-ommend effect sizes. For example, Cohen’sd gives the distance between means of twoconditions, in standard deviation units.However, it is possible to argue that evenCohen’s d is problematic because it con-founds variation due to randomness andsystematicity. Provided that the researcherhas obtained good reliability estimates ofthe dependent variables, Trafimow (inpress) demonstrated that it is possible todistinguish the variance due to random-ness, the independent variable, and system-atic effects due to variables not considered.Using this tripartite distinction, it also ispossible to obtain more focused effect sizesthat control for either randomness or forsystematic effects that are not of interest(due to variables not considered).

A second consideration is that the inter-vention attempted in an experiment might

have to be at some distance from how itactually would be implemented on a largescale. As an example, suppose thatresearchers find that to handle the mostpredictive beliefs, it is necessary to dosomething at an organizational level, suchas founding a journal whose purpose is totranslate important clinical science papersfrom journalese into language that is inter-esting and easy to understand. Short ofactually founding the journal and evaluat-ing its effects, a preliminary interventionstudy necessarily will be somewhat differ-ent. For example, practitioners might berandomly assigned to read specific articlestailored in this direction in the experimen-tal condition, but not in the control condi-tion, to determine whether the behavior ofconcern is influenced, and by how much.To what extent the findings from such apreliminary intervention study will sup-port broader conclusions about the likelyeffect of founding a journal may depend ona variety of factors, such as how close thetailored articles in the experiment would beto the real articles in the founded journal,the extent to which it would be easy forpractitioners to access the founded journal,and many others. My point is not thatresearchers should not conduct suchresearch, only that they should be awarethat a single study is unlikely to be defini-tive.

ConclusionThere has been much complaining on

the part of those knowledgeable about thescience of clinical psychology about the factthat practitioners mostly are uninfluencedby that science. Certainly, from the point ofview that therapy should be based on evi-dence, this is a deplorable state of affairs.But what has been lacking from the scien-tists themselves is (a) an admission thatthere is much wrong, as well as much that isright, with the science of clinical psychol-ogy; (b) strong efforts to fix what is wrongto provide a better case that practitionersought to be influenced; and (c) effortdevoted to finding out why practitionersfail to do what scientists think they shoulddo. Let me emphasize this last point. Ifresearchers do not know what determinesthe behaviors that practitioners perform orfail to perform, efforts to change practi-tioners’ behaviors are likely to fail. Thepoint of the present article is to focus onhow to find out that which is relevant andthat which is irrelevant, to provide a start-ing point in the right direction.

But although I obviously believe in thedirection advocated in the foregoing com-ments, it is important to be up front aboutthe difficulties. The first difficulty, as Iemphasized earlier, is to figure out pre-cisely what the behaviors of concern shouldbe. This includes specifying the target,action, time, and context of each behaviorbut it also includes specifying correspond-ing target, action, time and context for allprecursor variables. Although the two pre-liminary studies are reasonably straightfor-ward, and are practically guaranteed toprovide useful information, there also arecomplications with respect to performingand evaluating intervention studies. Forexample, what are the auxiliary assump-tions that allow the researcher to traversethe distance from the nonobservationalterms in the theory to the observationalterms used in the experimental hypothesis?Another problem is that the researcherneeds to figure out which type of effect sizeto use to index the size of the effect of theintervention. Although researchers may bein the habit of using a particular sort ofeffect size for a particular experimentalparadigm, the issue is not automatic, andresearchers should consider it carefullybefore coming to any conclusions. Finally,even if an intervention is quite successful inan experiment, there might be quite a dis-tance between the laboratory context andhow the intervention actually would beimplemented with real practitioners in realpractice sorts of contexts.

Although I have attempted to be upfront about the ambiguities that face theresearcher to whom change in practition-ers’ behaviors is an important concern, thisfocus should not be taken too pessimisti-cally. I am not arguing that the effort is notworthwhile. Nor am I arguing that the evi-dence has to come from a single, definitive,study. What I am saying, however, is thatchanging practitioner behavior is likely toprove extremely difficult, with an enor-mous amount of psychological inertia toovercome. If the reader nevertheless wouldlike to move in this direction, the fight willbe difficult and protracted. Still, very littlethat is worthwhile is obtained without afight, and so I hope and anticipate that sci-entific clinical psychology researchers willnot allow themselves to feel too discom-moded.

ReferencesAjzen, I. (1988). Attitudes, personality, and

behavior. Chicago: Dorsey.


Ajzen, I. (1991). The theory of plannedbehavior. Organizational Behavior andHuman Decision Processes, 50, 179–211.

Ajzen, I., & Fishbein, M. (1980). Under-standing attitudes and predicting socialbehavior. Englewood Cliffs, NJ: Prentice-Hall.

Ajzen, I., & Fishbein, M. (2005). The influ-ence of attitudes on behavior. In D.Albarracin, B. T. Johnson, & M. P. Zanna(Eds.), The handbook of attitudes (pp.173–221). Mahwah, NJ: Erlbaum.

Bakan, D. (1966). The test of significancein psychological research. PsychologicalBulletin, 66, 423–437.

Barrett, P.T. (2003) Beyond psychometrics:Measurement, non-quantitative struc-ture, and applied numerics. Journal ofManagerial Psychology, 3(18), 421-439.http://dx.doi.org/10.1108/02683940310484026

Carver, R. P. (1978). The case against sta-tistical significance testing. Harvard Edu-cational Review, 48, 378–399.

Carver, R. P. (1993). The case against sta-tistical significance testing, revisited. TheJournal of Experimental Education, 61,287–292.

Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, 997–1003.

Davidson, A. R. & Jaccard, J. (1975). Popu-lation psychology: A new look at an oldproblem. Journal of Personality andSocial Psychology, 31, 10312-1040.

Davidson, A. R., & Jaccard, J. (1979). Vari-ables that moderate the attitude behaviorrelation: Results of a longitudinalsurvey. Journal of Personality andSocial Psychology, 37, 1364-1376.

Duhem, P. (1954). The aim and structureof physical theory (P.P. Wiener, Trans.).Princeton, NJ: Princeton UniversityPress. (Original work published 1906)

Fishbein, M. (1963). An investigation ofthe relationships between beliefs about anobject and the attitude toward that object.Human Relations, 16, 233–239.

Fishbein, M. (1967). Attitude and the pre-diction of behavior. In M. Fishbein (Ed.),Readings in attitude theory and measure-ment (pp. 477–492). New York: Wiley.

Fishbein, M. (1980). Theory of reasonedaction: Some applications and implica-tions. In H. Howe & M. Page (Eds.),Nebraska Symposium on Motivation,1979 (pp. 65–116). Lincoln: University ofNebraska Press.

Fishbein, M., & Ajzen, I. (1975). Belief,attitude, intention and behavior: Anintroduction to theory and research.Reading, MA: Addison-Wesley.

Fishbein, M., & Ajzen, I. (2010). Predictingand changing behavior: The reasoned

action approach. New York, NY: Psychol-ogy Press (Taylor & Francis).

Garb, H. N., & Boyle, P. A. (2015). Under-standing why some clinicians use pseu-doscientific methods: Findings fromresearch on clinical judgment. In S. O.Lilienfeld, S. J. Lynn, & J. M. Lohr (Eds.),Science and pseudoscience in clinical psy-chology (2nd ed., pp. 19-41). New York:The Guilford Press.

Grice, J. W., Cohn, A., Ramsey, R. R., &Chaney, J. M. (2015). On muddled rea-soning and mediation modeling. Basicand Applied Social Psychology, 37(4),214–225. doi: 10.1080/01973533.2015.1049350

Gaudiano, B. A., Dalrymple, K. L., Wein-stock, L. M., & Lohr, J. M. (2015). Thescience of psychotherapy: Developing,testing, and promoting evidence-basedtreatments. In S. O. Lilienfeld, S. J. Lynn,& J. M. Lohr (Eds.), Science and pseudo-science in clinical psychology (2nd ed., pp.155-190). New York: The Guilford Press.

Gulliksen, H. (1987). Theory of mentaltests. Hillsdale: N.J: Lawrence Erlbaum.

Kass, R. E., & Raftery, A. E. (1995). Bayesfactors. Journal of the American Statisti-cal Association, 90, 773–795.

Kline, R. B. (2015). The mediation myth.Basic and Applied Social Psychology,37(4), 202–213. doi: 10.1080/01973533.2015.1049349

Krantz, D. H., Luce, R. D., Suppes, P., &Tversky, A. (1971). Foundations of mea-surement. New York: Academic Press.

Lakatos, I. (1978). The methodology of sci-entific research programmes. Cambridge,UK: Cambridge University Press.

Lederman, L. (1993). The God particle: Ifthe universe is the answer, what is thequestion? Boston, MA: Houghton Mifflin.

Lilienfeld, S. O., Lynn, L., S. J., & Lohr, J.M. (2015). Science and pseudoscience inclinical psychology (2nd ed.). New York:The Guilford Press.

Lord, F. M., & Novick, M. R. (1968). Statis-tical theories of mental test scores. Read-ing, MA: Addison-Wesley.

Luce, R. D., Krantz, D. H., Suppes, P., &Tversky, A. (1990). Foundations of mea-surement, Vol. III: Representation, axiom-atization, and invariance. New York:Academic Press.

McFall, R. M. (1991). Manifesto for a sci-ence of clinical psychology. Clinical Psy-chologist, 44, 75-88.

Meehl, P. E. (1967). Theory-testing in psy-chology and physics: A methodologicalparadox. Philosophy of Science, 34, 103-115.

Meehl, P. E. (1978). Theoretical risks andtabular asterisks: Sir Karl, Sir Ronald, andthe slow progress of soft psychology.

Journal of Consulting and Clinical Psy-chology, 46, 806–834.

Meehl, P. E. (1990). Appraising andamending theories: The strategy ofLakatosian defense and two principlesthat warrant using it. PsychologicalInquiry, 1, 108–141.

Meehl, P. E. (1997). The problem is episte-mology, not statistics: Replace signifi-cance tests by confidence intervals andquantify accuracy of risky numerical pre-dictions. In L. Harlow, S. A. Mulaik, & J.H. Steiger (Eds.), What if there were nosignificance tests? Mahwah, NJ: LawrenceErlbaum.

Michell, J. (1997). Quantitative science andthe definition of measurement in psy-chology. British Journal of Psychology, 88,355–383. doi: 10.1111/j.2044-8295.1997.tb02641.x

Michell, J. (2000). Normal science, patho-logical science and psychometrics.Theory and Psychology, 10, 639–667. doi:10.1177/0959354300105004

Michell, J. (2008a). Is psychometricspathological science? Measurement, 6, 7-24. doi: 10.1080/15366360802035489

Michell, J. (2008b). Rejoinder. Measure-ment, 6, 125-133.doi:10.1080/15366360802121917

Morris, D. M., Grice, J. W., & Cox, R. A.(2017). Scale imposition as quantitativealchemy: Studies on the transitivity ofneuroticism ratings. Basic and AppliedSocial Psychology, 39(1), 1-18.

Nunez, N., Poole, D. A., & Memon, A.(2003). Psychology’s two cultures revis-ited: Implications for the integration ofscience with practice. Scientific Review ofMental Health Practice, 2, 8-19.

Polusny, M. A., & Follette, V. M. (1996).Remembering childhood sexual abuse: Anational survey of psychologists' clinicalpractices, beliefs, and personal experi-ences. Professional Psychology: Research& Practice, 27, 41-52.

Poole, D. A., Lindsay, D. S., Memon, A., &Bull, R. (1995). Psychotherapy and therecovery of memories of childhoodsexual abuse: U.S. and British practition-ers’ opinions, practices, and experiences.Journal of Consulting and Clinical Psy-chology, 63, 426-437.

Roberts, F. S. (1979). Measurement theorywith applications to decision making, util-ity, and the social sciences. Reading, MA:Addison-Wesley Advanced Book Pro-gram.

Rozeboom, W. W. (1960). The fallacy ofthe null-hypothesis significance test. Psy-chological Bulletin, 57, 416–428.

Rozeboom, W. W. (1997). Good science isabductive, not hypothetico-deductive. InL. Harlow, S. A. Mulaik, & J. H. Steiger(Eds.), What if there were no significance

T R A F I M O W

S C I E N C E A N D E P I S T E M I C V I C E

January • 2018 19

tests? (pp. 335–391). Mahwah, NJ: Erl-baum.

Schmidt, F. L. (1996). Statistical signifi-cance testing and cumulative knowledgein psychology: Implications for the train-ing of researchers. Psychological Methods,1, 115–129.

Schmidt, F. L., & Hunter, J. E. (1997).Eight objections to the discontinuation ofsignificance testing in the analysis ofresearch data. In L. Harlow, S. A. Mulaik,& J. H. Steiger (Eds.), What if there wereno significance tests? (pp. 37–64).Mahwah, NJ: Erlbaum.

Spearman, C. (1904). The proof and mea-surement of association between twothings. The American Journal of Psychol-ogy, 15(1), 72–101.

Stevens, S. (1946). On the theory of scalesof measurement. Science, 103(2684),677–680.

Suppes, P., Krantz, D. H., Luce, R. D., &Tversky, A. (1989). Foundations of mea-surement (Vol. 2). New York: AcademicPress.

Tavris, C. (2015). The scientist-practi-tioner gap: Revisiting “A view from thebridge” a decade later. In S. O. Lilien-feld, S. J. Lynn, & J. M. Lohr (Eds.), Sci-ence and pseudoscience in clinical psy-chology (2nd ed., pp. ix-xx). New York:The Guilford Press.

Trafimow, D. (2003). Hypothesis testingand theory evaluation at the boundaries:

Surprising insights from Bayes’s theo-rem. Psychological Review, 110, 526–535.

Trafimow, D. (2006). Using epistemicratios to evaluate hypotheses: An impre-cision penalty for imprecise hypotheses.Genetic, Social, and General PsychologyMonographs, 132, 431-462.

Trafimow, D. (2017). Using the coefficientof confidence to make the philosophicalswitch from a posteriori to a priori infer-ential statistics. Educational and Psycho-logical Measurement, 77, 831-854.

Trafimow, D., & Finlay, K. A. (1996). Theimportance of subjective norms for aminority of people: Between-subjects andwithin-subjects analyses. Personality andSocial Psychology Bulletin, 22, 820-828.

Trafimow, D., & MacDonald, J. A. (2017).Performing inferential statistics prior todata collection. Educational and Psycho-logical Measurement, 77, 204-219.

Trafimow, D., & Marks, M. (2015). Editor-ial. Basic and Applied Social Psychology,37(1), 1-2.

Trafimow, D., & Marks, M. (2016). Editor-ial. Basic and Applied Social Psychology,38(1), 1-2.

Trafimow, D., & Sheeran, P. (1998). Sometests of the distinction between cognitiveand affective beliefs. Journal of Experi-mental Social Psychology, 34, 378–397

Trafimow, D., Sheeran, P., Conner, M., &Finlay, K. A. (2002). Evidence that per-ceived behavioral control is a multidi-mensional construct: Perceived control

and perceived difficulty. British Journalof Social Psychology, 41, 101-121.

Tavris, C. (2015). The widening scientist-practitioner gap: “A view from thebridge” a decade later. In S. Lilienfeld, S.Lynn, & J. Lohr (Eds.), Science and pseu-doscience in clinical psychology (2nd ed.,pp. ix to xx). New York: The GuilfordPress.

Valentine, J. C., Aloe, A. M., & Lau, T. S.(2015). Life after NHST: How to describeyour data without “p-ing” everywhere.Basic and Applied Social Psychology,37(5), 260-273. http://dx.doi.org/10.1080/01973533.2015.1060240

Wasserstein, R. L., & Lazar, N. A. (2016).The ASA's statement on p-values: con-text, process, and purpose. The AmericanStatistician. doi:10.1080/00031305.2016.1154108

. . .

The author has no funding or conflicts ofinterest to disclose.

Correspondence to David Trafimow,Ph.D., Department of Psychology, MSC3452, New Mexico State University, P.O.Box 30001, Las Cruces, NM 88003-8001;[email protected]

McFall (1991) in his classic “Manifesto fora Science of Clinical Psychology,” sug-gested that science is the only warrant forevaluating knowledge claims in clinicalpsychology. Certainly science has given riseto both an unprecedented growth inknowledge as well as powerful technologiesthat allow humans to apply this knowledgefor their desired ends. In addition, profes-sional expertise is founded on epistemicduties—a duty to know.

However, there are at least six problemsin this otherwise generally rosy pictureregarding science: (1) philosophers of sci-ence and others engaging in the study ofscience do not agree on how to define sci-ence or even if there is a single scientificmethod (Feyerabend, 1975; O’Donohue,

2013); (2) the social sciences, includingclinical psychology, have seemed to enjoymuch less scientific progress (e.g., discov-ery of scientific laws) than the natural sci-ences (Meehl, 1978); (3) not all of sciencehas produced beneficial results—for exam-ple, certain technologies have had harmfuleffects on the environment and weapons ofmass destruction have been produced; (4)there has been a grab bag of vexing andembarrassing problems in psychology,such as replicability failures, fraudulentdata, concerns that business interests suchas those of Big Pharma add considerablenoise to the literature (see for example,Antonuccio et al., 1999), such that at timeswhat may appear to be proper science isactually pseudoscience (see, for example,

Herbert et al. [2000], on EMDR, andO’Donohue, Snipes, & Soto [2016a & b] onAcceptance and Commitment Therapy—but also see Gregg & Hayes [2016] for arejoinder); (5) there are unresolved issuesregarding how disparate individual studiescan be properly and fairly aggregated andsummarized to accurately produce certainuseful summary statements such as“empirically supported treatments” (e.g.,see Chambless & Hollon, 1998); and (6)there are longstanding concerns aboutwhether scientific epistemology is com-plete or whether other ways of knowing arealso needed to complete our knowledge(see, for example, Hempel, 1965, on ethicsand Houts, 2009, on religious beliefs).

These are all important problems andconcerns relevant to the work of the cogni-tive behavior therapist. However, thispaper will focus on several key issuesdescribed in point 4 above, which may besummarized by the problem of pseudo-science. The basic notion is it is possible forresearch to be conducted in a way thatappears to be scientifically sound but actu-

Science and Epistemic Vice: The Manufactureand Marketing of Problematic EvidenceWilliam O’Donohue, University of Nevada, Reno


O ’D O N O H U E

ally misses some essential characteristic ofscience, so that one must conclude thatproper science actually has not been con-ducted but rather what has occurred ispseudoscience—literally false science. TheNobel Laurette Richard Feynman (1974)picturesquely called this cargo cult science:

In the South Seas there is a cargo cultof people. During the war they saw air-planes land with lots of good materials,and they want the same thing tohappen now. So they've arranged toimitate things like runways, to put firesalong the sides of the runways, to makea wooden hut for a man to sit in, withtwo wooden pieces on his head likeheadphones and bars of bamboo stick-ing out like antennas—he's the con-troller—and they wait for the airplanesto land. They're doing everythingright. The form is perfect. It looksexactly the way it looked before. But itdoesn't work. No airplanes land. So Icall these things cargo cult science,because they follow all the apparentprecepts and forms of scientific inves-tigation, but they're missing some-thing essential, because the planesdon't land. (p. 7)

A key underlying problem is that ifphilosophers of science have not produceda consensual characterization of what sci-ence is, it can be somewhat difficult toidentify some missing essential property ofepistemically sound science (O’Donohue,2013). For example, a popular candidatefor an essential feature of science is themaximization of criticism (Bartley, 1962).In this view good scientific research is anattempt to expose cherished beliefs tosevere criticism in order to efficiently iden-tify errors in one's web of belief. Genuinescience is not a craving to be correct, butrather a craving to efficiently learn whereour beliefs are wrong so that our errors canbe eliminated. The prominent philosopherof science Sir Karl Popper (1959), forexample, suggested that it is only throughsuch error elimination that knowledgegrows.

Thus, the best and most efficient way ofrooting out error in our beliefs is to exposethese to severe criticism through empiricaltests that can efficiently uncover error. Togive a general picture of the distinctionbetween severe vs. nonsevere tests, supposeone wanted to test the belief, “My ministernever swears.” The researcher could collectverbal samples from her sermons, from herspeeches in front of community groups,

and from her teaching Sunday school andso on. One could then examine these sam-ples to see if these refute the proposition“My minister never swears” by finding aninstance or instances of swearing. Thiswould be a test of the proposition—even anempirical one—but not a severe one.

Alternatively, the researcher couldsample from the minister’s golf games,after she stubs her toe, when she is intoxi-cated, when someone cuts her off in traffic,or when she is in a heated argument. Bothstudies could count as a test of the belief:but it is only the latter that counts as asevere test; it is simply much more likely toexpose the potential falseness of beliefunder test. It is a more risky test. The ques-tion then becomes, How severe have testssuch as random clinical trials been in cog-nitive behavior therapy? To what extentare behavior therapists designing and con-ducting tests that actually place their cher-ished beliefs at risk—or to what extent arethey practicing “cargo cult”—science inwhich there are “tests” but there is verylittle risk of their cherished belief beingshown as false? Are they looking at ser-mons for swearing or after toe stubbing? Itwill be argued that the general answer istwofold: first, research in cognitive behav-ior therapy generally has not been properlyevaluated on this key dimension—whichwill be argued is quite problematic; second,there are exemplars where at least someappear to be quite lacking on this dimen-sion.

The Case of “Scientific Research”and Big Pharma: Lessons Learned?

One of the best known recent examplesof such problematic science and the lack ofsevere testing is research that has been con-ducted by Big Pharma, particularly numer-ous for clinical trials of antidepressants(Antonuccio et al., 1999). Speaking gener-ally, this research used apparently soundmethodologies such as random clinicaltrials, decent sample sizes, double blinds,statistical analysis, and usually was pub-lished in high-impact peer-reviewed stud-ies—that is, with many of the apparentcharacteristics of sound science and indeedeven exceptional quality/high-prestige sci-ence. However, numerous critics haveastutely pointed out many methodologicalproblems with this research and thus ques-tioned the intellectual virtue of thisresearch (e.g., Antonuccio et al.; Greenberg& Fisher, 1994, Kirsch et al., 2008; Klein,2006). These problems occurred at anumber of levels: blinds were violated; neg-

ative results were file drawered; multipleoutcome measures were used but only theoutcome variables that failed to reach sig-nificance failed to be reported; side effectswere not fully reported; safety concernssuch as increased suicidality were not expli-cated; multiple statistical analysis wereconducted until supportive results werefound; statistical significance was conflatedwith clinical significance; process variableswere not directly measured or properlyreported, and so on. In addition, it isimportant to note that most of these prob-lems were not immediately apparent—these were hidden by researchers anduncovered only after often arduous inde-pendent investigation. Moreover, the drugresearchers themselves also had variouspersonal motivations that were oftenhidden: they were financially incented invarious ways by Big Pharma to find andreport positive results; they were offeredother inducements such as expense paidtrips to present results in luxurious confer-ences; and the allure of publishing in high-impact journals was also present amongother inducements.

Other critics have also pointed out addi-tional problems with research involvingother medications—e.g., that there wereoften deviations in the analysis planbetween protocols and published papers,and, interestingly, that the effect sizes ofdrug interventions are larger in the pub-lished literature compared with the corre-sponding data from the same trials submit-ted to FDA (Ioannidis, Munafo, Fusar-Poli,Nosek, & David, 2014). Ioanndis et al.(2014) nicely summarized other problemswith other drug studies:

For example, in a review of all random-ized controlled trials of nicotine replace-ment therapy (NRT) for smoking cessa-tion, more industry-supported trials (51%)reported statistically significant resultsthan nonindustry trials (22%); this differ-ence was unexplained by trial characteris-tics. Moreover, industry-supported trialsindicated a larger effect of NRT (summaryodds ratio 1.90, 95% CI 1.67 to 2.16) thannonindustry trials (summary odds ratio1.61, 95% CI 1.43 to 1.80). Evidence ofexcess significance has also been docu-mented in trials of neuroleptics. Compar-isons of published results against FDArecords shows that, while almost half of thetrials on antidepressants for depressionhave negative results in the FDA records,these negative results either remain unpub-lished or are published with distortedreporting that shows them as positive; thus,the published literature shows larger esti-


January • 2018 21

mates of treatment effects for antidepres-sants than the FDA data. A similar patternhas been recorded also for trials on antipsy-chotics.

This is a serious concern for obviousreasons—the pollution of the scientific lit-erature which can affect clinical decisionmaking and thus client welfare—but it alsois a parochial concern for cognitive behav-ior therapists because in many cases thesepsychotropic medications were often seenas in direct competition with cognitivebehavior therapies. The general scientificquestion could be phrased, “Which is moreefficacious, this medication, some CBT, orboth?” Any jimmying of results toward themedication not only distorted informationand placed patient welfare at risk but it alsohad direct implications for the rationalappraisal of the efficacy of CBT. Obviously,something is seriously amiss here—manyof the characteristics of science seem to bein place but yet all these efforts seem to beviolating what Meehl (1993) once attrib-uted to Bertrand Russell as the fundamen-tal orientation of an intellectually virtuousscientist— “the passion not to be fooledand not to fool anybody else.”

These criticisms of Big Pharma seemreasonable, fair, and important—but thiskind of scrutiny to date also seems to besomewhat one sided. Few are asking theextent to which CBT’s research house is inorder with regard to such epistemic vice. Atfirst blush, one would have to admit thatsome of the same personal incentives couldbe present in CBT research (although per-haps a bit less flush). CBT researchers canhave a financial interest to produce positiveresults—from paid trainings, from booksales, from academic promotions, andfrom additional salaries from grants, andso on. CBT researchers can also be inter-ested in other inducements such as fame,awards, professional offices, increased cita-tions, and publications in high-prestigejournals. The question becomes oughtbehavior therapy research and behaviortherapists also be scrutinized for their epis-temic virtue along lines similar to thescrutiny received by Big Pharma?

For example, a case study (see O’Dono-hue et al., 2016a, 2016b; and for a rejoinder,see Gregg & Hayes, 2016) of a series of pub-lications related to Acceptance and Com-mitment Therapy and diabetes self-man-agement found several similar problems,including: (a) a failure to report several keynegative results from the dissertation in asubsequent peer-reviewed journal publica-tion; (b) a series of overstatements and mis-statements by the researchers in subse-

quent publications exaggerating the posi-tive findings in the dissertation; (c) thedevelopment of a bibliotherapeutic inter-vention explicitly marketed to people withdiabetes (claiming to be “a proven pro-gram”) in which the reader is led to believethe bibliotherapy intervention they were touse had been shown to be effective and safein past research, when the bibliotherapyintervention had not even been studied atall; (d) the failure to accurately describe insubsequent publications, particularly in thepeer-reviewed journal publication, whatare at best equivocal findings regarding therole of putative ACT processes as mediat-ing these results. Instead, the opposite isfound: clear, but inaccurate, statementsabout ACT processes producing clinicallysignificant changes in diabetes self-man-agement when the original data simply donot warrant this; and (e) a lack of appropri-ate caution and qualification in interpret-ing the data relating to the effectiveness ofACT for diabetes self-management despitenumerous methodological shortcomings,including, but not limited to: therapist alle-giance effects, dependent measures withunknown psychometrics, no blinds, mini-mal follow-up, no safety measures, signifi-cant attrition, problems with alpha rateinflation, no comparison to key treatmentsas usual, and no replications. All of theseare serious problems and problems thatseem to be similar to those found in BigPharma’s problematic pseudo-scientificresearch. Interestingly, the existence ofthese problems sometimes occurred in acontext in which the authors were explic-itly reassuring readers that they wouldrefrain from excessive claims and wouldpoint out unresolved empirical issues, thusproviding readers with a false assurancethat good scientific practices were beingfollowed. This certainly raises clear issuesabout bias, pseudoscience, and intellectualvice

It is also important to note that there isan important second-order concern thatalso needs to be mentioned: Bias can occurnot only in the design and reporting of aparticular study, but it also can occur in theway studies are aggregated or how thatstudy is spoken about subsequently. Utter-ances like, “This and other studies showthat this treatment is scientifically proven”;“There are 200 RCTs proving the efficacyof X therapy” and so on each can also beexamined for bias and epistemic virtue. Itmay be particularly important to examinethese statements as these summary state-ments may be more influential in practical

decision making than statements about anindividual study.

There have been longstanding ques-tions about the epistemic virtue of otherresearch in psychotherapy—for example,with the refusal of proponents of facilitatedcommunication to accept evidence that fal-sified the notions that facilitated was effec-tive or that its hypothesized process vari-ables were operative (Lilienfeld et al., 2014):the refusals of proponents of EMDR toadequately test simple exposure canexplain positive results instead of fingerwaiving (Herbert et al., 2000), or whetherclaims for the efficacy of positive psychol-ogy have vastly outstripped the data (Eidel-son & Soldz, 2012). In all these cases,adherents are not disinterested—there arenumerous payoffs for ignoring reasonablecriticisms, data that is falsifying, and con-ducting research so that only weak tests areemployed that will produce “positive”results. However, like Big Pharma’s distort-ing research, there are serious conse-quences to clients and to the scientific liter-ature from such problematic studies.

Science and Virtue EpistemologyIf there is such scrutiny of epistemic

virtue of scientific practices, how ought thisto be understood? One such viable candi-date is virtue epistemology (Sosa, 2009).Virtue epistemology is a growing approachto understanding rational agency and theway knowledge can be legitimately gained.Kidd (2016) provides a useful summary:

The core conviction of virtue episte-mology is that enquiry is an activeprocess that can go better or worse,and that central among the factors thatdetermine how it goes are the charac-ters of the enquirers who perform it.Since enquiry is initiated and per-formed by epistemic agents, such asscientists or scholars, the stable cogni-tive and behavioural dispositions ofthose agents are surely crucial to thesuccess of that enquiry. (p. 10)

The list of possible epistemic virtues islenghty as rational belief formation can beevaluated on multiple dimensions: consci-entiousness, transparency, discernment,intellectual honesty, and intellectualhumility, for example. For our purposeshere critical epistemic virtues in researchare to honestly and transparently conductand report severe tests instead of gaming ofmethodology and scientific reporting toproduce weak or pseudo-tests to manufac-


O ’D O N O H U E

ture in an effort to report only “confirm-ing” results.

There is a growing recognition thatsuch biases occur in scientific research andthese epistemic problems need to be bothidentified and prevented. However, itseems that the field of clinical science andCBT generally has been somewhat of a lag-gard in this movement. For example, thereis no recognition of this in most conceptsof empirically supported treatments, suchas the well-known Chambless report(Chambless & Hollon, 1998). Instead, anyrandomly controlled trial with positiveresults seems to be taken without scrutinyfor bias or interest and is taken as soundevidence to gain the mantle of “empiricallysupported treatment.”

This has perhaps led to a problematicenterprise associated with research in cog-nitive behavior therapy: “If I can manufac-ture randomly controlled studies in sup-port of my therapy, I can gain the rewardsassociated with this.” Of course, the easiestway to do this is to avoid severe testing, asdiscussed above—one would report datafrom the sermon not from the golf game.To be more precise, O’Donohue et al.(2016b, p. 40) suggested that these method-ological moves would make the manufac-ture of such positive RCTS possible andwould all be problematic from the viewpoint of epistemic virtue:

1. Ensure that therapy allegianceeffects are operative in favor of theexperimental treatment; for example,by having one therapist stronglyaligned with a therapy orientationand the other not aligned with thecontrol condition.2. Do not use blind data collectors,therapists, or subjects. Give everychance that biases and expectationscan be communicated.3. Once these biases and expectationshave been instantiated, rely on self-report as a key outcome measure.4. Use a small convenience sample ofclients who only have relatively lowlevels of the clinical problem.5. In single subject experimentaldesigns run more than three subjectsbut report only the three that provideconfirming results.6. Stop collecting data once p < .05 isreached.7. Do not randomly assign or sampletherapists: use the more advanced,more talented, therapist in the experi-mental condition.

8. Use multiple outcome variables butin any discussion prioritize only thosethat show statistically significantresults. Interpret the nonsignificantresults as “minor” instead of falsifica-tions of any beliefs or hypotheses. Or,alternatively, completely fail to men-tion these in subsequent publications.9. Have a weak control condition—donot test for equivalence in initial cred-ibility; do not test for the presence ofany other key psychotherapeuticprocesses in the experimental condi-tion such as the nonspecifics. Espe-cially avoid a control that is evidence-based treatment as usual as this is aharder hurdle to beat. Ignore theiatrogenic effects that may realize ifany real patients are switched from amore robustly tested treatment asusual due to one’s weak test and exag-gerated results.10. Do not analyze for clinical signifi-cance. This is a tougher hurdle, so indiscussions conflate statistical signifi-cance with clinical meaningfulness.11. If the experimental therapy condi-tion fails to reach statistical signifi-cance on any outcome measure butthe means are in the favored direc-tion—report these positively astrends. This still gives a more favor-able impression to the original beliefsystem.12. Do not run many or any follow-ups after therapy is completed eventhough one may be treating a chroniccondition like diabetes. Relapse is acommon problem so the absence oflong-term follow-ups avoids thedetection of relapse which would be aless favorable study.13. If statistical tests show nonsignifi-cance find another statistical test thatshows a significant confirmatoryresult. Do not report in the publica-tion that a previous statistical test wasrun that showed nonsignificantresults.14. Use a small unrepresentativesample—which increases the odds ofa false positive result. However, makeclaims that the therapy works for abroad class of patients—seemingly alldiabetics, for example.15. Ignore initial differences ifrandom assignment fails to produceequivalent groups, particularly ifthese are in favor of the experimentaltreatment condition.

16. Do not attempt to search for anynegative side effects.17. Do not conduct a failure analysisand do not report the percentage ofpatients that did not change orbecame worse in the experimentalcondition.18. Be unclear in what exactly the keyprocesses are, e.g., “acceptance” and“commitment,” and how these wereinstantiated in the research.19. Have a vague, elastic model oftherapy process in which “accep-tance,” “emotional avoidance, “mind-fulness,” “valued action,” “deliteral-ization,” “psychological flexibility,”“recontextualization skills,” “cogni-tive entanglement,” “loss of corevalues,” “cognitive fusion,” “domina-tion of conceptualized self over ‘self ascontext,’” “relational frames” and soon are all intermixed so that it isunclear exactly what actually ought tooccur in treatment. Do not acknowl-edge that many of these allegedly keyconstructs were not actually tested inthe study.20. Do not provide an assessmentplan for each of these many con-structs in the study but still use theseconcepts in theoretical talk.21. Do not report any problems in thetheoretical background of the ther-apy—e.g., problems in the conceptu-alization or replication of relationalframe theory (see e.g., Roche, 2010).22. Use measures of unknown orproblematic validity.23. Run analyses on a variety of out-come measures such as change scores,and absolute differences at the end oftherapy and report those that showmore significant results.24. Do not conduct analyses on boththerapy completers and intent totreat. Generally, ignore attrition;especially do not interpret attrition asa problem for the experimental treat-ment condition.25. Make claims that one modality oftherapy (bibliotherapy) works eventhough another modality (a work-shop) was tested.26. In reporting results, simply do notreport some hypotheses that were notconfirmed.27. If all outcome measures are nega-tive, then use the file drawer.

January • 2018 23


28. In publications, make exagger-ated summary statements of the stateof the science such as “scientificallyproven” that ignore any design limita-tions, any outcome variables that failto reach significance, any failures, anyanalysis of relapse, etc.29. Make misleading statementstowards the positive, e.g., the ACT bib-liotherapy for diabetes has been stud-ied when it simply hasn’t. One can alsostate that ACT has shown its useful-ness in integrated care settings for dia-betes when there have been no studiesof this.30. Use honorific and obscurantist lan-guage to describe one’s approach toscience, e.g., “reticulated.”31. Keep a scorecard regardingnumber of RCTs supporting one’s pre-ferred position but an incomplete one.Do not report the scorecard of thecompetition such as standard cogni-tive behavior therapy.32. Do not mention that the resultshave not been replicated in an inde-pendent laboratory.33. When asked for therapy manualsto attempt to replicate, indicate thatthese are not available.34. Indicate that one is open to criti-cism but ignore this criticism.(O’Donohue et al., 2016b, p. 40)

Perhaps there has been too much emphasison cognitive biases such as heuristical errorsas affecting judgment of scientists and clini-cians (e.g., see Garb, 1989). The biases dis-cussed here provide a more comprehensiveand thus accurate view of the biases that canaffect science. Bertrand Russell (1950) in hisNobel Prize acceptance speech suggestedfour main desires that motivate muchbehavior, including scholarship: acquisitive-ness (“the wish to possess as much as possi-ble”); rivalry (“a much stronger motive”);vanity (“a motive of immense potency”);and love of power (“which outweighs themall”). We may note the tremendous degreeto which all four desires seem actively atwork in shaping science, including researchin cognitive behavior therapy. According toRussell, it is important to be clear-sighted onthis matter.

Recommendations to Identify Biasand Promote Intellectual VirtueFigure 1 illustrates common practices

and possible solutions across the workflowfor addressing multiple biases (from Ioan-nidis et al., 2014).

Specific Recommendationsfor Controlling Bias in More

Epistemically Virtuous ResearchOne seems to be confronted with the

fact that the epistemically virtuous scientistmight be rare or one should at least notassume that research being produced isbased on epistemic virtue. Instead, oneought to conduct, publish, and appraise allresearch. The following steps are recom-mended:

1. Epistemic vice and virtue aretaught as part of research methodsand ethics courses.2. All clinical trials are preregis-tered. This can allow a better assess-ment of the use of file drawer, p-hacking, as well as problematicdeviations from protocols and posthoc analyses.3. Part of peer review for journalsand grants is evaluating the extentto which methodological decisionswere made to construct a severe testvs. to manufacture a positive result.The steps described above as allow-ing weaker tests are made moretransparent and scrutinized and aregenerally reasons for rejection.4. Method sections are written toincrease transparency by includinga subsection in which the study’smethodological decisions are eluci-dated in more detail and sufficientinformation is provided to evaluatefor bias and severe testing.5. Replications are seen as havingincreased value and an importantpart of science. This would needbuy-in from journal editors andpromotion committees.

6. No research such as an RCTshould count as support for an hon-orific such as an “empirically sup-ported treatment” if the test is asuitably severe test.7. Summary statements about abody of research are also scruti-nized for their epistemic virtue.

ConclusionsIt is important that science be con-

ducted with an integrity where its essentialfunctions of error detection operatesinstead of in a manner in which only thetopography of science is present (“cargocult” science). Big Pharma provides animportant object lesson and more CBTresearch needs to be scrutinized for its epis-temic virtue. Perhaps this can result in theincreased growth of knowledge and over-come what Meehl (1978) has called “theslow growth of soft psychology” by a morethoroughgoing commitment to BertrandRussell’s recommended orientation for thevirtuous scientist, “the passion not to befooled and not to fool anybody else.”

ReferencesAntonuccio, D. O., Danton, W. G., DeNel-

sky, G. Y., Greenberg, R., & Gordon, J. S.(1999). Raising questions about antide-pressants. Psychotherapy and Psychoso-matics, 68(1), 3-14.

Bartley, W. W. (1962). The retreat to com-mitment. New York: Knopf.

Chambless, D. L., & Hollon, S. D. (1998).Defining empirically supported thera-pies. Journal of Consulting and ClinicalPsychology, 66(1), 7.

Eidelson, R., & Soldz, S. (2012). Does com-prehensive soldier fitness work: CSFresearch fails the test. Coalition for anEthical Psychology Working Paper, 1(5),1-13.

Feyerabend, P. (1975). Against method.New York: Verso.

Feynman, R. (1974). Cargo cult science.Engineering and Science, 37(7), 10-13.

Garb, H. N. (1989). Clinical judgment,clinical training, and professional experi-ence. Psychological Bulletin, 105, 387–396.

Greenberg, R. P., & Fisher, S. (1994). Sus-pended judgement seeing through thedouble-masked design: A commentary.Controlled Clinical Trials, 15(4), 244-246.

Gregg, J. A., & Hayes, S. C. (2016). Theprogression of programmatic research incontextual behavioral science: Responseto O’Donohue, Snipes, and Soto. Journalof Contemporary Psychotherapy, 46(1),27-35.

Figure 1. Common practices and possiblesolutions (retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4078993/figure/F2/)

F O L L E T T E


Hempel, C. (1965). Aspects of scientificexplanation. New York: The Free Press.

Herbert, J. D., Lilienfeld, S. O., Lohr, J. M.,Montgomery, R. W., T O'Donohue, W.,Rosen, G. M., & Tolin, D. F. (2000). Sci-ence and pseudoscience in the develop-ment of eye movement desensitizationand reprocessing: Implications for clini-cal psychology. Clinical PsychologyReview, 20(8), 945-971.

Houts, A.C. (2009). Reformed theology isa resource in conflicts between psychol-ogy and religious faith. In N. Cummings,W. O’Donohue, & J. Cummings (Eds.),Psychology’s war on religion. Phoenix,AZ: Zeig, Tucker, Thiesen.

Ioannidis, J. P., Munafo, M. R., Fusar-Poli,P., Nosek, B. A., & David, S. P. (2014).Publication and other reporting biases incognitive sciences: Detection, prevalence,and prevention. Trends in Cognitive Sci-ences, 18(5), 235-241.

Kidd, I. J. (2016). Why did Feyerabenddefend astrology? Integrity, virtue, andthe authority of science. Social Episte-mology, 30(4), 464-482.

Kirsch, I., Deacon, B. J., Huedo-Medina, T.B., Scoboria, A., Moore, T. J., & Johnson,B. T. (2008). Initial severity and antide-pressant benefits: A meta-analysis of data

submitted to the Food and Drug Admin-istration. PLOS Medicine, 5(2), e45.

Klein, D. F. (2006). The flawed basis forFDA post-marketing safety decisions:The example of anti-depressants andchildren. Neuropsychopharmacology,31(4), 689.

Lilienfeld, S. O., Marshall, J., Todd, J. T., &Shane, H. C. (2014). The persistence offad interventions in the face of negativescientific evidence: Facilitated communi-cation for autism as a case example. Evi-dence-Based Communication Assessmentand Intervention, 8(2), 62-101.

Meehl, P. E. (1978). Theoretical risks andtabular asterisks: Sir Karl, Sir Ronald, andthe slow progress of soft psychology.Journal of Consulting and Clinical Psy-chology, 46(4), 806.

Meehl, P. E. (1993). Philosophy of science:Help or hindrance? PsychologicalReports, 72(3), 707-733.

McFall, R. M. (1991). Manifesto for a sci-ence of clinical psychology. The ClinicalPsychologist, 44(6), 75-88.

O’Donohue, W (2013). Clinical psychologyand the philosophy of science. New York:Springer.

O’Donohue, W., Snipes, C., & Soto, C.(2016a). A case study of overselling psy-

chotherapy: An ACT intervention fordiabetes management. Journal of Con-temporary Psychotherapy, 46(1), 15-25.

O’Donohue, W., Snipes, C., & Soto, C.(2016b). The design, manufacture, andreporting of weak and pseudo-tests: Thecase of ACT. Journal of ContemporaryPsychotherapy, 46(1), 37-40.

Popper, K.R. (1959). The logic of scientificdiscovery. London: Hutchinson.

Russell, B. (1950). What desires are politi-cally important. Retrieved from Nobel-prize.org.

Sosa, E. (2009). A virtue epistemology.Oxford: Oxford University Press.

. . .

The author has no funding or conflicts ofinterest to disclose.Correspondence to William O'Donohue,Ph.D., Director, Victims of Crime Treat-ment Center, Department of Psychology,University of Nevada, Reno, Reno, NV89557; [email protected]

TO MANY OF US it is perplexing as to whyproviders or utilizers of interventionsintended to help people in distress ignoreresearch findings that document effica-cious interventions in favor of unsup-ported pseudoscientific therapies. Clinicalscience programs certainly present thereports on empirically supported treat-ments (Chambless, 2015; Chambless et al.,1998; Chambless & Hollon, 1998; Chamb-less & Ollendick, 2001). In research meth-ods or philosophy of science courses, thedemarcation criteria for differentiating sci-ence from pseudoscience are often taught(Lilienfeld, Lynn, & Lohr, 2015a; Schermer,2002). Some classes will offer classicdebates about whether this distinctionbetween the two can be reliably made (cf.Laudan, 1983; Mahner, 2013; Pigliucci,2013). Even without philosophical tutor-ing, at some point it is clear that one hasdeparted from science into pseudoscience(Lilienfeld, 2011, p. 109).

Our interest in this issue gets rekindledwhen a practice that represents a signifi-cant cost to society uses the trappings ofscience to establish credibility and attractdisciples. At some point our scientificvalues are sufficiently offended to cause usto decry pseudoscientific practices, non-science, or antiscience. Examples have beenidentified for decades (e.g., Beyerstein,2001). Researchers have proposed methodsfor identifying harmful practices (Dimid-jian & Hollon, 2010). There may be debateabout what is meant by harm. Tragically,there are some cases where harm is indis-putable, therapy is abusive, and deathsoccur (e.g., Advocates for Children inTherapy, 2017; Chaffin et al., 2006; Mercer,2014; Singer & Lalich, 1996).

Outside of therapeutic interventions,fraudulent science poses threats to the col-lective well-being of larger groups of indi-viduals. Recent examples of people beingmisled by fraud are those who oppose vac-

cinations based on false information link-ing vaccinations to the development ofautism (see Rao & Andrade, 2011, for asynopsis and timeline of Wakefield'sretracted report; Wakefield et al., 1998).This report fed into personal beliefs andheuristic errors of parents that place at risknot only their children, but other childrenwho cannot be immunized. At a larger levelof analysis, climate change deniers placemultiple species at risk of extinction. Onething we know is that once misinformationis received, it is extremely hard to correct(Chan, Jones, Jamieson, & Albarracín,2017).

If we emphasize science in the trainingof our students and make available lists ofempirically supported treatments, why dopeople make use of alternative treatments?It might be useful to ask the question ofwhy people are not persuaded by scienceand go on to create and consume such ther-apies. Paraphrasing Skinner, it behooves usto study the behavior of the person,because the person is always right. Let usleave aside such factors as greed, gullibility,lack of training, motivated reasoning,naiveté, hopelessness, etc. Let us ask thequestion of why our appeal to scientific evi-dence is not sufficiently convincing to keep

Pseudoscience Persists Until Clinical ScienceRaises the BarWilliam C. Follette, University of Nevada, Reno

January • 2018 25

P S E U D O S C I E N C E P E R S I S T S

the audience for pseudoscience sufficientlysmall.

ContextSo why do pseudoscientific practices

persist? It is not because the issue has notbeen well articulated. For many yearsLilienfeld and colleagues and many othershave provided thoughtful critiques of pseu-doscience in clinical psychology (Lilienfeld,Lynn, & Lohr, 2003, 2015b). Yet practition-ers, some psychologists and some not,invent and practice dubious interventionseven though other therapies may have doc-umented efficacy. Perhaps there are othersufficiently potent contextual features thatmake the science less clearly persuasivethus allowing the production and con-sumption of pseudoscientific practiceinstead. Let’s consider some reasons whyclinical science has not preempted alterna-tive practices.

The Choice of the Medical ModelPerhaps one reason the quality of our

science is not sufficiently convincing isbecause the field took a wrong turn in 1980.Psychology misestimated the effects of gen-erally acquiescing to an implicit medicalmodel when accepting DSM-III as thedominant nosology to organize researchand practice in clinical psychology (seeKirk & Kutchins, 1992, for a discussion).Several negative effects ensued (see Follette& Houts, 1996, for a critique and alterna-tive; Follette, Houts, & Hayes, 1992).

• The research strategy. One ill-effect ofthis decision was that DSM influencedresearchers to construe distressing behav-ior as one of hundreds of disorders ratherthan different topographies of a muchsmaller number of functional classes ofbehavior that rested on common psycho-logical principles. DSM-III claimed to beatheoretical (American Psychiatric Associ-ation, 1980, p. 7). With the exception ofPTSD, there were almost no statements ofetiology of clinical problems. The resultwas that treatments were developed to treatdisorders with little regard to commonali-ties that produced or maintained distress.What emerged was treatment X for depres-sion. The same basic treatment X was laterdeveloped for (applied to) anxiety, andthen other disorders. Less attention waspaid to common processes for the develop-ment of these disorders from a psychologi-cal science perspective. The fact that thesame basic treatments worked across sev-eral diagnostic categories should have beentaken as an occasion for us to question

either our understanding of the treatmentsfor the “disorders” or the validity of theunderlying nosology itself.

One effect of adopting an atheoreticalnosology seemed to be a reluctance todevelop and test theories of mechanisms ofetiology or change. Yet specifying testablemechanisms of change is a hallmark of sci-ence that distinguishes it from pseudo-science. Treatments often did have theoriesof etiology or change, but theory testingwas not the focus of treatment studies.Instead, studies were often of the “horserace” variety where the winner was what-ever treatment produced significantlymore reduction in symptoms. The generalconclusion has been that the races oftenended in ties where many treatments werebetter than a waitlist control, and mostwere equivalent to each other. These typesof studies are still done and mostly producesimilar results. The question of interest inclinical trials was usually whether one gotto the finish line but not how. Since scien-tific programs receiving significant fundingwere being judged by whether they pro-duced improvement, pseudoscientifictreatments can often show some amount ofself-reported improvement as well.Because randomized controlled trials werenot judged by the evidence testing the the-ories on which treatments were designed,pseudoscientific theories have not suc-cumbed to the criticism that the theoriesbehind them are invalid or untestable.Instead, pseudoscientific interventions areidentified by the apparent absurdity of therationale. We will discuss falsifiability later.

• Inclusion and outcome measures. Toreceive treatment clients had to have adiagnosable disorder. Without a diagnosis,people who were unhappy with life cir-cumstances, relationships, prejudice, ordidn’t understand the relationship betweenthe environment (writ large) with how suc-cessfully they achieved valued goals werenever the focus of study. By focusing ondisorders rather than including well-beingas part of the assessment of outcomes, clin-ical researchers largely ceded these latterissues to others.

Accepting the presence or absence of adisorder as an outcome measure producesa methodological problem. Effect sizes aregenerally reported with respect to somemeasure of change in the degree of distress(e.g., reduction in depression or anxietyscores or no longer meeting criteria for adiagnosis). This choice of dependent mea-sures creates problems for arguing for verylarge effects. First, the best result one could

achieve is no symptoms of a disorder. For-mally, this is an instrumentation threat tointernal validity due to a floor effect. If thegoal of an intervention is “cure” a disorder,then the best one can do is have zeroamount of the disorder. Others haveargued that psychology could offer a morerobust model of psychological health thatwould conceptually allow for a richer mea-surement model of outcome (Bonow &Follette, 2009; Follette, Bach, & Follette,1993). Unfortunately, for a long time out-come assessment did not differentiatebetween an instance where depressivesymptoms were gone and a secondinstance where depressive symptoms weregone and the patient was more involvedwith family, enjoyed an engaging socialnetwork, experienced more control overlife, or worked and played with greater sat-isfaction.

If an alternative treatment offers an out-come that is more than misery manage-ment but also includes a richer life experi-ence, one can appreciate the appeal ofimprovement claims beyond “diseased ornot” as an outcome. There is no assertionhere that the claims offered by alternativetreatments were valid, only that the scopeof outcomes addressed by alternative treat-ments can be more appealing. Morerecently, there have been thoughtful con-tributions on the treatment quality andoutcome assessment measurementdomains (e.g., Lambert, 2017; Thornicroft& Slade, 2014), but these additions are lateto the game. In the meantime, interven-tions that made claims to improve the qual-ity of life or enhance control had an oppor-tunity to proliferate.

• Dissemination. If there were a reducedaudience for pseudoscientific interven-tions, the problem would fade to a manage-able level. Lilienfeld and colleagues havediscussed sources of resistance to evidence-based practice by psychotherapists (Lilien-feld, Ritschel, Lynn, Cautin, & Latzman,2013). Beyond the difficulties with psy-chotherapists being convinced by data,others struggle with how to translate scien-tific information to a variety of audiences(Kaslow, 2015). However, the way clinicalscientists disseminate information cannotpossibly be as influential on consumerbehavior as how purveyors of pseudo-science approach the task. One of the crite-ria used to identify pseudoscience is the useof testimonials, a practice that is prohibitedby the APA in Standard 5 of the EthicalPrinciple of Psychologists and Code ofConduct because of concerns about vul-


nerability to undue influence (AmericanPsychological Association, 2017). Practi-tioners outside the purview of APA are notalways so constrained.

Social psychologists and persuasionexperts have long identified the potency ofpersonal narratives compared to how sci-entific information is usually conveyed tothe public or policy makers. Almost everypresidential State of the Union addressnow includes a policy initiative bolsteredby a vivid story that refers to a specific indi-vidual who embodies the need for thepolicy or policy change. Ronald Reagan’svivid description of a “welfare queen”during the 1976 presidential campaign isone such example used to illustrate the can-didate’s assertion that reform was neededto protect against wanton abuse. In 2009President Obama made references to threespecific instance of individuals who were orwould be impacted by policy changesduring his State of the Union address.When discussing the improved state of theeconomy in his 2015 State of the Unionaddress, then President Obama detailed astory of the Erler family, who fell on hardtimes and recovered in parallel with theeconomy. When the need for change isadvocated or accomplishments touted, thepresident names such a person or familywho is often in the audience who standsand receives an ovation. Certainly, thesubtle but powerful influence of socialmedia on public attitudes has been thefocus of much attention since the last elec-tion, attesting to the power of repetitionand volume over facts.

Thus far, clinical scientists have notidentified the optimal, ethical ways tobetter disseminate scientifically groundedpractices. Pseudoscience practitioners orcomplementary alternative medicineproviders operate under a different dissem-ination model. While scientists mightargue that the use of testimonials is a wayto identify pseudoscience, the public viewssuch testimonials as influential, crediblesources of information. The issues relatedto pharmaceutical “direct to consumer”advertising is more complex than can beaddressed here, but it is easy to observe thecorrelation between advertising and salesof a drug. Even not considering pharma-ceutical marketing, woe to anyone watch-ing late-night television who fails to havethe correct pillow, doesn’t hang by theirfeet, or does not partake of the cornucopiaof dietary and vitamin supplements toimprove, well, just about everything. I amunequivocally not advocating for the aban-donment of ethical dissemination prac-

tices. I raise the issue that psychology per-suasion science would predict that the dis-semination practices of pseudosciencewould be more effective than those used bypsychology clinical scientists. This state ofaffairs is especially ironic given that socialpsychology provides some of the foremostexperts in persuasion and influence (e.g.,Pratkanis, 1995, 2007).

• Summary. The initial acceptance of amedical model and the emphasis on effi-cacy rather than the testing of psychologi-cal theory placed clinical science at a disad-vantage. By failing to make model testing aprimary focus of study, clinical trials didnot follow its own methodology for distin-guishing science from pseudoscience. Themodel did not include measures of well-being and improved adaptability but ratheronly reducing a limited set of symptoms.This constrained the ability to show largertreatment effects and a richer domain oftreatment benefits. Though there havebeen efforts to disseminate findings topractitioners and the public, the methodsof doing so are less effective than those whopropose and advocate alternative treat-ments.

Empirically Supported Treatments(ESTs)

Following the evidence-based medicinemovement in England, in 1995 APA estab-lished the Task Force on Promotion andDissemination of Psychological Proce-dures with the laudable goal of identifyingand disseminating treatments with knownefficacy (Chambless & Ollendick, 2001).How does this decision by APA, and Divi-sion 12, contribute to the context thatmight paradoxically lead to the discountingof scientifically supported therapies? Theenormity of the task of sifting through theliterature and reliably identifying ESTsrequired a focus on evidence that a therapyworked. Treatments were not evaluated onhow they worked, whether one worked sig-nificantly better than other ESTs, how clin-ically meaningful the observed changeswere, or what, if anything, differentiatedone therapy from another and contributedto a better outcome (Follette, 1995; Follette& Beitz, 2003; Follette & Houts, 1996; Fol-lette et al., 1992; Jacobson, Follette, &Revenstorf, 1984; Kazdin, 2007, 2014).

• Mechanisms. Because mechanisms ofan intervention were not the primary focusof study, it was and is possible for the“same” therapy to be reinvented underanother name and subsequently appear on

the EST list. Let us consider the roads takenby two therapies, now both with some levelof empirical support but dubious theoreti-cal underpinnings. The first example is of atherapy that ultimately produced evidenceof efficacy but initially was wrapped inobscurant language and contained unnec-essary treatment elements. The secondexample is of a treatment with an initiallywell-received theoretical foundation andevidence of efficacy, but eventually hasmaintained evidence of efficacy but its pro-posed mechanism of change has been sub-stantially challenged. Both are on the list ofESTs.

Eye Movement Desensitization andReprocessing therapy for the treatment fortrauma and anxiety (EMDR; Shapiro,1998) garnered considerable criticism, inpart, because, among other issues, one ofthe initially identified treatment compo-nents included having clients track thetherapist’s finger movements that werelearned by participating in training andcertification programs. However, dataaccumulated that the eye-movement com-ponent of the intervention was not neces-sary (Hyer & Brandsma, 1997). To manyresearchers, the important element of thetherapy was exposure and habituation.Many considered the initial explanation touse obscurant language, invoke untestablemechanisms, and resulted in monetarygain for the developer. These and other fea-tures of the therapy satisfied some thatEMDR passed the demarcation criteria forpseudoscience.

Over time an EMDR journal hasformed, and studies of EMDR were con-ducted that met criteria for inclusion as anEST. The rationale for how EMDR workedhas also changed (see references in Perkins& Rouanzoin, 2002, for some of the argu-ments about mechanisms). It is nowargued that the intervention results inchanges in adaptive information process-ing (Oren & Solomon, 2012). The mecha-nism of action still may seem to rely ontechnical, obscurant language (Oren &Solomon, 2012, pp. 200-201), but in arecent report EMDR does not appear to bean outlier in terms of clinical efficacy whencompared to several other therapies forPTSD (Cusack et al., 2016).

The point of presenting EMDR is not tosay whether it was or is pseudoscience, or isnow more normal psychological science.The point is that many treatments forPTSD make use of in vivo or imaginalexposure, and many refer to changes ininformation processing that results fromthe exposure component. Because empiri-

F O L L E T T E

January • 2018 27


cally supported treatments are not requiredto demonstrate mechanisms of change nordefine ways to identify its essential treat-ment components, nor describe ways inwhich a therapy is essentially different foranother therapy, nor the conditions underwhich the treatment and its underlyingtheory could be fundamentally challenged,the list of therapies continues to grow.More important, until such requirementsare established, there is nothing to keeppseudoscientific treatments from compet-ing. The effort to identify ESTs was laud-able. Now a more refined strategy isrequired beyond just showing that a partic-ular treatment produces change (e.g.,David & Montgomery, 2011; Follette &Beitz, 2003; Lilienfeld, 2011; Tolin, McKay,Forman, Klonsky, & Thombs, 2015). Untilthese features of treatments are defined,there is little to dissuade treatment devel-opers from adding superfluous but mar-ketable components to principle-basedtreatment elements and creating a “new”therapy where the purported mechanism iscompletely unrelated to how an interven-tion actually works.1

Now turning to cognitive therapy (CT)for depression: Beck, in his classic publica-tions (Beck, 1967; Beck, Rush, Shaw, &Emery, 1979), highlighted the role of dys-functional cognitions in depression. Thetheory highlighted the importance of thecognitive triad of a negative view of the self,the world, and the future as well as dys-functional attributional styles in depres-sion. Notions of core self-schemas evolvedand CT evolved into an intervention withan articulated mechanism for the treat-ment of depression that was plausible andwas efficacious. However, there have beensome important studies along the way thathave challenged the purported mecha-nisms underlying CT.

In 1996 Jacobson and colleagues con-ducted a component analysis of CT com-paring the behavioral activation compo-nent of CT, CT with behavioral activationand skills to modify automatic thoughts,and the full version of CT including behav-ioral activation, skills to modify automaticthoughts, plus the addition of focus on coreschemas. That study, involving 150 outpa-tients, showed that the complete version ofCT did no better than its componentsincluding behavioral activation, which had

little to do with the direct correction of cog-nitive distortions. In 2006 another studydemonstrated that behavioral activationperformed better than CT (Dimidjian et al.,2006).

In 2001, a study was published thatexamined the relationship between depres-sion, anxiety, and dysfunctional attitudes(DAs) in 521 patients receiving a 12-weekcourse of CBT (Burns & Spangler, 2001).Using structural equation modeling, thestudy examined four hypotheses:

(1) changes in DAs lead to changes indepression and anxiety during treat-ment (the cognitive mediationhypothesis); (2) changes in depressionand/or anxiety lead to changes in DAs(the mood activation hypothesis); (3)DAs and negative emotions have reci-procal causal effects on each other (thecircular causality hypothesis); and (4)there are no causal links between DAsand emotions—instead, a third vari-able simultaneously activates DAs,depression and anxiety (the “commoncause” hypothesis) . . . This commoncause accounted for all the correlationsbetween the attitude and mood vari-ables, and also appeared to mediate theeffects of psychotherapy and medica-tion on dysfunctional attitudes,depression, and anxiety. (p. 337)

This study was particularly interestingbecause the first author, having written asuccessful self-help book making use ofCBT principles (Burns, 1980), had a strongallegiance to CBT. The findings wereclearly reported and cast doubt about themechanism of change for CBT.

What were the consequences of thesefindings? Certainly cognitive behaviortherapy was not deleted from the EST list,but our understanding of how it works isnow known to be wrong or incomplete atbest. Beck’s most recent theoretic model ofdepression has changed considerably (Beck& Bredemeier, 2016). It is now a multicom-ponent model featuring several interactingsystems at different levels of analysis. It isan elaborated diathesis-stress model thatseems difficult to falsify. The language nowstates that “depression can be viewed as anadaptation to conserve energy after theperceived loss of an investment in a vital

resource . . .” (p. 596). In an article pub-lished in the APS Observer, the authors arequoted as saying, “Our model suggests thatany intervention that targets key predis-posing, precipitating, or resilience factorscan reduce risk or alleviate symptoms”[Italics in original] (Observer, 2016, April).One can take these statements to allow fora variety of interventions to claim to influ-ence the system Beck and Bredemeierdescribe.

The point of describing these lines oftheory and treatment development is thatthe route from theory to treatment or treat-ment to theory unfolds over time. Compo-nents were shown to be misunderstood orsuperfluous in both cases. Both interven-tions appeal to some combination of prac-titioners and patients. Neither are effectivein all cases. Beck’s and Bredemeier’s for-mulation is accepting of a vast number ofinterventions. Therapy designers can con-struct (fabricate) all kinds of explanationsfor how an intervention targets resilienceor any other component. The reference inthe theory to conserving energy almostinvites “energy therapies” to justify theintervention in spite of the underlyingexplanation for energy therapies beingconsidered as classic pseudoscience bymany.

Both EMDR and CBT have empiricalsupport. But what is the basis for the sup-port? It cannot be that the theoretical basiswas always (or ever was) correct. In the caseof EMDR, the treatment charitably had animprobable theoretical basis. The treat-ment had a component, the finger move-ments, that seems superfluous. Yet becauseeventually there was evidence of efficacy,the treatment persisted while the theoreti-cal explanation morphed. Over time, datawere accumulated to qualify EMDR as anEST. In the other instance, a treatment withan initially plausible theoretical basis andgood initial support was later shown to beeffective but for reasons not entirely under-stood from an initial examination of howthat intervention was thought to achieve itseffect.

At some point both treatments madetheir way onto the EST list. At some pointthe theoretical rationales for both failed. Inboth cases the treatments remain on theEST list.

The Practice of Clinical ScienceResearch

One of the features of a psychotherapythat gets labeled as pseudoscience is that itis, in principle, not subject to falsification.In an oft-cited paper Platt (1964) argued

1 Space constrains don’t allow for discussions of placebo effects (Kirsch, 2008; Stewart-Williams,2004), common factors, or the Dodo bird arguments (cf. Honyashiki, et al., 2014; Rosenzweig,1936; Wampold, 2015).


F O L L E T T E

that for the social sciences to advance at amore rapid rate, it needed to utilize stronginference tests, similar to those in physics.Such tests pit competing theories againsteach other. Ideally, one would identify twotheories that would make opposite predic-tions in a particular experimental condi-tion, run the experiment, and the resultshould be that one hypothesis is falsifiedand eliminated from further consideration.

Such experiments are actually difficultto conduct because in order to test a pri-mary hypothesis, all related assumptions orauxiliary hypotheses must be valid. If anexperiment (or clinical trial) does not per-form as predicted, it may not be that thetherapy or theory is incorrect, but thatthere may have been a problem with themeasurement instruments, training,fidelity, adherence, etc., that could accountfor the outcome rather than a problem withthe underlying theory or therapy design(see Curd & Cover, 1998, for a discussionof the Duhem Quine thesis that raises thisissue).

Consider an elementary school scienceteacher who intends to show her class thatwater boils at 100° C. During the demon-stration water boils at some other tempera-ture. Rather than concluding to the classthat known gas laws have been falsified, shewould have to determine that all the neces-sary auxiliary assumptions were met, i.e.,that the thermometer was accurate, that thewater was free of impurities, and the exper-iment was conducted at 1 standard atmos-phere of pressure. If any of those assump-tions were shown to be false, the claim thatwater boils at 100° C is never directlytested, and the theory could not be falsified.

In clinical psychology the problem isconsiderably more difficult because theauxiliary hypotheses usually involve hypo-thetical constructs of cause and hypotheti-cal constructs of effects that are not directlymeasured. Depression, adherence, compe-tence, alliance or outcomes do not have thesame potential to be directly assessed (oreven have a consensus definition) that tem-perature, water purity, or the atmosphericpressure at the time of the experiment do.

• Programmatic research. For severaldecades the gold standard for program-matic research has been the randomizedcontrolled trial (RCT). The RCT usuallyfollows earlier phases of research todemonstrate feasibility and gather the datanecessary to plan the larger-scale RCT. Oneof the decisions researchers have made wasto use highly selected participants wherethe participants were eligible for inclusion

if they met the diagnostic criteria for oneselected disorder but showed no other clin-ical problem. Such studies have been usedin RCTs to identify efficacious treatments.The logic is that if the treatment does notwork on a “pure” sample for which it wasdesigned, it is probably not likely to pro-duce an effect large enough to pursue inmore complicated cases. In many suchstudies the comparison is made betweenthe active treatment and a no treatmentcontrol and then to another active treat-ment or treatment as usual.

While this strategy has identified manyempirically supported therapies for specificdisorders, the scientific evidence has notbeen sufficient to persuade the majority ofpractitioners to use ESTs. As in evidence-based medicine, the adoption of ESTs byprimary practitioners has been limitedbecause practitioners do not treat highlyselected samples without other complicat-ing factors. Explaining with precision andscope, how to apply the science purportedto underlie the treatment when applied tomore complicated cases has not been per-suasive (see Lilienfeld, et al., 2013, for anelaborated discussion of resistance to adoptevidence-based practices).

• Has the EST effort been progressive?Recently, NIMH has recognized that areliance on the medical model, and DSM inparticular, has not served the researchagenda well. Noting that perhaps the focuson efficacy research may have been a mis-take, now effectiveness research is favored,where it is hoped that less restricted criteriafor inclusion might lead to larger, moregeneral principles of intervention andresults that will have more reported applic-ability to practitioners.

Additionally, NIMH is now interestedin identifying mechanisms of change, notjust evidence that change occurs, but how itoccurs. These changes, for better and forworse, recognize that the earlier strategyfor advancing science has not yielded theresults one might anticipate given the timeand money invested (Cuthbert & Insel,2013; Insel, 2014; Insel & Gogtay, 2014).Without statements of mechanisms ofchange that are falsifiable, judgment aboutone of the central issues in the demarcationproblem are almost impossible to adjudi-cate.

That NIMH has abandoned theresearch strategy used from the 1980s tothe beginning of this decade suggests thatthe clinical research strategy has not deliv-ered a convincing, progressive science. Thechange described in the Research Domain

Criteria, for better or worse, now seeks toidentify mechanisms of change or influ-ence (Cuthbert & Insel, 2013; Insel, 2014;National Institute on Mental Health, 2011).

How progressive have our treatmentdevelopment programs actually been?Lakatos (1974) recognized that research istypically programmatic. Research pro-grams do not initially start with fully devel-oped theories and therefore may not ini-tially be experimentally supported. Heallows for modifications to either the coretheory or the auxiliary hypotheses toaccount for contrary findings. Lakatos sug-gested that as long as modifications to thetheory (a) account for findings that pro-vided counter-evidence to the theory, and(b) provide for novel predictions notentailed in the prior version of the theory,such theory revisions are permissible andindicative of a progressive research pro-gram. Modifications to the theory that didnot accomplish both goals and were notsupported by empirical research were adhoc modifications and indicative of adegenerating program of research.

In clinical science it seems rare that atheory is refuted, though it is easy to findindividual articles attempting to do so (seeabove discussion of EMDR and CT). It isdifficult to identify the process where atherapy is falsified and discarded. Hosts ofauxiliary hypotheses are invoked to explainapparent deficiencies in the theory. Modifi-cations are offered but rarely evaluated asto whether they are ad hoc or progressive(consider the history of modifications tothe learned helpless model of depression).Perhaps this is one reason why pseudosci-entific therapies persist—there is no goodmodel for discarding a practice or definingthe acceptability of a modification to atheory or practice. As Paul Meehl oncestated of theories, “Most of them suffer thefate that General MacArthur ascribed toold generals—They never die, they justslowly fade away” (Meehl, 1978, p. 807).Tools for the evaluation of mediators andmoderators have been developed andrefined (MacKinnon, 2008), but no con-sensus exists about how to conceptuallycompare the results of such analyses withrespect to how comparably sized mediationeffects advance our understanding of howone theory prevails over another orwhether the magnitude of a mediator issufficient in size to be conceptually mean-ingful.

• Summary. Clinical science hasfocused on efficacy studies that have notbeen convincing to practitioners. The strat-


January • 2018 29

egy has been replaced by a call for moremeaningful effects and an understandingof mechanisms. As of yet, there is no agree-ment about how to identify when a theoryhas been refuted. Without being able todefine the criteria for discarding a theoryon the basis of evidence, it makes it difficultto argue that scientific and pseudoscientificpractices actually meet different standards.Currently the difference between a theorybeing not falsifiable versus not knowingwhen or how to falsify theories that rest onhypothetical constructs may be a distinc-tion without a difference.

Social FactorsSo far commentary on our failure to

mount a powerful methodological attackon pseudoscience has focused on ourresearch and analytic shortcomings. How-ever, there are social influences that under-mine the perceived value of making use ofevidence-based practices.

• Financial support. Many products andpractices that are considered to be exam-ples of pseudoscience fall under the rubricof complementary alternative medicine(CAM). There is considerable variability incosts of complementary alternative treat-ments. The Affordable Care Act does notallow insurance companies to discriminateagainst health providers with a recognizedstate license. If an insurance policy pro-vides for mental health services, then a con-sumer has the possibility of finding alicensed practitioner to deliver nontradi-tional therapy and get reimbursed. Reim-bursement varies by state. Even whereinsurance may cover some licensed service,it does not allow for reimbursement fornonlicensed treatments such as aromather-apy, Ayurveda, cryotherapy, reflexology,vibroacoustic therapy, crystal therapy, andthe like. However, lax restrictions implythat all permitted choices share the sameevidence for efficacy. Of course, they donot. There are no data on exactly howmuch money people spend on pseudosci-entific alternatives to psychotherapy or forwhat problems people seek such servicesthat may be out of the purview of ESTs. Forcomplementary or alternative medicine,data do indicate that consumer out-of-pocket spending is about $34B or 1.5% oftotal health care expenditures in the U.S.Approximately 2/3 of those expenses werefor self-care purchases (NIH NationalCenter for Complementary and IntegrativeHealth, 2007). In addition to public policymaking it appear that all reimbursed ser-vices are equal, the fact that consumers pay

for these services establishes an expectationthat they will be beneficial.

• Reasons for seeking alternative care.There have been attempts to identify rea-sons why patients seek CAM treatments. In1994, a small sample of physicians weresurveyed in Washington, New Mexico, andIsrael. That study reported that in the lastyear, 60% of physicians made referral toalternative providers. The referrals foralternative care included spinal manipula-tion, naturopathy, spiritual healing, andmovement therapy, among other forms ofinterventions. The rationale for referralsincluded patient requests, cultural beliefs,failure of conventional treatment, andphysician beliefs that patients had nonor-ganic disease (Borkan, Neher, Anson, &Smoker, 1994). In a 1996 study that utilizedphone interviews, a sample of CAM utiliz-ers were characterized as unconventionaland reported a lack of confidence in con-ventional medical treatment (McGregor &Peay, 1996).

The CDC National Health InterviewSurvey interviewed over 30,000 U.S. adultsand examined the utilization of 27 CAMtreatments (Barnes, Powell-Griner,McFann, & Nahin, 2004). The studyreported that 36% of adults used someform of CAM treatment in the last 12months (62% when prayer was included).Mind-body interventions were among the10 most common CAM therapies utilizedwithin the last 12 months of the data col-lection. Respondents with anxiety ordepression were the most frequently iden-tified users of CAM for those who self-identified as having a mental disorder.CAM users reported a variety of reason forusing CAM treatments including beliefthat a combined approach would be useful,conventional medical professionals sug-gested it, belief that it would be interesting,cost, and believing conventional treatmentwould not be helpful.

Another study by Kessler and col-leagues utilized a nationally representativephone sample with over 2,000 respondents(Kessler et al., 2001). Two findings wereparticularly interesting. First, a majority ofthose with “anxiety attacks” and “severedepression” reported the use of CAM treat-ments. Second, the proportion of anxiousand depressed respondents who reportedCAM treatments “very helpful” was com-parable to those who rated conventionaltreatment the same. The authors state that,“No evidence was found for significantvariation in the perceived helpfulness ofcomplementary and alternative therapies

on the basis of whether the respondent alsoused conventional therapy” (p. 291).

Reports on the efficacy of treatment forboth depression and anxiety in the psy-chotherapy literature are variable, but areasonable estimate is that about half thepeople respond significantly and half donot. Clinical science cannot yet provideoutcome data so convincing as to negatethe demand for alternatives. It does notseem likely that thoughtful instruction tothe public will help them discern thethreats to validity and heuristic errors thateven clinicians make when assessing actualversus spurious therapeutic effectiveness(Lilienfeld, Ritschel, Lynn, Cautin, & Latz-man, 2014).

• Summary. In addition to problems inbeing able to mount a strong theory-basedargument against the use of alternativetreatments, there are social and culturalfactors that support the continued use ofsuch interventions. Social policy makesmany practices seem equivalent; profes-sionals may actually refer to alternativepractitioners; there is distrust of conven-tional treatments; combined treatmentsmay be presumed to offer the best of bothworlds; alternative treatments may addressimportant issues consumers believe are notaddressed by more conventional treat-ments.

ConclusionThe application of criteria to identify

the differences between clinical science andpseudoscience have been noted. One ofthose important features is the ability to fal-sify the theoretical basis for an interven-tion. There is nothing in this paper thatprevents one from identifying absurd prac-tices. In clinical psychology the predomi-nant research strategy has focused on effi-cacy and not tests of the underlying theoryupon which the intervention is based. Evenif we can describe a method for rejecting aclinical theory, with few exceptions, wehave not done so. That means that con-sumers look for perceived benefits and notfor scientific justification when choosing atreatment. Social influences support thenotion that treatments are equivalent.Until we take on the task of definingwhether a research program is progressiveor not, we will be lacking the strongestargument against consumers using pseu-doscientific interventions.


F O L L E T T E

ReferencesAdvocates for Children in Therapy.

(2017). Opposing abusive and unvali-dated psychotherapy. Retrieved October31, 2017, from http://www.chil-drenintherapy.org/index.html

American Psychiatric Association. (1980).Diagnostic and statistical manual ofmental disorders (3rd ed.). Washington,DC: Author.

American Psychological Association.(2017). Ethical principles of psychologistsand code of conduct (2002, AmendedJune 1, 2010 and January 1, 2017). fromhttps://www.apa.org/ethics/code/ethics-code-2017.pdf

Barnes, P. M., Powell-Griner, E., McFann,K., & Nahin, R. L. (2004). Complemen-tary and alternative medicine use amongadults: United States, 2002. Seminars inIntegrative Medicine, 2, 54-71.

Beck, A. T. (1967). Depression: Causes andtreatment. Philadelphia: University ofPennsylvania Press.

Beck, A. T., & Bredemeier, K. (2016). Aunified model of depression: Integratingclinical, cognitive, biological, and evolu-tionary perspectives. Clinical Psychologi-cal Science, 4, 596-619.

Beck, A. T., Rush, A. J., Shaw, B. F., &Emery, G. (1979). Cognitive therapy ofdepression. New York: Guilford.

Beyerstein, B. L. (2001). Fringe psy-chotherapies: The public at risk. The Sci-entific Review of Alternative Medicine, 5,70-79.

Bonow, J. T., & Follette, W. C. (2009).Beyond values clarification: Addressingclient values in clinical behavior analysis.Behavior Analyst, 32, 69-84.

Borkan, J., Neher, J. O., Anson, O., &Smoker, B. (1994). Referrals for alterna-tive therapies. Journal of Family Practice,39, 545-551.

Burns, D. D. (1980). Feeling good: The newmood therapy. New York: Morrow.

Burns, D. D., & Spangler, D. L. (2001). Dochanges in dysfunctional attitudes medi-ate changes in depression and anxiety incognitive behavioral therapy? BehaviorTherapy, 32, 337-369.

Chaffin, M., Hanson, R., Saunders, B. E.,Nichols, T., Barnett, D., Zeanah, C., ...Miller-Perrin, C. (2006). Report of theAPSAC Task Force on Attachment Ther-apy, Reactive Attachment Disorder, andAttachment Problems. Child Maltreat-ment, 11, 76-89.

Chambless, D. L. (2015). Bringing identifi-cation of empirically supported treat-ments into the 21st century. Clinical Psy-chology: Science & Practice, 22, 339-342.

Chambless, D. L., Baker, M., Baucom, D.H., Beutler, L. E., Calhoun, K. S., Crits-

Christoph, P., ... Woody, S. A. (1998).Update on empirically validated thera-pies, II. The Clinical Psychology, 51, 3-16.

Chambless, D. L., & Hollon, S. D. (1998).Defining empirically supported thera-pies. Journal of Consulting and ClinicalPsychology, 66, 7-18.

Chambless, D. L., & Ollendick, T. H.(2001). Empirically supported psycho-logical interventions: Controversies andevidence. Annual Review of Psychology,52, 685-716.

Chan, M.-p. S., Jones, C. R., Jamieson, K.H., & Albarracín, D. (2017). Debunking:A meta-analysis of the psychological effi-cacy of messages countering misinforma-tion. Psychological Science, 28, 1531-1546.

Curd, M., & Cover, J. (1998). The DuhemThesis and the Quine Thesis. In M. Curd& J. Cover (Eds.), Philosophy of science:The central issues (pp. 302-319). NewYork: Norton.

Cusack, K., Jonas, D. E., Forneris, C. A.,Wines, C., Sonis, J., Middleton, J. C.,...Gaynes, B. N. (2016). Psychologicaltreatments for adults with posttraumaticstress disorder: A systematic review andmeta-analysis. Clinical PsychologyReview, 43, 128-141.

Cuthbert, B. N., & Insel, T. R. (2013).Toward the future of psychiatric diagno-sis: The seven pillars of RDoC. BMCMedicine, 11, 1-8.

David, D., & Montgomery, G. H. (2011).The scientific status of psychotherapies:A new evaluative framework for evi-dence‐based psychosocial interventions.Clinical Psychology: Science and Practice,18, 89-99.

Dimidjian, S., & Hollon, S. D. (2010). Howwould we know if psychotherapy wereharmful? American Psychologist, 65, 21-33.

Dimidjian, S., Hollon, S. D., Dobson, K. S.,Schmaling, K. B., Kohlenberg, R. J.,Addis, M. E., ... Jacobson, N. S. (2006).Randomized trial of behavioral activa-tion, cognitive therapy, and antidepres-sant medication in the acute treatment ofadults with major depression. Journal ofConsulting and Clinical Psychology, 74,658-670.

Follette, W. C. (1995). Correcting method-ological weaknesses in the knowledgebase used to derive practice standards. InS. C. Hayes, V. M. Follette, R. M. Dawes& K. E. Grady (Eds.), Scientific standardsof psychological practice: Issues and rec-ommendations (pp. 229-247). Reno, NV:Context Press.

Follette, W. C., Bach, P. A., & Follette, V.M. (1993). A behavior analytic view ofpsychological health. The Behavior Ana-lyst, 16, 303-316.

Follette, W. C., & Beitz, K. (2003). Addinga more rigorous scientific agenda to theempirically supported treatment move-ment. Behavior Modification. SpecialIssue: Empirically supported treatments,27, 369-386.

Follette, W. C., & Houts, A. C. (1996).Models of scientific progress and the roleof theory in taxonomy development: Acase study of the DSM. Journal of Con-sulting and Clinical Psychology, 64, 1120-1132.

Follette, W. C., Houts, A. C., & Hayes, S.C. (1992). Behavior therapy and the newmedical model. Behavioral Assessment,14, 323-343.

Honyashiki, M., Furukawa, T. A., Noma,H., Tanaka, S., Chen, P., Ichikawa, K., etal. (2014). Specificity of CBT for Depres-sion: A Contribution from MultipleTreatments Meta-analyses. CognitiveTherapy and Research, 38, 249-260.

Hyer, L., & Brandsma, J. M. (1997). EMDRminus eye movements equals good psy-chotherapy. Journal of Traumatic Stress,10, 515-522.

Insel, T. R. (2014). The NIHM ResearchDomain Criteria (RDoC) Project: Preci-sion medicine for psychiatry. The Ameri-can Journal of Psychiatry, 171, 395-397.

Insel, T. R., & Gogtay, N. (2014). NationalInstitute of Mental Health clinical trials:New opportunities, new expectations.JAMA Psychiatry, 71, 745-746.

Jacobson, N. S., Dobson, K. S., Truax, P.A., Addis, M. E., Koerner, K., Gollan, J.K., et al. (1996). A component analysis ofcognitive-behavioral treatment fordepression. Journal of Consulting andClinical Psychology, 64, 295-304.

Jacobson, N. S., Follette, W. C., & Reven-storf, D. (1984). Psychotherapy outcomeresearch: Methods for reporting variabil-ity and evaluating clinical significance.Behavior Therapy, 15, 336-352.

Kaslow, N. J. (2015). Translating psycho-logical science to the public. AmericanPsychologist, 70, 361-371.

Kazdin, A. E. (2007). Mediators and mech-anisms of change in psychotherapyresearch. Annual Review of Clinical Psy-chology, 3, 1-27.

Kazdin, A. E. (2014). Moderators, media-tors and mechanisms of change in psy-chotherapy. In W. Lutz, S. Knox, W.Lutz, & S. Knox (Eds.), Quantitative andqualitative methods in psychotherapyresearch (pp. 87-101). New York, NY:Routledge/Taylor & Francis Group.

Kessler, R. C., Soukup, J., Davis, R. B.,Foster, D. F., Wilkey, S. A., Van Rompay,M. I., et al. (2001). The use of comple-mentary and alternative therapies to treatanxiety and depression in the United

January • 2018 31


States. American Journal of Psychiatry,158, 289-294.

Kirk, S. A., & Kutchins, H. (1992). The sell-ing of DSM: The rhetoric of science in psy-chiatry. New York: Aldine de Gruyther.

Kirsch, I. (2008). Challenging receivedwisdom: Antidepressants and theplacebo effect. McGill Journal of Medi-cine, 11, 219-222.

Lakatos, I. (1974). Falsification and themethodology of scientific research pro-grammes. In I. Lakatos & A. Musgrave(Eds.), Criticism and the growth of knowl-edge (pp. 91-196). Cambridge: Cam-bridge University Press.

Lambert, M. J. (2017). Maximizing psy-chotherapy outcome beyond evidence-based medicine. Psychotherapy and Psy-chosomatics, 86, 80-89.

Laudan, L. (1983). The demise of thedemarcation problem. In R. S. Cohan &L. Laudan (Eds.), Physics, Philosophy,and Psychoanalysis (pp. 111-127). Dor-drecht: Reidel.

Lilienfeld, S. O. (2011). Distinguishing sci-entific from pseudoscientific psychother-apies: Evaluating the role of theoreticalplausibility, with a little help from Rev-erend Bayes. Clinical Psychology: Science& Practice, 18, 105-112.

Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M.(2015a). Science and pseudoscience inclinical psychology: Initial thoughts,reflections, and considerations. In S. O.Lilienfeld, S. J. Lynn & J. M. Lohr (Eds.),Science and pseudoscience in clinical psy-chology (2nd ed., pp. 1-16). New York:Guilford Press.

Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M.(Eds.). (2003). Science and pseudosciencein clinical psychology. New York: Guil-ford Press.

Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M.(Eds.). (2015b). Science and pseudo-science in clinical psychology. New York:Guilford Press.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J.,Cautin, R. L., & Latzman, R. D. (2013).Why many clinical psychologists areresistant to evidence-based practice: Rootcauses and constructive remedies. Clini-cal Psychology Review, 33, 883-900.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J.,Cautin, R. L., & Latzman, R. D. (2014).Why ineffective psychotherapies appearto work: A taxonomy of causes of spuri-ous therapeutic effectiveness. Perspectiveson Psychological Science, 9, 355-387.

MacKinnon, D. P. (2008). Introduction tostatistical mediation analysis. New York:Lawrence Erlbaum.

Mahner, M. (2013). How to demarcateafter the (alleged) demise of the demarc-tion problem. In M. Pigliucci & M.Boudry (Eds.), Reconsidering the demar-

cation problem: Philosophy of pseudo-science (pp. 29-43). Chicago: Universityof Chicago Press.

McGregor, K. J., & Peay, E. R. (1996). Thechoice of alternative therapy for healthcare: Testing some propositions. SocialScience & Medicine, 43, 1317-1327.

Meehl, P. E. (1978). Theoretical risks andtabular asterisks: Sir Karl, Sir Ronald, andthe slow progress of soft psychology.Journal of Consulting and Clinical Psy-chology, 46, 806-834.

Mercer, J. (2014). Alternative psychothera-pies: Evaluating unconventional mentalhealth treatments. Lanham, MD:Rowman & Littlefield.

National Institute on Mental Health.(2011). NIMH Researach Domain Crite-ria (RDoC). Retrieved fromhttp://www.nimh.nih.gov/research-fund-ing/rdoc/nimh-research-domain-crite-ria-rdoc.shtml

NIH National Center for Complementaryand Integrative Health. (2007, September24, 2017). The use if complementary andalternative medicine in the United States:Cost data. Retrieved Nov 13, 2017, fromhttps://nccih.nih.gov/news/camstats/costs/costdatafs.htm - over

Observer. (2016, April). Beck proposesintegrative theory of depression.Retrieved November 7, 2017, fromhttps://www.psychologicalscience.org/observer/beck-proposes-an-integrative-theory-of-depression

Oren, E., & Solomon, R. (2012). EMDRtherapy: An overview of its developmentand mechanisms of action. RevueEuropéenne de PsychologieAppliquée/European Review of AppliedPsychology, 62, 197-203.

Perkins, B. R., & Rouanzoin, C. C. (2002).A critical evaluation of current viewsregarding eye movement desensitizationand reprocessing. Journal of Clinical Psy-chology, 58, 77-97.

Pigliucci, M. (2013). The demarctionproblem: A (belated) response to Laudan.In M. Pigliucci & M. Boudry (Eds.), Phi-losophy of pseudoscience: Reconsideringthe demarcation problem (pp. 9-28).Chicago: University of Chicago Press.

Platt, J. (1964). Strong inference. Science,146, 347-353.

Pratkanis, A. R. (1995). How to sell a pseu-doscience. Skeptical Inquirer, 19-25.

Pratkanis, A. R. (Ed.). (2007). The scienceof social influence: Advances and futureprogress. New York: Psychology Press.

Rao, T. S. S., & Andrade, C. (2011). TheMMR vaccine and autism: Sensation,refutation, retraction, and fraud. IndianJournal of Psychiatry, 53, 95-96.

Rosenzweig, S. (1936). Some implicitcommon factors in diverse methods of

psychotherapy. American Journal ofOrthopsychiatry, 6, 412-415.

Schermer, M. (2002). How thinking goeswrong: Twenty-five fallacies that lead usto believe weird things Why people believeweired things: Pseudoscience, superstition,and other confusions of our time (pp. 44-62). New York: Henry Holt.

Shapiro, F. (1998). Eye movement desensi-tization and reprocessing (EMDR):Accelerated information processing andaffect-driven constructions. Crisis Inter-vention & Time-Limited Treatment, 4,145-157.

Singer, M. T., & Lalich, J. (1996). "Crazy"therapies: What are they? Do they work?San Francisco, CA: Jossey-Bass.

Stewart-Williams, S. (2004). The placebopuzzle: Putting together the pieces.Health Psychology, 23, 198-206.

Thornicroft, G., & Slade, M. (2014). Newtrends in assessing the outcomes ofmental health interventions. World Psy-chiatry, 13, 118-124.

Tolin, D. F., McKay, D., Forman, E. M.,Klonsky, E. D., & Thombs, B. D. (2015).Empirically supported treatment: Rec-ommendations for a new model. ClinicalPsychology: Science and Practice, 22, 317-338.

Wakefield, A. J., Murch, S. H., Anthony,A., Linnell, J., Casson, D. M., Malik, M.,... Walker-Smith, J. A. (1998). Ileal-lym-phoid-nodular hyperplasia, non-specificcolitis, and pervasive developmental dis-order in children. Lancet, 351, 637-641.

Wampold, B. E. (2015). How importantare the common factors in psychother-apy? An update. World Psychiatry, 14,270-277.

. . .

The author has no funding or conflicts ofinterest to disclose.

Correspondence to William C. Follette,Ph.D., Department of Psychology/298,University of Nevada, Reno, Reno, NV89557-0062; [email protected]


IN CLINICAL PSYCHOLOGY, implementa-tion research has focused on sustained useof evidence-based psychosocial treatments(EBPT) by testing different implementa-tion strategies and models to improvetreatment delivery and patient outcomes.However, an integral and foundational stepof implementation has been understudiedand frequently overlooked until recently:the de-implementation of less effectivepractices. De-implementation is defined asceasing the use of previously implementedpractices (Niven et al., 2015). Fortunately,strategies for de-implementation areemerging, which pave the way for imple-mentation of EBPTs. We suggest that thisrecent work can be extended to the chal-lenge of reducing harmful, pseudoscientificpractices. In this article, we will address thedanger of harmful, pseudoscientific prac-tice and how it can inhibit the implementa-tion of EBPTs. Using de-implementationmodels from the medical sector and clinicalpsychology implementation research, wewill also briefly outline the steps required tode-implementing harmful practice in themental health sector (e.g., Elshaug, Watt,Moss & Hiller, 2009; Niven et al.).

The Problem: Harmful,Pseudoscientific Practices

Before embarking on the process of de-implementing harmful, pseudoscientificpsychotherapies, it is critical to definepseudoscience. A pseudoscientific practiceis one that uses vague and broad scientificlanguage yet falsely promotes the reliabil-ity, efficacy, or effectiveness of the practice(Hansson, 2013). Lilienfeld and his col-leagues (2015) go further to point out spe-cific tendencies of pseudoscience: “1. Anoveruse of ad hoc hypotheses designed toimmunize claims from falsification… 2.Absence of self-correction… 3. Evasion ofpeer review… 4. Emphasis on confirma-tion rather than refutation… 5. Reversed

burden of proof… 6. Absence of connectiv-ity… 7. Overreliance on testimonial andanecdotal evidence… 8. Use of obscuran-tist language… 9. Absence of boundaryconditions… [and/or] 10. The mantra ofholism” (pp. 7-10). This article focuses onpseudoscientific practices that include oneor more of the above tendencies. Morespecifically, we define a harmful, pseudo-scientific practice as one that has empiricalevidence of long-term physical or emo-tional harm on patients or other individu-als in the patients’ lives (Lilienfeld, 2007).

Conversion therapy, a treatment usedto change sexual orientation in the mid-1900s, is a common example of a harmful,pseudoscientific practice. The AmericanPsychological Association (APA) and theNational Association of Social Workers,among other associations, deemed conver-sion therapy to do more harm than good topatients (APA, 2009; Jenkins & Johnston,2004). For example, conversion therapy isunsuccessful 70% of the time and fre-quently leads to depression, avoidance ofintimacy, de-masculinization, and loss ofreligion. Implementing evidence-basedpsychosocial treatments can serve as a solu-tion to stop the use of harmful, pseudosci-entific practice like conversion therapy(Hansson, 2013). However, to forego thede-implementation of the harmful practicebefore implementing a new EBPT mayresult in unsuccessful implementation(Niven et al., 2015). For example, if a clini-cian’s case conceptualization or selection ofinterventions remained more consistentwith the theory and practices of conversiontherapy, while attempting to deliver anEBPT, the EBPT would be unlikely toachieve the desired results, and their confi-dence in the EBPT would remain low.

A Potential Solution:De-Implementation of Harmful,

Pseudoscientific PracticesAlthough de-implementation is most

commonly studied within the realm of low-value medical treatments, the existing de-implementation frameworks can be used toguide de-implementation within mentalhealth (e.g., Elshaug, et al., 2009; Henshall,Schuller, & Mardhani-Bayne 2012; Ibar-goyen-Roteta, Gutiérrez-Ibarluzea, &Asua, 2010; Montini & Graham, 2015;Niven et al., 2015; Polisena et al., 2013;Prasad & Ioannidis, 2014). Although wefocus in this article on harmful, pseudosci-entific practices, the guidelines below canbe used for de-implementation of ineffec-tive practices that are not necessarily pseu-doscientific or harmful.

Guideline 1: Identify and Prioritize theHarmful, Pseudoscientific Practice

Before implementation leaders canbegin to de-implement a harmful, pseudo-scientific practice, they need to identifywhich practice(s) to target. A high-levelanalysis should initially take place whereimplementation leaders conduct a meta-analysis of existing data on therapeutic out-comes with a focus on articles that provideevidence of a harmful treatment (i.e.,declining patient outcomes, increase ofsymptoms after treatment, etc.; Lilienfeld,2007). After an exhaustive list of harmful,pseudoscientific practices is created, thepractices should be reviewed to determinewhich to target first. The following areaspects to consider when deciding whichharmful, pseudoscientific practice to de-implement first:

• Evidence base. Priority should be givento a harmful practice with the most datadocumenting patient harm (Elshaug et al.,2009; Henshall et al., 2012; Ibargoyen-Roteta et al., 2010). In the case of psy-chotherapeutic practices, harm can includeworsened patient outcomes, and/or emo-tional and physical harm to a patient’sfamily or friends. Moreover, it is integral tothe prioritization of de-implementation toconsider the populations in which theharm is documented, while focusing onresearch evidence that includes partici-pants with demographics most related tothe system’s specific patient population.Prioritization based on the existing evi-dence and highest impact changes is key tosuccessful de-implementation of harmful,pseudoscientific practice, as it is importantto be realistic about the amount of change

De-implementation of Harmful, Pseudo-Scientific Practices: An Underutilized Stepin Implementation ResearchClara Johnson, National Center for PTSD

Shannon Wiltsey-Stirman, National Center for PTSD and StanfordUniversity

Heidi La Bash, National Center for PTSD

D E I M P L E M E N T A T I O N O F P S E U D O S C I E N T I F I C P R A C T I C E S

January • 2018 33

that is feasible in the short term within anorganization, particularly if multiple newinterventions will need to be introduced toreplace existing practices.

• Severity of functional impairment.Implementation leaders and other stake-holders must consider the degree to whichthe functional impairment of the patientsengaged in the harmful, pseudoscientificpractice interferes with daily life (Elshauget al., 2009; Ibargoyen-Roteta et al., 2010;Polisena et al., 2013). Leaders should focusde-implementation efforts on the practicesdelivered to patients who experience thelowest quality of life.

• Financial burden and resource alloca-tion. Priority should be given to harmful,pseudoscientific practices that pose anextreme financial burden on patients, clin-icians, clinics, and/or insurance companiescompared to an alternative EBPT (Elshauget al., 2009; Henshall et al., 2012; Polisenaet al., 2013). Generally speaking, it is morelikely that stakeholders will support de-implementation efforts if implementationleaders can report on expected savings. Ifthe alternative EBPT costs clinicians lessmoney to train, patients less money tocomplete, and clinics less money to pro-vide, the successful de-implementation ofthe old, harmful, pseudoscientific practicewill be more likely. While there are costsassociated with all treatments, some thera-pies make more sense to de-implementbecause of the extent of the costs. For amedical example, the radical mastectomywas a popular yet expensive surgery forbreast cancer in the late 1800s to early1900s (Montini & Graham, 2015).Researchers later discovered other saferand lower cost methods of removing suchtumors. Hospitals and facilities were able tosuccessfully de-implement radical mastec-tomies because they recognized financialincentives of the change (Montini &Graham).

• Policy mandates. All individualsinvolved in the de-implementation processshould consider harmful practices that runcounter to policy, mandates or clinicalpractice guidelines a priority to increasepatient, clinician, and clinic buy-in forchange (Elshaug et al., 2009; Polisena et al.,2013). While external motivation may notalways be the best way to promote change,mandating change can still help persuadeclinicians to de-implement the practicesthat cause harm to patients.

• Existing alternatives. As we will dis-cuss later, presenting an alternative EBPTis one of the most useful tools in increasingproponent buy-in and sustaining the de-implementation of a harmful, pseudoscien-tific practice.

Guideline 2: Increase Proponent Buy-inAfter leaders identify the harmful, pseu-

doscientific practice to de-implement, theyshould begin to increase provider buy-infor the change. As described by Niven andcolleagues (2015), the engagement of stake-holders is a critical step in the de-imple-mentation process. First, implementationleaders must ascertain what pressures andbarriers exist for proponents of the pseudo-scientific practice before intervening. Evenif some clinics and systems mandate apolicy change, it is important to engageindividual clinicians. We suggest firstlearning about the core values of clinicianswho provide the identified harmful, pseu-doscientific practice. These values are typi-cally related to providing patients with thebest possible care to increase the likelihoodof recovery. Once those leading the imple-mentation effort identify the core values,they can frame the need to de-implementthe therapy in terms that reflect thosevalues.

To further increase clinician buy-in,Lilienfeld and colleagues (2013) recom-mend involving clinicians in the dissemi-nation of information regarding the pseu-doscientific nature of the targeted practiceand to present the alternative EBPT.Researchers can involve clinicians inreviewing research and evaluation data (onboth pseudoscientific and evidence-basedtreatments) to increase the clinician’s basicunderstanding of the effectiveness of thecurrent practice. Moreover, researchersneed to present research findings that pointto a practice’s lack of evidence in an easy-to-understand manner. Often cliniciansresist de-implementing pseudoscientificpractice or implementing an EBPT becauseof the complexity in which a researcherpresents the findings (Lilienfeld et al.).

Developing a system of patient outcomemeasurement that feels relevant to the clin-icians and their patients may increase clin-ician support to de-implement certainharmful practices. Clinicians can use thissystem to see firsthand whether the prac-tice is working. In the case of harmful,pseudoscientific practice, the clinician willnotice that the patient is not improvingbased on measured outcomes. Systematicmeasurements can help clinicians recog-nize their current strategies are not yielding

the desired effects, and can facilitate sup-port for de-implementing the practice anda willingness to try something new.

Guideline 3: Identify Barriers and Facil-itators to De-implementation

Another guideline to consider is identi-fying barriers and facilitators to the de-implementation of the harmful practicewithin a specific clinic to help inform de-implementation efforts. We suggest incor-porating all types of stakeholders in thisstep to gather facilitators and barriers spe-cific to different levels (e.g., patient level,clinician level, facility level, etc.). Bringingin the perspective of each stakeholder willachieve two goals: (a) to increase supportand understanding of the change early onand (b) to measure feasibility across differ-ent levels. For example, patients can bestexplain their needs within a therapeuticcontext, while clinicians can express theirgoals and concerns, and clinic leaders canbring up structural issues within the clinicthat may interfere with de-implementa-tion. Looping individual stakeholders in atthis point can be extremely helpful inbuilding support for change throughoutthe entire de-implementation and imple-mentation process. Frameworks and mea-sures exist to guide this assessment (cf.Aarons et al., 2011; Rabin et al., 2016).Once the stakeholders identify a compre-hensive understanding of barriers andfacilitators to de-implement, the imple-mentation leaders then need to select,tailor, and implement the de-implementa-tion intervention depending directly on thestakeholders, facility, clinicians, andpatient-level needs within a specific clinic(Powell et al., 2017).

Guideline 4: Develop a Sustaining De-Implementation Strategy

The next step is to determine a de-implementation strategy or, more likely, aset of strategies. Below we provide possiblestrategies to develop and sustain the de-implementation of a harmful, pseudoscien-tific practice.

• Strategy 1: Implement an alternativeEBPT. Part of the de-implementationprocess is to give hope to clinicians that abetter alternative exists. If implementationleaders do not present a new practice withevidence, clinicians will likely feel no needto stop the old practice, and may in fact feelpressure to offer something else, defaultingto the practices they know best. Theoptions for new practices should be pre-sented in an easy-to-understand manner,

J O H N S O N E T A L .


and then compared with the old practice interms of the clinic's goals and mission. If anevidence-based alternative does not existfor the specific clinic-level needs, it may bewise to collect practice-level data or partneron research to test and refine practices thatare identified as the best available alterna-tives, perhaps benchmarking against previ-ous program evaluation data on the pseu-doscientific practice (see Strategy 3).

• Strategy 2: Provide consultation.Implementation leaders should provide aspace in which clinicians can give feedbackand ask questions about the de-implemen-tation process. By creating this space, clin-icians will feel involved in the process andideally will align with the need to de-imple-ment the harmful, pseudoscientific prac-tice. Those tasked with implementation ofeffective practices should also provide con-sultation on how to de-implement the ther-apy and replace it with the new alternative.For example, clinicians could meet once aweek and present their cases to understandwhat to do in place of the old pseudoscien-tific practice. Consultation is often studiedunder the context of training clinicians inEBPTs (Beidas, Edmunds, Marcus, &Kendall, 2012; Nadeem, Gleacher, &Beidas, 2013). Nonetheless, implementa-tion researchers should incorporate con-sultation in the earlier phases of de-imple-mentation to enhance the implementationprocess. Ongoing consultation and sup-port, or the development of internalresources to support evidence-based prac-tice, is likely to be necessary to ensure thatthe practice changes are sustained.

• Strategy 3: Evaluate patient outcomes.Comparing pre- and postpatient outcomesmay help clinicians see the benefits first-hand of de-implementing the harmful,pseudoscientific practice. Before doingthis, researchers may need to developbetter methods of outcome measurement.This should take place in two ways: at theclinician level and at the clinic or systemlevel. For example, clinicians who seepatients with anxiety disorders could usethe Beck Anxiety Inventory (BAI) or Gen-eral Anxiety Disorder 7-item (GAD-7) tocompare patient outcome before and afterchanging from a pseudoscientific practiceto an EBPT. Measures of quality of life andfunctioning and client satisfaction are alsoimportant to examine. Using outcometracking to evaluate patient outcomes givesclinicians an empirical way to see positivechange among patients. Measurement-based care may also allow clinicians to feel

in control, in that they could change thetreatment plan if the measures revealedpatient improvement, worsening, or nochange.

Clinic-wide outcome monitoring willlikely help all stakeholders see the improve-ment in patient outcome before and afterthe de-implementation of the pseudoscien-tific practice. This will also let clinicians seethat the clinic leadership is observing thedegree of patient improvement and theclinician’s role in achieving the improve-ment. By collecting and presenting the pre-post comparison data, clinics can furtherevaluate the extent to which of the de-implementation improves patient out-comes. If the results show little improve-ment or an increase in patient symptomsand other valued outcomes, clinics can usethe data to rethink and reevaluate theimplementation strategies or the new prac-tices that have been identified.

• Strategy 4: Incent the delivery of effec-tive alternatives. Emerging evidence sug-gests that clinicians and clinics are morelikely to implement EBPTs when theyreceive external rewards for doing so(Andrzejewski, Kirby, Morral, & Iguchi,2001; Carise, Cornely, & Gurel, 2002). At apolicy level, possible strategies to facilitatede-implementation include incentives suchas preferential contracting with agenciesthat use EBPTs (McLellan, Kemp, Brooks,& Carise, 2008), block grants to fund initialEBPT implementation, and enhancedreimbursement rates for EBPTs (Magna-bosco, 2006). These incentives mightincrease an organization leader’s supportfor discontinuation of pseudoscientificpractices and a transition to EBPTs.Research also suggests that incenting clini-cians to deliver EBPTs can lead toimproved adherence and intention todeliver EBPTs (Garner et al., 2012). Suchrewards could be contingent on demon-strating that EBPTs have in fact replacedharmful practices.

Guideline 5: Sustain the ImplementedEffective Treatment

To ensure the permanent de-imple-mentation of a harmful, pseudoscientificpractice, implementation leaders need tofocus their efforts on sustaining the imple-mentation of the alternative EBPT. If clini-cians begin to drift from the EBPT thatreplaced the de-implemented practice, theymay fall back on old harmful practices.Ongoing support and the use of implemen-tation strategies to promote and sustainnew and effective practices are essential to

de-implementation efforts. Overviews andframeworks for implementation and sus-tainability will provide a richer under-standing of the process of implementation(Aarons et al., 2011; Damschroder et al.,2009; Kilbourne, Nuemann, Puncus, Bauer& Stall, 2007; Stirman, Gutner, Langdon, &Graham, 2016).

ConclusionAlthough relatively understudied

within the mental health sector, de-imple-mentation of harmful, pseudoscientificpractice is a critical initial step in a success-ful implementation process of an EBPT.Failing to attend to harmful, pseudoscien-tific practice within a clinic or system canlead to an eventual return to the harmfulpractice. Clinicians should therefore workto cease the use of harmful, pseudoscien-tific practices to achieve the overarchinggoal of therapy: improve patient outcomes.The present article highlights guidelinesand frameworks based from medicalresearch that may guide the de-implemen-tation of pseudoscientific practice: (a) iden-tify and prioritize the harmful, pseudosci-entific practice; (b) increase proponentbuy-in; (c) identify barriers and facilitatorsof the de-implementation; (d) develop asustaining de-implementation strategy;and (e) sustain the implemented effectivetreatment. These guidelines can also beused to support de-implementation ofthose practices that are not pseudoscien-tific or harmful, but less effective thanestablished EBPTs. We advise that any timeEBPTs are to be implemented, that boththe less effective, and the potentially harm-ful existing practices be identified, and thatimplementation efforts focus on strategiesfor de-implementation of these practices aswell as implementation of new practices.

A major limitation of the present articleis the lack of research specific to de-imple-mentation of harmful, pseudoscientificpractice within the field of clinical psychol-ogy. We recognize the need to study suchguidelines and frameworks within themental health context. We encourage fur-ther attention to de-implementation inboth research and practice contexts, as itmay be necessary to ensure the delivery ofeffective care.

ReferencesAarons, G. A., Hurlburt, M., & Horwitz, S.

M. (2011). Advancing a conceptualmodel of evidence-based practice imple-mentation in public service sectors.Administration and Policy in Mental

January • 2018 35

D E I M P L E M E N T A T I O N O F P S E U D O S C I E N T I F I C P R A C T I C E S

Health and Mental Health ServicesResearch, 38(1), 4-23.

American Psychological Association.(2009). Report of the American Psycholog-ical Association’s Task Force on Appropri-ate Therapeutic Responses to Sexual Ori-entation. Washington, DC: AmericanPsychological Association. Retrievedfrom http://www.apa.org/pi/lgbt/resources/therapeutic-response.pdf.

Andrzejewski, M. E., Kirby, K. C., Morral,A. R., & Iguchi, M. Y. (2001). Technologytransfer through performance manage-ment: The effects of graphical feedbackand positive reinforcement on drugtreatment counselors’ behavior. Drugand Alcohol Dependence, 63, 179–186.

Beidas, R. S., Edmunds, J. M., Marcus, S.C., & Kendall, P. C. (2012). Training andconsultation to promote implementationof an empirically supported treatment: Arandomized trial. Psychiatric Services,63(7), 660-665.

Carise, D., Cornely, W., & Gurel, O.(2002). A successful researcher– practi-tioner collaboration in substance abusetreatment. Journal of Substance AbuseTreatment, 23, 157–162.

Damschroder, L. J., Aron, D. C., Keith, R.E., Kirsh, S. R., Alexander, J. A., &Lowery, J. C. (2009). Fostering imple-mentation of health services researchfindings into practice: A consolidatedframework for advancing implementa-tion science. Implementation Science,4(1), 50.

Elshaug, A. G., Watt, A. M., Moss, J. R., &Hiller, J. E. (2009). Policy perspectives onthe obsolescence of health technologies inCanada. Ottawa, Canada: CADTH.

Garner, B. R., Godley, S. H., Dennis, M. L.,Hunter, B. D., Bair, C. M., & Godley, M.D. (2012). Using pay for performance toimprove treatment implementation foradolescent substance use disorders:results from a cluster randomized trial.Archives of Pediatrics & Adolescent Medi-cine, 166(10), 938–944.

Haldeman, D. (2002). Therapeutic anti-dotes: Helping gay and bisexual menrecover from conversion therapies. Jour-nal of Gay and Lesbian Psychotherapy, 5,119–132.

Hansson, S. O. (2013). Defining pseudo-science and science. The Philosophy ofPseudoscience, 61-77.

Henshall, C., Schuller, T., & Mardhani-Bayne, L. (2012). Using health technol-ogy assessment to support optimal use oftechnologies in current practice: the chal-lenge of “disinvestment”. InternationalJournal of Technology Assessment inHealth Care, 28(3), 203-210.

Ibargoyen-Roteta, N., Gutiérrez-Ibarluzea,I., & Asua, J. (2010). Guiding the process

of health technology disinvestment.Health Policy, 98(2), 218-226.

Jenkins, D., & Johnston, L. (2004). Unethi-cal treatment of gay and lesbian peoplewith conversion therapy. Families inSociety, 85(4), 557-561.

Kilbourne, A. M., Neumann, M. S., Pincus,H. A., Bauer, M. S., & Stall, R. (2007).Implementing evidence-based interven-tions in health care: application of thereplicating effective programs frame-work. Implementation Science, 2(1), 42.

Lilienfeld, S. O. (2007). Psychologicaltreatments that cause harm. Perspectiveson Psychological Science, 2(1), 53-70.

Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M.(2015). Science and pseudoscience inclinical psychology: Initial thoughts,reflections, and considerations. Scienceand Pseudoscience in Clinical Psychology(2nd ed., pp. 1-16). New York: GuilfordPress.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J.,Cautin, R. L., & Latzman, R. D. (2013).Why many clinical psychologists areresistant to evidence-based practice: Rootcauses and constructive remedies. Clini-cal Psychology Review, 33(7), 883-900.

Magnabosco, J. (2006). Innovations inmental health services implementation:A report on state-level data from the U.S.evidence-based practices project. Imple-mentation Science, 1, 1–11.

McLellan, A. T., Kemp, J., Brooks, A., &Carise, D. (2008). Improving publicaddiction treatment through perfor-mance contracting: The Delaware experi-ment. Health Policy, 87(3), 296–308.

Montini, T., & Graham, I. D. (2015).“Entrenched practices and other biases”:Unpacking the historical, economic,professional, and social resistance to de-implementation. Implementation Science,10(1), 24.

Nadeem, E., Gleacher, A., & Beidas, R. S.(2013). Consultation as an implementa-tion strategy for evidence-based practicesacross multiple contexts: Unpacking theblack box. Administration and Policy inMental Health and Mental Health Ser-vices Research, 40(6), 439-450.

Niven, D. J., Mrklas, K. J., Holodinsky, J.K., Straus, S. E., Hemmelgarn, B. R., Jeffs,L. P., & Stelfox, H. T. (2015). Towardsunderstanding the de-adoption of low-value clinical practices: a scoping review.BMC Medicine, 13(1), 255.

Polisena, J., Clifford, T., Elshaug, A. G.,Mitton, C., Russell, E., & Skidmore, B.(2013). Case studies that illustrate disin-vestment and resource allocation deci-sion-making processes in health care: asystematic review. International Journalof Technology Assessment inHealth Care, 29(2), 174-184.

Powell, B. J., Beidas, R. S., Lewis, C. C.,Aarons, G. A., McMillen, J. C., Proctor,E. K., & Mandell, D. S. (2017). Methodsto improve the selection and tailoring ofimplementation strategies. The Journal ofBehavioral Health Services & Research,44(2), 177-194.

Prasad, V., & Ioannidis, J. P. (2014). Evi-dence-based de-implementation for con-tradicted, unproven, and aspiring health-care practices. Implementation Science,9(1), 1.

Rabin, B. A., Lewis, C. C., Norton, W. E.,Neta, G., Chambers, D., Tobin, J. N.,Brownson, R.C., & Glasgow, R. E. (2016).Measurement resources for dissemina-tion and implementation research inhealth. Implementation Science, 11(1), 42.

Stirman, S. W., Gutner, C. A., Langdon,K., & Graham, J. R. (2016). Bridging thegap between research and practice inmental health service settings: Anoverview of developments inimplementation theory and research.Behavior Therapy, 47(6), 920-936.

. . .

This project was supported by1R01MH106506-01A1.Correspondence to Clara Johnson,National Center for PTSD, 795 WillowRoad, Bldg. 334, Rm. C119, Menlo Park, CA94025; [email protected]


MENTAL HEALTH PRACTITIONERS havelong struggled to assert their authority onmatters psychological. Even now thatmental health services have become morewidely available than they once were, prac-titioners suffer by comparison to the med-ical profession. Seeking psychological ser-vices is often stigmatized in a way thatmedical treatment is not (Sartorius, 2007;Schulze, 2007). In addition, physicians gainan air of authority from their highly techni-cal subject matter. In contrast, everyonewitnesses human behavior every day. Whatcould be so difficult about knowing whypeople act the way they do?

Indeed, the problem is much larger thanjust mental health professions. Today, thedenial of authority extends to almostanyone claiming to be an expert. Scien-tists—who should be afforded some creditin return for their extensive training andthe quality of their data—are often at oddswith the views of the general public. Arecent Pew Research Center poll found anastonishing 51-point gap between theviews of U.S. adults and members of theAmerican Association for the Advance-ment of Science (AAAS) on the safety ofgenetically modified foods (GMOs; Funk &Rainie, 2015). Eighty-nine percent ofAAAS members said GMOs were safe.(They can’t all work for Monsanto!) Simi-larly, the Pew study found that 87% of sci-entists agreed with the statement, “Climatechange is mostly due to human activity,”compared with only 50% of U.S. adults.

A recent book decries the “death ofexpertise” (Nichols, 2017b), and there is noshortage of anecdotal evidence to certifythe death. The United States recentlyelected a real estate developer with no priorgovernment experience to be president,and he went on to appoint a number ofpeople to high-level positions who weresimilarly lacking in expertise relevant totheir assignments. As just one example, thenew administration appointed a formerconservative radio talk-show host to thehighest science position in the Departmentof Agriculture—a man whose only sciencedegree was a B.A. in political science (Geil-ing, 2017; Nichols, 2017a). The new presi-dent came into power by campaigning

against the “elites,” repeatedly asserting, “Ialone can fix it” (Jackson, 2016).

So how should we respond to thesechallenges? Michael Bowen (2017), writingfor the World Economic Forum’s YoungScientists Community, asserts that we areconfronted with “a populist backlashagainst scientific consensus and expertopinion” and urges scientists to strengthentheir resolve and fight back with facts. Butit seems like scientists have been fightingback with facts and evidence for a whilenow, with minimal results. It has been 19years since Andrew Wakefield publishedhis infamous study in The Lancet, purport-ing to show a relationship between theMMR vaccine and the incidence of autism.Many failures to replicate Wakefield’sresults followed, and 7 years ago, theBritish General Medical Council revokedWakefield’s medical license and The Lancetwithdrew his 1998 article (Offit, 2010).Much ink has been spilled and wordsspoken in an effort to use facts to convinceparents that vaccination is safe and impor-tant, but a 2015 poll found that only 84% ofAmericans thought vaccination of youngchildren was very or extremely important,down from 93% fourteen years earlier(Newport, 2015). In 2014 the Centers forDisease Control reported a record 663cases of measles, the “greatest number ofcases since measles elimination was docu-mented in the U.S. in 2000” (Centers forDisease Control, 2017).

Lest hubris begin to set in, it should beacknowledged that therapists are far fromimmune to nonscientific practices. Recentevidence shows that many practicing psy-chologists and social workers are usingtechniques that are unsupported by scien-tific evidence (Barnett & Shale, 2012; Pig-notti & Thyer, 2009; Stapleton et al., 2015).As a result, considerable effort needs to beaimed at healing ourselves (Lilienfeld et al.,2013). But putting that issue aside, let’sassume you are a behavior therapist com-mitted to evidence-based practice (EBP)who is confronted with a client who isequally committed to Reiki, chelation ther-apy, or homeopathic medicines. What is atherapist to do?

In the long term, the solution to theseconflicts may come from better public edu-

cation in science and critical thinking. Inaddition, Lilienfeld, Lynn, and Lohr (2014)offered a number of suggestions forreforming the standards and training ofclinical psychologists. But these societaland professional reforms will not come intime for therapists who have credulousclients in their offices today. Understand-ing this, I will discuss four possible strate-gies for dealing with unscientific clientbeliefs: adopting, avoiding, reasoning, andcollaborating.

AdoptingAlthough it may seem odd to consider

adopting the unscientific ideas of yourclients, it is not without precedent. Con-fronted with a client who has a particularworldview, therapists have been known toincorporate client beliefs into the treatmentplan. Sweat lodge ceremonies have beenrecommended as part of treatment forposttraumatic stress disorder in NativeAmericans, and other practitioners havesuggested praying with or for clients duringtherapy (Meichenbaum, n.d.; Silver &Wilson, 1988). Therapists who adopt thesemethods may have the admirable goals ofacknowledging cultural or religious differ-ences or wanting to make clients feel moreat home, but the ABCT is an organizationcommitted to EBPs (ABCT, 2017). With-out convincing empirical support, thesepractices represent an ethical dilemma forthe therapist. Furthermore, if therapistshope to project a consistently evidence-based image to the public, adopting non-scientific methods will only muddy thewaters and make it harder to distinguishthe profession from other non-evidence-based practitioners. Finally, in the case ofsweat lodge ceremonies and a number ofother nontraditional methods, there maybe substantial safety concerns (Dougherty,2009). As a result, adopting nonscientificclient beliefs as part of therapy is not a rec-ommended strategy.

AvoidingFrom a utilitarian viewpoint it might be

reasonable to say nothing. As long as theclient is faithfully following through withyour treatment recommendations andmaking progress, a pragmatic strategymight be to avoid confronting the client’smisconceptions and say as little as possibleabout the pseudoscientific methods beingused or advocated by the client. Whentherapists are directly asked about non-EBP treatments, they are under an ethicalobligation to provide accurate information,

What’s a Therapist to Do When Clients HavePseudoscientific Beliefs?Stuart Vyse, Stonington, Connecticut

January • 2018 37

W H E N C L I E N T S H A V E P S E U D O S C I E N T I F I C B E L I E F S

but given that the primary goal is improv-ing client well-being, saying nothing maysometimes be an option.

However, biting one’s tongue will rarelybe a comfortable choice because the thera-pist risks appearing to give credence to anunsupported treatment, and just as in thecase of the “adopting” strategy, it is impor-tant to present the profession as consis-tently guided by evidence. But, in the inter-est of keeping positive momentum going,individual therapists may choose to avoidunnecessary battles. Unfortunately, some-times the client’s unsupported remediesobstruct the implementation of evidence-based interventions and/or are potentiallyharmful. In these cases, avoidance is not anoption.

ReasoningIn addition to a rejection of experts, the

current era has seen a decline in the valueof rational argument. Indeed, sophistryappears to be enjoying a period of growth.During the 2016 U.S. presidential cam-paign the eventual winner was greatlyrewarded for his use of derogatory nick-names for his political rivals, a practice thathas continued during his presidency(Estepa, 2017), and formerly trusted newssources are now routinely labeled “fakenews.”

As difficult as the current environmentappears to be, a discussion with clientsabout basic research methods and levels ofevidence—or lack of evidence—supportingvarious methods is worth trying. It wouldbe impractical to administer a full course incritical thinking; however, some therapistshave had success giving clients readingmaterial about both EBPs and non-EBPs(Kay, 2015). But what if those early conver-sations don’t go smoothly? What’s a thera-pist to do?

If there is a benefit of the current cli-mate of rampant credulity it is the emer-gence of a growing literature on the bestmethods for debunking misinformation. In2012, Lewandowsky and colleagues pub-lished a very useful qualitative review, andrecently Chan, Jones, Jamieson, and Albar-racin (2017) published a meta-analysis ofthe effectiveness of various debunkingmethods. These studies point to a numberof recommendations about how to success-fully counter misinformation, and severalof these may be useful to the clinician whohopes to steer a client towards EBPs:

• Avoid reviewing any evidence in sup-port of unsupported treatments. In the

interest of fairness, a therapist mightadmit that homeopathic medicines havean intuitive appeal and that many effec-tive medicines were similarly derivedfrom naturally occurring herbs and com-pounds, but this would be a mistake. Theresearch on debunking suggests that anyrecounting of arguments in support ofmisinformation tends to solidify a mentalmodel, making it more difficult to quashwith new information.

• Don’t just say the misinformation iswrong; provide an alternative formula-tion. The debunking of misinformationleaves a void that is an obstacle to a lastingeffect. As time passes, the client is likely torefill the hole with the same old myth. Asa result, it is important to supply the clientwith information about EBPs that isexplained in some detail, along with theavailable evidence to back it up. As aresult, when debunking homeopathy, thetherapist should point out that the activeingredients are far too diluted to be effec-tive, but it is also important to create anew theory of the client’s problemthrough the lens of a sound empiricalresearch. Be prepared to report what sci-ence has to say about the client’s concern.

• Try to keep the explanation of EBPssimple and clear. Somewhat paradoxi-cally, as important as it is to create a newevidence-based theory of the problem,debunking research suggests that anoverly elaborate explanation can backfire.If the misinformation is simpler andclearer than the more valid alternative,the myth may survive. Unfortunately, itcan be difficult to keep the description ofan EBP simple. For example, when a ther-apist is confronted with a parent who iscommitted to the use of facilitated com-munication in the treatment of a childwith autism, the elaborateness of anapplied behavior analysis (ABA) protocolis going to come up short in relation to thefar simpler explanation, “Jenny has amotor problem. She needs help steadyingher hand on the keyboard.”

• If there is a choice between giving infor-mation in printed or video form, choosevideo. A recent study showed that whenfact-checking information was presentedin either long-form written format or in avideo, the video presentation was moreeffective in debunking misinformation(Young, Jamieson, Poulsen, & Goldring,2017). Given the number of professionalvideos that are available both commer-

cially and on free websites (e.g.,YouTube.com), it is likely that therapistscan find useful material to present toclients.

CollaboratingIf the rational approach does not

quickly move the client in a constructivedirection, a more empirical strategy cansometimes work. The therapist and clienthave an important common goal, helpingthe client. If sharing accurate informationdoes not shake the client from unsupportedor pseudoscientific beliefs, then offering tocollaborate on an empirical test can behelpful. Rather than continuing to arguewith the client—or sending the clientaway—the therapist can offer to join forcesin an evaluation of the treatment options.In brief, the therapist might simply say,“OK, I can see you’re not convinced. Let’sperform a test with the understanding thatwhatever method works best will be theone we choose.”

This strategy has been successfullyemployed by Shannon Kay (2015), a tal-ented behavior analyst who has workedwith many parents of children withautism.1 Autism continues to be a “fadmagnet” (Metz, Mulick, & Butter, 2015),attracting a seemingly endless stream ofpseudoscientific treatments. As a result,Kay reported that, by the time she arrivedon the scene of a newly diagnosed case, thechild’s parents were often already usingprism glasses or sensory integration ther-apy. In those cases where she was unable towin parents over by sharing informationand readings, she offered to conduct asingle-participant study testing an appliedABA approach against the methods beingused by the parents. And, of course, thesubject of the research was the most impor-tant person of all, the child everyone wastrying to help.

Kay described her experiences and pro-vided data from three case studies in achapter for the book Controversial Thera-pies for Autism and Intellectual Disabilities(Foxx & Mulick, 2015). In each of the threecases, she used an alternating treatmentsdesign and trained the parents in data col-lection. In all three instances, the unsup-ported therapy being used at the time wasshown to have a negative effect on thechild’s behavior, rather than a positive one,and the parents and educational team

1Shannon Kay is a former student of mine.


V Y S E

members quickly reversed their positionsand endorsed a plan based on ABA.

When working with adults on issuesother than autism treatment, it may beimpractical to implement a test of compet-ing therapies, and when a test is possible, areversal design (e.g., ABAC) may be moreappropriate than the alternating treatmentdesign employed by Kay (2015). But intro-ducing the client to some of the basics ofresearch design and objective data collec-tion can be very useful. Furthermore, itappears that one of the important featuresof Kay’s approach is putting aside thestruggle to assert one’s authority as a thera-pist and offering to solve the dispute in acollaborative fashion. Understandably,some therapists may find it objectionableto agree to test a previously unsupportedtherapy. Furthermore, the empirical testapproach is not without risks. Clients canrarely be blinded to the experimental con-ditions, and client expectations about pseu-doscientific therapies can lead to measur-able placebo effects. In the unlikely event ofa positive outcome for a non-EBP, the ther-apist would be confronted with a thornydilemma. It is best not to gamble if the out-come is in doubt. But in those cases wherea collaboratively designed test seems bothfeasible and potentially effective, it may bemore convincing than talk.

It is difficult to be optimistic about theprospect of a near future free of supersti-tion and pseudoscience. In recent decadeswe have experienced an explosion in accessto information; however, much of the easi-est information to find is false. The Pewstudies cited above suggest that manypeople are unable to judge the quality ofinformation and, as a result, are unpre-pared to separate out the misinformation.Furthermore, as the Dunning-Krugereffect suggests, it is often the least informedpeople who are the most convinced theyare right (Kruger & Dunning, 1999). As aresult, further research on debunkingstrategies will be needed, and for the fore-seeable future, therapists will continue tocome across clients who espouse unscien-tific therapies.

A final thought. Fad therapies appear toreproduce at alarming rates and, in somecases, are all but impervious to rationalattack. Despite the recent blows to theauthority of experts of all kinds, behaviortherapists are in an excellent position tospeak publicly on these topics. It is unlikelythat pseudoscience and superstition willever be permanently vanquished, butbehavior therapists who seek out publicspeaking opportunities, comment in the

media, or write for the general public canhelp to counter the misinformation in theircommunities. Although many profession-als feel most comfortable speaking aboutthe EBPs they have been trained to use,reducing the level of psychological snakeoil in the marketplace will take additionalefforts. According to research cited above,effective debunking will require therapiststo inform people about the current scien-tific understanding of the disorders theytreat and to call out the unsubstantiatedtreatments that are sometimes used.Taking these extra steps may eventuallyreduce the number of clients who come toyou under the influence of pseudoscientifictheories and will have the added benefit ofpublicly reinforcing the point that, in con-trast to other approaches, behavior therapyis a rigorous evidence-based discipline.

ReferencesABCT. (2017). About psychological treat-

ment. Retrieved September 28, 2017, fromhttp://www.abct.org/Help/?m=mFindHelp&fa=WhatIsEBPpublic

Barnett, J.E., & Shale, A. J. (2012). The inte-gration of complementary and alternativemedicine (CAM) into the practice of psy-chology: A vision for the future. Profes-sional Psychology: Research and Practice,43(6), 576–585.

Bowen, M. (2017, June 22). As scientists, wemust fight fake news with truth. Retrievedfrom https://www.weforum.org/agenda/2017/06/as-scientists-we-must-fight-fake-news-with-truth/

Centers for Disease Control. (2017, August23). Measles Cases and Outbreaks.https://www.cdc.gov/measles/cases-out-breaks.html

Chan, M. P. S., Jones, C. R., Hall Jamieson,K., & Albarracín, D. (2017). Debunking: Ameta-analysis of the psychological efficacyof messages countering misinformation.Psychological Science, 28, 1531-1546.https://doi.org/10.1177/0956797617714579.

Dougherty, J. (2009, October 11). Deaths atsweat lodge bring soul-searching. NewYork Times. Retrieved September 28,2017, from http://www.nytimes.com/2009/10/12/us/12lodge.html

Estepa, J. (2017, September 21). It's not just'Rocket Man.' Trump has long history ofnicknaming his foes. Retrieved September21, 2017, from https://www.usatoday.com/story/news/politics/onpoli-tics/2017/09/21/its-not-just-rocket-man-trump-has-long-history-nicknaming-his-foes/688552001/

Foxx, R. M. & Mulick , J. A. (2015). (Eds.).Controversial therapies for autism andintellectual disabilities: Fad, fashion, and

science in professional practice (2nd ed.).New York: Routledge.

Funk, C., & Rainie, L. (2015, January 29).Public and scientists' views on science andsociety. Pew Research Center.http://www.pewinternet.org/2015/01/29/public-and-scientists-views-on-science-and-society/

Geiling, N. (2017, July 20). Trump officiallynominates climate-denying conservativetalk radio host as USDA's top scientist.Retrieved fromhttps://thinkprogress.org/sam-clovis-offi-cially-nominated-still-not-a-scientist-d47be4ffb1a8/

Jackson, D. (2016, July 22). Donald Trumpaccepts GOP nomination, says 'I alone canfix' system. Retrieved fromhttps://www.usatoday.com/story/news/politics/elections/2016/07/21/donald-trump-republican-convention-accep-tance-speech/87385658/

Kay, S. (2015). Helping parents separate thewheat from the chaff: Putting autismtreatments to the test. In R. M. Foxx & J.A. Mulick (Eds.), Controversial therapiesfor autism and intellectual disabilities: Fad,fashion, and science in professional practice(pp. 196-208). New York: Routledge.

Kruger, J., & Dunning, D. (1999). Unskilledand unaware of it: how difficulties in rec-ognizing one's own incompetence lead toinflated self-assessments. Journal of Per-sonality and Social Psychology, 77(6),1121-1134.

Lewandowsky, S., Ecker, U. K., Seifert, C.M., Schwarz, N., & Cook, J. (2012). Misin-formation and its correction: Continuedinfluence and successful debiasing. Psy-chological Science in the Public Interest,13(3), 106-131.

Lilienfeld, S. O., Lynn, S. J., Lohr, J. M.(2014). Science and pseudoscience in clin-ical practice: Concluding thoughts andconstructive remedies. In S. O. Lilienfeld,S. J. Lynn, & J. M. Lohr (Eds.), Science andpseudoscience in clinical psychology (pp.527-532). New York: Guilford.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J.,Cautin, R. L., & Latzman, R. D. (2013).Why many clinical psychologists are resis-tant to evidence-based practice: Rootcauses and constructive remedies. ClinicalPsychology Review, 33(7), 883-900.

Meichenbaum, D. (N.D). Trauma, spiritu-ality and recovery: Toward a spiritually-integrated psychotherapy. Unpublishedmanuscript. Retrieved from http://melissainstitute.com/documents/spirituality_psychotherapy.pdf

Metz, B., Mulick, J. A., & Butter, E. M.(2015). Autism: A Twenty-First Centuryfad magnet. In R. M. Foxx & J. A. Mulick(Eds.), Controversial therapies for autismand intellectual disabilities: Fad, fashion,

and science in professional practice (pp.169-194). New York: Routledge.

Newport, F. (2015, March 06). In U.S., Per-centage Saying Vaccines Are Vital DipsSlightly. Retrieved fromhttp://www.gallup.com/poll/181844/percentage-saying-vaccines-vital-dips-slightly.aspx

Nichols, T. (2017a, September/October).How We Killed Expertise. Poltico Maga-zine. http://www.politico.com/magazine/story/2017/09/05/how-we-killed-exper-tise-215531

Nichols, T. (2017b). The death of expertise:The campaign against established knowl-edge and why it matters. New York, NY:Oxford University Press.

Offit, P. A. (2010). Deadly choices: How theanti-vaccine movement threatens us all.New York: Basic Books.

Pignotti, M., & Thyer, B. A. (2009). Use ofnovel unsupported and empirically sup-ported therapies by licensed clinical socialworkers: An exploratory study. SocialWork Research, 33(1), 5-17.

Sartorius, N. (2007). Stigma and mentalhealth. The Lancet, 370(9590), 810-811.

Schulze, B. (2007). Stigma and mentalhealth professionals: A review of the evi-dence on an intricate relationship. Inter-national Review of Psychiatry, 19(2), 137-155.

Silver, S. M., & Wilson, J. P. (1988). NativeAmerican healing and purification ritualsfor war stress. In J. P. Wilson, Z. Harel, &B. Kahana (Eds.), Human adaptation toextreme stress: From the Holocaust to VietNam (pp. 337-355). New York: PlenumPress.

Stapleton, P. H., Chatwin, E., Boucher, E.,Crebbin, S., Scott, S., Smith, D., & Purkis,G. (2015). Use of complementary thera-pies by registered psychologists: An inter-national study. Professional Psychology:Research and Practice, 46(3), 190–196.

Young, D. G., Jamieson, K. H., Poulsen, S.,& Goldring, A. (2017). Fact-checkingeffectiveness as a function of format andtone: Evaluating FactCheck.org andFlackCheck.org. Journalism & Mass Com-munication Quarterly. https://doi.org/10.1177/1077699017710453.

. . .

The author has no funding or conflicts ofinterest to disclose.Correspondence should be addressed tothe author at [email protected]

RECENTLY, A MAJOR PROFESSIONAL orga-nization sponsored a webinar whereby theattendees would learn about the underly-ing mechanisms and procedures for Emo-tional Freedom Techniques (EFT; seeMoran & Keynes, 2012, for overview).What was notable about this webinar offer-ing was not so much the topic as the factthat the sponsoring organization indicatesa commitment to promoting scientificfoundations of assessment and treatment.One might even make a case for the scien-tific basis of EFT, given that there areclaims in the literature of efficacious out-come with the method (Clond, 2016).However, most readers of this journalknow what’s coming next: namely, thatEFT, as a member of the broader class ofenergy therapies, lacks (a) an underlyingtheoretical basis for different psychopatho-logical states and (b) an empirical basis forthe mechanisms of treatment efficacy. Andyet, offerings like the aforementionedwebinar proliferate, available through awide range of organizations that are other-wise solidly science-minded.

Energy therapies are hardly the onlyexample of treatment methods that lackany scientifically compelling underlyingmechanisms of psychopathology orexplanatory structures for the interventionmethods. Indeed, there are far too many toenumerate here. Those who practiceapproaches that the scientific communityhave declared science-based smugly1 deni-grate these other approaches as nonscien-tific or, worse, pseudoscientific. Despitethis divide, these approaches proliferate,and many practitioners offer treatmentsthat lack qualities that we might call scien-tific.

Further, mental health practitioners arenot the only professional group to fall preyto pseudoscientific theories. One famousexample is the pursuit of achieving coldfusion in the lab, with the most recentunsubstantiated claim coming in 1989,despite the lack of a compelling theoreticalbasis for predicting the phenomenon couldbe produced (Beaudette, 2002). Philoso-

phers of science have struggled with theproblem of pseudoscience, citing a demar-cation problem suggesting a continuum ofsorts from that which can be definitivelytermed science to that which is squarelypseudoscience (Popper, 1957).

While all sciences seem to be suscepti-ble to pseudoscience, psychotherapyapproaches may be at particular risk. Theaim of this paper is to suggest some expla-nations for this problem, and some modestrecommendations for remediation.

Therapy Allegiance: A SpecialProblem in Mental Health Delivery

Since you are reading this article, youare most likely an adherent to the theoriesthat underlie cognitive and behavioraltherapies. Asked to describe the approachto a friend or colleague in another profes-sion, you might offer a detailed litany ofjustifications for the approaches based onyour intimate knowledge of the theory andits accumulated empirical support. If askedon follow-up why this approach to treat-ment is so special and different from tradi-tional psychotherapy, you might go so faras to explicate paradigmatic differencesaround the degree that each therapeuticapproach values data (discussed by a psy-chodynamic theorist; Bornstein, 2005). Butwhat happens should this same person askwhat made you choose this therapeuticapproach over all the others that are outthere? You might very well offer an expla-nation that sounds like cold hard rational-ity—the data made you do it! The approachis evidence-based, and I’m an evidence-based person! But the research suggeststhat these explanations are as likely ex postfacto explanations as they may be a prioridecisions.

Research has suggested that the deci-sion to align with CBT comes more frompersonal factors, whereas traditional psy-chotherapy approaches come more fromtraining experiences (Buckman & Barker,2010). That is, if you have a particularlycompelling personal training experience,

The Seductive Allure of Pseudoscience inClinical PracticeDean McKay, Fordham University

1I count myself among the smug.

January • 2018 39


M C K A Y

you may be more likely to adopt a psycho-dynamic approach to treatment, whereas ifyou possess specific personality character-istics (low Openness to Experience, highConscientiousness), you are more likely tochoose CBT. Notice that neither justifica-tion is derived from such factors as “finddata compelling” or “possess skepticalideas about therapy research methods.”Digging a bit deeper into this single inves-tigation, we find that a vast swath of practi-tioners who adhere to psychodynamicapproaches are self-described as being par-ticularly attuned to inner experiences andto find meaning in symbolic processes.

This suggests that the factors that leadto self-identification with one or anothertherapeutic approach is less about com-pelling data and more about a feeling state,an irrational basis unmoored from any sci-entific findings.2 Long before survey dataidentified variables that explained theroutes for how therapists sorted themselvesinto different theoretic camps, it was recog-nized that the therapy approach one prac-ticed strongly influenced outcomes in oth-erwise controlled research (see Leykin &DeRubeis, 2009, for detailed discussion).

This means that should you have a goodtraining experience, and are the kind ofperson who ascribes strong meaning toinner experiences, and receives training ina pseudoscientific method, you may be anew adherent to that approach. And oncethat happens, adherence to that method isdifficult to shake.

Tribalism in Therapeutic ApproachesThe factors that go into group affiliation

are complex and wide ranging, certainly farbeyond the scope of this article. However,in the self-sorting process that takes placefollowing the determination of therapeuticorientation, it can be expected that wechoose groups with whom to affiliate thatwe anticipate having similar values(Wagner, 1995). These values can be fur-ther crystallized as we further identify withthe group. So what happens when thebroad outlines of the values of the groupare threatened? In the case of our discus-sion, what happens when the purveyor of

pseudoscientific methods is called out forproffering a nonscientific approach?

Douglas (1966) described a robustsocial process, evident in religion, groupdynamics, and close-knit tribal communi-ties, whereby external threats are identifiedand specific remedies are developed andsanctioned by the group. Practitioners ofall stripes are members of "tribes," and willseek out assistance from the tribe whenthreatened. Accordingly, the purveyor ofpseudoscientific methods will find supportfrom their "tribe" of like-minded providerswhen attacked for their practices. The sci-entific community is not a part of thisequation since that is not the tribe that willbe available to them. And without externalstructures that might restrict their practice,pseudoscientific approaches will likely con-tinue and even thrive. The methods ofassistance vary widely based on group-specific customs that develop to create asense of group purity and cohesion.

Market Forces SupportDifferent Tribes

Travel to areas of the desert southwestin the United States and one finds a widerange of New Age practices. For example,Sedona, AZ has numerous practitioners ofphysical and mental healing that relies onthe local “crystal vortex” (Dannelly, 1995).This specific region is said to possess spe-cial qualities, and the crystals in the redrock formations distinctive to the townconverge with mystic energies that pro-mote a healing process. Aside from thestunning beauty of the place, there is littleto support the idea that the local vortexpossesses special healing properties.Nonetheless, people suffering from alltypes of maladies seek “treatment” fromwhat are effectively faith healers.

These approaches persist for a variety ofreasons, one of which involves strongmarket forces that support their demand.The various pseudoscientific practicesroughly correspond to so-called New Agepractices. Research suggests that segmentsof the population find these practices com-pelling and includes endorsement of magi-cal ideation (Farias, Claridge, & Lalljee,2005). As further evidence that there would

be a sizeable market for New Ageapproaches, look no further than the con-siderable sales of the book The Secret(Byrne, 2006), a bestseller with a centralthesis that the way to a better life is thatsimply thinking positive thoughts will inand of itself change oneself and the world.Imagine for a moment now that, as a CBTpractitioner, you include in your treatmentplan an effort to directly challenge thought-action fusion (Shafran & Rachman, 2004),the specific cognitive distortion that think-ing something bad increases its likelihood,and you learn your client subscribes to themodel described in The Secret. At the veryleast the discussion that will follow will beawkward.

We can then conclude that practitionerswho offer pseudoscientific approaches maydo so as a consequence of true identifica-tion with a group that endorses these meth-ods (tribalism), and that it is perpetuatedthrough a market that supports it. Attackson these approaches are met with credulity,counter-attacks, and retrenchment. Howoften have you heard some variation on thefollowing counter-argument: “I’m notgoing to worry about which theory ormechanism is at work, I just do what Iknow is effective.” This ultimate tribalretreat allows for retention of the approachwithout concern for science, and retainsthe claim that what they do works. Youmight even be on the receiving end of acounter-accusation that because of a slav-ish reliance on science, you are lacking incompassion (discussed in McKay, 2017).

Making PseudoscienceUnappealing to the Masses

This has not been an exhaustive consid-eration of all the ways pseudoscience isappealing to clinicians and consumers. Butsome of the factors that make it appealing,and difficult to dislodge, come more frompersonal preferences and sociologicalforces than from cold hard facts. Amongthe challenges are: demarcation; illusoryeffectiveness in psychotherapy; and publiceducation in science.

The Demarcation ProblemIt was noted earlier that the demarca-

tion problem in science has been a persis-tent challenge in rooting out pseudosciencefrom science. Indeed, some philosophers ofscience declared it hopeless to pursue anylonger (Lauden, 1983). Since that time, ahealthy reemergence of interest in estab-lishing a specific boundary between whatconstitutes science and what belongs in the

2Self-disclosure moment: During my undergraduate years I self-identified as psychodynamic inorientation. It was only in graduate school that I found I had a talent for exposure, discoveredaccidentally over a dinner outing, and later crystallized during a training experience. So it seemsthat reinforcement and determinism shaped my professional trajectory rather than some clear-eyed and deliberate planning.

January • 2018 41

P S E U D O S C I E N C E I N C L I N I C A L P R A C T I C E

category of pseudoscience has sprouted(Pigliucci & Boudry, 2013). In assessing theimportance of this approach, it has beensuggested that pseudoscience is actuallyessential for understanding science itselfsince it permits a clarification of definitionfor what counts as evidence (Ladyman,2013).

In some ways, psychotherapy researchis ahead on this matter. We have begun toreckon with this problem by directly andunambiguously identifying practices thatare pseudoscientific (such as the aforemen-tioned energy therapies) by specifying thecharacteristics of questionable practices(see Lilienfeld, Lynn, & Lohr, 2014). Ofcourse, this optimism is tempered by themere fact that pseudoscientific approachesare not only still practiced, but that train-ing in these approaches continues to prolif-erate.

Illusory Effectiveness and the PublicPseudoscientific therapy approaches

can retreat into pure empiricism to supportthe claims of efficacy. A long-standing andwell-known problem in psychotherapy isthat virtually any treatment performsbetter than waitlist (Eysenck, 1952). Earlycompilations of the outcomes of treatmentsuggested that all interventions had com-parable efficacy (Smith & Glass, 1977). Thisled to a defense of common factors and abroad therapeutic relationship as centralmechanisms in efficacy since it appearedthat all treatment were on comparablefooting, an argument that continues toattract supporters (Shedler, 2010).

The practice community engages in awide range of errors in reasoning that canlead to the development and adoption ofpractices that lack scientific merit. Lilien-feld, Ritschel, Lynn, Cautin, and Latzman(2014) described a taxonomy of these prob-lems, termed causes of spurious therapeu-tic effectiveness. This taxonomy has threebroad categories: perception of clientchange in its actual absence; misinterpreta-tions of actual client change derived fromextratherapeutic factors; and misinterpre-tation of client change resulting from non-specific factors.

However, the public has begun to iden-tify problematic therapeutic approaches forthemselves. It has been suggested for sev-eral years now that more and more clini-cians recognize that clients request evi-dence-based treatment, and cognitive-behavior therapy in particular (McKay,2014). The stated adoption of CBT meansthat increased attention must be paid to

fidelity of treatment delivery. This isimportant to the dissemination effort.Namely, the public has to trust that treat-ments delivered in everyday practice willmimic the scientific findings of efficacy forCBT from carefully controlled investiga-tions, or come as close as reasonably possi-ble. Otherwise, how can we disseminatethat this is evidence-based if clients cannotreadily access genuine CBT?

Public Education in ScienceAn old commercial for Syms clothing

store intoned, “An educated consumer isour best customer.” In a similar vein, edu-cated consumers will be the best customersfor CBT as well as for the future of scientif-ically informed psychotherapy. However,unlike in clothing, this will mean that con-sumers will need to be better educatedabout the science of treatment, and whatcounts as evidence.

This requires that the public have somelayman’s understanding of causation intreatment. On this we might be a bit lessoptimistic. First, the problem of differentlevels of analysis germane to psychopathol-ogy remains elusive to practitioners of thevarious mental health disciplines. Forexample, Kendler (2012) made a persuasivecase that there are numerous levels ofanalysis appropriate for consideration intreatment, ranging from genetics upthrough and including culture. However,from the policy side, recent research fund-ing priorities such as the Research DomainCriteria favor biological mechanism expla-nations over other levels of analysis (Inselet al., 2010). By favoring single levels ofanalysis over multifaceted contributions topsychopathology, the public is less likely toappreciate putative causes and correlatessince the assumption across all mentalhealth problems is that biological factorsare causative, even if the evidence is notdemonstrated.

Second, the public is not in an advan-taged position to recognize the differencebetween a hierarchical view of causes ofpsychopathology and resultant treatmentcompared to a more situational-con-strained perspective. This is largely becausesome concepts persistently escape under-standing by the general public. For exam-ple, in a series of experiments it was shownthat participants were equally likely todraw causal inferences from experimentaldata as from nonexperimental data. Fur-ther, causal inferences were more frequentwhen it conformed to intuitively heldnotions (Bleske-Rechek, Morrison, & Hei-dtke, 2015). This difficult situation means

that public science education is moreessential than ever if consumers are goingto be able to parse fraud from fact in thepursuit of good treatment.

Understanding is the first step in devel-oping an action plan. At this point there isstill an inadequate understanding of whatcompels well-intentioned clinicians toadopt practices that have dubious efficacy,questionable scientific foundations, andsimply lack clear and compelling mecha-nisms for actions. There are some promis-ing options for consideration here thatinclude individual preferences, groupprocesses, and market forces. Hopefully, byclarifying the role each of these play, poli-cymakers will design methods to combatpseudoscientific practices as a means toprotect an unsuspecting public.

ReferencesBeaudette, C.G. (2002). Excess heat and

why cold fusion research prevailed. SouthBristol, ME: Oak Grove Press.

Bleske-Rechek, A., Morrison, K.M., & Hei-dtke, L.D. (2015). Causal inference fromdescriptions of experimental and non-experimental research: Public under-standing of correlation-versus-causation.Journal of General Psychology, 142, 48-70.

Bornstein, R.F. (2005). Connecting psy-choanalysis to mainstream psychology:Challenges and opportunities. Psychoan-alytic Psychology, 3, 323-340.

Buckman, J.R., & Barker, C. (2010). Thera-peutic orientation preferences in traineeclinical psychologists: Personality ortraining? Psychotherapy Research, 20,247-258.

Byrne, R. (2006). The secret. New York:Simon & Schuster.

Clond, M. (2016). Emotional freedomtechniques for anxiety: A systematicreview with meta-analysis. Journal ofNervous and Mental Disease, 204, 388-395.

Dannelly, R.D. (1995). Sedona: Beyond thevortex. Flagstaff, AZ: Light Technology.

Douglas, M. (1966). Purity and danger.London: Routledge.

Eysenck, H.J. (1952). The effects of psy-chotherapy: An evaluation. Journal ofConsulting Psychology, 16, 319-324.

Farias, M., Claridge, G., & Lalljee, M.(2005). Personality and cognitive predic-tors of New Age practices and beliefs.Personality and Individual Differences,39, 979-989.

Insel, T., Cuthbert, B., Garvey, M.,Heinssen, R., Pine, D.S., Quinn, K.,Wang, P., (2010). Research domain crite-ria (RDoC): Toward a new classificationframework for research on mental disor-


L I L I E N F E L D E T A L .

ders. American Journal of Psychiatry, 167,748–751.

Kendler, K.S. (2012). The dappled nature ofcauses of psychiatric illness: Replacing theorganic-functional/hardware-softwaredichotomy with empirically based plural-ism. Molecular Psychiatry, 17, 377-388.

Ladyman, J. (2013). Toward a demarcationof science from pseudoscience. In M.Pigliucci & M. Boudry (Eds.), Philosophyof pseudoscience: Reconsidering thedemarcation problem (pp. 45-59).Chicago: University of Chicago Press.

Lauden, L. (1983). The demise of thedemarcation problem. In R.S. Cohen &L. Lauden (Eds.), Physics, philosophy, andpsychoanalysis (pp. 111-127). New York:Springer.

Leykin, Y., & DeRubeis, R.J. (2009). Alle-giance in psychotherapy outcomeresearch: Separating association frombias. Clinical Psychology: Science & Prac-tice, 16, 54-65.

Lilienfeld, S.O., Lynn, S.J., & Lohr, J.M.(2014). Science and pseudoscience in clini-cal psychology (2nd ed). New York: Guil-ford.

Lilienfeld, S.O., Ritschel, L.A., Lynn, S.J.,Cautin, R.L., & Latzman, R.D. (2014).Why ineffective psychotherapies appearto work: A taxonomy of causes of spuri-ous therapeutic effectiveness. Perspectiveson Psychological Science, 9, 355-387.

McKay, D. (2014). “So you say you are anexpert”: False CBT identity harms ourhard earned gains. the Behavior Thera-pist, 37, 213-216

McKay, D. (2017). Presidential Address:Embracing the repulsive: The case fordisgust as a functionally central emo-tional state in the theory, practice, anddissemination of cognitive-behaviortherapy. Behavior Therapy, 48, 731-738.

Moran, C., & Keynes, M. (2012). Introduc-ing emotional freedom techniques.London: Speechmark Publishing.

Pigliucci, M., & Boudry, M. (2013). Philos-ophy of pseudoscience: Reconsidering thedemarcation problem. Chicago: Univer-sity of Chicago Press.

Popper, K. (1957). Philosophy of science:A personal report. In C.A. Mace (Ed.),British Philosophy in Mid-Century (pp.

155-191). New South Wales: Allen &Unwin.

Shafran, R., & Rachman, S. (2004).Thought-action fusion: A review. Journalof Behavior Therapy and ExperimentalPsychiatry, 35, 87-107.

Shedler, J. (2010). The efficacy of psycho-dynamic psychotherapy. American Psy-chologist, 65, 98-109.

Smith, M.L., & Glass, G.V. (1977). Meta-analysis of psychotherapy outcome stud-ies. American Psychologist, 32, 752-760.

Wagner, W. (1995). Social representations,group affiliation, and projection: Know-ing the limits of validity. European Jour-nal of Social Psychology, 25, 125-139.

. . .

The author has no funding or conflicts ofinterest to disclose.Correspondence to Dean McKay, Ph.D.,Department of Psychology, 441 East Ford-ham Road, Bronx, NY 10458;[email protected]

AS CLINICAL PSYCHOLOGISTS and othermental health professionals, our priorityshould be crystal clear: to ensure that indi-viduals suffering from mental illnessreceive the highest quality psychologicalcare. Nevertheless, survey data on thera-pists’ treatment selection make abundantlyevident that we are falling woefully short inthis regard (Lilienfeld, Ritschel, Lynn,Cautin, & Latzman, 2013). Our disciplinehas long been marked by a science-practicegap, a wide schism between the best avail-able research evidence bearing on the effi-cacy and validity of psychological tech-niques, on the one hand, and their routineuse in clinical practice, on the other (Lilien-feld, Lynn, Ritschel, Cautin, & Latzman,2013; Tavris, 2014).

To take merely a handful of salientexamples, a survey of 51 licensed therapistsin Wyoming (Hipol & Deacon, 2013)revealed that fewer than one third adminis-

tered exposure and response prevention(ERP) for obsessive-compulsive disorder(OCD), even though ERP is the clear-cutscientific intervention of choice for OCD.Many of these therapists availed them-selves of treatments boasting minimal sci-entific support for OCD, such as psychody-namic therapy, art therapy, and ThoughtField Therapy, the latter being one of sev-eral energy therapies (more on that soon).In a survey of 130 Canadian therapists whotreat patients with eating disorders (vonRansom, Wallace, & Stevenson, 2013), only23% reported using cognitive-behavioraltechniques, even though these methods areamong the few empirically supported ther-apies (ESTs) for eating disorders. More-over, even among therapists who claimedto administer cognitive-behavioral meth-ods for eating disorders, sizeable pluralitiesor minorities did not make regular use ofstandard cognitive-behavioral techniques,

such as cognitive restructuring or stimuluscontrol methods (von Ranson et al., 2013).Other survey data indicate that up to half ofpeople who meet diagnostic criteria formajor depression receive no formal psy-chological treatment, and fewer than 10%of those who do receive interventions con-sistent with scientific evidence (Layard &Clark, 2014).

Over the past decade or so, the standardremedy for bridging the science-practicegap has been evidence-based practice(EBP), which is an overarching approachto clinical decision-making (Straus,Glasziou, Richardson, & Haynes, 2010).EBP integrates three legs within a “three-legged stool”: (a) the best available data onpsychotherapy outcome (and to a lesserextent, process), (b) client preferences andvalues, and (c) clinical expertise (Ander-son, 2006; Spring, 2007). EBP emanatedfrom the evidence-based medicine move-ment, which was launched in McMasterUniversity in Canada in the late 1980s andearly 1990s (Guyatt et al., 1992). Later, thismovement emigrated to the U.K. (Sackett,Rosenberg, Gray, Haynes, & Richardson,1996), American medicine, and, belatedly,American psychology. Although theAmerican Psychological Association(APA, 2006) has declined to adopt a stanceon which, if any, of the three legs of the EBPstool should be accorded highest priority intreatment selection, the Canadian Psycho-

Why Evidence-Based Practice Isn’t Enough:A Call for Science-Based PracticeScott O. Lilienfeld, Emory University, University of Melbourne

Steven Jay Lynn, Binghamton University

Stephen C. Bowden, University of Melbourne

January • 2018 43

logical Association (2012) has advocatedthat the first leg—research evidence—should take precedence above the others, aposition that we strongly endorse. Someauthors have extended this three-leggedstool from psychological interventions toevidence-based assessment (Bowden, 2017;Hunsley & Mash, 2007), an issue to whichwe briefly return (see “ConcludingThoughts”).

Ideally, the research leg of EBP shouldenhance the quality of mental health careby aligning clinical practice more closelywith scientific evidence (Kazdin, 2008;Lilienfeld et al., 2013). As a consequence, ifEBP is functioning as intended, it shouldhelp to stem the tide of pseudoscientificand otherwise questionable interventionand assessment techniques (see Lilienfeld,Lynn, & Lohr, 2014, and Thyer & Pignotti,2016, for reviews).

In this commentary, we contend thatalthough EBP has been a laudable and nec-essary first step toward ensuring high-qual-ity mental health care, it is not sufficient.More provocatively, we maintain that insome noteworthy respects, EBP has failedand will continue to do so. Hence, themental health disciplines need to adopt anapproach that is at once considerablybroader and more rigorous than EBP,namely, science-based practice (SBP). As wewill demonstrate, SBP incorporates all thefundamental elements of EBP but goes wellbeyond it in one significant respect—whichwe soon discuss.

Our call is not entirely novel, as similararguments have been advanced in medi-cine. For example, Gorski and Novella(2014) advocated for science-based medi-cine (SBM) as a more stringent and all-encompassing alternative to evidence-based medicine. We gratefully acknowl-edge the influence of their thinking on ouranalysis and adapt their terminology tomental health practice (see also Hall, 2011;Sampson & Atwood, 2005). Furthermore,as we note in a later section (“The Remedy:Science-Based Practice”), a few authors inthe psychotherapy literature have antici-pated our core arguments (e.g., David &Montgomery, 2011; see also Lilienfeld,2011).

Nevertheless, to our knowledge, we arethe first to call explicitly for a wholesaletransition from EBP to SBP in clinical psy-chology and allied mental health domains,such as psychiatry, counseling, social work,and psychiatric nursing. Moreover,because several of the substantive issuesand details of this approach’s pragmaticimplementation differ in medicine as

opposed to psychology, an independentanalysis of SBP as opposed to SBM is war-ranted.

The Recent Impetus forScience-Based Practice

The awareness that EBP has its note-worthy shortcomings is similarly not new.For example, some authors have observedthat the APA task force on EBP was con-spicuously vague when it came to opera-tionalizing the meaning of “evidence”(Stuart & Lilienfeld, 2007). Nevertheless,the limitations of EBP have becomeincreasingly evident in the last few years.Indeed, this article was precipitated largelyby a series of relatively recent events thathave raised troubling questions regardingthe capacity of EBP to curtail the continuedspread of pseudoscience in mental healthpractice. We highlight three developmentsin particular.

• The APA and several other national psy-chological associations continue toaccredit sponsors for continuing educa-tion (CE) courses and workshops onintervention techniques that are premisedon dubious or blatantly implausible theo-retical rationales, such as energy therapies(e.g., Thought Field Therapy and Emo-tional Freedom Techniques; e.g., seehttp://www.eftuniverse.com/certifica-tion/accreditation-information). Energytherapies are based on the highly suspectand probably unfalsifiable suppositionsthat (a) humans are surrounded by invis-ible and unmeasurable energy fields, and(b) blockages or other disturbances inthese fields produce anxiety disorders andother psychiatric conditions. Similarproblems extend to social work, wherelicensed practitioners can obtain CE cred-its for a host of energy therapies, primaltherapy (colloquially termed primalscream therapy), and internal family sys-tems therapy (Thyer & Pignotti, 2016),the latter of which posits that the humanmind comprises largely distinct subper-sonalities, each with its distinctive way ofviewing oneself and the world.

• In 2014, the Substance Abuse andMental Health Services Administration(SAMHSA), an agency within the U.S.government, added Thought Field Ther-apy to its list of National Registry of Evi-dence-based Programs and Practices.This registry is intended to educate thepublic regarding efficacious interventionsfor substance use disorders and other psy-

chological conditions (Satel & Lilienfeld,2016). The rationale for the inclusion ofThought Field Therapy is that this tech-nique has been demonstrated in multiplecontrolled studies to be efficacious forenhancing resilience and self-concept,and for diminishing trauma- and anxiety-related symptoms, depressive symptoms,and so on, when compared to wait-listcontrol conditions (http://nrepp.samhsa.gov/ProgramProfile.aspx?id=60).

• A growing number of practitioners ofhighly dubious interventions are noweagerly claiming the “evidence-based”mantle and advertising themselves usingthis moniker (Mercer & Pignotti, 2007).For example, websites for, and articles on,the following interventions describe thesetreatments explicitly as “evidence-based”:therapeutic drumming (https://wakeup-world.com/2015/04/07/6-evidence-based-ways-drumming-heals-body-mind-and-soul/); animal-assisted therapyfor eating disorders (https://www.remu-daranch.com/index.php); Thought FieldTherapy (https://www.thoughtfieldther-apy.net/tft-recognized-by-nrepp/); Emo-tional Freedom Techniques (Church,2013); Imago Relationship Therapy(https://www.newharbinger.com/evi-dence-based-therapies); Jungian sandplaytherapy (http://sandplayassociation.com/faqs/); primal therapy (http://primalther-apy.com.au/frequently-asked-ques-tions/); dance movement therapy(https://www.hochschule-heidelberg.de/en/academics/masters-degree/dance-movement-therapy/), abre-active hypnosis for PTSD (Barabasz,2013); acupuncture for clinical depression(https://www.alternativementalhealth.com/evidence-based-uses-of-chinese-medical-therapies-in-the-treatment-of-depressed-mood/); and neurolinguisticprogramming (NLP; Zaharia, Reiner, &Schütz, 2015). Furthermore, recentunpublished survey data suggest thatlarge majorities of practitioners whoadminister non-evidence-based interven-tions for anxiety disorders neverthelessdescribe themselves as offering “evi-dence-based” services (Deacon, personalcommunication).

Superficially, it might seem straightfor-ward to address all three of the aforemen-tioned trends by means of logic alone. Afterall, one might presume, energy therapies—to take merely one example—cannot pos-sibly be evidence-based given that theirtheoretical foundation is exceedingly

C A L L F O R S C I E N C E - B A S E D P R A C T I C E



implausible scientifically. Almost surely, itis not true that humans are surrounded byinvisible and unmeasurable energy fields,let alone that blockages or disruptions inthese fields are the central causes of psy-chological distress. Hence, this reasoningcontinues, energy therapies cannot possi-bly satisfy the research leg of EBP.

Nevertheless, given how this leg of EBPis presently operationalized in Americanclinical psychology, proponents of energytherapies are equipped with an effectiverebuttal: If one relies exclusively on con-trolled outcome data on energy therapies,one can make a reasonable case that theseinterventions are in fact supported byresearch evidence. Why? Because con-trolled studies reveal that energy therapiestypically outperform wait-list control con-ditions (Feinstein, 2008, 2012). Indeed,when the first author, among others, hasasked members of the APA EducationDirectorate why sponsors who offeredcourses on energy therapies were approvedfor CE credit, they referred in part to thepublished research support for these inter-ventions.

Of course, energy therapy critics couldrespond with considerable justificationthat this apparent efficacy almost certainlyderives from nonspecific influences, suchas placebo effects, regression effects, spon-taneous remission, and perhaps mostimportant, the incidental repeated expo-sure that accompanies the intervention(Bakker, 2013; Pignotti & Thyer, 2009).Nevertheless, the APA Division 12 (Societyof Clinical Psychology) criteria for ESTs,which constitutes by far the most influen-tial instantiation of the research prong ofEBP, require only that a treatment mustoutperform a no-treatment control condi-tion in two or more randomized controlledtrials or systematic within-subject designs(Chambless & Hollon, 1998; http://www.div12.org/psychological-treatments/frequently-asked-questions/). Energy ther-apies may very well meet this lax criterion.Hence, the APA Education Directorate,which approves CE sponsors, may have itshands tied when it comes to approvingsuch interventions. The same problemarises for a number of the other interven-tions listed three paragraphs earlier. Usingthe current EST criteria, a host of otherpseudoscientific and otherwise question-able interventions, such as animal-assistedtherapies of many stripes (e.g., dolphin-assisted therapy and equine-assisted thera-pies; see Anestis, Anestis, Zawilinski, Hop-kins, & Lilienfeld, 2014; Marino &Lilienfeld, 2007), dance therapies for severe

psychopathology, and NLP may also clearthe evidence-based research bar.

When it comes to proponents of thesetreatments claiming evidence-based status,some readers might reasonably contendthat it is unfair to lay the blame on EBP. Allconcepts can be misused, as the principle ofabusus non tollit usum (the abuse of aclaim does not invalidate its proper use)reminds us. Neverthleless, in many casesthese proponents can legitimately lay claimto fulfilling the research leg of the EBP stoolgiven the current EST criteria, which focusexclusively on outcome data. Hence, EBPleaves the door wide open for preciselysuch misuse.

The Remedy: Science-Based PracticeFortunately, there is at least a partial

solution to the aforementioned problems:science-based practice (SBP). In SBP, as inscience-based medicine (SBM; Gorski &Novella, 2014), treatment outcome data arenot the only source of data bearing on theresearch evidence for interventions.Instead, in SBP, treatment outcome dataare considered along with broader researchevidence bearing on the plausibility of thetreatment’s theoretical rationale when eval-uating an intervention’s scientific status.That is, in SBP, all forms of research evi-dence are relevant when evaluating the sci-entific status of an intervention. If the treat-ment is based on a grossly implausibletheoretical rationale, one that runs counterto what research has consistently demon-strated about how the natural world works,it should not be regarded as fully evidence-based, even if supported by promising out-come data.

By the workings of the “natural world,”we include the laws of physics in additionto well-established principles regarding thefunctioning of the human mind. As notedearlier, energy therapies conflict sharplywith research evidence derived fromphysics. Or, to take an example from themore psychological realm, primal therapyrests on the supposition that mentalanguish in adulthood results from therepression of unbearable psychologicalpain emanating from traumatic experi-ences in infancy or early childhood, insome cases the trauma of birth. Such paincan purportedly be released and expungedby repeated screaming. There is no com-pelling or even suggestive evidence for anyof these assertions (Singer & Lalich, 1996).

As noted earlier, some authors haveanticipated our arguments. In medicine,Gorski and Novella (2014) and Sampson

and Atwood (2005), among others, advo-cated for a Bayesian approach, in whichtreatment outcome data are integratedwith the a priori likelihood of the treat-ment’s efficacy (“Bayesian prior probabil-ity”; see also Lilienfeld, 2011) in ascertain-ing an intervention’s scientific status.Further, in a useful analysis, David andMontgomery (2011) proposed that the ESTcriteria be expanded to incorporate evi-dence for a given psychotherapy’s theoret-ical rationale. Specifically, they suggestedthat parallel criteria be employed to evalu-ate the plausibility of a treatment’s theoret-ical rationale as that currently employed toevaluate its empirical status, namely, twowell-conducted supportive studies. Yetbecause theories are underdetermined byscientific evidence (Laudan, 1990), twosupportive studies are almost always insuf-ficient to provide compelling evidence for atreatment’s theoretical rationale. Anotherlimitation of David and Montgomery’sframework is its invocation of a categoricalcutoff for theoretical support (two studies),which does not necessitate consideration ofthe full body of high-quality scientific evi-dence bearing on the evidence for andagainst a treatment’s rationale (Lilienfeld,2011).

To be sure, the second limitationapplies to the Division 12 criteria for ESTsas well. In this respect, we side with Tolin,McKay, Forman, Klonsky, and Thombs(2015), who maintained that the currentEST criteria should be superseded by amuch more comprehensive approach topsychotherapy and assessment methodsevaluation that includes all relevant data ontreatment outcomes, along with a carefulanalysis of the methodological rigor of therelevant studies (see also Miller &Wilbourne's 2002 “mesa grande” approachto evaluating the strength of evidence foralcohol use disorder treatments; and thetheoretically motivated approach to cogni-tive ability assessment of Riley, Combs,Davis & Smith, 2017). In SBP, the sameprinciple should hold for the evaluation ofresearch evidence for the treatment ratio-nale, namely, a comprehensive analysis ofall relevant high-quality data.

In an important but largely neglectedarticle, entitled “Psychotherapy Is the Prac-tice of Psychology,” Sechrest and Smith(1994) argued that the practice of psy-chotherapy, as well as psychotherapyresearch, must be informed by broaderknowledge of psychology, includingresearch in neuroscience, affect, cognition,learning, social psychology, personality,culture, development, and other subfields.

C A L L F O R S C I E N C E - B A S E D P R A C T I C E

January • 2018 45

Their article is worth quoting from atlength:

A psychologically integrated psy-chotherapy will not be merely eclectic,for it will be guided by both the scien-tific theory and evidence available atany one time. . . . In our view . . . psy-chology is making great strides inknowledge about many aspects ofbehavior, e.g., in the workings of thebrain, in the genetic bases for behavior,in cognitive functions, in the course ofhuman development over the life span,and so on. These gains in knowledgeprovide a large, sound data base richwith implications for psychotherapy.It will be a shame if psychotherapycontinues as a fragmented enterpriseon the borders of psychology, limitedboth conceptually and scientifically byself-imposed insulation from what byits origins is its birthright. (p. 27)

Similar considerations apply to SBP.To properly appraise psychotherapies, weneed to consider not merely how well theywork when compared against wait-list con-trol conditions, but also whether they aregrounded in adequate scientific founda-tions, including basic psychological sci-ence.

SBP should help to solve several press-ing problems. First, SBP should begin tocurb the continued infiltration of pseudo-science into clinical practice, as many andarguably most poorly supported interven-tions rest on highly questionable theoreti-cal premises. Second, SBP offers a cogentcounterargument to assertions that scien-tifically dubious interventions that outper-form wait-list control conditions shouldqualify for CE credits or clinical practiceguidelines. Third, SBP renders it difficultfor advocates of energy therapies and otherhighly dubious interventions to dub them-selves “evidence-based,” which they canoften do now with some justification givencurrent EBP standards.

Potential ObjectionsWe can envision several potential objec-

tions to SBP; we briefly address four here.First, critics of SBP might contend that “if atreatment works, it works.” So, if we wishto be blindly empirical, we should regardenergy therapies as roughly equivalent towell-established ESTs in evidentiarystrength, as the controlled outcome datafor the former interventions are also sup-portive. Setting aside the question of

whether the outcome evidence for energytherapies is as persuasive as its advocatescontend, which is doubtful (see Pignotti &Thyer, 2009), this argument neglects thecrucial point that interventions with bla-tantly implausible theoretical rationales areunlikely to be both “efficacious and spe-cific” (Chambless & Hollon, 1998).That is,they are unlikely to display efficacy aboveand beyond nonspecific ingredients, suchas placebo effects, effort justification, or thegeneralized effects of attention and inter-personal support (Lilienfeld et al., 2014).As a consequence, they are far less likely tobe deserving of further research investiga-tion compared with other interventions,not to mention more efficacious than stan-dard interventions.

A second objection is that scientists aresometimes mistaken about how the naturalworld works, so it is illegitimate to considerresearch evidence bearing on a treatment’stheoretical rationale when evaluating itsscientific status. Scientific knowledgechanges, in some cases radically. As onefamiliar example, German geophysicistAlfred Wegener was dismissed by somescientists as a crackpot after introducinghis theory of continental drift in 1912, asthe idea that the continents move struckthem as preposterous. As we know, how-ever, Wegener was later vindicated by stud-ies in plate tectonics, paleontology, andother disciplines (McComas, 1995). But forevery Wegener, there are at least a thou-sand inventors of would-be perpetualmotion devices and mind-readingmachines (Sagan, 1995). More important,SBP, like EBP (see Gibbs & Gambrill, 2002;Lilienfeld et al., 2013), is not ossified, as itevolves in accord with new evidence. Ifphysicists were to uncover compelling evi-dence for the existence of energy fields sur-rounding the human body, or if psycholo-gists were to uncover compelling evidencefor the existence of internal subpersonali-ties, then energy therapies and internalfamily systems therapy, respectively, mightwarrant consideration as meeting SBP cri-teria.

A third objection is that we have notoffered explicit criteria for SBP status akinto those for ESTs. To this objection, weplead guilty, as we do not intend to proposea specific operationalization of SBP here,although we hope to do so in a future com-munication. At this juncture, we will sayonly that to meet full SBP status, the twoprongs of (a) controlled research outcomeevidence and (b) evidence for the scientificrationale are both necessary, though nei-ther in isolation is sufficient. The full

details of a proposed SBP operationaliza-tion for mental health care, however, haveyet to be fleshed out.

A fourth objection is that the theoreticalrationale for many well-established orpromising psychological treatments,including exposure treatments, remain indispute or are incompletely understood(Lilienfeld, 2011). Nevertheless, our goal inthis brief communication is modest:namely, to present SBP as an overarchingframework that can serve as a partial safe-guard against interventions whose theoret-ical rationales are markedly at variancewith well-replicated scientific evidence. Weare far less concerned about interventionswhose rationales are inadequately under-stood than those whose rationales areexceedingly implausible from a scientificstandpoint. In this respect, SBP should beable to function as a partial bulwark againstthe ongoing intrusion of pseudoscienceinto clinical work, evidence-based practiceguidelines, graduate education and train-ing, and continuing education courses.

Concluding ThoughtsEBP has been an essential step toward

grounding the field of clinical psychologymore firmly in science. Nevertheless, it hasnot gone far enough, as it has failed to oper-ate as an effective safeguard against thepenetration of pseudoscience into myriaddomains, including continuing educationcourses, clinical practice guidelines, andthe marketing and promotion of interven-tions. SBP, although not a panacea, shouldnudge the field in the direction of astronger scientific foundation. By incorpo-rating evidence from all relevant science,including the natural sciences (e.g., physics,chemistry), rather than merely treatmentoutcome evidence, SBP should help to pre-vent advocates of treatments based ongrossly implausible theoretical rationalesfrom laying claim to the coveted evidence-based mantle.

Although we have focused our analysison psychological treatment, many or mostof the same considerations we have raised(e.g., Sechrest & Smith, 1994) apply inequal force to psychological assessment(see Bowden, 2017). For example, in neu-ropsychological assessment, good scientifictheory plays a critical role in test scoreinterpretation. Neuropsychological assess-ments rooted in stronger theory are notonly likely lead to more interpretableassessments, but are also likely to reducedecision errors, because the assessment ismotivated by a theory that will have under-



gone more rigorous evaluation and replica-tion (Riley et al., 2017). One theory thataccounts for a vast array of neuropsycho-logical data is the Cattell-Horn-Carroll(CHC) model, an integration of the empir-ical work of the three eponymous authorsover many decades (McGrew, 2009) thathas been validated across diverse popula-tions and clinical conditions (Jewsbury,Bowden, & Duff, 2016; Jewsbury, Bowden,& Strauss, 2016). This model articulatesseveral different cognitive ability con-structs that have historically been groupedunder the broad rubric of “executive func-tion.” The latter atheoretical grouping isillustrated by the cognitive ability taxon-omy of DSM-5 (American PsychiatricAssociation, 2013). Nevertheless, if multi-ple constructs are assessed and interpretedtogether exclusively as a global construct,the risks of confounded assessments andclinical decision errors are exacerbated(Jewsbury & Bowden, 2017).

In closing, we encourage more explicitintegration of SBP into CE courses as wellas into graduate training and education,including clinical supervision and formalcoursework. Current and would-be mentalhealth professionals need to conceptualizeand evaluate clinical practice, includingpsychotherapy and assessment, within thebroader context of basic and applied sci-ences. This more encompassing perspec-tive on evidence should help to immunizethem against the seductive charms of pseu-doscience.

ReferencesAmerican Psychological Association Pres-

idential Task Force on Evidence-BasedPractice (2006). Evidence-based practicein psychology. American Psychologist, 61,271–285. http://dx.doi.org/10.1037/0003-066X.61.4.271.

American Psychiatric Association. (2013).Diagnostic and statistical manual ofmental disorders (DSM-5; 5th ed.).Washington, DC: Author.

Anderson, N. B. (2006). Evidence-basedpractice in psychology. American Psy-chologist, 61, 271-285.

Anestis, M. D., Anestis, J. C., Zawilinski, L.L., Hopkins, T. A., & Lilienfeld, S. O.(2014). Equine‐related treatments formental disorders lack empirical support:A systematic review of empirical investi-gations. Journal of Clinical Psychology,70, 1115-1132.

Bakker, G. M. (2013). The current status ofenergy psychology: Extraordinary claimswith less than ordinary evidence. TheClinical Psychologist, 17(3), 91-99.

Barabasz, A. (2013). Evidence based abre-active ego state therapy for PTSD. Ameri-can Journal of Clinical Hypnosis, 56, 54-65.

Bowden, S. C. (Ed.). (2017). Neuropsycho-logical assessment in the age of evidence-based practice: Diagnostic and treatmentevaluations. New York, NY: Oxford Uni-versity Press.

Canadian Psychological Association.(2012). Evidence-based practice of psycho-logical treatments: A Canadian perspec-tive (Report of the CPA Task Force onEvidence-Based Practice of PsychologicalTreatments). Ottawa, Ontario: Author.http://www.cpa.ca/docs/File/Practice/Report_of_the_EBP_Task_Force_FINAL_Board_Approved_2012.pdf

Chambless, D. L., & Hollon, S. D. (1998).Defining empirically supported thera-pies. Journal of Consulting and ClinicalPsychology, 66, 7-18.

Church, D. (2013). Clinical EFT as an evi-dence-based practice for the treatment ofpsychological and physiological condi-tions. Psychology, 4, 645-654.

David, D., & Montgomery, G. H. (2011).The scientific status of psychotherapies:A new evaluative framework for evi-dence-based psychosocial interventions.Clinical Psychology: Science and Practice,18, 89–99.

Deacon, B. (2017, June). Message to theSociety for a Science of Clinical Psychol-ogy (SSCP) listserv.

Feinstein, D. (2008). Energy psychology: Areview of the preliminary evidence. Psy-chotherapy: Theory, Research, Practice,Training, 45, 199-213.

Feinstein, D. (2012). Acupoint stimulationin treating psychological disorders: Evi-dence of efficacy. Review of General Psy-chology, 16, 364-380.

Gibbs, L., & Gambrill, E. (2002). Evidence-based practice: Counterarguments toobjections. Research on Social WorkPractice, 12, 452-476.

Gorski, D.H., & Novella, S.P. (2014). Clin-ical trials of integrative medicine: Testingwhether magic works? Trends in Molecu-lar Medicine, 20, 473-476.

Guyatt, G., Cairns, J., Churchill, D., Cook,D., Haynes, B., Hirsh, J., ... Sackett, D.(1992). Evidence-based medicine: A newapproach to teaching the practice ofmedicine. Journal of the American Med-ical Association, 268, 2420-2425.

Hall, H. (2011). Evidence-based medicine,tooth-fairy science, and Cinderella medi-cine. Skeptic, 17(1), 4–5.

Hipol, L.J., & Deacon, B.J. (2013). Dissem-ination of evidence-baed practices foranxiety disorders in Wyoming: A surveyof practicing psychotherapists. BehaviorModification, 37, 170-188.

Hunsley, J., & Mash, E. J. (2007). Evi-dence-based assessment. Annual Reviewof Clinical Psychology, 3, 29-51.

Jewsbury, P. A., & Bowden, S. C. (2017).Construct validity has a critical role inevidence-based neuropsychologicalassessment. In S. C. Bowden (Ed.), Neu-ropsychological assessment in the age ofevidence-based practice: Diagnostic andtreatment evaluations (pp. 33-63). NewYork: Oxford University Press.

Jewsbury, P. A., Bowden, S. C., & Duff, K.(2016). The Cattell–Horn–Carroll modelof cognition for clinical assessment. Jour-nal of Psychoeducational Assessment, 35,1-21.

Jewsbury, P. A., Bowden, S. C., & Strauss,M. E. (2016). Integrating the switching,inhibition, and updating model of execu-tive function with the Cattell-Horn-Car-roll model. Journal of Experimental Psy-chology: General, 145, 220-245.

Kazdin, A. E. (2008). Evidence-basedtreatment and practice: Newopportuni-ties to bridgeclinical research and prac-tice, enhance the knowledge base, andimprove patient care. American Psycholo-gist, 63, 146–159.

Larrabee, G. J. (2000). Associationbetween IQ and neuropsychological testperformance: Commentary on Tremont,Hoffman, Scott, and Adams (1998). TheClinical Neuropsychologist, 14, 139-145.

Laudan, L. (1990). Demystifying underde-termination. Minnesota Studies in thePhilosophy of Science, 14(1990), 267-297.

Layard, R., & Clark, D. M. (2014). Thrive:The power of evidence-based psychologicaltherapies. London: Penguin UK.

Lilienfeld, S. O. (2011). Distinguishing sci-entific from pseudoscientific psychother-apies: Evaluating the role of theoreticalplausibility, with a little help from Rev-erend Bayes. Clinical Psychology: Scienceand Practice, 18, 105-112.

Lilienfeld, S.O., Lynn, S.J., & Lohr, J.M.(Eds.). (2014). Science and pseudosciencein clinical psychology. New York: Guil-ford.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J.,Cautin, R. L., & Latzman, R. D. (2013).Why many clinical psychologists areresistant to evidence-based practice: Rootcauses and constructive remedies. Clini-cal Psychology Review, 33, 883-900.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J.,Cautin, R. L., & Latzman, R. D. (2014).Why ineffective psychotherapies appearto work: A taxonomy of causes of spuri-ous therapeutic effectiveness. Perspectiveson Psychological Science, 9, 355-387.

Marino, L., & Lilienfeld, S. O. (2007). Dol-phin-assisted therapy: More flawed dataand more flawed conclusions. Anthro-zoös, 20, 239-249.

January • 2018 47

L E G A L R E C O U R S E

McComas, W. F. (1996). Ten myths of sci-ence: Reexamining what we think weknow about the nature of science. SchoolScience and Mathematics, 96, 10-16.

McGrew, K. S. (2009). CHC theory andthe human cognitive abilities project:Standing on the shoulders of giants ofpsychometric intelligence research. Intel-ligence, 37, 1-10.

Mercer, J., & Pignotti, M. (2007). Shortcutscause errors in systematic research syn-theses: Rethinking evaluation of mentalhealth interventions. Scientific Review ofMental Health Practice, 5, 59-77.

Miller, W. R., & Wilbourne, P. L. (2002).Mesa Grande: A methodological analysisof clinical trials of treatments for alcoholuse disorders. Addiction, 97, 265-277.

Pignotti, M., & Thyer, B. A. (2009). Use ofnovel unsupported and empirically sup-ported therapies by licensed clinicalsocial workers: An exploratory study.Social Work Research, 33, 5-17.

Riley, E.N., Combs, H.L., Davis, H.A.,Smith, G.T. (2017). Theory as evidence:Criterion validity in neuropsychologicaltesting. In S. C. Bowden (Ed.), Neu-ropsychological assessment in the age ofevidence-based practice: Diagnostic andtreatment evaluations (pp. 15-31). NewYork: Oxford University Press.

Sackett, D. L., Rosenberg, W. M. C., Gray,J. A. M., Haynes, R. B., & Richardson, W.S. (1996). Evidence-based medicine:What it is and what it isn't. British Med-ical Journal, 312, 71–72.

Sagan, C. (1995). The demon-hauntedworld: Science as a candle in the dark.New York: Random House.

Sampson, W., & Atwood, K. (2005). Prop-agation of the Absurd: Demarcation ofthe Absurd revisited. Medical Journal ofAustralia, 183, 580-581.

Satel. S.L., & Lilienfeld, S.O. (2016, March29). You won’t believe the government issupporting this crackpot therapy.https://www.forbes.com/sites/sallysa-tel/2016/03/29/you-wont-believe-the-government-is-supporting-this-crackpot-mental-health-therapy/#20eafcd34732

Sechrest, L., & Smith, B. (1994). Psy-chotherapy is the practice of psychology.Journal of Psychotherapy Integration, 4,1-29.

Singer, M., & Lalich, J. (1996). Crazy thera-pies: What are they? How do they work?San Francisco: Jossey-Bass.

Spring, B. (2007). Evidence-based practicein clinical psychology: What it is; why itmatters; what you need to know. Journalof Clinical Psychology, 63, 611–631.

Straus, S. E., Glasziou, P., Richardson,W.S., & Haynes,R. B. (2010). Evidence-basedmedicine: How to practice and teach EBM(4th ed.). New York: Churchill Liv-ingston.

Stuart, R. B., & Lilienfeld, S. O. (2007). Theevidence missing from evidence-basedpractice. American Psychologist, 62, 615-616.

Tavris, C. (2014). The scientist-practi-tioner gap: Revisiting “A view from thebridge” a decade later. Foreword in S.O.Lilienfeld, S.J. Lynn, & J. Lohr (Eds.), Sci-ence and pseudoscience in clinical psy-chology (pp. ix-xx). New York: Guilford.

Thyer, B. A., & Pignotti, M. (2016). Scienceand pseudoscience in social work. NewYork: Oxford University Press.

Tolin, D. F., McKay, D., Forman, E. M.,Klonsky, E. D., & Thombs, B. D. (2015).Empirically supported treatment: Rec-ommendations for a new model. ClinicalPsychology: Science and Practice, 22, 317-338.

Von Ranson, K. M., Wallace, L. M., &Stevenson, A. (2013). Psychotherapiesprovided for eating disorders by commu-nity clinicians: infrequent use of evi-dence-based treatment. PsychotherapyResearch, 23, 333-343.

Zaharia, C., Reiner, M., & Schütz, P.(2015). Evidence-based neuro linguisticpsychotherapy: A meta-analysis. Psychia-tria Danubina, 27, 355-363.

. . .

The authors have no funding or conflicts ofinterest to disclose.Correspondence to Scott O. Lilienfeld,Ph.D., Department of Psychology, Room473, Emory University, 36 Eagle Row,Atlanta, GA, 30322; [email protected]

SINCE THE EARLY 90s, the field of psy-chotherapy has witnessed a proliferation ofpseudotherapies—seemingly scientifictreatments that are not actually based onscientific principles (Lilienfeld, Lynn, &Lohr, 2015b). As a result, consumers andmental health professionals are increas-ingly vulnerable to pseudoscientific pro-motions. Pseudotherapies have infiltratedpopular psychological discourse andthreaten to erode the scientific foundationsof clinical psychology. They also posepotential harms to the public. The majorityof pseudotherapies are ineffective ratherthan iatrogenic (Lilienfeld, 2007). How-ever, even when not overtly harmful, theymay inflict indirect harm by deprivingindividuals of time and money that could

have been allocated to other efficacious andeffective treatments (Lilienfeld et al.,2015b).

The current measures in place to pro-tect the field of clinical psychology and thepublic from pseudotherapies are inade-quate. Rather than issuing practice guide-lines and sanctioning the practitioners ofthese treatments, the American Psycholog-ical Association (APA) and various gov-ernmental organizations have arguablycontributed to their proliferation. Forinstance, the APA offers continuing educa-tion credits for Jungian sandtray therapyand psychological theater (Lilienfeld, Lynn,& Lohr, 2015a). Recently, the SubstanceAbuse and Mental Health Services Admin-istration (SAMHSA) added Thought Field

Therapy to its National Registry of Evi-dence-Based Programs and Practices as aneffective treatment for trauma and otherconditions (Lilienfeld & Satel, 2016).

The fact that we cannot rely on profes-sional associations and national agencies toprotect us from ineffective and harmfulpseudotherapies underscores the impor-tance of legal recourse for the consumerand the concerned professional.

But what legal recourse do psycholo-gists and consumers have? The answervaries by jurisdiction.

This article examines the legal strategiesthat have been used to curtail the practiceof pseudoscientific therapies. In particular,it examines two legal approaches that havebeen used to curtail the practice of sexualorientation change efforts (SOCE), a cate-gory of pseudoscientific therapies designedchange a person’s sexual orientation fromlesbian, gay, or bisexual (LGB) to hetero-sexual.

The first strategy is the enactment oftargeted legislation by states to prohibit thepractice of SOCE. The second is the use of

Pseudotherapies in Clinical Psychology:What Legal Recourse Do We Have?Lisa A. Napolitano, CBT/DBT Associates, New York

N A P O L I T A N O


the state’s consumer fraud laws to sueSOCE practitioners. These approaches arecompared and evaluated as potentialmodels for legal approaches that can beused to curtail the practice of SOCE andother pseudotherapies.

Legislative Bans Targeting SOCEand the California Model

In 2012, California enacted legislationmaking it illegal for a mental healthprovider to practice SOCE with a minorunder the age of 18. SB 1172 was landmarklegislation, making California the first statein the nation to restrict the practice ofSOCE. Under SB 1172, the practice ofSOCE is considered unprofessional con-duct and provides grounds for the therapistto lose his or her license.

The basis for California’s legislation wasthe state’s “compelling interest in protect-ing the physical and psychological well-being of minors . . . and in protecting itsminors against exposure to serious harmscaused by sexual orientation changeefforts” (S.B. 1172, § 1 (n), 2011-2012, S.Reg. Sess. (Cal. 2012)).

First Amendment ChallengesImmediately after it was enacted, the

legislation was challenged by mental healthpractitioners on First Amendmentgrounds as an infringement of protectedspeech (Pickup v. Brown, 2014).

The primary issue raised by this FirstAmendment challenge was whether theCalifornia legislation regulated a therapist’sprofessional conduct or whether it inhib-ited constitutionally protected speech.

The regulation of professional speech isnot a well-defined area of First Amend-ment law. The court acknowledged thatmental health professionals have a FirstAmendment right to express their opinionsin public. However, this protection dimin-ishes for speech uttered in the context ofthe therapist-client relationship. Further, it“ultimately ceases when it is uttered in acontext exclusively regulated by theaccepted standards of professional con-duct” (Victor, 2014, p. 1555).

The court ultimately decided the legis-lation was a regulation of conduct that onlyincidentally regulates speech. The rationalefor this decision was twofold. First, thecourt noted that California has the author-ity to prohibit licensed mental health pro-fessionals from providing therapies that thelegislature has deemed harmful. Second,the fact that speech is used to carry outthose therapies does not transform the reg-

ulation into one of speech (Victor, 2014).As such, it is outside the scope of FirstAmendment protection.

Infringement on Parental RightsThe California legislation was also chal-

lenged as an infringement of parents’ rightsto control their children’s upbringing andmake important medical decisions forthem. In other words, parents should havethe right to choose SOCE for their children(Pickup v. Brown, 2014).

The court acknowledged that althoughparents have a constitutionally protectedright to make decisions regarding the care,custody, and control of their children, thisright is not without limitations. If thechild’s mental health is jeopardized, thestate has the right to intervene to protectthis child.

For this challenge, the central issuebefore the court was whether parents’ fun-damental rights include the right to choosea therapy for their children that the statehas deemed harmful. The court stated thatparents could not compel the state topermit licensed mental health profession-als to engage in unsafe practices and cannotdictate standard of care in California basedon their own views. It concluded that thefundamental rights of parents do notinclude the right to choose a specific med-ical or mental health treatment that thestate has reasonably deemed harmful(Pickup v. Brown, 2014).

California Legislation as a Model forOther States

Although SB 1172 passed constitutionalmuster, similarly crafted legislation wouldbe vulnerable to constitutional challenges.

One of the reasons for this vulnerabilityis that the state may not be able to meet itsburden in showing that SOCE or anotherpseudotherapy causes harm that the statehas a compelling interest to protect against.California increased the likelihood of meet-ing this burden by recognizing two cate-gories of harm: (a) the cause or exacerba-tion of psychiatric disorders such asanxiety, depression, and suicidal behavior;and (b) the internalization of stigma andimpeded development of a positive LGBidentity.

To establish the first type of harm, theLegislature relied heavily on a report by aTask Force of the APA that surveyed allexisting literature on SOCE. The reportfound compelling evidence that physicallyinvasive forms of SOCE, such as aversiontherapy or conversion therapy, causeharmful mental health effects such as

increased anxiety, depression, suicidality,and loss of sexual functioning (APA, 2009).

However, the risks of SOCE methodsthat exclusively involve talk therapy andexclude physical techniques are less clearlydocumented. For these forms of SOCE, theAPA report concluded there is only anec-dotal evidence of harmful outcomes and itcould not definitively state how likely it isthat harm would occur from them (APA,2009). For this reason, the California legis-lature included the second type of harm.Reasoning that because SOCE is premisedon the notion that LGB status is pathologi-cal, the legislature concluded that SOCEimpedes the development of a healthy self-concept and self-acceptance, and con-tributes to the internalization of stigma.

Other States Adopt the CaliforniaModel to Target SOCE

Several states have adopted the Califor-nia model to target SOCE. However, theneed to produce clinical evidence ofSOCE’s harmfulness has limited the scopeof these legislative bans to conversion ther-apy because there is less evidence for theharms caused by noninvasive forms. Addi-tionally, these bans tend to be limited to thepractice of conversion therapy with minorswho are generally viewed as in greater needof the state’s protection than adults.

To date, eight states and the District ofColumbia have enacted legislation thatnarrowly prohibits the practice of conver-sion therapy with minors, rather thanSOCE more broadly and with all ages. Theyinclude Illinois, Nevada, New Mexico, NewJersey, Oregon, Rhode Island, and Ver-mont. New York is considering similar leg-islation.

Connecticut has enacted the broadestlegislation against conversion therapy,banning the practice with adults, as well aschildren. It has also banned the expendi-ture of public funds on this pseudotherapy.

As in California, legislation in NewJersey was challenged on First Amendmentgrounds by practitioners. The court ruledthat the state’s interest in protecting thepublic from harm outweighed the thera-pists’ free speech interests (King v. Gover-nor of New Jersey, 2014).

Limitations of California ModelAlthough California’s legislation seems

like a promising model for other states tocurtail the practice of SOCE and otherpseudotherapies, it has significant limita-tions.

First, adults tend not to be protectedfrom the harms of pseudotherapies by leg-

L E G A L R E C O U R S E

January • 2018 49

islative bans. Even in states with legislativebans, SOCE may be offered by any mentalhealth practitioner to an adult with theexception of Connecticut. Second, this reg-ulatory scheme does not prevent unli-censed providers, such as religious leaders,from administering SOCE or otherpseudotherapies. Nor does it preventlicensed mental health practitioners fromreferring children and adults to unlicensedpractitioners of SOCE and other pseudo-therapies.

From a legal perspective, California’slegislation and similarly modeled legisla-tion are particularly vulnerable to FirstAmendment challenges because it pro-hibits a type of speech partly on ideologicalgrounds.

To withstand constitutional challenge,legislation enacted by the states must showthe state has a compelling interest to pro-tect the public from harm. Accordingly, inthe absence of clear evidence of harm, thelegislative approach to curtail the practiceof therapies is not effective. Because thevast majority of pseudotherapies aremerely ineffective rather than harmful, thisapproach is not optimal.

Consumer Fraud Acts andthe New Jersey Model

Given the limitations inherent in a leg-islative approach, a more promising strat-egy for the restriction of SOCE and otherpseudotherapies may be the use of thestates’ Consumer Fraud Acts.

Pseudotherapies lack scientific evidenceof efficacy and effectiveness. Yet, mostpractitioners of these treatments make mis-leading claims to the public about theirsuccess for treating problems. For exam-ple, SOCE practitioners misleadingly holdthemselves out as being able to “convert”patients from LGB to heterosexual. RogerCallahan and other practitioners of VoiceTherapy, a variant of Thought Field Ther-apy, have claimed 97% to 98% cure rates forall emotional disorders (Callahan & Calla-han, 2000).

These misleading claims by therapistsarguably fall under a broader existing legalregime that defines them as fraud (Victor,2014). An antifraud approach casts a widernet than targeted legislation and could beused to address pseudotherapies that aremerely ineffective rather than harmful. Italso closes many of the loopholes that existunder legislative bans. This approach canbe used to restrict the practice of pseudo-therapies by unlicensed practitioners andpractitioners who work with adults. Addi-

tionally, it is less vulnerable to constitu-tional challenge.

Ferguson v. JonahThe same year the California legislation

was enacted to restrict SOCE, anotherapproach was being taken in New Jerseyagainst this pseudotherapy.

In November 2012, a lawsuit was filedagainst a SOCE practitioner group calledJews Offering New Alternatives for Healing(JONAH) by a group of former patients.

The patients alleged that JONAH’spromise to cure them of their homosexual-ity was fraudulent and deceptive in viola-tion of New Jersey’s Consumer Fraud Act(CFA). Ferguson v. JONAH (2015) is alandmark case—the first consumer fraudclaim filed against conversion therapists inthe nation (Dubrowski, 2015).

The bases for the fraud claim were threekey misrepresentations made by JONAH:first, homosexuality is a mental disorder;second, sexual orientation can be changed;and third, that JONAH’s practices werewell grounded in science and that there was“empirical evidence” supporting their effi-cacy. Other misrepresentations includedthat the program’s success rate was 66%and that it worked on a specified timeframe (Dubrowski, 2015).

In addition to advertisements for indi-vidual and group therapy, the main evi-dence of fraud came from JONAH’s listserve and emails to potential clients. Theplaintiffs also testified that they had beenpersonally assured they had a two out ofthree chance of changing their sexual ori-entation.

Bolstering the plaintiffs’ case for fraudwas the 2009 APA report discrediting anytreatment model that purports to changesexual orientation. After a systematicreview of the research on the efficacy ofsexual change efforts, the APA’s reportconcluded that claims of the effectivenessof SOCE for changing sexual orientationare not supported.

The APA also filed an amicus brief forthe plaintiffs stating that the consensus ofmental health professionals and research-ers is that homosexuality is a normalexpression of sexuality (Dubrowski, 2015).

To counter the claims of fraud, JONAHsubmitted reports from six experts includ-ing four conversion therapists, one medicaldoctor, and one rabbi. They all testified thathomosexuality is not universally acceptedas normal. Rather, they asserted thathomosexuality is a learned response tochildhood “wounds” and is addressablethrough therapy.

The court found JONAH’s expert testi-mony inadmissible. New Jersey, like manyother states, has adopted the Frye test todetermine the admissibility of expert testi-mony (United States v. Frye, 1923). Underthis test, the reliability of expert testimonydepends on whether it has general accep-tance in its field. The court found that “theoverwhelming weight of scientific author-ity concludes that homosexuality is not adisorder” (Ferguson v. Jonah, 2015, p. 19).It also noted that a “group of a few closelyassociated experts cannot incestuously val-idate one another in order to establish thereliability of their shared theories” (Fergu-son v. Jonah, p. 26).

The New Jersey court ruled thatJONAH had violated the consumer fraudact by stating that homosexuality is not anormal variant of sexuality. After only 3hours of deliberation, the jury found thedefendants were guilty of unconscionableconsumer fraud.

In addition to attorneys' fees and dam-ages, the plaintiffs were granted injunctiverelief and the JONAH clinic was perma-nently closed (Dubrowski, 2015).

The verdict in JONAH has beendescribed as a potential “coup de grace tothe remaining providers of conversiontherapy in the United States” (Dubrowski,p. 79).

The case provides a powerful model forlawsuits in other states that can be used tocurtail the practice of conversion therapy,and potentially other pseudotherapies.

Implementing Ferguson to Target SOCEEvery state has a consumer protection

law that grants private citizens the right toenforce it through civil causes of action.Accordingly, implementation of the Fergu-son model means that individual victims ofSOCE would bring lawsuits against SOCEpractitioners for deceptiveness-based pro-fessional conduct.

This antideception approach to SOCE isconsistent with the ways in which manystates currently regulate the advertising oflicensed therapists. For example, Californiahas a provision that prohibits public com-munications by psychologists that containfalse, fraudulent, or misleading statements.This includes any claims intended toinduce or likely to induce services thatcannot be substantiated by reliable, peer-reviewed, and published scientific studies.Under these provisions, an offending prac-titioner can be de-licensed (Victor, 2014).

This strategy has been used successfullyin Arizona to limit the practice of VoiceTechnology (VT), a variant of Thought


N A P O L I T A N O

Field Therapy (TFT). In 1999, the ArizonaBoard of Psychologists sanctioned a psy-chologist for making false advertisingclaims of a 95% success rate for VT andforbid him from practicing both VT andTFT (Pignotti, 2007).

An antideception approach can be usedto expand the protection of legislative bansagainst SOCE. For example, adult citizensof California who are currently not pro-tected by the legislative ban can sue SOCEpractitioners for fraud.

Similarly, in states with narrower leg-islative bans against conversion therapyonly, the antideception approach expandsprotection against SOCE broadly for adultsand children.

Under some state antideception laws,it’s not necessary that the individual havereceived the services to file the complaint.For example, California law provides that“anyone who thinks that a psychologist hasacted illegally or irresponsibly can file acomplaint” (Victor, 2014, p. 1574).

The primary disadvantage to the use ofthe consumer fraud acts to curtail the prac-tice of SOCE is its slow pace. Cases must bebrought on an individual basis. However, ifa sufficient number of individual com-plaints are brought, the state board of psy-chology could adopt a regulation that clar-ifies that SOCE advertising and SOCEefforts within a doctor-patient relationshipare covered under the state’s definition ofunprofessional conduct.

The Comparative Benefits of anAntifraud Approach for TargetingSOCE and Other PseudotherapiesBecause there is consensus within the

mental health establishment that homosex-uality is not a disorder and that SOCEcannot change sexual orientation, SOCEfalls squarely within the ambit of an antide-ception regime (Victor, 2014). However,for other pseudotherapies that lack thisconsensus (e.g., TFT, past life regressiontherapy), an anti-deceptive approach maybe more challenging.

Nevertheless, in comparison to targetedlegislation, the use of state consumer fraudlaws to target pseudotherapies has severaladvantages.

First, logistically it’s easier to use exist-ing laws rather than have new laws passed.If a state doesn’t have a consumer fraudlaw, it’s likely easier to convince legislatorsto pass general antideception statutes thana targeted ban against a particular pseudo-therapy.

Second, this approach is less vulnerableto First Amendment challenges. Fraudu-lent or deceptive advertising is widely con-sidered to be outside the scope of the FirstAmendment and the government may banit. For this reason, there is a generalassumption that states may prohibit mis-leading advertising. States also have theauthority to regulate the conduct of psy-chologists and other licensed mental healthprofessionals. Restricting speech that isincidental to the regulation of professionalconduct is not considered a free speechrestriction at all. Consequently, prohibitinga psychologist from making deceptivepromises about a treatment’s efficacywould likely survive First Amendmentchallenges (Dubrowksi, 2015; Victor,2014).

Third, unlike a legislative ban, there isno need to show that the pseudotherapy isharmful and the state has a compellinginterest to protect against it. Because themajority of pseudotherapies are merelyineffective rather than harmful, a largernumber can be targeted with this approach.

Fourth, every state’s consumer fraudlaw provides a plaintiff who wins their caseequitable relief. This means the court canenjoin or stop the offending therapist fromcontinuing to perpetrate the fraud on thepublic. In JONAH, a permanent injunctionwas issued closing the clinic and prohibit-ing the JONAH therapists from ever prac-ticing conversion therapy again (Dubrow-ski, 2015).

Fifth, many states’ consumer fraud lawsdo not require that the practitioner of apseudotherapy knew or intended hisactions to be fraudulent. Consequently,actions brought under the ConsumerFraud Acts could target pseudotherapypractitioners who seem to believe firmly intheir treatments and the pseudoscientificbasis for them rather than seek to defraudthe public (Dubrowksi, 2015). For exam-ple, Roger Callahan is the creator of TFTand frequently described as a “truebeliever” in the effectiveness of his treat-ment, despite a lack of any scientific evi-dence to support it (Pignotti, 2007). Simi-larly, the SOCE practitioners at JONAHmay have genuinely believed in the effec-tiveness of their treatment.

For states that do require a showing ofintent, the burden can be met with experttestimony that there is no science to sup-port the efficacy of the pseudotherapy.Alternatively, expert testimony that there isa general consensus in the psychologicalcommunity that the practitioner’s state-

ments are false would suffice (Dubrowski,2015).

The intent requirement poses a poten-tial obstacle for those seeking to pursue apseudotherapy fraud action. While con-sensus within the psychological commu-nity certainly exists for conversion andrebirthing therapies, it does not for manyother pseudotherapies. The APA, forexample, offers continuing education cred-its for attachment therapy and EMDR.

ConclusionThe proliferation of pseudotherapies

poses harm to consumers of therapy andclinical psychologists. The inadequate pro-tection by professional associations andgovernmental agencies underscores theimportance of exploring legal remedies.

A review of the two primary legal strate-gies that have been used to curtail the prac-tice of SOCE suggests that pseudotherapiescould be effectively targeted through thestate’s antideception laws. This strategyseems to be an effective alternative to leg-islative bans and is relatively impervious tochallenge on constitutional grounds. Theuse of the states’ consumer fraud acts alsoobviates the need to establish the treatmentis harmful. Without the burden of showingharm, this strategy casts a wider net thantargeted legislation and can be used totarget a broader number of pseudothera-pies.

ReferencesAmerican Psychological Association, Task

Force on Appropriate TherapeuticResponses to Sexual Orientation. (2009).Report of the APA Task Force on Appro-priate Therapeutic Responses to SexualOrientation, 22-25. http://www.apa.org/pi/lgbt/resources/sexual-orientation.aspx.

Callahan, R.J., & Callahan, J. (2000). Stopthe nightmares of trauma. Chapel Hill,NC: Professional Press.

Dubrowski, P. (2015). The Ferguson v.Jonah verdict and a path towardsnational cessation of gay-to-straight“conversion therapy.” Northwestern Uni-versity Law School, 110, 77-99.

Ferguson v. JONAH, No. L-5473-12 (N.J.Sup. Ct. Law Div. 2015).

King v. Governor of the State of New Jersey,767 F. 3d 216 (3d Cir. 2014).

Lilienfeld, S. O. (1998). Pseudoscience incontemporary clinical psychology: Whatit is and what we can do about it. TheClinical Psychologist, 51(4), 3-9.

January • 2018 51

Lilienfeld, S. O. (2007). Psychologicaltreatments that cause harm. Perspectiveon Psychological Science, 2, 53-56.

Lilienfeld, S. O., Lynn, S.J., & Lohr, J.(2015a). Science and pseudoscience inclinical psychology: Initial thoughts,reflections, and considerations. In S.O.Lilienfeld, S.J. Lynn, & J.M. Lohr (Eds.),Science and Pseudoscience in ClinicalPsychology (pp., 1-16). New York: Guil-ford.

Lilienfeld, S. O., Lynn, S.J., & Lohr, J.(2015b). Science and pseudoscience inclinical psychology: Concludingthoughts, and constructive remedies. InS.O. Lilienfeld, S.J. Lynn, & J.M. Lohr(Eds.), Science and Pseudoscience in Clin-ical Psychology (pp., 527-532). NewYork: Guilford.

Lilienfeld, S. O., & Satel, S. (2016). Youwon’t believe the government is support-ing this crackpot mental health therapy.Forbes.com

Mercer, J. (2015). Attachment therapy. InS.O. Lilienfeld, S.J. Lynn, & J.M. Lohr(Eds.), Science and Pseudoscience in Clin-ical Psychology (pp., 466-499). NewYork: Guilford.

Pickup v. Brown, 740 F. 3d 1208 (9th Cir.2014).

Pignotti, M. (2007). Thought field ther-apy: A former insider’s experience.Research on Social Work Practice, 17(3),392-407.

Report of the American PsychologicalAssociation Task Force on AppropriateTherapeutic Responses to Sexual Orien-tion (2009). American PsychologicalAssociation, 22-25.

S.B. 1172, 2011-2012, S. Reg Sess (Cal.2012) (codified at Cal. Bus. & Prof. Code§ 865-865.2 (West 2013)

United States v. Frye, 293 F. 1013, 1014(D.C. Cir., 1923).

Victor, J. (2014). Regulating sexual orien-tation change efforts: The Californiaapproach, its limitations, and potentialalternatives. The Yale Law Journal, 123,1532-1585.

. . .

The author has no funding or conflicts ofinterest to disclose.Correspondence to Lisa Napolitano, J.D.,Ph.D., 501 Madison Avenue, Suite 303, NewYork, NY 10022;[email protected]

EXPOSING PSEUDOSCIENTIFIC practicescomes with a price. Although I knew thiswhen I began exposing such practices, Igreatly underestimated the magnitude ofvitriolic attacks from proponents of suchpractices; this has greatly impacted mycareer.

What follows is an account of my expe-rience in writing about the harmful effectsof “attachment therapies” and holding andcoercive restraint therapies used inaddressing behavioral problems, mostlywith foster and adopted children. Suchpractices are lacking in scientific support,and, in some cases, have resulted in greatharm, including death. Some critics havecharacterized this as torture. One of themost egregious examples of this is that of10-year-old Candace Newmaker, who in2000 was smothered to death by two unli-censed therapists in a rebirthing session,that consisted of placing pillows on top ofher and having four adults sitting on top ofher small frame, ignoring pleas that shecould not breathe (Mercer, Sarner, & Rosa,2003). However, despite the fact that achild died and a law was subsequentlypassed (Candace’s Law outlawingRebirthing Therapy), similar and equallytroubling practices continued (see Thyer &Pignotti, 2015, Chapter 3, for an overview).

Enough concern was raised about thesetypes of attachment therapies that a specialtask force was convened by the AmericanPsychological Association and the Profes-sional Society on the Abuse of Children(APSAC; Chaffin et al., 2006), which com-piled a report to review and evaluate thesepractices. The APSAC report noted con-cern about:

. . . a variety of coercive techniques areused, including scheduled holding,binding, rib cage stimulation (e.g.,tickling, pinching, knuckling), and/orlicking. Children may be held down,may have several adults lie on top ofthem, or their faces may be held sothey can be forced to engage in pro-longed eye contact. Sessions may lastfrom 3 to 5 hours, with some sessionsreportedly lasting longer. (Chaffin etal., 2006, p. 79)

The report also noted that not allattachment-based interventions were dan-gerous, and that some of the more focusedshorter-term goal-directed interventionshave some evidence of efficacy. However,the more broadly focused and extensiveinterventions were the ones of concernbecause of their potential to do harm. Con-clusions and recommendations of theAPSAC task force included the following:

a. Treatment techniques or attach-ment parenting techniques involvingphysical coercion, psychologically orphysically enforced holding, physicalrestraint, physical domination, pro-voked catharsis, ventilation of rage,age regression, humiliation, withhold-ing or forcing food or water intake,prolonged social isolation, or assum-ing exaggerated levels of control anddomination over a child are con-traindicated because of risk of harmand absence of proven benefit andshould not be used.b. Prognostications that certain chil-dren are destined to become psy-chopaths or predators should never bemade based on early childhood behav-ior. These beliefs create an atmosphereconducive to overreaction and harshor abusive treatment. Professionalsshould speak out against these andsimilar unfounded conceptualizationsof children who are maltreated.c. Intervention models that portrayyoung children in negative ways,including describing certain groups ofyoung children as pervasively manipu-lative, cunning, or deceitful, are notconducive to good treatment and maypromote abusive practices. In general,child maltreatment professionalsshould be skeptical of treatments thatdescribe children in pejorative termsor that advocate aggressive techniquesfor breaking down children’s defenses.(Chaffin et al., 2006, p. 86)

Nevertheless, such practices still contin-ued to be promoted and used by bothlicensed and unlicensed practitioners. Itwas out of concern for the harm (and thepotential for harm) being done that the

Exposing Pseudoscientific Practices:Benefits and HazardsMonica Pignotti, Independent Scholar

P I G N O T T I


nonprofit organization Advocates for Chil-dren in Therapy (ACT), was formed in2003. Its mission statement is as follows:

Advocates for Children in Therapy(ACT) is a not-for-profit organiza-tion concerned with the methodsused in the treatment of children’smental health. Specifically withrespect to psychotherapy, parentingtechniques, and other mental-healthpractices applied to children, ACTadvocates humane, non-violent andscientifically validated treatments,and opposes the use of unvalidatedpractices, especially those known tobe inhumane and abusive by:

• Raising general public awarenessof the dangers and cruelty of suchpractices;• Opposing governmental supportand subsidy for such practices;• Alerting professional organiza-tions to inappropriate advocacyand promotion of such practices,such as in continuing educationprograms;• Urging appropriate authorities toestablish and then enforce stan-dards of care and professionalethics to effectively ban the use ofsuch practices;• Assisting, with information andadvice, in the prosecution of thosewho criminally defraud parentsand damage children by using suchpractices or by recommendingtheir use; and,• Obtaining some measure of jus-tice for the victims of such practicesthrough restitution and compensa-tion from the perpetrators (Advo-cates for Children in Therapy, n.d.,para. 1-2).

Obviously, these are all laudable goals thatfew ethical mental health professionals,especially those who take an evidence-based approach, would disagree with, inspite of the virulent attacks this organiza-tion has received from proponents of themethods that they exposed.

In 2006, I was honored to accept, as aservice to the profession, a position on theirBoard of Directors, where I served for 4years (2006–2010). Even though I had pub-lished peer-reviewed articles, the audiencefor such articles was small, and the generalpublic, especially potential consumers ofsuch practices, needed to be educated sothey could make informed choices abouttreatments for their children.

My interest in understanding andexposing the dangers of pseudoscientificpractices predates my mental healthdegrees, and began with my 6-year per-sonal experience in Scientology. After leav-ing Scientology in the late 1970s, I washighly motivated to understand how suchgroups operated to attract and retain mem-bers, as well as their practices, particularlywhen it came to the extraordinary mentalhealth claims being made that were basedlargely on testimonials and anecdotes,utterly lacking in scientific evidence. Out ofmy desire to learn and practice therapiesthat were noncoercive and actually helpedpeople, I obtained an M.S.W. from Ford-ham University in 1996. Following gradua-tion, I worked for 5 years at Saint Vincent'sHospital in Geriatric and Palliative Careresearch. Additionally, I had a private prac-tice in New York City as a certified socialworker. Unfortunately, since I did notcompletely understand evidence-basedpractice at the time, around 1997, I becameinvolved with Thought Field Therapy,which was invented by licensed psycholo-gist Roger Callahan (see Callahan & Calla-han, 2000, for a full description). Ulti-mately, this resulted in my conducting amuch-needed controlled experiment (Pig-notti, 2005) that showed this practice wasnot what it was claimed to be.

During my time in the Ph.D. programat Florida State University’s (FSU) Collegeof Social Work, I continued to publish arti-cles related to the exposure and critique ofpseudoscientific practices, including ques-tionable attachment and holding therapiesand coercive restraint therapies (Mercer &Pignotti, 2007; Pignotti & Mercer, 2007). Itwas these articles that made proponents ofsuch therapies aware of my work, which,needless to say, did not please them. Forexample, one of the proponents of suchtherapies contacted the dean at the univer-sity where one of the critics worked, alleg-ing that she was mentally unbalanced, hadpersonal problems, and was transsexual(which, in addition to the obvious bigotryin such an accusation, was not true).Although I was aware already of some indi-viduals who had been the target of viciouspersonal attacks and harassment after crit-icizing questionable mental health prac-tices, I was not expecting what was tofollow. In the summer of 2009, I defendedmy dissertation and was teaching classes atFSU’s College of Social Work.

In the spring of 2009, a blogger who wasa survivor of attachment therapy as a childand was exposing it on her blog, received athreatening email from one of the thera-

pists we had been criticizing, which sheshared with me. The email warned that hewould be exposing our sexual problemsand Scientology past. Weeks later, somevery derogatory anonymous postings, donethrough anonymous remailers impossibleto trace, began appearing on public Inter-net groups about me and my colleagues atACT. Bizarre advertisements were run onCraigslist and BackPage about me. Some ofthese advertisements were in the erotic sec-tion under my name. I had to cancel thetext function on my cell phone because Iwas receiving obscene texts from men whowere answering the ads. I received a phonecall from a man who wanted me to work athis erotic establishment. Some of the post-ings claimed that I was having an affairwith my Ph.D. dissertation chair and otherfaculty members and that I had been firedfrom FSU (I was not) and that I was unfit toteach. On the contrary, my course evalua-tions were acceptable, there had never beenany complaints against me, and I simplystopped teaching there after I graduated, asall Ph.D. students did (FSU does not hiretheir own former students). One personeven posted a bad review of me on “RateMy Professors” at the FSU PensacolaCampus (I never taught on that campus).

At the same time, one of the therapistswhom my colleagues in ACT had criticizedwrote a letter to the Dean of the College ofSocial Work complaining about me. Afterexplaining the situation to the dean, hechose to take no action. This was the sameindividual mentioned previously who hadalso complained to the dean of the univer-sity where one of my colleagues was a Pro-fessor Emerita, again fortunately, to noavail. An anonymous individual also sentbizarre emails to faculty members andother Ph.D. students at FSU about me. Thecontent was so incoherent that none of mycolleagues believed what was alleged inthem, but being a target was not helpful tomy reputation. Very little was known aboutcyberstalking at the time, and there werepeople who tended to take a "blame thevictim" attitude, wondering how I hadgotten myself into this situation. Somepeople believed the old adage "wherethere's smoke, there's fire," and were unableto entertain the idea that the targets of suchattacks were completely innocent and wereinstead being attacked for doing somethingto try to help others.

These attacks were happening at thesame time as I was on the job market for atenure-track faculty position, and this wasbeing brought up at some of my on-campus interviews. Although the faculty I

E X P O S I N G P S E U D O S C I E N T I F I C P R A C T I C E S

January • 2018 53

interviewed with were largely sympathetic,my situation was also difficult for most ofthem to understand and, to make thingseven worse, there were some who weresympathetic to some of the people I hadcriticized. Universities aligning themselveswith pseudoscientific practices is, unfortu-nately, not uncommon, and I had written apiece (Pignotti, 2007) exposing this prac-tice at a top-ranked school of social workwhere energy-tapping therapies were beingtaught. At the time, a well-meaningmember of the profession who was a strongproponent of evidence-based practice hadwarned me about this, saying that the socialwork profession was a small world andthere could be consequences for writingsuch pieces. I had dismissed this warning,believing that the better, more evidence-based establishments would see value inwhat I did and in fact, some of them did,but ultimately, not enough to want to hireme. In fact, one director of research of areputable university, after I had presentedmy research and other evidence to docu-ment the problem of pseudoscience in thesocial work profession, still dismissed thenotion that the profession had any problemand implied that my area of interest ininvestigating such usage by practitionerswas not a valid one. Even though I pointedout the high percentage of licensed clinicalsocial workers that are in private practice,he maintained that the agencies were allusing evidence-based practice now andthat there was no problem in the profes-sion, which ran contrary to my own inves-tigations and research (Pignotti & Thyer,2009, 2012). I had to wonder how manyothers with whom I had interviewed agreedwith this research director, but were toopolite to be as blunt as this particular indi-vidual had been.

After 5 years of a job search and over100 applications, I failed to obtain a facultyposition of any kind. While there is no wayfor me to prove a direct cause-and-effectrelationship, what I do know is that everyone of my peers in my Ph.D. cohort whosought faculty positions obtained them—and even though my credentials, teaching,and research experience were at leastequivalent to theirs (I had more peer-reviewed publications than anyone in mycohort), I was not able to obtain such aposition. I believe this was the consequenceof the focus of my work in exposing andpseudoscientific practices. In addition tothe more general problem faculty mighthave had with my involvement, the bizarrematerial on the Internet, even though noneof them believed it was true, was likely

something that no one wanted inflictedupon any faculty at an establishment thatmight have hired me. It was as if I had acontagious disease; although it was not myfault, people were sympathetic, it was notsomething anyone wanted to be around.

Everything came to a head in December2010 when one of the therapists sued me,along with five of my colleagues, fordefamation and interference with business.Interestingly, while the lawsuit was underway, the anonymous postings, which hadbeen occurring on an almost daily basis,almost completely stopped. Fortunately,we were able to have the suit escalated to afederal court, where the case was dismissedbefore the discovery phases and trial. Afterthe lawsuit was dismissed, the anonymousattacks resumed with a vengeance, includ-ing a fake posting about me on a sitedesigned to expose adulterers, where some-one accused me of having an affair with herhusband in a city I wasn’t even living in. Iwas only able to get the site to agree to takeit down after I proved to them they hadcopied the story from elsewhere andchanged the person’s name to mine. Whenthe therapist lost the lawsuit, he put up aderogatory document about all of us on hisbusiness’ website, saying that as a publicfigure, his lawsuit against us had beenunwinnable. Actually, the case was dis-missed due to failure to state a claim uponwhich relief can be granted and jurisdic-tion. The judge had noted that he would belikely considered a public figure, had thecase progressed, but that was not thereason the case was dismissed. Rather, FirstAmendment rights to express our opinionsabsent factually false statements gave himno case. In addition to being a victory forInternet free speech, this is also a victoryfor academic freedom.

The attacks continued through 2011and finally, by 2012, suddenly lessenedwith only an occasional blog post. How-ever, when it came to my academic career,the damage had been done. Being 3 yearsout from graduation made it even moredifficult for me to obtain a faculty position,and although I continued to try, after hun-dreds of applications, I did not obtain a fac-ulty position. I continue to write and pub-lish on understanding and exposingpseudoscience and disseminating evi-dence-based practice, but I make my livingoutside the profession.

What are the lessons learned from thisexperience? I have been asked if I would dowhat I did again, now knowing what theconsequences could be. Essentially, myanswer would be yes: I do not think I

could be fulfilled being in a professionwhere I had to keep silent in order to getahead; I would feel as though I sold out. Ifmy colleague and other experienced socialwork faculty members who had warned meabout the “small world” of the professionhad been right, that I needed to remainsilent until getting a position and then get-ting tenure, I could not have lived with thatdecision.

Are there things I would do differently?Of course. I would have posted less lengthyresponses and explanations to my critics onmy blog—the feedback I received indicatedthat such responses did not help and mademe look unbalanced. I would have insteadlimited myself to one statement refutingthe lies that were posted about me. It is dif-ficult to determine, though, whether thatwould have made a difference or lessenedthe attacks, as my colleagues who weresilent when attacked were still just asviciously attacked as I was, the only differ-ence being, they were not seeking facultypositions or already had tenure, so did notsuffer the consequences I had.

Hillary Clinton, who has been the targetof a much more highly publicized andbroader attacks, recently expressed regretthat she hadn’t been more vociferous in herresponses and fought back harder. Beingsilent did not stop the attacks, nor did ithelp her win the election. These types ofattacks place the target in a double-bind sit-uation. If we fight back hard, we are por-trayed as mentally imbalanced and any-thing we say, no matter how seeminglyinnocuous, can further be twisted and dis-torted. On the other hand, if we do notfight back, people believe that the accusa-tions must have some truth in them or thatthe person has something to hide or does-n’t care.

Since cyber abuse of this kind is a rela-tively new phenomenon, we really do notknow what tactics would be effective instopping it. There is still a tendency toblame the victims, thinking that if only wehad behaved differently, this would nothave happened. Hopefully, there will bemore empirical study of this phenomenonto find out what works and what doesn’twork when dealing with cyber abuse.

I have also been asked what advice Iwould give to students interested in acade-mic careers who are concerned about theproblem and wish to expose it. What Iwould advise is to make an informedchoice. I had been exposing various pseu-doscientific practices for years with verylittle consequences, until I angered thewrong people by criticizing their particular

practice. That is a risk that anyone takeswho chooses to expose such practices andsomeone who wants an academic careerneeds to realize that it could interfere withtheir ability to get hired or get tenure. Oneoption would be to, instead, as a studentand a new graduate, focus on disseminat-ing evidence-based approaches and leavethe exposure of pseudoscience to otherswith more secure positions, but even then,there have been severe consequences as wecan see with what Elizabeth Loftus had toendure as a result of her research on recov-ered memory. This is an informed personalprofessional choice that each person willneed to make for themselves, weighing theimportance of their values, the benefits andpossible consequences. What I do know isthat if such risks are not taken, these prac-tices will continue to be promoted, espe-cially on the Internet, and writing for jour-nals that only academic colleagues read isnot enough. Fortunately there is a recenttrend in the academic community, withprojects such as the Public Voices Fellow-ship (OpEd Project, n.d.), to encouragepeople in various academic disciplines,especially those that impact the public, whoregularly publish little-read peer-reviewedarticles, to extend their writing and voicingof opinions to the larger community in theform of op-eds, blogs, Tweets, and othermedia where their knowledge and exper-tise is so badly needed.

Exposing pseudoscience and other mis-information disseminated to the public istruly an interdisciplinary effort that allhealth and mental health professions, aswell as perhaps sociologists and anthropol-ogists, could be involved in. I am not alonein challenging such practices and I amdeeply grateful to all my colleagues whohave taken these risks, in spite of all theconsequences they suffered.

ReferencesAdvocates for Children in Therapy (n.d.).

Mission Statement. Retrieved fromhttp://www.childrenintherapy.org/mission.html on July 8, 2017.

Callahan, R.J. & Callahan, J. (2000). Stopthe nightmares of trauma. Chapel Hill:Professional Press.

Chaffin, M., Hanson, R., Saunders, B. E.,Nichols, T., Barnett, D., & Zeanah, C. ...Miller-Perrin, C. (2006). Report of theAPSAC task force on attachment ther-apy, reactive attachment disorder, andattachment problems. Child Maltreat-ment, 11, 76-89.

Mercer, J. & Pignotti, M. (2007). Shortcutscause errors in systematic research syn-thesis: Rethinking evaluation of mentalhealth interventions. The ScientificReview of Mental Health Practice, 5, 59-77.

Mercer, J., Sarner, L., & Rosa, L. (2003).Attachment therapy on trial: The tortureand death of Candace Newmaker. West-port, CT: Praeger Publishers/GreenwoodPublishing Group.

OpEd Project (n.d.). Public Voices Fellow-ship. Retrieved on October 1, 2017 fromhttps://www.theopedproject.org/public-voices-fellowship/

Pignotti, M. (2005). Thought field therapyvoice technology vs. random meridianpoint sequences: A single-blind con-trolled experiment. The Scientific Reviewof Mental Health Practice, 4(1), 72-81.

Pignotti, M. (2007). Questionable inter-ventions taught at top-ranked school ofsocial work. The Scientific Review ofMental Health Practice, 5, 78-82.

Pignotti, M. & Mercer, J. (2007). Holdingtherapy and Diadic Developmental Psy-chotherapy are not supported andacceptable practices: A systematicresearch synthesis revisited. Research onSocial Work Practice, 17, 513-519.

Pignotti, M. & Thyer, B. A. (2009). The useof novel unsupported and empiricallysupported therapies by licensed clinicalsocial workers. Social Work Research, 33,5-17.

Pignotti, M. & Thyer, B. (2012). Novel andempirically supported therapies: Patternsof usage among licensed clinical socialworkers. Behavioural and Cognitive Psy-chotherapy, 40, 331-349.

Thyer, B. A. & Pignotti, M. (2015). Scienceand pseudoscience in social work practice.New York: Springer.

. . .

The author has no funding or conflicts ofinterest to disclose.Correspondence to Monica Pignotti,Ph.D., [email protected]

FULL TIME THERAPIST POSITIONAVAILABLE Dec 2017Due to our continued growth, MountainValley Treatment Center seeks an addi-tional licensed clinician to join our excep-tional clinical team as a primary therapistat its beautiful new campus in Plainfield,NH. Mountain Valley, a short term resi-dential treatment program, serves maleand female adolescents and emergingadults, 13 – 20 years old, from around theglobe with debilitating anxiety and OCD.Our newest campus, located nearHanover, NH and Dartmouth College,provides a unique professional and treat-ment environment as well as a locale toconveniently implement in-vivo expo-sure exercises.

Mountain Valley adds clinically inten-sive CBT and ERP within an experientialeducation program and mindfulness-based milieu. Our program was recentlyfeatured in the October 15, 2017 NewYork Times Magazine as well as otherlocal and national media venues high-lighting the great work being done.

Primary Therapists manage a caseloadof three to five private pay residents overtheir 90-day treatment stay, providingindividual, group, and family therapy.Designing and implementing exposureswith their clients both on our 25-acrecampus and within local communitiesprovides a unique professional experi-ence unmatched at any other residentialtreatment setting.

The ideal candidate will have, at a mini-mum, a Master’s degree, be currentlylicensed or license-eligible in NewHampshire, and have an understandingof CBT based modalities such as DBT,ACT and ERP. Prior experience servingclients with OCD and anxiety disorderspreferred.

Mountain Valley offers above averagesalary, full benefits package, relocationand temporary housing assistance, andsponsored professional developmentopportunities such as attending ABCT,ADAA and IOCDF conferences. Casualwork and team focused environment.Mountain Valley supports the profes-sional growth of all its staff.

Please contact Don Vardell, ExecutiveDirector at [email protected] for more information orto apply.

C L A S S I F I E D


P I G N O T T I

I nominate the following individuals:

P R E S I D E N T- E L E C T ( 2 0 1 8 – 2 0 1 9 )

R E P R E S E N TAT I V E -AT- L A R G E ( 2 0 1 8 – 2 0 2 1 )and liaison to Convention and Education Issues

S E C R E TA RY-T R E A S U R E R ( 2 0 1 9 – 2 0 2 2 )

Every nomination counts! Encourage col-leagues to run for office or consider runningyourself. Nominate as many full members asyou like for each office. The results will be tal-lied and the names of those individuals whoreceive the most nominations will appear onthe election ballot next April. Only thosenomination forms bearing a signature andpostmark on or before February 1, 2018, willbe counted.

Nomination acknowledges an individual'sleadership abilities and dedication to behav-ior therapy and/or cognitive therapy, empiri-cally supported science, and to ABCT. Whencompleting the nomination form, please takeinto consideration that these individuals willbe entrusted to represent the interests ofABCT members in important policy decisionsin the coming years.Only full and new mem-ber professionals can nominate candidates.Contact the Leadership and Elections Chairfor more information about serving ABCT orto get more information on the positions.Complete, sign, and send form to:David Pantalone, Ph.D., Leadership &Elections Chair, ABCT, 305 SeventhAve., New York, NY 10001.

N A M E ( printed) S I G N AT U R E ( required)

Nominate the Next Candidates for ABCT Office

Good governance requires participation of the mem-bership in the elections. ABCT is a membership organiza-tion that runs democratically. We need your participationto continue to thrive as an organization.

NOTE: To be nominated for President-Elect of ABCT, itis recommended that a candidate has served on theABCT Board of Directors in some capacity; served as acoordinator; served as a committee chair or SIG chair;served on the Finance Committee; or have made othersignificant contributions to the Association as deter-mined by the Leadership and Elections Committee.Candidates for the position of President-Elect shallensure that during his/her term as President-Elect andPresident of the ABCT, the officer shall not serve asPresident of a competing or complementary professionalorganization during these terms of office; and the candi-date can ensure that their work on other professionalboards will not interfere with their responsibilities toABCT during the presidential cycle.

This coming year we need nominations for three electedpositions: President-Elect, Secretary-Treasurer, and Repre-sentative-at-Large. Each representative serves as a liaisonto one of the branches of the association. The representa-tive position up for 2018 election will serve as the liaison toConvention and Education Issues Coordinator.

A thorough description of each position can be found inABCT’s bylaws: www.abct.org/docs/Home/byLaws.pdf.

Three Ways to Nominate" Mail the form to the ABCT office

(address above)" Fill out the nomination form by hand

and fax it to the office at 212-647-1865" Fill out the nomination form by hand

and then scan the form as a PDF file andemail the PDF as an attachment to ourcommittee: [email protected].

The nomination form

with your original

signature is

required,

regardless

of how

you get

it to

us.

✹

January • 2018 55


ABCT awards recognition& 2017

(left) David DiLilloOutstanding Service to ABCT

(right) Marsha LinehanLifetime Achievement Award

(2016)

(NOTE: The 2017 LifetimeAchievement is awarded to

Dianne L. Chambless)

(left) President Gail Steketee withPresident’s New Researcher

Christian A. Webb

(right) Graduate StudentResearch Grant,

Hannah Lawrence (l),and Honorable MentionAmanda L. Sanchez (r),

(below) Alexandra Kredlow acceptingthe Virginia Roswell StudentDissertation Award

Leonard Krasner StudentDissertation: ShannonMichelle Blakey

John R. Z. Abela StudentDissertation:Carolyn Spiro

Anne Marie AlbanoEarly Career Award:Carmen P. McLean

51st Annual Convention | November 17 | San Diego

ABCT awards recognition& 2017

(left) Outstanding Training Program:Lee Cooper, Director, Clinical SciencePh.D. Program, Virginia PolytechnicInstitute

(right) Jennifer P. ReadOutstanding Contribution to Research

Student TravelAward WinnerDev Crasta

Elsie Ramos FirstAuthor MemorialPoster Award Winners(left to right)Awards Committeemember Sara Elkins,with Kate Kysow,Chloe Hudson, &Christian Goans

ADAA TravelAward Winners(left to right)President Gail Steketee,Andrea Niles,Amy Sewart, JennieKuckertz, and AwardsChair Katherine Baucom

January • 2018 57


t o b e p r e s e n t e d a t t h e 5 2 n d A n n u a l C o n v e n t i o n i n Wa s h i n g t o n , D C

The ABCT Awards and Recognition Committee, chaired by Cassidy Gutner, Ph.D., of Boston University School of Medicine,is pleased to announce the 2018 awards program. Nominations are requested in all categories listed below. Given the numberof submissions received for these awards, the committee is unable to consider additional letters of support or supplementalmaterials beyond those specified in the instructions below. Please note that award nominations may not be submitted by cur-rent members of the ABCT Board of Directors.

Career/Lifetime AchievementEligible candidates for this award should be members of ABCT in good standing who have made significant contributionsover a number of years to cognitive and/or behavior therapy. Recent recipients of this award include Thomas H.Ollendick, Lauren B. Alloy, Lyn Abramson, David M. Clark, Marsha Linehan, and Dianne L. Chambless. Applicationsshould include a nomination form (available at www.abct.org/awards), three letters of support, and the nominee’s cur-riculum vitae. Please e-mail the nomination materials as one pdf document to [email protected]. Include“Career/Lifetime Achievement” in the subject line. Nomination deadline: March 1, 2018

Outstanding MentorThis year we are seeking eligible candidates for the Outstanding Mentor award who are members of ABCT in good stand-ing who have encouraged the clinical and/or academic and professional excellence of psychology graduate students,interns, postdocs, and/or residents. Outstanding mentors are considered those who have provided exceptional guidance tostudents through leadership, advisement, and activities aimed at providing opportunities for professional development,networking, and future growth. Appropriate nominators are current or past students of the mentor. Previous recipients ofthis award are Richard Heimberg, G.Terence Wilson, Richard J. McNally, Mitchell J. Prinstein, Bethany Teachman, andEvan Forman. Please complete the nomination form found online at www.abct.org.Then e-mail the completed form andassociated materials as one pdf document to [email protected].. Include “Outstanding Mentor” in your subjectheading. Nomination deadline: March 1, 2018

Distinguished Friend to Behavior TherapyEligible candidates for this award should NOT be members of ABCT, but are individuals who have promoted the missionof cognitive and/or behavioral work outside of our organization. Applications should include a letter of nomination, threeletters of support, and a curriculum vitae of the nominee. Recent recipients of this award include Mark S. Bauer,VikramPatel, Benedict Carey, and Patrick J. Kennedy. Applications should include a nomination form (available atwww.abct.org/awards), three letters of support, and the nominee’s curriculum vitae. Please e-mail the nomination mate-rials as one pdf document to [email protected]. Include “Distinguished Friend to BT” in the subject line.Nomination deadline: March 1, 2018

Mid-Career InnovatorEligible candidates for the Mid-Career Innovator Award are members of ABCT in good standing who are at the associate pro-fessor level or equivalent mid-career level, and who have made significant innovative contributions to clinical practice orresearch on cognitive and/or behavioral modalities.The previous recipient was Carla Kmett Danielson. Please complete thenomination form found online at www.abct.org.Then e-mail the completed form and associated materials as one pdf docu-ment to [email protected].. Include “Mid-Career Innovator” in the subject line.Nomination deadline: March 1, 2018

Call for Award Nominations#!$"!!!!!!!!!!!!!!!!

January • 2018 59

Anne Marie Albano Early Career Award for Excellencein the Integration of Science and Practice

Dr. Anne Marie Albano is recognized as an outstanding clinician, scientist, and teacher dedicated to ABCT’s mission. She isknown for her contagious enthusiasm for the advancement of cognitive and behavioral science and practice.The purpose ofthis award is to recognize early career professionals who share Dr. Albano’s core commitments.This award includes a cashprize to support travel to the ABCT Annual Meeting and to sponsor participation in a clinical treatment workshop. Eligibilityrequirements are as follows: (1) Candidates must be active members of ABCT, (2) New/Early Career Professionals within thefirst 5 years of receiving his or her doctoral degree (PhD, PsyD, EdD). Preference will be given to applicants with a demon-strated interest in and commitment to child and adolescent mental health care. Applicants should submit: nominating coverletter, CV, personal statement up to three pages (statements exceeding 3 pages will not be reviewed), and 2 to 3 supportingletters. Application materials should be emailed as one pdf document to [email protected].. Include candidate'slast name and “Albano Award” in the subject line. Nomination deadline: March 1, 2018

Student Dissertation Awards• Virginia A. Roswell Student Dissertation Award ($1,000) • Leonard Krasner Student Dissertation Award ($1,000)• John R. Z. Abela Student Dissertation Award ($500)

Each award will be given to one student based on his/her doctoral dissertation proposal. Accompanying this honor will be amonetary award (see above) to be used in support of research (e.g., to pay participants, to purchase testing equipment)and/or to facilitate travel to the ABCT convention. Eligibility requirements for these awards are as follows: 1) Candidatesmust be student members of ABCT, 2) Topic area of dissertation research must be of direct relevance to cognitive-behavioraltherapy, broadly defined, 3) The dissertation must have been successfully proposed, and 4) The dissertation must not havebeen defended prior to November 2017. Proposals with preliminary results included are preferred.To be considered for theAbela Award, research should be relevant to the development, maintenance, and/or treatment of depression in childrenand/or adolescents (i.e., under age 18). Self-nominations are accepted or a student's dissertation mentor may complete thenomination.The nomination must include a letter of recommendation from the dissertation advisor. Please complete thenomination form found online at www.abct.org/awards/. Then e-mail the nomination materials (including letter of recom-mendation) as one pdf document to [email protected]. Include candidate’s last name and “Student DissertationAward” in the subject line. Nomination deadline: March 1, 2018

President’s New Researcher AwardABCT’s 2017-18 President, Sabine Wilhelm, Ph.D., invites submissions for the 40th Annual President’s New ResearcherAward.The winner will receive a certificate and a cash prize of $500.The award will be based upon an early program ofresearch that reflects factors such as: consistency with the mission of ABCT; independent work published in high-impact jour-nals; and promise of developing theoretical or practical applications that represent clear advances to the field.While nomina-tions consistent with the conference theme are particularly encouraged, submissions will be accepted on any topic relevant tocognitive behavior therapy, including but not limited to topics such as the development and testing of models, innovativepractices, technical solutions, novel venues for service delivery, and new applications of well-established psychological princi-ples. Requirements: candidates must be the first author, and self-nominations are accepted; 3 letters of recommendation mustbe included; the author's CV, letters of support, and paper must be submitted in electronic form.E-mail the nomination materials (including letter of recommendation) as one pdf document to [email protected]. Includecandidate’s last name and “President's New Researcher” in the subject line. Nomination deadline: August 1, 2018

Nominations for the following award are solicited from members of the ABCT governance:

Outstanding Service to ABCTPlease complete the nomination form found online at www.abct.org/awards/.Then e-mail the completed form and associatedmaterials as one pdf document to [email protected]. Include “Outstanding Service” in the subject line.Nomination deadline: March 1, 2018

The ABCT Convention is designed for scientists, practitioners, students, and schol-ars who come from a broad range of disciplines. The central goal is to provide edu-cational experiences related to behavioral and cognitive therapies that meet theneeds of attendees across experience levels, interest areas, and behavioral andcognitive theoretical orientations. Some presentations offer the chance to learnwhat is new and exciting in behavioral and cognitive assessment and treatment.Other presentations address the clinical-scientific issues of how we develop empir-ical support for our work. The convention also provides opportunities for profes-sional networking. The ABCT Convention consists of General Sessions, Targetedand Special Programming, and Ticketed Events.

ABCT uses the Cadmium Scorecard system for the submission of general ses-sion events. The step-by-step instructions are easily accessed from the AbstractSubmission Portal, and the ABCT home page. Attendees are limited to speaking(e.g., presenter, panelist, discussant) during no more than FOUR events. As you pre-pare your submission, please keep in mind:

• Presentation type: Please see the two right-hand columns on this page fordescriptions of the various presentation types.

• Number of presenters/papers: For Symposia please have a minimum of fourpresenters, including one or two chairs, only one discussant, and 3 to 5 papers.The chair may present a paper, but the discussant may not. For PanelDiscussions and Clinical Round tables, please have one moderator and betweenthree to five panelists.

• Title: Be succinct.

• Authors/Presenters: Be sure to indicate the appropriate order. Please ask allauthors whether they prefer their middle initial used or not. Please ask allauthors their degree, ABCT category (if they are ABCT members), and their emailaddress. (Possibilities for “ABCT category” are current member; lapsed memberor nonmember; postbaccalaureate; student member; student nonmember; newprofessional; emeritus.)

• Institutions: The system requires that you enter institutions before enteringauthors. This allows you to enter an affiliation one time for multiple authors. DONOT LIST DEPARTMENTS. In the following step you will be asked to attach affilia-tions with appropriate authors.

• Key Words: Please read carefully through the pull-down menu of alreadydefined keywords and use one of the already existing keywords, if appropriate.For example, the keyword “military” is already on the list and should be usedrather than adding the word “Army.” Do not list behavior therapy, cognitive thera-py, or cognitive behavior therapy.

• Objectives: For Symposia, Panel Discussions, and Clinical Round Tables, writethree statements of no more than 125 characters each, describing the objectivesof the event. Sample statements are: “Described a variety of disseminationstrategies pertaining to the treatment of insomnia”; “Presented data on noveldirection in the dissemination of mindfulness-based clinical interventions.”

Overall: Ask a colleague to proof your abstract for inconsistencies or typos.

Thinking about submitting anabstract for the ABCT 52ndAnnual Convention in DC? Thesubmission portal will be opened fromFebruary 14–March 14. Look for moreinformation in the coming weeks to assistyou with submitting abstracts for the ABCT51st Annual Convention. The deadline forsubmissions will be 11:59 P.M. (EST),Wednesday, March 14, 2018. We look for-ward to seeing you in Washington, DC!

Preparingto Submit

an Abstract

ABCT’s 52nd Annual ConventionNovember 15–18, 2018 • Washington, DC


General SessionsThere are between 150 and 200 generalsessions each year competing for yourattention. An individual must LIMIT TO6 the number of general session submis-sions in which he or she is a SPEAKER(including symposia, panel discussions,clinical roundtables, and research spot-lights). The term SPEAKER includes rolesof chair, moderator, presenter, panelist,and discussant. Acceptances for any givenspeaker will be limited to 4. All generalsessions are included with the registrationfee. These events are all submittedthrough the ABCT submission system.The deadline for these submissions is11:59 PM, Wednesday, March 15, 2017.General session types include:

SymposiaIn response to convention feedbackrequesting that symposia include morepresentations by established research-ers/faculty along with their graduatestudents, preference will be given tosymposia submissions that include non-student researchers and faculty mem-bers as first-author presenters.

Symposia are presentations of data,usually investigating the efficacy or effec-tiveness of treatment protocols. Symposiaare either 60 or 90 minutes in length.They have one or two chairs, one discus-sant, and between three and five papers.No more than 6 presenters are allowed.

Panel Discussionsand Clinical Round TablesDiscussions (or debates) by informedindividuals on a current important topic.These are organized by a moderator andinclude between three and six panelistswith a range of experiences and attitudes.No more than 6 presenters are allowed.

Spotlight Research PresentationsThis format provides a forum to debutnew findings considered to be ground-breaking or innovative for the field. Alimited number of extended-format ses-sions consisting of a 45-minute researchpresentation and a 15-minute question-and-answer period allows for more in-depth presentation than is permitted bysymposia or other formats.

Poster SessionsOne-on-one discussions betweenresearchers, who display graphic repre-sentations of the results of their studies,and interested attendees. Because of thevariety of interests and research areas ofthe ABCT attendees, between 1,200 and1,400 posters are presented each year.

Targeted and SpecialPrograming

Targeted and special programing eventsare also included with the registration fee.These events are designed to address arange of scientific, clinical, and profes-sional development topics. They also pro-vide unique opportunities for networking.

Invited Addresses/PanelsSpeakers well-established in their field, orwho hold positions of particular impor-tance, share their unique insights andknowledge.

Mini WorkshopsDesigned to address direct clinical care ortraining at a broad introductory level andare 90 minutes long.

Clinical Grand RoundsClinical experts engage in simulated livedemonstrations of therapy with clients,who are generally portrayed by graduatestudents studying with the presenter.

Research and Professional DevelopmentProvides opportunities for attendees tolearn from experts about the developmentof a range of research and professionalskills, such as grant writing, reviewingmanuscripts, and professional practice.

Membership Panel DiscussionOrganized by representatives of theMembership Committees, these eventsgenerally emphasize training or careerdevelopment.

Special SessionsThese events are designed to provide use-ful information regarding professionalrather than scientific issues. For morethan 20 years, the Internship andPostdoctoral Overviews have helpedattendees find their educational path.Other special sessions often includeexpert panels on getting into graduate

school, career development, informationon grant applications, and a meeting ofthe Directors of Clinical Training.

Special Interest Group (SIG) MeetingsMore than 39 SIGs meet each year toaccomplish business (such as electing offi-cers), renew relationships, and often offerpresentations. SIG talks are not peer-reviewed by the Association.

Ticketed EventsTicketed events offer educational oppor-tunities to enhance knowledge and skills.These events are targeted for attendeeswith a particular level of expertise (e.g.,basic, moderate, and/or advanced).Ticketed sessions require an additionalpayment.

Clinical Intervention TrainingOne- and two-day events emphasizing the“how-to” of clinical interventions. Theextended length allows for exceptionalinteraction.

InstitutesLeaders and topics for Institutes areselected from previous ABCT workshoppresentations. Institutes are offered as a 5-or 7-hour session on Thursday, and aregenerally limited to 40 attendees.

WorkshopsCovering concerns of the practitioner/educator/researcher, these remain ananchor of the Convention. Workshops areoffered on Friday and Saturday, are 3hours long, and are generally limited to 60attendees.

Master Clinician SeminarsThe most skilled clinicians explain theirmethods and show videos of sessions.These 2-hour sessions are offeredthroughout the Convention and are gen-erally limited to 40 to 45 attendees.

Advanced Methodology and StatisticsSeminarsDesigned to enhance researchers’ abilities,they are 4 hours long and limited to 40attendees.

Continuing EducationSee pp. 64-65 for a complete description.

Understanding the ABCT Convention

January • 2018 61

Workshops & Mini WorkshopsWorkshops cover concerns of the practitioner/ educator/researcher. Workshops are 3hours long, are generally limited to 60 attendees, and are scheduled for Friday andSaturday. Please limit to no more than 4 presenters. Mini Workshops address directclinical care or training at a broad introductory level. They are 90 minutes long and arescheduled throughout the convention. Please limit to no more than 4 presenters. Whensubmitting for Workshops or Mini Workshop, please indicate whether you would like tobe considered for the other format as well. | For more information or to answer anyquestions before you submit your abstract, contact Lauren Weinstock, WorkshopCommittee Chair: [email protected]

InstitutesInstitutes, designed for clinical practitioners, are 5 hours or 7 hours long, are generallylimited to 40 attendees, and are scheduled for Thursday. Please limit to no more than 4presenters. | For more information or to answer any questions before you submit yourabstract, contact Christina Boisseau, Institute Committee Chair:[email protected]

Master Clinician SeminarsMaster Clinician Seminars are opportunities to hear the most skilled clinicians explaintheir methods and show taped demonstrations of client sessions. They are 2 hours long,are limited to 40 attendees, and are scheduled Friday through Sunday. Please limit tono more than 2 presenters. | For more information or to answer any questions before yousubmit your abstract, contact Courtney Benjamin Wolk, Master Clinician SeminarCommittee Chair: [email protected]

Research and Professional DevelopmentPresentations focus on "how to" develop one's own career and/or conduct research,rather than on broad-based research issues (e.g., a methodological or design issue,grantsmanship, manuscript review) and/or professional development topics (e.g., evi-dence-based supervision approaches, establishing a private practice, academic produc-tivity, publishing for the general public). Submissions will be of specific preferred length(60, 90, or 120 minutes) and format (panel discussion or more hands-on participationby the audience). Though this track is not new for 2018, this is the first time that RPDabstracts are due at the earlier deadline, along with ticketed events/mini workshops,and will also be submitted through the same portal. Please limit to no more than 4 pre-senters, and be sure to indicate preferred presentation length and format. | For moreinformation or to answer any questions before you submit your abstract, contactCole Hooley, Research & Professional Development Chair:[email protected]

#!"" 52nd Annual ConventionNovember 15–18, 2018 | Washington, DC

Submissions will nowbe accepted through

the online submissionportal, which will be

open until February 1.

Submit a 250-wordabstract and a CV foreach presenter. Forsubmission require-

ments and informationon the CE session selec-tion process, please visitwww.abct.org and click

on “Convention andContinuing Education.”

Submission deadline: February 1, 2018

ticketedsessions

for Ticketed Sessions


Submission deadline: February 14, 2018

ABCT has always celebrated advances in clinical science. We now find our-selves at the cusp of a new era, marked by technological advances in a range ofdifferent disciplines that have the potential to dramatically affect the clinicalscience we conduct and the treatments we deliver. These innovations arealready influencing our investigations of etiological hypotheses, and are simi-larly opening new frontiers in the ways that assessments and treatments aredeveloped, patients access help, clinicians monitor response, and the broaderfield disseminates evidence-based practices. Building on the strong, theoreticaland practical foundations of CBT, we have the exciting opportunity to use ourmultidisciplinary values to identify new and emerging technologies that couldcatapult our research on mental health problems and well-being to the nextlevel.

The theme of ABCT's 52nd Annual Convention, "Cognitive Behavioral Science,Treatment, and Technology," is intended to showcase research, clinicalpractice, and training that:

• Uses cutting-edge technology and new tools to increase our under standingof mental health problems and underlying mechanisms;

• Investigates how a wide range of technologies can help us improve evidence-based practices in assessment and the provision of more powerful interven-tions; and

• Considers the role technology can have in training a new generation ofevidence-based treatment providers at home and across the globe.

The convention will highlight how advances in clinical science can be strength-ened and propelled forward through the integration of multidisciplinarytechnologies.

Submissions may be in the form of symposia, clinical round tables, paneldiscussions, and posters. Information about the Convention and how tosubmit abstracts will be on ABCT's website, www.abct.org , after January 1,2018.

Call for Papers

Deadlinefor submissions:

March 14, 2018

Portal opensFebruary 14, 2018

See p. 60for informationabout preparing

your abstract

Theme:COGNITIVE

BEHAVIORALSCIENCE,

TREATMENT,and

TECHNOLOGY

generalsessions

Program Chair: Kiara R. Timpano, Ph.D.

52nd Annual ConventionNovember 15–18, 2018 • Washington, DC

January • 2018 63

At the ABCT Annual Conventions, there are Ticketed events(meaning you have to buy a ticket for one of these beyond thegeneral registration fee) and General sessions (meaning you getin by paying the general registration fee), the vast majority ofwhich qualify for Continuing Education credit. See the end ofthis document for the current list of bodies that have approvedABCT as a CE sponsor. Note that we do not currently offerCMEs. Attendance at each continuing education session in itsentirety is required to receive CE credit. No partial credit isawarded; late arrival or early departure will preclude awarding ofCE credit. For those who have met all requirements according tothe organizations which have approved ABCT as a CE sponsor,certificates will be mailed early in the new year following theAnnual Convention.

Ticketed Events Eligible for CEAll Ticketed Events offer CE in addition to educational opportu-nities to enhance knowledge and skills. These events are targetedfor attendees with a particular level of expertise (e.g., basic, mod-erate, and/or advanced). Ticketed sessions require an additionalpayment beyond the general registration fee. For ticketed eventsattendees must sign in and sign out and complete and return anindividual evaluation form to be awarded CE. It remains theresponsibility of the attendee to sign in at the beginning of thesession and out at the end of the session.

Clinical Intervention Trainings (CITs)One- and two-day events emphasizing the "how-to" of clini-cal interventions. The extended length allows for exceptionalinteraction. Participants attending a full day session can earn7 continuing education credits, and 14 CE credits for the two-day session.

InstitutesLeaders and topics for Institutes are selected from previousABCT workshop presentations. Institutes are offered as a 5-or 7-hour session on Thursday, and are generally limited to40 attendees. Participants in the full-day Institute can earn 7continuing education credits, and in the half-day Institutescan earn 5 CE credits.

WorkshopsCovering concerns of the practitioner/educator/researcher,these remain an anchor of the Convention. Workshops areoffered on Friday and Saturday, are 3 hours long, and are gen-erally limited to 60 attendees. Participants in theseWorkshops can earn 3 CE credits per workshop.

Master Clinician Seminars (MCS)The most skilled clinicians explain their methods and showvideos of sessions. These 2-hour sessions are offered through-out the Convention and are generally limited to 40 to 45attendees. Participants in these seminars can earn 2 CE cred-its per seminar.

Advanced Methodology and Statistics Seminars (AMASS)Designed to enhance researchers' abilities, there are generallytwo seminars offered on Thursday or during the course of theConvention. They are 4 hours long and limited to 40 atten-dees. Participants in these courses can earn 4 CE credits perseminar.

General Sessions Eligible for CEThere are 200 general sessions each year competing for yourattention. All general sessions are included with the registrationfee. Most of the sessions are eligible for CE, with the exception ofthe poster sessions, Membership Panel Discussions, the SpecialInterest Group Meetings (SIG), and a few other sessions. You areeligible to earn 1 CE credit per hour of attendance.

General sessions attendees must sign in and sign out andanswer particular questions in the CE booklet regarding eachsession attended. The booklets must be handed in to ABCT atthe end of the Convention. If the booklet is not completed andhanded in, CE credit will not be awarded.

General session types that are eligible for CE include:Clinical Grand RoundsClinical experts engage in simulated live demonstrations oftherapy with clients, who are generally portrayed by graduatestudents studying with the presenter.

Invited Panels and AddressesSpeakers well-established in their field, or who hold positionsof particular importance, share their unique insights andknowledge on a broad topic of interest.

Mini-WorkshopsThese 90-minute sessions directly address evidence-basedclinical skills and applications. They are offered at an intro-ductory level and clinical care or training issues.

Panel Discussions and Clinical Round TablesDiscussions (or debates) by informed individuals on a currentimportant topic. These are organized by one moderator andinclude between three and five panelists with a range of expe-rience and attitudes. The total number of speakers may notexceed 6.

Spotlight Research PresentationsThis format provides a forum to debut new findings consid-ered to be groundbreaking or innovative for the field. A lim-ited number of extended-format sessions consisting of a 45-minute research presentation and a 15-minute question-and-answer period allows for more in-depth presentation than ispermitted by symposia or other formats.

SymposiaPresentations of data, usually investigating the efficacy oreffectiveness of treatment protocols. Symposia are either 60or 90 minutes in length. They have one or two chairs, one dis-cussant, and between three and five papers. The total number

ABCT and Continuing Education


General Sessions NOT Eligible for CEMembership Panel DiscussionOrganized by representatives of the MembershipCommittees, these events generally emphasize training orcareer development.

Poster SessionsOne-on-one discussions between researchers, who displaygraphic representations of the results of their studies, andinterested attendees. Because of the variety of interests andresearch areas of the ABCT attendees, between 1,400 and1,600 posters are presented each year.

Special Interest Group (SIG) MeetingsMore than 39 SIGs meet each year to accomplish business(such as electing officers), renew relationships, and oftenoffer presentations. SIG talks are not peer-reviewed by theAssociation.

Special SessionsThese events are designed to provide useful informationregarding professional rather than scientific issues. For morethan 20 years the Internship and Postdoctoral Overviewshave helped attendees find their educational path. Other spe-cial sessions often include expert panels on getting into grad-uate school, career development, information on grant appli-cations, and a meeting of the Directors of Clinical Training.These sessions are not eligible for CE credit.

Other SessionsOther sessions not eligible for CE are noted as such on theitinerary planner and in the program book.

How Do I Get CE at the ABCT Convention?The CE fee must be paid (see registration form) for a personal-ized CE credit letter to be distributed. Those who have includedCE in their preregistration will be given a booklet when they pickup their badge and registration materials at the ABCTRegistration Desk. Others can still purchase a booklet at the reg-istration area during the convention. The current fee is $99.00.We do not charge a fee that is hidden within general registration.

Which Organizations Have Approved ABCT as a CESponsor?

PsychologyABCT is approved by the American PsychologicalAssociation to sponsor continuing education for psycholo-gists. ABCT maintains responsibility for this program and itscontent. Attendance at each continuing education session inits entirety is required to receive CE credit. No partial creditis awarded; late arrival or early departure will precludeawarding of CE credit.

For ticketed events attendees must sign in and sign outand complete and return an individual evaluation form. Forgeneral sessions attendees must sign in and sign out andanswer particular questions in the CE booklet regarding eachsession attended. The booklets must be handed in to ABCT atthe end of the Convention. It remains the responsibility of theattendee to sign in at the beginning of the session and out atthe end of the session.

Social WorkABCT program is approved by the National Association ofSocial Workers (Approval # 886427222-7448) for 34 contin-uing education contact hours.

Continuing Education (CE) Grievance ProcedureABCT is fully committed to conducting all activities in strictconformance with the American Psychological Association’sEthical Principles of Psychologists. ABCT will comply with alllegal and ethical responsibilities to be non-discriminatory inpromotional activities, program content and in the treatment ofprogram participants. The monitoring and assessment of com-pliance with these standards will be the responsibility of theCoordinator of Convention and Continuing Education Issues inconjunction with the Director of Education and MeetingServices.

Although ABCT goes to great lengths to assure fair treatmentfor all participants and attempts to anticipate problems, there willbe occasional issues which come to the attention of the conven-tion staff which require intervention and/or action on the part ofthe convention staff or an officer of ABCT. This proceduraldescription serves as a guideline for handling such grievances.

All grievances must be filed in writing to ensure a clear expla-nation of the problem. If the grievance concerns satisfaction witha CE session the Director of Outreach and Partnerships shalldetermine whether a full or partial refund (either in money orcredit for a future CE event) is warranted. If the complainant isnot satisfied, their materials will be forwarded to theCoordinator of Convention and Continuing Education Issuesfor a final decision.

If the grievance concerns a speaker and particular materialspresented, the Director of Outreach and Partnerships shall bringthe issue to the Coordinator of Convention and ContinuingEducation Issues who may consult with the members of the con-tinuing education issues committees. The Coordinator will for-mulate a response to the complaint and recommend action ifnecessary, which will be conveyed directly to the complainant.For example, a grievance concerning a speaker may be conveyedto that speaker and also to those planning future educationalprograms.

Records of all grievances, the process of resolving the griev-ance and the outcome will be kept in the files of the Director ofEducation and Meeting Services. A copy of this GrievanceProcedure will be available upon request.

If you have a complaint, please contact the ABCT centraloffice at (212) 646-1890 for assistance, or [email protected].

January • 2018 65

CBT Medical Educator DirectoryAnother indispensable resourcefrom ABCT—an online directory ofCBT educators who have agreed tobe listed as potential resources toothers involved in training physi-cians and allied health providers. Inparticular, the educators on this listhave been involved in providingeducation in CBT and/or the theo-ries underlying such interventionsto medical and other allied healthtrainees at various levels. The listingis meant to connect teachers acrossinstitutions and allow for the shar-ing of resources.

Inclusion Criteria1. Must teach or have recentlytaught CBT and/or CB interventionsin a medical setting. This mayinclude psychiatric residents, med-ical students, nursing, pharmacy,dentistry, or other allied health pro-fessionals, such as PT, OT, or RD.Teachers who exclusively train psy-chology graduate students, socialworkers, or master’s level thera-pists do not qualify and are not list-ed in this directory.2. “Teaching” may include directtraining or supervision, curriculumdevelopment, competency evalua-tion, and/or curriculum administra-tion. Many professionals on the listhave had a central role in designingand delivering the educationalinterventions, but all educationalaspects are important.3. Training should take place or beaffiliated with an academic training

facility (e.g. medical school, nursingschool, residency program) and notoccur exclusively in private consul-tations or paid supervision.Please note that this list is offered asa service to all who teach CBT to themedical community and is notexhaustive.To Submit Your Name

for Inclusion in the MedicalEducator DirectoryIf you meet the above inclusion cri-teria and wish to be included on thislist, please send the contact infor-mation that you would like includ-ed, along with a few sentencesdescribing your experience withtraining physicians and/or alliedhealth providers in CBT to BarbaraKamholz at [email protected] and include “MedicalEducator Directory” in the subjectline.

DisclaimerTime and availability to participatein such efforts may vary widelyamong the educators listed. It is upto the individuals seeking guidanceto pick who they wish to contact andto evaluate the quality of theadvice/guidance they receive. ABCThas not evaluated the quality ofpotential teaching materials andinclusion on this list does not implyendorsement by ABCT of any partic-ular training program or profes-sional. The individuals in this listingserve strictly in a volunteer capaci-ty.http://www.abct.org

Resources for Professionals

Teaching Resources

CBT Medical Educator Directory

!

!

!

ABCT’sMedicalEducatorDirectory }66 the Behavior Therapist

findCBT.org

Find a CBT Therapist

ABCT’s Find a CBT Therapistdirectory is a compilation of prac‐

titioners schooled in cognitive and

behavioral techniques. In addition

to standard search capabilities

(name, location, and area of exper‐

tise), ABCT’s Find a CBT Therapist

offers a range of advanced search

capabilities, enabling the user to

take a Symptom Checklist, review

specialties, link to self‐help books,

and search for therapists based on

insurance accepted.

We urge you to sign up for the

Expanded Find a CBT Therapist(an extra $50 per year). With this

addition, potential clients will see

what insurance you accept, your

practice philosophy, your website,

and other practice particulars.

To sign up for the Expanded Find

a CBT Therapist, click MEMBER

LOGIN on the upper left‐hand of the

home page and proceed to the

ABCT online store, where you will

click on “Find CBT Therapist.”

For further questions, call the

ABCT central office at 212‐647‐

1890.

s h i p p i n g & h a n d l i n gU.S./Canada/Mexico 1–3 videos: $5.00 per video

4 or more videos: $20.00

Other countries 1 video: $10.002 or more videos: $20.00

Name on Card

Card Number CVV Expiration

Signature

complex cases

master clinicians

live sessions

`äáåáÅ~ä

dê~åÇ

oçìåÇë

! Steven C. Hayes, Acceptance and Commitment Therapy! Ray DiGiuseppe, Redirecting Anger Toward Self-Change! Art Freeman, Personality Disorder! Howard Kassinove & Raymond Tafrate, Preparation, Change,

and Forgiveness Strategies for Treating Angry Clients! Jonathan Grayson, Using Scripts to Enhance Exposure in OCD! Mark G. Williams, Mindfulness-Based Cognitive Therapy and the Prevention

of Depression! Donald Baucom, Cognitive Behavioral Couples Therapy and the Role

of the Individual! Patricia Resick, Cognitive Processing Therapy for PTSD

and Associated Depression! Edna B. Foa, Imaginal Exposure! Frank Dattilio, Cognitive Behavior Therapy With a Couple! Christopher Fairburn, Cognitive Behavior Therapy for Eating Disorders! Lars-Goran Öst, One-Session Treatment of a Patient With Specific Phobias! E. Thomas Dowd, Cognitive Hypnotherapy in Anxiety Management! Judith Beck, Cognitive Therapy for Depression and Suicidal Ideation

3-SESSION SERIES

! DOING PSYCHOTHERAPY: Different Approaches to ComorbidSystems of Anxiety and Depression

(Available as individual DVDs or the complete set)

! Session 1 Using Cognitive Behavioral Case Formulation in Treating a ClientWith Anxiety and Depression (Jacqueline B. Persons)

! Session 2 Using an Integrated Psychotherapy Approach When Treating aClient With Anxiety and Depression (Marvin Goldfried)

! Session 3 Comparing Treatment Approaches (moderated by Joanne Davilaand panelists Bonnie Conklin, Marvin Goldfried, Robert Kohlenberg,and Jacqueline Persons)

TO ORDER

}

O R , O R D E R O N L I N E AT www.abc t .o rg | c l i ck on ABCT STORE

ABCT’S T R A I N I N G V I D E O S Deepen

yourunderstandingVisa | MasterCard | American Express

Individual DVDs— $55 each • “Doing Psychotherapy”: Individual sessions — $55 / set of three—$200

January • 2018 67

PRSRT STDU.S. POSTAGE

PAIDHanover, PAPermit No. 4

the Behavior TherapistAssociation for Behavioraland Cognitive Therapies305 Seventh Avenue, 16th floorNew York, NY 10001-6008212-647-1890 | www.abct.org

ADDRESS SERV ICE REQUESTED

This may be your last issue of tBT.Renew your ABCT membership before January 31.

www.abct.org

€¦ · the behaviortherapist issn 0278-8403 volume 41, no. 1•january 2018 [continued on p. 3]...

Documents