echo chambers and epistemic bubbles - for pdf · 2 as corrupt and untrustworthy. many of us have...
TRANSCRIPT
1
Echo Chambers and Epistemic Bubbles
C. Thi Nguyen (Thisisthepenultimatedraft.Pleasecitethepublishedversion,publishedinEpisteme:https://doi.org/10.1017/epi.2018.32. Anearlierandmuch-shortenedversionofthisessay,writtenforageneralaudience,originallyappearedinAeonMagazineasEscapetheEchoChamber.)
AbstractDiscussionofthephenomenaofpost-truthandfakenewsoftenimplicatestheclosedepistemicnetworksofsocialmedia.Therecentconversationhas,however,blurredtwodistinctsocialepistemicphenomena.Anepistemicbubbleisasocialepistemicstructureinwhichotherrelevantvoiceshavebeenleftout,perhapsaccidentally.Anechochamberisasocialepistemicstructurefromwhichotherrelevantvoiceshavebeenactivelyexcludedanddiscredited.Membersofepistemicbubbleslackexposuretorelevantinformationandarguments.Membersofechochambers,ontheotherhand,havebeenbroughttosystematicallydistrustalloutsidesources.Inepistemicbubbles,othervoicesarenotheard;inechochambers,othervoicesareactivelyundermined.Itiscrucialtokeepthesephenomenadistinct.First,echochamberscanexplainthepost-truthphenomenainawaythatepistemicbubblescannot.Second,eachtypeofstructuresrequiresadistinctintervention.Mereexposuretoevidencecanshatteranepistemicbubble,butmayactuallyreinforceanechochamber.Finally,echochambersaremuchhardertoescape.Onceintheirgrip,anagentmayactwithepistemicvirtue,butsocialcontextwillpervertthoseactions.Escapefromanechochambermayrequirearadicalrebootingofone’sbeliefsystem.
Keywords:Epistemology,socialepistemology,echochambers,testimony,philosophyoftestimony,trust,expertise,filterbubbles,politicalpolarization,post-truth,socialmedia.
Somethingseemstohavegonewrongwiththeflowofinformation.Recentanalysesof
FacebookfeedsandTwitternetworksrevealthattheiruser’sinformationalinputisbeing
radicallyfiltered,thatusersarebeingexposedlargelytoargumentsandviewswithwhich
theyalreadyagree(An,QuerciaandCrowcroft2014;Saez-Trumper,CastilloandLalmas
2013).What’smore,wholesegmentsofthepopulationhavedismissedthemainstreammedia
2
ascorruptanduntrustworthy.Manyofushavestartedtowonder:arewetrappedinecho
chambersofourownmaking?1
Therecentconversation,however,hasblurredtwodistinct,butinterrelated,social
epistemicphenomena,whichIwillcallepistemicbubblesandechochambers.Bothare
problematicsocialstructuresthatleadtheirmembersastray.Bothreinforceideological
separation.Buttheyaredifferentintheirorigins,mechanismsforoperation,andavenuesfor
treatment.Botharestructuresofexclusion–butepistemicbubblesexcludethrough
omission,whileechochambersexcludebymanipulatingtrustandcredence.However,the
modernconversationoftenfailstodistinguishbetweenthem.
Loosely,anepistemicbubbleisasocialepistemicstructureinwhichsomerelevantvoices
havebeenexcludedthroughomission.Epistemicbubblescanformwithnoillintent,through
ordinaryprocessesofsocialselectionandcommunityformation.Weseektostayintouch
withourfriends,whoalsotendtohavesimilarpoliticalviews.Butwhenwealsousethose
samesocialnetworksassourcesofnews,thenweimposeonourselvesanarrowedandself-
reinforcingepistemicfilter,whichleavesoutcontraryviewsandillegitimatelyinflatesour
epistemicself-confidence.
Anechochamber,ontheotherhand,isasocialepistemicstructureinwhichother
relevantvoiceshavebeenactivelydiscredited.MyanalysisbuildsonKathleenHallJamieson
andFrankCapella’swork,withsomephilosophicalaugmentation.AccordingtoJamiesonand
Capella,anechochamber’smemberssharebeliefswhichincludereasonstodistrustthose
outsidetheechochamber.Echochambersworkbysystematicallyisolatingtheirmembers
1 An earlier version of this article appeared as ‘Escape the Echo Chamber’ in Aeon Magazine (Nguyen 2018b). That earlier version was written with a general audience in mind. The present article contains fuller versions of the core definitions and arguments, and some additional material, especially on the topics of responsibility and escape.
3
fromalloutsideepistemicsources(JamiesonandCappella2008,163-236).Thismechanism
bearsastrikingresemblancetosomeaccountsofcultindoctrination.Bydiscrediting
outsiders,echochambersleavetheirmembersoverlydependentonapprovedinsidesources
forinformation.Inepistemicbubbles,othervoicesaremerelynotheard;inechochambers,
othervoicesareactivelyundermined.(Thisisaconceptualdistinction;acommunitycan
practicebothformsofexclusiontovaryingdegrees.)
Thecontemporarydiscussionhasbeenmostlyfocusedonthephenomenonofepistemic
bubbles.CassSunstein’sfamousdiscussionsofgrouppolarization,extremism,andthe
Internetlargelyfocusonmattersofconstrictedinformationflowandomittedviewpoints
(Sunstein2001,2009b,2009a).EliPariser’sTheFilterBubblefocusesentirelyonfiltration
effectsfrompersonalizationtechnology,asinGooglesearches,andself-selected
informationalnetworks,asinFacebook(Pariser2011).Popularconversationhastendedto
followPariser’sfocusontechnologically-mediatedfiltration.Theterm“echochamber”has,in
recentusage,beenreducedtoasynonymforsuchbubblesandtheirconstrictedinformation
flow.Whenthespecificallytrust-orientedmanipulationsofechochambersarediscussed,
theyareusuallylumpedinwithepistemicbubblesaspartofoneunifiedphenomenon.
Thisisamistake;itisvitaltodistinguishbetweenthesetwophenomena.Anepistemic
bubbleisanepistemicstructureemergingfromtheinformationalarchitectureof
communities,socialnetworks,media,andothersourcesofinformationandargument.Itisan
impairedinformationaltopology—astructurewithpoorconnectivity.Anechochamber,on
theotherhand,isanepistemicstructurecreatedthroughthemanipulationoftrust;itcan
existwithinahealthyinformationaltopologybyaddingasuperstructureofdiscreditand
authority.Ihopetoshow,contratherecentfocusonepistemicbubbles,thatechochambers
poseasignificantanddistinctivethreat–perhapsevenamoredangerousone–thatrequires
4
averydifferentmodeofrepair.Furthermore,echochamberscanexplainwhatepistemic
bubblescannot:theapparentresistancetoclearevidencefoundinsomegroups,forexample,
climatechangedeniersandanti-vaccinationgroups.
Itmaybetemptingtotreatmembersofechochambersasmeresheep,andaccusethem
ofproblematicacquiescencetoepistemicauthority.Butthataccusationreliesonan
unreasonableexpectationforradicalepistemicautonomy.Contemporaryepistemology,
especiallysocialepistemology,hastaughtusthattrustinothersisnecessaryand
ineradicable.Weare,asJohnHardwigsays,irredeemablyepistemicallydependentoneach
other(Hardwig1985,1991).Echochamberspreyonourepistemicinterdependence.Thus,in
somecircumstances,echochambermembersdonothavefullepistemicresponsibilityfor
theirbeliefs.Onceoneistrappedinanechochamber,onemightfollowgoodepistemic
practicesandstillbeledfurtherastray.Andsomepeoplecanbetrappedinechochambers
becauseofcircumstancesbeyondtheircontrol—forexample,theycanberaisedinthem.
Whichleadstothemostimportantquestions:howcanonetellifoneisinanechochamber?
Andhowcanoneescape?Iwillarguethat,forthosetrappedwithinanechochamber,
prospectsfordetectionarepoorandtheescapepathdaunting.Detectingandescapingfrom
echochamberswillrequirearadicalrestructuringofamember’srelationshiptotheir
epistemicpast,whichmaybemorethanwecanreasonablyexpectofoneanother.
EpistemicBubbles
Let’sstartwithasimplepictureofhowmanyofusconductourepistemiclivesrightnow.
IgetmuchofmynewsviaFacebook.IhaveselectedmostofmyFacebookfriendsforsocial
reasons;theyaremyfriendsandcolleagues.Asignificantconduitformylearningabout
5
eventsintheworldisbypeoplere-postingarticlesthattheyhavefoundnewsworthyor
interesting.WhenIgooutsideofFacebook,Iusuallyturntosourceswhich,byandlarge,are
affiliatedwithmyownpoliticalbeliefsandintellectualculture.
Thisprocessimposesafilteronmyuptakeofinformation.Selectionandexclusionarenot
badinandofthemselves—theworldisoverstuffedwithsupposedsourcesofinformation,
manyofthemterrible.Thebettermyfilter,themoreitwillfocusmyattentiononrelevant,
useful,andreliableinformation.Butthebarefactthateachindividualmemberofthesystem
isthemselvesreliablewillnotguaranteeanybroadnessorcompletenessofcoverage.
Suppose,forexample,thatmysocialnetworkwascomposedentirelyofintelligent,reliable
professorsofaestheticswhoseinterestswerelargelyfocusedonnewdevelopmentsinopera,
ballet,andavant-gardecontemporaryart.Throughthissystem,Imightlearnaboutallthe
excitingnewdevelopmentsintheNewYorkArtscene,butentirelymiss,say,relevant
developmentsinrap,orthefactthatmycountrywasslowlyslidingintofascism.Thesystem
lackswhatGoldbergcallscoverage-reliability—thecompletenessofrelevanttestimonyfrom
acrossone’swholeepistemiccommunity(Goldberg2011,93-4).Badcoveragecannotonly
leaveoutrelevantfactsandevidence;itcanalsofailtobringrelevantargumentstoour
attention.Thus,badcoveragecanstarveusofadequateexposuretorelevantarguments.
Noticethatbadcoverageisanepistemicflawofepistemicsystemsandnetworks,notof
individuals.
Icannowspecifymyuseof“epistemicbubble”withgreaterprecision.Anepistemic
bubbleisasocialepistemicstructurewhichhasinadequatecoveragethroughaprocessof
exclusionbyomission.Epistemicbubblesformbyleavingoutrelevantepistemicsources,
ratherthanactivelydiscreditingthem.Thereareattwoprimaryforcesencouragingthis
omission.First,thereisanepistemicagent’sowntendencytoseeklike-mindedsources.This
6
phenomenonissometimescalled“selectiveexposure”bysocialscientists(Nelsonand
Webster2017).Inmanycontemporarycases,suchasFacebook,theprocessofomissioncan
occurinadvertently.ItypicallyputpeopleinmyFacebookfeedforsocialreasons—becauseI
likethemorIfindthemfunny.Butsocialselectiondoesnotguaranteegoodcoverage
reliability;infact,thetypicalbasesofsocialselectionareinimicaltogoodcoverage
reliability.2Weusuallylikepeoplewhoaresimilartous,andsuchsimilaritymakescoverage
gapsmorelikely.Friendsmakeforgoodparties,butpoorinformationnetworks.Wenow
haveastraightforwardaccountofonewayinwhichepistemicbubblescanform.Wecan
buildastructureforonesetofpurposes—maintainingsocialrelations,finding—andthen
proceedtouseitforanentirelydifferentpurpose,forwhichitfunctionsbadly:information
gathering.
Second,therearetheprocessesbywhichanepistemicagent’sinformationallandscapeis
modifiedbyotheragents.Thismightinclude,say,systematiccensorshipormediacontrolby
thestateorotheractors.Themostworrisomeoftheseexternalforces,atthemoment,seems
tobethealgorithmicpersonalfilteringofonlineexperiences(Pariser2011;Watson2015).
Internetsearchengines,forexample,willtrackpersonalinformationforeachparticularuser,
andadapttheirsearchresultstosuiteachuser’sinterest.Certainly,newspapersandother
traditionalmediatechnologiesdoplaceexternalfiltersontheirreaders,butthemodern
instantiationisparticularlypowerfulandtroubling.AsBoazMillerandIsaacRecordargue,
Internettechnologiescreatehyper-individualized,secretfilters.Thesecrecyisparticularly
threatening.Manyusersdonotknowabouttheexistenceofalgorithmicpersonalfiltering.
Evenamongstthosethatdo,mostdonothaveaccesstotheparticularitiesofthealgorithms
2 For an overview of empirical research on personal similarity and polarization, see (Sunstein 2009a, 83-5). Curiously, Sunstein notes the group polarization literature has thought relatively little about the impact of personal similarity.
7
doingthefiltering;thus,theveryopacityoftheprocessmakesitharderforauserto
successfullyevaluateandepistemicallycompensateforsuchfiltering(MillerandRecord
2013).Thus,mostuserssignificantlyunderestimatethedegreetowhichtheirexposureto
information,viasearchresults,hasalreadybeentailoredtopresentsearchresultstowhich
theuserwillalreadybeamenable.
Boththeagent-drivenprocessofselectiveexposure,andtheexternalitiesofalgorithmic
filtering,contributetothecreationofepistemicbubbles.Iintroducetheterm“epistemic
bubble”heretoindicateabroadersetofphenomena.Pariserintroducedtheterm“filter
bubble”toreferspecificallytotechnologicallymediatedfiltering,especiallyviaalgorithmic
matching.Epistemicbubblesarethosestructureswhichomitrelevantvoicesbyanymeans,
technologicalorotherwise.Epistemicbubblesincludefilterbubbles,butalsonon-
technologicalselectionprocesses,suchasphysicallysortingoneselfintoneighborhoodsof
like-mindedpeople(Bishop2009).
TheaccountI’vegivenofepistemicbubblesfocusesonthewaytheyomitrelevant
information,butthatomissioncanalsothreatenuswithbootstrappedcorroboration.Usersof
socialnetworksandpersonalizedsearchtechnologieswillencounteragreementmore
frequentlyandsobetemptedtoover-inflatetheirepistemicself-confidence.Thisdanger
threatensbecause,ingeneral,corroborationisoftenaverygoodreasontoincreaseone’s
confidenceintherelevantbeliefs(Nguyen2010,2018a).Butcorroborationoughttoonly
haveweightifitaddssomethingepistemically,ratherthanbeingamerecopy.Toborrowan
examplefromWittgenstein:imaginelookingthroughastackofidenticalnewspapersand
treatingeachnextnewspaperheadlinesayingpasareasontoincreaseyourbeliefthatp
(Wittgenstein2010,100).Thisisclearlyamistake;thefactthatanewspaperclaimsthatp
hassomeepistemicweight,butthenumberofcopiesofthatnewspaperoneencounters
8
oughtnotaddanyextraweight.Similarly,imaginespeakingtoabunchofacolytesofGuru
Jane,whorepeatanythingthatGuruJanesayswithoutanyfurtherthought.Thefactthatall
theseacolytesrepeatGuruJane’stestimonyshouldaddnoextraweight.Solongasthe
disciplinesrepeatanythingGuruJanesays—solongastheyaremereconduitsfor
information,ratherthansourcesofinformation—theyaresimplyanothersortofcopy.
Butcopyingisn’ttheonlyroutetoaproblematicformofnon-independence.SupposeI
believethatthePaleodietisthebestdiet.IproceedtoassembleabodyofpeerswhoItrust
preciselybecausetheyalsobelievethatPaleoisthebestdiet.Inthatcase,theexistenceof
perfectagreementonPaleo’samazingnessthroughoutthatgroupoughttocountforfarless
thanitmightforothergroupsthatIhadnotassembledonthatbasis.Evenifallthegroup
membersarrivedattheirbeliefsindependently,theiragreementisalreadyguaranteedbymy
selectionprinciple.TothedegreethatIhavepre-selectedthemembersinmyepistemic
networkfromagreementwithsomesetofbeliefsofmine,thentheiragreementwiththatset
ofbeliefsandanyotherbeliefsthatitentailsoughttobeepistemicallydiscounted.3Ifwefail
tosodiscount,weareignoringapernicioushiddencircularityinourcorroborativeprocess.It
iseasytoforgettodiscountbecausethebootstraphereisobscuredbytimeandinterface.But
wemustactivelyadjustfortheincreasedlikelihoodofagreementinsideourbubbles,orwe
willunwarrantedlybootstrapourconfidencelevels.4
3 This is a relative of the problem, from statistics, of failing to identify dependent variables.
4 I am not claiming, as Alvin Goldman does, that all cases of non-independent testimony should be discounted Goldman 2001, 98-104). Goldman’s analysis, and the principle of non-independence, have been significantly challenged (Coady 2006; Lackey 2013). David Coady and Jennifer Lackey have demonstrated that we can trust the weight of non-independent agreement when we have good reason to think that the non-independent agreers had good epistemic meta-reasons for agreeing. For example, suppose that all scientists agreed that climate change was coming. Their agreement is non-independent — the majority of these scientists have not analyzed the data for themselves, but trust the expert specialists in climate change. But still, the weight of numbers matters here because the trusting scientists have good epistemic reasons for picking who to trust. But in the case I’ve described, the weight of numbers here does not emanate from a good epistemic process. In other words, my claim doesn’t depend on the claim that all forms of non-independence are problematic, only that some are, for the reasons I adduce.
9
Tosummarize:anepistemicbubbleisanepistemicnetworkthathasinadequatecoverage
throughaprocessofexclusionbyomission.Thatomissionneednotbemaliciousoreven
intentional,butmembersofthatcommunitywillnotreceivealltherelevantevidence,norbe
exposedtoabalancedsetofarguments.
EchoChambers
Luckilyforus,epistemicbubblesarerelativelyfragile.Relevantsourceshavesimplybeen
leftout;theyhavenotbeendiscredited.Itispossibletopopanepistemicbubblebyexposing
amembertorelevantinformationorargumentsthattheyhavemissed.Echochambers,on
theotherhand,aresignificantlymorerobust.
MyanalysisherecombinesempiricalworkandanalysisfromJamiesonandCappellaon
thenatureofright-wingecho-chamberswithrecentinsightsfromsocialepistemology.
JamiesonandCappellastudiedechochambersbuiltaroundparticularcharismatic
personalities—RushLimbaughandthenewsteamofFoxNews,andcertainothermembers
ofconservativetalkradio.TheirdataandanalysissuggestthatLimbaughusesmethodsto
activelyisolatehiscommunityoffollowersfromotherepistemicsources.Limbaugh’s
consistentattacksonthe“mainstreammedia”servetodiscreditallpotentialsourcesof
knowledgeortestimonybesidesLimbaughandaselectinner-cadreofotherapproved
sources(JamiesonandCappella2008,140-76).Limbaughalsodevelopswhattheycalla
privatelanguage,fullofalternatemeaningsforfamiliartermsandnewjargon(forexample,
“SJWs”),inordertoexaggeratetheinsularityandseparatenessofthein-group.Finally,
Limbaughprovidescounter-explanationsofallcontraryviews,intendednotonlytoattack
eachparticularview,butalsotounderminethegeneraltrustworthinessandintegrityof
10
anybodyexpressingacontraryview.Theresultingworld-viewisoneofhighlyopposed
forces;onceonehassubscribedtoLimbaugh’sview,onehasreasontothinkthatanybody
whodoesnotalsosubscribeisactivelyopposedtothesideofright,andtherebymorally
unsoundandsogenerallyuntrustworthy(177-90).JamiesonandCappellasuggestthatthis
makesafollowerdependentonasinglesourceorgroupofsources,andmakesthemhighly
resistanttoanyoutsidesources.Theyofferthefollowingdefinitionofanechochamber:an
echochamberisaboundedandenclosedgroupthatmagnifiestheinternalvoicesand
insulatesthemfromrebuttal(76).
Iwillusetheterm“echochamber”herefollowingtheiranalysis,butIadaptthedefinition
slightlyforphilosophicaluse.Iuse“echochamber”tomeananepistemiccommunitywhich
createsasignificantdisparityintrustbetweenmembersandnon-members.Thisdisparityis
createdbyexcludingnon-membersthroughepistemicdiscrediting,whilesimultaneously
amplifyinginsidermembers’epistemiccredential.Finally,echochambersaresuchthatin
whichgeneralagreementwithsomecoresetofbeliefsisapre-requisiteformembership,where
thosecorebeliefsincludebeliefsthatsupportthatdisparityintrust.
By“epistemicdiscrediting”,Imeanthatnon-membersarenotsimplyomittedornot
heard,butareactivelyassignedsomeepistemicdemerit,suchasunreliability,epistemic
maliciousness,ordishonesty.By“amplifyingepistemiccredentials”,Imeanthatmembersare
assignedveryhighlevelsoftrust.Ofcourse,thesetwoprocessescanfeedbackintoone
another.Solongasanechochamber’strustedinsiderscontinuetoclaimthatoutsidersare
untrustworthy,thentheinnertrustwillreinforcetheoutwarddistrust.Andsolongas
outsidersarelargelydistrusted,thentheinsiderswillbeinsulatedfromvariousformsof
counter-evidenceandrebuff,thusincreasingtheirrelativecredence.Onceasufficient
disparityincredencebetweeninsidersandoutsidershasbeenestablished,solongastrusted
11
insiderscontinuetoholdandespouseepistemicallydismissivebeliefstowardsoutsiders,
thentheechochambers’beliefssystemwillbeextremelydifficulttodislodge.
Comparethisprocessofcredencemanipulationtotheprocessofomissionfoundin
epistemicbubbles.Inonestandardscenario,Iaddothersastrustedmembersofmy
epistemicnetworkbasedonagreement.Iamthenlesslikelytoencounteranoutsidevoice—
butwhenIdoactuallyhavesuchanencounterwithanoutsider,Ihavenobackgroundreason
todismissthem.Bubblesrestrictaccesstooutsiders,butdon’tnecessarilychangetheir
credibility.Echochambers,ontheotherhand,workbyofferingapre-emptivediscredit
towardsanyoutsidesources.5
Theresultisaratherstrikingparalleltothetechniquesofisolationtypicallypracticedin
cultindoctrination.Thestandardtechniquesofcultindoctrination,byatraditionalaccount,
aretheaggressiveemotionalisolationofcultmembersfromallnon-cultmembers,which
amplifiesindoctrinatedmember’sdependencyonthecult(Singer1979;Langone1994;
Lifton1991).6Newcultmembersarebroughttodistrustallnon-cultmembers,which
providesanepistemicbufferagainstanyattemptstoextracttheindoctrinatedpersonfrom
thecult.Thisisnothinglikehowepistemicbubbleswork.Epistemicbubblesmerelyleave
5 Note that the kind of echo chambers here are different from those I described in (Nguyen 2018a). There, I explored what I called “personal” echo chambers, which can be generated through entirely good-intentioned individual action under very special epistemic conditions, which I called a “cognitive island”. A cognitive island is any cognitive domain in which there is no possible empirical method to check whether a purported expert is really an expert. For example, the moral domain and the aesthetic domain might be cognitive islands. What I’ve discussed in this paper is something different – what I’ve called “social” echo chambers. They are malicious and essentially social structures which can exist in any empirical domain, not just on a cognitive island. I will offer a synthesis of these two conceptions of echo chambers, with an eye towards elucidating the similarities in their underlying structures, in a future work. 6 Note that this view of cult indoctrination is standard among mental health professionals and social workers, but has been resisted by some theorists, especially from the fields of religious studies and sociology, as a method of de-legitimizing and repressing minority religions (Robbins and Anthony 1982). For a good overview of the debate, see (Coates 2010). For the purposes of my argument, it does not matter if cults actually work this way; what is interesting is that echo chambers work by the mechanism that is, under a traditional account, attributed to cults.
12
theirmembersignorant,butignorancecanbefixedwithsimpleexposure.Thefunctionofan
echochamber,ontheotherhand,istocredentiallyisolateitsmembersbyamanipulationof
trust.Bythis,Imeanthatmembersarenotjustcutoff,butareactivelyalienatedfromanyof
theusualsourcesofcontraryargument,consideration,orevidence.Membershavebeen
preparedtodiscreditanddistrustanyoutsidesources;thus,mereexposuretorelevant
outsideinformationwillhavenoeffect.
Infact,echochamberscanavailthemselvesofanotherepistemicprotectivemechanism:
theycancontainwhatI’llcalladisagreement-reinforcementmechanism.Memberscanbe
broughttoholdasetofbeliefssuchthattheexistenceandexpressionofcontrarybeliefs
reinforcestheoriginalsetofbeliefsandthediscreditingstory.Atoyexample:supposeIama
cultleader,andIhavetaughtmyfollowerstobelievethateveryhumanexceptthemembers
ofourgrouphasbeeninfestedandmind-controlledbyalienghostsfromMars.Ialsoteach
myfollowersthatthesealienghostsfromMarshateourgroupforknowingthetruth,andso
willconstantlyseektoundermineourknowledgeoftheirexistencethroughmechanismslike
callingusa‘cult’andcallinguslunatics.EndreBegbyhasofferedacarefulanalysisofthis
particularsortofdisagreement-reinforcementmechanism,whichhecalls“evidential
preemption.”SupposethatItellmyfollowerstoexpectoutsiderstofalselyclaimthatthere
arenoghostsfromMars.Whenmyfollowersdoconfrontsuchcontraryclaimsfrom
outsiders,thosecontraryclaimsareexactlywhattheyexpectedtohear.Thus,newcontrary
testimonyisneutralized,becauseitwaspredictedbypastbeliefs.This,saysBegby,functions
asakindofepistemicinoculation.Thereisalsoasecondaryeffect.Whenmyfollowershear
exactlywhatIpredicted,thenmyclaimshavebeenverified,andsomyfollowerswillhave
somereasontoincreasetheirtrustinme.Thus,theechochamber’sbeliefsystemnotonly
neutralizestheepistemicimpactofexposuretooutsiderswithcontrarybeliefs;theexistence
13
ofthosecontrarybeliefswillactivelycorroboratethepre-emptorandsoincreasethe
credenceleveloftheentireechochamber(Begby2017).Thiscreatesafeedbackmechanism
withintheechochamber—inmakingunderminingpredictionsaboutcontrarytestimony,
insideauthoritiesnotonlydiscreditthatcontrarytestimony,butincreasetheir
trustworthinessforfuturepredictions.
Oncesuchasystemofbeliefsissetup,itcanbeverydifficulttodislodge—itisself-
reinforcing,bounded,andbuilttodiscountanycontraryinput.Infact,thoughmydefinitionof
echochambersisconceptuallyseparablefromsuchadisagreement-reinforcement
mechanism,everyplausiblereal-worldcandidateforanechochamberI’veeverencountered
includessomeversionofadisagreement-reinforcementmechanism.Foradepressingreal-
worldexample,considerPizzagate.Pizzagateisaconspiracytheorythatboiledoutofaright-
wingonlineforumonReddit,whichincludedbeliefsthatCometPingPong,apizzarestaurant,
wasthesiteofachildsextraffickingringownedbyaliberalconspiracyinvolvingHillary
ClintonandBarackObama.Eventually,EdgarWelch,amemberofthatforum,forcibly
enteredthepizzaparlorarmedwithanassaultrifletoinvestigate;whenhesatisfiedhimself
thattherestaurantcontainednochildslaves,hegavehimselfuptothepolice.Theonline
forum,however,didnottakethisascontraryevidence.Instead,theyleanedontheirbelief
thattheliberalconspiracyhadtotalcontrolofthemainstreammedia,andwaswillingto
stagefakeeventstodiscredittheright-wing.TheforumtookWelch’sclaimsthattherewasno
sextraffickingringasevidencethatWelchwasapaidactor,andthusasfurtherconfirmation
oftheexistenceofapowerfulcabalofliberalchildsextraffickers(Mengus2016;Vogtand
Goldman2016).
Conspiracytheoriesfunctionhereinafascinatinginversiontocorroborative
bootstrapping.Incorroborativebootstrapping,themistakeistotreatproblematically
14
dependentlyselectedinsidersasiftheywereindependent,andthusoverweighttheir
testimony.Whenanechochamberusesaconspiracytheoryinthismanner,theyare
attributingaproblematicformofnon-independencetooutsiderswhoareactually
independent,andtherebyunderweightingoutsidetestimony.Anechochamberhereworks
bydiscreditingtheapparentindependenceof,say,differentclimatechangescientistsby
claimingthatalltheirvarioustestimoniesareproblematicallyderivedfromasinglesource.
Incidentally,Iamnotclaimingherethatconspiracytheoriesarealwaysornecessarily
incorrectortheproductofepistemicvice.Asothershaveargued,believinginconspiracy
theoriesisn’tbadperse,becausesomeconspiracytheoriesaretrue(Coady2012,110-137;
Dentith2017).Butthefactthatconspiracytheoriescanfunctiontoreinforcetheboundaries
ofechochambers—thoughtheydonotnecessarilydoso—mightexplainpartofconspiracy
theories’badrap.Becauseoftheireffectivenessinsettingupdisagreement-reinforcement
mechanisms,conspiracytheoriesareoftenconscriptedasapowerfultoolinthebad
epistemicbehaviorofcertaingroups.
Itisimportanttonotethattheepistemicmechanismsbywhichechochamberswork,
thoughproblematic,arenotsuigeneris.Theyareperversionsofnatural,useful,andnecessary
attitudesofindividualandinstitutionaltrust.Theproblemisn’tthatwetrustanddistrust
groupsandinstitutions.Infact,wemustdoso.EljiahMillgramcallsittheproblemofhyper-
specialization.Humanknowledgehassplinteredintoavastsetofspecializedfieldsthat
dependoneachother.Noonehumancanmanagethatinformation;weareforcedtotrust
eachother(Millgram2015,27-44).7Noneofusisinapositiontoreliablyidentifyanexpertin
7 Though this paper relies on insights from modern work in the epistemology of testimony, I have tried to rely only on uncontroversial claims from that field, and not on the technical details of any particular view. In particular, I have attempted to construct my analysis so as to be independent of the debate on whether or not testimony is a basic source of knowledge. I have also attempted to make the paper compatible with the major accounts of trust.
15
mostspecialistfieldsoutsideofourown.Iam,onmyown,helplesstoevaluatethevirtuesof
antibioticsortheexpertiseofaparticulardoctororsurgeon.Iam,instead,reliantonavast
networkofinstitutionallicensingpracticesinordertochoosemyhealthcaresources—
includingjournalpeerreview,medicalboardexams,universityhiringpractices,andthelike
(Nguyen2017a).Often,ItrustviawhatPhilipKitchercallsindirectcalibration—Itrust
mechanicalengineersbecausetheymakethingsthatwork,butIknowthatmechanical
engineerstrustappliedphysicists,andIknowthatappliedphysiciststrusttheoretical
physicists,soIacquiretrustthroughalongchainoffield-widelinks(Kitcher1993,320-3).I
evenuselitmustests:thefactthatanypersonorgroupisinfavorof,say,sexualorientation
conversiontherapyisenoughformetodiscreditthemonanysocialormoraltopics.Wemust
resorttosuchtacticsinordertonavigatethehyper-specializedworld(Nguyenforthcoming).
Echochambersfunctionparasiticallyonthesepracticesbyapplyingdiscreditswithout
regardfortheactualepistemicworthofthediscreditedinstitutionsorindividuals.The
discreditisinsteadappliedstrategicallyanddefensively,towardsalloutsiderssolelyonthe
basisoftheirbeingoutsiders.Oncethediscreditingbeliefsareinplace,theensuingbeliefs
andactiontheechochambers’membersaresurprisinglyclosetorational.Infact,wecan
easilyimaginealternativescenariosinwhichaverysimilarsetofbeliefswereappropriate
andveristic.IfIwasananti-NaziinGermanyduringtheriseoftheNaziparty,Iwoulddowell
tomaintainthebeliefsthatthemostpeoplewerecorrupt,untrustworthy,andoutto
maliciouslyunderminemyowntruebeliefs.Butifsuchbeliefsbecomeimplantedinan
inappropriatecontext,theycanleadtheirbelieversentirelyastray.
Epistemicbubblescaneasilyformaccidentally.Butthemostplausibleexplanationforthe
particularfeaturesofechochambersissomethingmoremalicious.Echochambersare
excellenttoolstomaintain,reinforceandexpandpowerthroughepistemiccontrol.Thus,itis
16
likely(thoughnotnecessary)thatechochambersaresetupintentionally,oratleast
maintainedforthisfunctionality.Myaccountthusbearssomeresemblancetosomeworkon
testimonialinjusticeandtheepistemologyofignorance,butitisimportantlydistinct.
MirandaFrickerhasarguedforakindoftestimonialinjustice,basedonagapbetweenactual
reliabilityandperceivedcredibility.Forexample,saysFricker,beingwhiteandbeingmale
arebothbonusestocredibility.Sincecredibilityisasourceofpower,anybodywith
credibilitywillseektoincreaseit,usingthatverycredibility.Thus,saysFricker,credibility
gapscanbeturnedintoepistemictoolsofsocialoppression(Fricker2011).Similarly,Charles
Millsarguesthatthereisanactivepracticeofignoranceamongmembersofoppressive
groups,suchaswhiteAmericans.Itistothebenefitofthoseinpowertoactivelyignoremany
aspectsoftheexistenceofoppressedgroups(Mills2007;Alcoff2007,47-57).Myaccountis
compatiblewith,butindependentfrom,Fricker’sandMills’accounts.Echochamberscanand
surelyareusedtomaintainsocialoppressionthroughenhancingcredibilitygapsand
supportingpracticesofactiveignorance.Thesystematicmistrustofanechochambersisa
powerfultoolforperpetuatingepistemicinjusticeandactiveignorance.However,the
conceptofanechochamberdoesnotrequirethattheybedeployedonlyinpoliticalcontexts,
nordoesitrequirethattheyonlybedeployedonlyintheserviceofoppression.Echo
chamberscouldpotentiallyexistamongtheoppressed,andsurelyexistinapoliticalcontexts.
IbelieveIhavewitnessedechochambersformingaroundtopicssuchasanti-vaccination,
multi-levelmarketingprograms,particulardiets,exerciseprograms,liberalactivism,
therapeuticmethodologies,philosophiesofchild-rearing,particularacademicsub-
disciplines,andCrossfit(Weathers2014).
17
‘Post-truth’andthepowerofechochambers
Ithasoftenbeenclaimed,duringandaftertheAmericanpoliticalseasonof2016,thatwe
haveentereda‘post-truthera’.Notonlydosomepoliticalfiguresseemtospeakwithablatant
disregardforthefacts,theirsupportersseemunswayedbyreasonorcontraryevidence.To
many,itseemsasifavastswathoftheelectoratehasbecomeentirelyunmooredfromany
interestinfactsorevidence.Callthisthe“totalirrationality”explanationofthepost-truth
phenomenon.
Butechochambersofferanalternativeexplanationfortheapparentpost-truthmood.It
seemslikelythatthereisatleastonevastpartisanecho-chamberpresentinthepolitical
landscape.JamiesonandCappella’sstudyisadecadeold,butsourceslikeBreitbartandAlex
Jones’Infowarsseemlikeclearextensionsofthesameright-wingechochamber.(Otherecho
chamberssurelyexistelsewhereonthepoliticalspectrum,though,tomymind,theleft-wing
echochambershavebeenunabletoexertasimilarlevelofpoliticalforce.)Inthatcase,the
accountofechochambersI’veofferedhassignificantexplanatoryforce.Theapparent“post-
truth”attitudecanbeexplained,atleastinpart,astheresultofcredencemanipulations
wroughtbyechochambers.Inhealthyepistemiccommunities,thereissomethinglikean
upperceilingonthecredencelevelaccordedtoanyindividual.Ahealthyepistemicnetwork
willsupplyasteadystreamofcontraryevidenceandcounterarguments;thus,nosingle
individualorgroupwillevergounchallenged.Epistemicbubblesmakethediscoveryof
mistakessignificantlylesslikely,andsotendtoexaggeratethecredencelevelsofepistemic
sourcesinsidethebubble.Butwhenanechochamberisinplaceandalloutsidesourceshave
beeneffectivelydiscredited,thatceilingdisappearscategorically.Echochamberscancreate
runawaycredencelevelsforapprovedindividuals.Byremovingdisconfirmationsand
discorroborationfromthesystemthroughthesystematicdiscreditingofoutsiders,echo
18
chamberscancreateexceptionallyhigh—oneistemptedtosayunnaturallyhigh—levelsof
trust.Thatpotentialforrunawaycredenceisbuiltrightintothefoundationsofanyecho
chamber,andarisesfromaninteractionbetweenthetwomaincomponentsofanyecho
chamber.First,anechochamberinvolvesasignificantdisparityoftrustbetweentheinsiders
andtheoutsiders.Second,anechochamberinvolvesbeliefs,espousedbytheinsiders,
reinforcingthatdisparity.Theessentialfeaturesofechochambersseemdesignedtoself-
reinforcetheirpeculiararrangementoftrust.
Noticethatepistemicbubblesalonecannotexplainthepost-truthphenomenon.Since
epistemicbubblesworkonlyviacoveragegaps,theyofferlittleinthewayofexplanationfor
whyanindividualwouldrejectclearevidencewhentheyactuallydoencounterit.Coverage
gapscannotexplainhowsomebodycould,say,continuetodenytheexistenceofclimate
changewhenactuallyconfrontedwiththeoverwhelmingevidence.Onewouldbetempted,
then,toaccuseclimatechangedeniersofsomekindofbruteerror.Butechochambersoffer
anexplanationofthephenomenonwithoutresortingtoattributionsofbruteirrationality.
Climatechangedeniershaveenteredanepistemicstructurewherebyalloutsidesourcesof
evidencehavebeenthoroughlydiscredited.Enteringthatepistemicstructuremightitself
involvevariousepistemicmistakesandvices—butherethestorycanbeoneoftheslow
accumulationofminormistakes,whichgraduallyembedthebelieverinaself-reinforcing,
internallycoherent,butultimatelymisleadingepistemicstructure.
Similarly,somehavesuggestedthat,inthepost-truthera,manypeople’sinterestinthe
truthhasevaporated.Onceagain,thisaccountofechochamberssuggestsalessdamningand
moremodestexplanation.Anechochamberdoesn’terodeamember’sinterestinthetruth;it
merelymanipulatestheircredencelevelssuchthatradicallydifferentsourcesandinstitutions
willbeconsideredpropersourcesofevidence.Thisphenomenonstandsinstarkcontrastto
19
accountsofobfuscatoryspeech.Take,forinstance,Orwelliandoublespeak—deliberately
ambiguous,euphemism-filledlanguage,designedtohidetheintentofthespeaker(Orwell
1968).Doublespeakisapracticethatevincesnointerestincoherence,clarity,ortruth.But
bymyaccount,weshouldexpectdiscoursewithinechochamberstobeentirelydifferent—
weshouldexpectsuchdiscoursetobecrispandclear,andtopresentunambiguousclaims
aboutwhatisthecase,whatsecretconspiraciesareinplace,andwhichsourcesaretobe
entirelydistrusted.Andthisispreciselywhatwefind(JamiesonandCappella2008,3-41,140-
176).Consider,forexample,Breitbart’sattacksonothermediasources.Onearticlebegins:
“MainstreammediaoutletscontinuetoprintfalseanddefamatorydescriptionsofBreitbart
Newsinanakedlypoliticalefforttomarginalizeagrowingcompetitor”(Pollak2017).Thisis
notthedouble-speakofadministratorsandbureaucrats—thisisaclear,strident,and
unambiguouslywordeddiscredit.
Onemightbetemptedtosay:butjustgivethemtherealevidence!Youcan’tdiscredit
neutralevidence!Butthisresponseradicallyunderestimatesthedegreeoftrustandsocial
processinginvolvedinmostpresentationsofevidence.ExceptforempiricalevidenceImyself
havegathered,allotherpresentationsofevidencerelyontrust.Mybeliefintherealityof
climatechangedependsonenormousamountsofinstitutionaltrust.Ihavenotgatheredthe
climatechangeevidencemyself;Imostlyjusttrustsciencejournalistswho,inturn,trust
institutionalcredentialingsystems.EvenifIhadbeenon,say,acoresamplingexpeditionto
theArctic,Iwouldbeunabletoprocessthatinformationformyself,orevenvetwhether
somebodyelsehasproperlyprocessedit.Eventheclimatologistwhoactuallyprocessesthat
informationmustalsodependontrustingavastarrayofotherexperts,includingstatisticians,
chemists,andtheprogrammersoftheirdataanalysissoftware.Mostso-called“neutral
20
evidence”dependsonlongandrobustchainsoftrust(Millgram2015,27-44).Membersofan
echochamberhaveacquiredbeliefswhichbreaktheusualarrangementsoftrust.
Butdespitetheirevidentexplanatoryforce,echochambershavebeenlargelyneglected
byrecentempiricalresearch.Muchoftherecentresearchoncausesofbeliefpolarization
focusesonthecausalroleofindividualpsychology,suchasthetendencytowardslazinessin
thescrutinyofone’sownbeliefs(Troucheetal.2016).Similarly,recentstudiesonclimate
changedenialfocusonstudyingtherelationshipbetweenanindividual’sstatedpolitical
beliefsandtheirreactionstoclimatechangeinformation,withoutinquiringintothesocial
epistemicstructuresinwhichtheindividualsareembedded(Corner,WhitmarshandXenias
2012).Famously,DanKahanandDonaldBramanarguefortheculturalcognitionthesis—
thatis,thatculturalcommitmentsarepriortofactualbeliefs,andthatnon-evidentially
formedculturalvaluesinformwhichfuturepresentationsofevidencewillbeadmittedas
weighty(KahanandBraman2006).Thoughthevaluesmayoriginallycomefroman
individual’sculture,KahanandBramanfocustheiranalysisonhowthoseacquiredvalues
functioninindividualreasoningtocreatepolarization.Theypaylittleattentiontothe
continuingroleofthecontingentsocialstructuresinwhichtheindividualisembedded.
Thedirectliteratureonechochambersandepistemicbubblesisnewandrelativelysmall,
comparedtothesizableliteratureonindividualbeliefpolarization.Unfortunately,evenin
thatliterature,echochambersandepistemicbubbleshaveoftenbeenconfused.Theyare
usuallyaddressedinthepopularmediatogether,andtheterms‘epistemicbubble’and‘echo
chamber’aretypicallyusedinterchangeably(El-Bermawy2016).Thesameblurringhas
occurredinthetreatmentofthephenomenainacademicepistemologyinthesurprisingly
smallliteratureonechochambers.Forexample,BertBaumgaertner,inhisanalysisofecho
chambersviacomputermodeling,lumpstogetherundertheheading‘echochamber’both
21
JamiesonandCappella’saccountofrightwingechochambersandEricGilbertetal’s
treatmentofblogsasechochambers(Baumgaertner2014).ButwhereJamiesonand
Cappella’ssubjectofstudyisechochambers,Gilbert’sdiscussionconcernshowblog
communitieshavemerelyexcludedrelevantvoicesthroughsocialpracticesofconnecting
withlike-mindedindividuals—clearlywhatI’mcallingepistemicbubbles(Gilbert,
BergstromandKarahalios2009).Inbothpopularandacademiccases,furtheranalysishas
focusedhasbeenonepistemicbubbles;thestructuresofdiscreditinvolvedinJamiesonand
Cappella’sechochambershavebeenlargelyneglected.8
Foraparticularlyprominentexample,considerSunstein’sinfluentialaccountofthe
relationshipofInternettechnologies,grouppolarizationandtheriseofpoliticalextremism.
Thoughheprofessestocoverbothfilterbubblesandechochambers,hisworkfocusesalmost
entirelyonepistemicbubbleeffects:constrictedinformationflow,lackofexposureto
alternatearguments,andbootstrappedcorroboration(Sunstein2009b,xi,19-06,2009a,1-
98).Thepointhereisaboutmorejustthanhischoiceofwords:hissubjectsofanalysis
include,amongotherthings,Facebookfriendgroups,hategroups,extremistonlinepolitical
forums,conspiracytheorists,andterroristgroups(99-125,2009b,46-96).Clearly,thislist
includesprimecandidatesforbothepistemicbubblesandechochambers.Buthisanalysis
focusesalmostentirelyontheeffectsofbootstrappedcorroborationandlackofexposure.For
Sunstein,theprimarymechanismdrivingpolarizationandextremismisthelossoftruly
publicforums,becausetechnologyhasover-empoweredpeople’stendencytoself-select
sourcesofferingfamiliarviews.Thus,hissolutionistore-create,inthenewmedia
environment,thekindofgeneralpublicforumswherepeoplemightbemorelikelyto
8 One exception is the careful treatment of aesthetic echo chambers in (Robson 2014), which brought Jamieson and Cappella’s work to my attention, and was instrumental in this paper’s development.
22
serendipitouslyencountercontraryviewsandarguments.Hissolutionsincludegovernment-
fundedpublicnewswebsiteswithdiversecoverageandvoluntaryworkbycorporationsand
individualstobursttheirbubbles.Hisrecommendationsforrepairlargelyhavetodowith
increasingexposure(Sunstein2009a,135-48,2009b,19-45,190-211).But,again,ifwhat’s
goingonisactuallyanechochambereffect,exposureisuselessorworse.
Theblurringofthetwoconceptshasalsoleadtosomeproblematicdismissalsofthe
wholeclusterofphenomena.Anumberofrecentarticlesinsocialscience,communications,
andmediastudieshavearguedthatthewholesetofworriesaboutbubblesandecho
chambersiswildlyoverstated.Thesearticlessharethesameargumentativepattern.First,
theyusetheterms“filterbubble”and“echochamber”interchangeably,andaddress
themselvestothesameclusterofphenomenaasSunstein,treatingthemassingular.Infact,
JamesNelsonandJamesWebsterconflateJamiesonandCappella’sanalysisofechochambers
andPariser’sanalysisoffilterbubbles,anderroneouslyattributetoJamiesonandCappella
theviewthatpoliticalpartisansonlyseekoutandencountermediafromsourceswith
matchingpoliticalalignments—thatis,NelsonandWebsterattributetoanepistemic
bubblesaccounttoJamiesonandCappella,whereJamiesonandCappella’sactualtextis
clearlyanechochambersaccount(NelsonandWebster2017,2).Moreimportantly,these
recentarticlesproceedtoargueagainsttheexistenceoffilterbubblesandechochambersby
demonstratingthat,throughtheanalysisofempiricaldataaboutmediaconsumption,most
peopleinfactexposethemselvestomediafromacrossthepoliticalspectrum.Nelsonand
Webster,forexample,argueagainstJamiesonandCapella,claimingthatfilterbubblesand
echochambersdon’texist.NelsonandWebstersupporttheirclaimwithdatashowingthat
bothliberalsandconservativesvisitthesamemediasitesandspendcomparableamountsof
timeatthosesites(6-7).Again,thismissesthemark—thisisevidenceonlyagainstthe
23
existenceofepistemicbubbles,andnotagainsttheexistenceofechochambers.Similarly,Seth
Flaxmanetalseekstoproblematizetheexistenceoffilterbubblesandechochamberswith
datathatsocialmediaplatformsseemtoactuallyincreasepeople’sexposuretomediafrom
acrossthepoliticaldivide(Flaxman,GoelandRao2016).Again,thesedataonlyconcernthe
exposureandomission,andonlyweighagainsttheexistenceofepistemicbubbles.Theysay
nothingaboutwhetherechochambersexist.Echochambers,recall,arestructuresofstrategic
discrediting,ratherthanbadinformationalconnectivity.Echochamberscanexistevenwhen
informationflowswell.Infact,echochambersshouldhopethattheirmembersareexposedto
mediafromtheoutside;iftherightdisagreementreinforcementmechanismsareinplace,
thatexposurewillonlyreinforcetheechochambers’members’allegiance.Weoughtnot
concludethen,fromdatathatepistemicbubblesdonotexist,thatechochambersalsodonot
exist.
Wecanseenowcrucialitistokeepthesetwocategoriesdistinct.Epistemicbubblesare
ratherramshackle—theygoupeasily,buttheyareeasytotakedown.Sincethereisno
systematicdiscreditingofoutsiders,simpleexposuretoexcludedvoicescanrelievethe
problem.Echochambers,ontheotherhand,aremuchhardertoescape.Echochamberscan
starttoseemalmostlikelivingthings—thebeliefsystemsprovidestructuralintegrityand
resilience.Mereexposurewillbeineffective.JamiesonandCappellaofferevidenceofthis
effect:oncelistenersarecaughtinRushLimbaugh’slanguage,framing,anddiscredentialing
ofthemainstreammedia,theirbeliefscansurvivefrequentcontactwithcontraryviewpoints.
Limbaugh’stechnique,sayJamiesonandCappella,servestoinsulateandinoculatehis
audiencefrombeingaffectedbyexposuretocontraryviewpoints(JamiesonandCappella
2008,163-190).Infact,iftheappropriatedisagreement-reinforcementmechanismsarein
place,exposurewillsimplystrengthentheattackedbeliefsystems.Thus,anoutsider’s
24
attempttobreakanechochamberasifitwereamerebubbleislikelytobackfireand
reinforcetheechochamber’sgrip.9
Responsibilityandthepossibilityofescape
Sowhat,then,aretheepistemicresponsibilitiesofanagenttodiscoverwhethertheyare
inoneofthesesocialepistemictraps,andwhataretheirprospectsforactuallydiscovering
theirpredicamentandsuccessfullyescaping?Toanswerthis,wemustconsidertwodistinct
questions:
Theescaperoutequestion:Isthereanywayoutofanechochamberorepistemic
bubble?
Theescaperesponsibilityquestion:Couldonebehaveepistemicallyvirtuously,andyet
stillremaincaughtwithinanechochamberorepistemicbubble?Inotherwords,towhat
degreeisanepistemicagentembeddedwithinsuchastructureblameworthy,orblameless,
forthefaultinessoftheirbeliefs?
Thefirstquestionasksaboutthepossibleexistenceofanescaperoute.Thesecondasks
whetherthereisanescaperoutethatwemightreasonablyexpectanepistemicallyvirtuous
agenttodiscoverandenact.Thesearedistinctquestions,becauseanescaperoutemightturn
9 Sunstein does briefly note the empirical data for the disagreement-reinforcement effect in passing, but then seems to ignore it in proposing his solutions (Sunstein 2009a, 54-5)
25
outtobepossible,butsodifficulttodiscoverorusethatitwasbeyondwhatwemight
reasonablyexpectofanagentofmoderateepistemicvirtue.
Forepistemicbubbles,theanswersarestraightforward.AsI’veargued,epistemicbubbles
arequiteeasytoshatter.Onejustneedsexposuretoexcludedinformation.Insofarasthat
informationisavailable,butsimplynotpartofone’sstandardnetwork,thenmembersof
epistemicbubblesarefailingtoliveuptotheirepistemicduties,whichincludeproactively
gatheringrelevantdata.Totranslateintocontemporaryterms:ifyou’resubjecttoan
epistemicbubblebecauseyougetallyournewsfromFacebookanddon’tbothertolookat
othersources,youare,indeed,blameworthyforthatfailure.Ifonefindsthelanguageof
epistemicvirtuesandvicesappealing,thenwecansaythatmembersofepistemicbubbles
arecommittingtheviceofepistemiclaziness.
Answeringthesetwoquestionsismuchmoredifficultforechochambers.Recall:where
encounteringexcludedvoicesandevidencewillshatteranepistemicbubble,suchencounters
arelikelytoreinforceanechochamber.Let’sgrantthatintentionallyconstructinganecho
chamber,asJamiesonandCappellaclaimthatRushLimbaughdid,isepistemically(and
morally)blameworthy.Furthermore,activelyenteringanechochamberseemsepistemically
blameworthyinmanycircumstances.Foragentinfullpossessionofawiderangeof
informationalsources,toabandonmostofthemandplacetheirtrustinanechochamberfor,
say,anincreasedsenseofcomfortandsecurity,issurelysomeformofepistemicvice.There
issomeevidencethatthismaybethecase;JamiesonandCappellasuggestthatpeopleenter
echochambersforthesakeofthecommunitybondsandthesenseofbelongingtoanin-
group(JamiesonandCappella2008,180).
Buttherearemanycasesinwhichtheagentseemsplausiblyblameless.Imagineaperson
raisedinanechochamber.Theirearliestepistemiccontacts—let’ssaytheirparents,
26
relatives,andclosefamilyfriends—areallfirmlycommittedmembersoftheechochamber.
Supposethatthechildiseitherhome-schooledbythoseechochambermembersorsenttoa
schoolthatreinforcesthebeliefsofthatparticularechochamber.Itakeitthatitisreasonable
forachildtotrusttheirparentsandthoseofseemingepistemicauthority,andthatachildis
epistemicallyblamelessforhavingdoneso(Goldberg2013).Thus,whenthatchildeventually
comesintocontactwiththelargerepistemicworld—say,asateenager—theecho
chamber’sbeliefsarefullyinplace,suchthattheteenagerdiscreditsallsourcesoutsideof
theirechochamber.
Itseems,atfirstglance,thatourteenagercouldbeactingverymuchlikeareasonable
epistemicagent.Theycould,infact,beepistemicallyvoracious:seekingoutnewsources,
investigatingthem,andevaluatingthemusingtheirbackgroundbeliefs.Theyinvestigatethe
reliabilityofpurportedexpertsanddiscreditexpertswhentheyhaveapparentlygoodreason
todoso,usingtheirbackgroundbeliefs.Ourteenagerseems,infact,tobebehavingwith
manyepistemicvirtues.Theyarenotatalllazy;theyareproactiveinseekingoutnew
sources.Theyarenotblindlytrusting;theyinvestigateclaimsofepistemicauthorityand
decideforthemselves,usingalltheevidenceandbeliefsthattheypresentlyaccept,whether
toacceptordenythepurportedexpertiseofothers.Theyhavetheories,whichtheyhave
acquiredbyreasonablemethods,predictingthemaliciousnessofoutsiders;theyincrease
theirtrustinthosetheorieswhentheirpredictionsareconfirmed.10
Theworryhereisthatagentsraisedwithinanechochamberare,throughnofaultoftheir
own,epistemicallytrapped—theirearnestattemptsatgoodepistemicpracticesare
transformedintosomethingepistemicallyharmfulbythesocialstructureintowhichthey
havebeenembeddedandwhichtheyhaveingested.PaulSmarthasarguedforthepossibility
10 For a parallel argument about the invidiousness of background prejudicial beliefs, see (Begby 2013).
27
ofatransformativesocialepistemicphenomenonwhichhedubs“mandevillianintelligence,”
inhonorofBernardMandeville.Mandevillearguedthat,intherightsocialcontext,individual
vicescouldleadtocollectiveeconomicprosperity.Foracertainkindofeconomictheorist,
capitalismissuchatransformativestructure—individualsactselfishly,butthestructureof
themarkettransformsthatselfishnessintovirtuouscollectiveaction.AccordingtoSmart,
thereisanepistemicanalog:themandevillianintelligence,whichtransformstheindividual
epistemicvicesofitsmembersintoacollectiveepistemicvirtuebyvirtueofthesocial
structureintowhichtheyareembedded(Smart2017).Intellectualstubbornness,for
example,mightbeanintellectualviceforindividuals.Butsetthosestubbornindividualsina
properlyarrangedsocialstructure(like,perhaps,academia)andyoumightgetacollective
systemthatproperlyexploreseveryrelevantnookandcrannywithoptimalthoroughness.
Butechochambersaretheveryopposite;theyarereverse-mandevillianintelligences.Echo
chambersaresocialepistemicstructureswhichconvertindividuallyepistemicallyvirtuous
activityintocollectiveepistemicvice.Infact,thereverse-mandevilliannaturecontributesto
thestickinessoftheechochambertrap.Ifourteenagerself-reflectsontheirepistemic
practices,whattheywillseemightberathergratifying.Theirepistemicbehaviormightvery
wellbeearnest,vigorous,andengaged.Itistheirexternalcontext—thesocialepistemic
systemintowhichtheyhavebeenunluckilyraised—whichmakessuchbehavior
problematic.
ContrastthisaccountwithQuassimCassam’streatmentofOliver,hisfictional9/11
conspiracytheorist.Oliverbelievesthatthecollapseofthetwintowerswasaninsidejob,and
heishappytoprovidereasonsandpointtosupportingevidencefromagreatmany
conspiracytheoristwebsites.SaysCassam:Oliverisobviouslymistaken—Oliverrelieson
outrageous,baselessclaimsfromclearlydiscreditedsources.Thebestexplanationfor
28
Oliver’sbeliefsisintermsofepistemicvice—thatis,intermsofOliver’sbadintellectual
charactertraits.Oliveris“gullible,cynical,andprejudiced,”saysCassam.Oliverisgullible
withregardtohisconspiracytheoristsites,cynicalwithregardtothemainstreammedia,and
hisprejudiceconsistsof,amongotherthings,intellectualpride,wishfulthinking,closed-
mindedness,andalackofthoroughness(Cassam2016,162-4).AndIcertainlygrantthat
suchepistemicvicescanleadtothesesortsofbeliefs.Butthestoryofourhaplessteenager
offersanalternateepistemicpathtosuchbeliefsandsuchnarrowcastedtrust—onein
whichepistemicallyvirtuouscharactertraitshavebeenwrong-footedbythesocialepistemic
structureinwhichtheagenthasbeenembedded.Thecrucialdifferencebetweenthereverse-
mandevillianaccountandCassam’saccountiswherethebruntoftheresponsibilitylies.In
Cassam’saccount,theresponsibilitylieswiththeindividual,andtheirownintellectualhabits
andpractices.11Inareverse-mandevillianaccount,asignificantpartoftheresponsibilitylies
withthesocialstructureinwhichtheactorsareembedded.Theepistemicviceisafeatureof
thecollectiveintelligence,ratherthanoftheindividual.Or,ifoneisaversetothinkingin
termsofcollectiveintelligences,here’saconceptuallyminimalwaytoputtheclaim:echo
chambersarelocalbackgroundconditionsthatturngenerallygoodepistemicpracticesinto
locallyunreliableones.
Butthepossibilityofatrulyfaultlessepistemicagent,whollymisledbyanechochamber,
alsodependsonthelackofanaccessibleescaperoute.So:arethereescaperoutesfroman
11 Note, however, that Cassam distinguishes between epistemic responsibility and epistemic blameworthiness, and does not take blameworthiness to necessarily follow from responsibility (168-9). Cassam leaves room for the view that the individual’s intellectual vices were epistemically responsible for their bad beliefs, but that the individual wasn’t blameworthy for those vices, because the vices were inculcated in them at an early age. However, my complaint still stands, for I contest Cassam’s claim that the responsibility is in the individual.
29
echochamber,andhowreasonableisittoexpectechochambermemberstodiscoverand
makeuseofthem?
Hereisonepossibleescaperoute.ConsiderThomasKelly’sdiscussionofbelief
polarization.Beliefpolarizationisthetendencyofindividuals,oncetheybelievethatp,to
increasetheirbeliefthatp.Kellyarguesthatbeliefpolarizationworksbythemechanismthat,
onceanagenthasacquiredabelief,theytendtosubjectcounter-argumentstogreater
scrutinyandgivesupportingargumentsarelativepass.Thus,theircriticalreflectionislikely
toreinforcepreviouslyheldbeliefs.Kellynotesthatthebeliefpolarizationviolatesthe
CommutativityofEvidencePrinciple:
TheCommutativityofEvidencePrinciple:totheextentthatwhatitisreasonableforonetobelievedependsonone'stotalevidence,historicalfactsabouttheorderinwhichthatevidenceisacquiredmakenodifferencetowhatitisreasonableforonetobelieve.(Kelly2008,616)
Inshort,beliefpolarizationmakesitmatterwhatordertheyreceivedtheevidence,but
thehistoricalorderingofevidenceoughtnotmatter.Notethatourepistemicallyhapless
teenagerhasalsoviolatedtheCommutativityofEvidencePrinciple.Forthem,itmattersvery
muchwhatorderthattheyreceivedtheevidence.Iftheyhadbeenraisedoutsidetheecho
chamberandfedabroaderdietofepistemicsourcesbeforeencounteringtheechochamber,
thentheywouldlikelyhavefoundtheechochamber’sworld-viewtobeproblematic.But
sinceourteenagerencounteredtheechochamberandassimilateditsbeliefsfirst,theiruseof
backgroundbeliefstovetnewsourcesleadsthemtocontinuallyincreasetheirtrustinthe
echochamberandtheirdistrustofoutsiders.Evenifourechochamberedteenagereventually
cametoencounterallthesameevidenceastheirepistemicallyfree-rangecounterpart,their
earlyeducationwithintheechochamberwouldbedecisive.Solongaseachnewpieceof
30
evidenceisassessedusingthecurrentlyheldsetofbeliefs,thenearlyeducationinanecho
chamberbecomesdomineeringlypowerful.
However,theCommutativityofEvidencePrinciplesuggestsawayout.Inordertofree
themselvesoftheechochamber’sgrip,ourteenagerneedstoundotheinfluenceofthe
historicalorderingoftheirencounterswiththeevidence.Howcouldtheypossiblydothis?
Ourteenagerwouldhavetosuspendbeliefinalltheirparticularbackgroundknowledgeand
restarttheknowledge-gatheringprocess,treatingalltestimonyasequallyviable.Theywould
needto,inasense,throwawayalltheirbeliefsandstartoveragain.Thissuggestsaprocess
that,initsoutlines,mightsoundawfullyfamiliar.Ourescaperouteturnsouttobea
somethinglikeamodifiedversionofDescartes’infamousmethod.
Whatproceedsfromthispointisadmittedlysomethingofafantasy,butperhapsitisa
fantasyfromwhichwecaneventuallydrawsomesortofmoral.Thestoryofthehistoryof
Westernepistemologymightbecartoonishlysummarizedthusly:Descarteshadadreamof
radicalintellectualautonomy.Byhisaccounting,hecametorealizethatmanyofthebeliefshe
hadacquiredinhisearlylifewerefalse,andthatthoseearlyfalsebeliefsmighthaveinfected
anynumberofotherbeliefs.Hisresponsewasthatfamedmethod:togetridofhisbeliefsand
startoveragain,trustingno-oneandnothingandonlypermittingthosebeliefsofwhichhe
wasentirelycertain.CallthistheCartesianepistemicreboot.Butifrecentepistemologyhas
taughtusanything,it’sthatthistotalrebootisnothingbutapipedream.Anysortof
reasonableepistemiclifeisessentiallyimpossiblewithouttrustingthetestimonyofothers
(Burge1993;Faulkner2000;Goldberg2010;Zagzebski2012;Hardwig1985,1991).
ButrecallthatthereasonDescarteswantedtodiscardeverythingandstartoverfrom
scratch—themotivationforhisproject,andnotthemethod—wasexplainedinthevery
firstlineof“Meditation1”:Hewasworriedbythefalsehoodshehadlearnedinchildhoodand
31
theshakinessoftheedificethathadbeenbuiltfromthosefalsehoods(Descartes1984,24).
Ourteenagerfacesaproblemquitesimilarinstructure.Thecredentialingstructureoftheir
upbringingisflawed;thatcredentialingstructurehasinfluencedanynumberoftheirother
beliefs,andthedegreeofthatinfluenceisimpossibletotrack.Furthermore,theselater
beliefs,approvedbytheechochambers’credentialedsources,willoftenservetoreinforce
thatcredentialingstructure.Theperniciouseffectofanechochambercannotbeattackedone
beliefatatime.Anysinglebeliefthatourteenagerre-consideredwouldcomeunderthe
influenceofthenetworkoftheflawedbackgroundbeliefsthatsustainsanechochamber.
Whattheyneedissomewaytostartover.Inordertoundotheinfluenceofhistorical
ordering,anepistemicagentwillhavetotemporarilysuspendbeliefinalltheirbeliefs,in
particulartheircredentialingbeliefs,andstartfromscratch.Butwhentheystartfromscratch,
theyneednotdisregardthetestimonyofothers,norneedtheyholdtoDescartes’stringent
demandforcertainty.Let’scallthisprocedurethesocialepistemicreboot.Inthesocial
epistemicreboot,theagentispermitted,duringthebeliefre-acquisitionprocess,totrustthat
thingsareastheyseemandtotrustinthetestimonyofothers.Buttheymustbeginafresh
socially,byre-consideringalltestimonialsourceswithpresumptiveequanimity,without
deployingtheirpreviouscredentialingbeliefs.Furthermore,theymustdiscardalltheirother
backgroundbeliefs,becausethosepotentiallyarosefromtheflawedcredentialstructureof
theechochamber,andverylikelyhavebeendesignedtosupportandreinforcethatvery
credentialstructure.Ourrebootermusttakeonthesocialepistemicposturethatwemight
expectofacognitivenewborn:oneoftentative,butdefeasible,trustinallapparent
testimonialsources(Burge1993)(Nguyen2011).Thismethodwill,ifsuccessfullyapplied,
undothehistoricaldependenceofourepistemicagentandremovetheundueinfluenceofthe
32
echochamber.Thesocialepistemicrebootis,theoreticallyatleast,theescaperoutewe’ve
beensearchingfor.12
Thisreboot,describedinsuchclinicalterms,mightseemratherfantastical.Butitisnot,I
think,utterlyunrealistic.Considerthestoriesofactualescapeesfromechochambers.Take,
forexample,thestoryofDerekBlack,whowasraisedbyaneo-Nazifather,groomedfrom
childhoodtobeaneo-Nazileader,andwhobecameateenagedbreakoutstarofwhite
nationalisttalkradio.WhenBlackleftthemovement,hewentthroughyears-longprocessof
self-transformation.Hehadtocompletelyabandonhisbeliefsystem,andhespentyearsre-
buildingaworld-viewofhisown,immersinghimselfbroadlyandopen-mindedlyin
everythinghe’dmissed—popculture,Arabicliterature,thepronouncementsofthe
mainstreammediaandtheUSgovernment,rap—allwithanoverallattitudeoftrust(Saslow
2016).
Ofcourse,allwehaveshownsofaristhatthesocialepistemicrebootwould,ifpulledoff,
undotheeffectsofanechochamberedupbringing.Whetherornotanepistemicagentmight
reasonablybeexpectedtoreboot,orblameworthyforfailingtoreboot,isaseparateand
significantlymoredifficultquestion.First,asocialepistemicrebootmightbepsychologically
impossible,oratleastbeyondwhatwecouldreasonablyexpectofnormalepistemicagents.
Second,whatreasonwouldanepistemicagenthavetoundertakeasocialepistemicreboot?
Suchanundertakingwouldbejustifiedonlyiftheagenthadasignificantreasontothinkthat
theirbeliefsystemwassystematicallyflawed.Butechochambermembersdon’tseemlikely
tohaveaccesstoanysuchapparentreason.Afterall,theyhaveclearandcoherent
explanationsforalltheevidenceandtestimonytheyencounter.Ifthisisallrightthenwe
12 Note that the social epistemic reboot would also undo the effects of Begby’s evidential preemption, since that preemption also depends on the historical ordering of received data.
33
arriveataworryingconclusion:thatechochambersmay,theoretically,beescapable,butwe
havelittlereasontoexpectmembersofechochambersrealizethattheyaremembersof
somethingthatneedsescaping.
Whatcouldhopedowehave,then,ofmotivatingareboot?DerekBlack’sownstorygives
usahint.Blackwenttocollegeandwasshunnedbyalmosteveryoneinhiscollege
community.ButthenMatthewStevenson,aJewishfellowundergraduate,begantoinvite
BlacktohisShabbatdinners.Stevensonwasunfailinglykind,open,andgenerous,andhe
slowlyearnedBlack’strust.ThiseventuallyleadtoamassiveupheavalforBlack—aslow
dawningrealizationofthedepthstowhichhehadbeensystematicallymisled.Blackwent
throughaprofoundtransformation,andisnowananti-Nazispokesperson.
TheturningpointseemstobepreciselythatStevenson,anoutsider,gainedBlack’strust.
Andthisisexactlywhereweshouldexpecttheturningpointtobe.Sinceechochamberswork
bybuildingdistrusttowardsoutsidemembers,thentheroutetounmakingthemshould
involvecultivatingtrustbetweenechochambermembersandoutsiders.Inordertomotivate
thesocialepistemicreboot,anechochambermemberneedstobecomeawareofhowinthe
echochamber’sgriptheyare,andformingatrustrelationshipwithanoutsidermightcould
mediatethatawareness.Buthowthattrustcouldbereliablycultivatedisaverydifficult
matter,andatopicforfutureinvestigation.Wehave,however,arrivedatatentativemoralof
thestory.Echochambersworkbyamanipulationoftrust.Thus,theroutetoundoingtheir
influenceisnotthroughdirectexposuretosupposedlyneutralfactsandinformation;those
sourceshavebeenpreemptivelyundermined.Itistoaddressthestructuresofdiscredit--to
34
worktorepairthebrokentrustbetweenechochambermembersandtheoutsidesocial
world.13
Bibliography
Alcoff,LindaMartín.2007.Epistemologiesofignorance:Threetypes.InRaceandEpistemologiesofIgnorance,Ed.ShannonSullivanandNancyTuana.Albany:SUNYPress.
An,Jisun,DanieleQuercia,andJonCrowcroft.2014.Partisansharing:Facebookevidenceandsocietalconsequences.InProceedingsoftheSecondACMConferenceonOnlineSocialNetworks.http://cosn.acm.org/2014/files/cosn033f-anAembTS.pdf.
Baumgaertner,Bert.2014.Yes,no,maybeso:Averitisticapproachtoechochambersusingatrichotomousbeliefmodel.Synthese191(11):2549-2569.
Begby,Endre.2013.Theepistemologyofprejudice.Thought:AJournalofPhilosophy2(2):90-99.
———.2017.Evidentialpreemption.AmericanPhilosophicalAssociationPacificDivision2017Meeting.
Bishop,Bill.2009.TheBigSort:WhytheClusteringofLike-mindedAmericaIsTearingUsApart.Boston:HoughtonMifflinHarcourt.
Burge,Tyler.1993.Contentpreservation.ThePhilosophicalReview102(4):457-488.Cassam,Quassim.2016.Viceepistemology.TheMonist99(2):159.Coady,David.2006.Whenexpertsdisagree.Episteme:AJournalofSocialEpistemology3
(1-2):68-79.———.2012.WhattoBelieveNow.WestSussex:Wiley-Blackwell.Coates,DominiekD.2010.Post-involvementdifficultiesexperiencedbyformermembers
ofcharismaticgroups.JReligHealth49(3):296-310.Corner,Adam,LorraineWhitmarsh,andDimitriosXenias.2012.Uncertainty,scepticism
andattitudestowardsclimatechange:Biasedassimilationandattitudepolarisation.ClimaticChange114(3-4):463-478.
Dentith,MatthewR.X.2017.Theproblemofconsparicism.Argumenta5:1-16.Dentith,MatthewRX.2015.Wheninferringtoaconspiracytheorymightbethebest
explanation.SocialEpistemology16(5-6):572-591.Descartes,René.1984.ThePhilosophicalWritingsofDescartes,Vol.2,TranslatedbyJohn
Cottingham,RobertStoothoff,andDugaldMurdoch.Cambridge:CambridgeUniversityPress.El-Bermawy,Mostafa.Yourfilterbubbleisdestroyingdemocracy.
https://www.wired.com/2016/11/filter-bubble-destroying-democracy/(accessedFebruary2,2017).
Faulkner,Paul.2000.Thesocialcharacteroftestimonialknowledge.TheJournalofPhilosophy97(11):581-601.
Flaxman,Seth,SharadGoel,andJustinM.Rao.2016.Filterbubbles,echochambers,andonlinenewsconsumption.PublicOpinionQuarterly80:298-320.
13 I’d like to thank Kara Barnette, Endre Begby, Anthony Cross, Melissa Hughs, Eric Stencil, Matt Strohl, Shannon Mussett, Bekka Williams, the anonymous reviewers, and many others for their assistance with this paper.
35
Fricker,Miranda.2011.Rationalauthorityandsocialpower:Towardsatrulysocialepistemology.InSocialEpistemology-EssentialReadings.Ed.AlvinGoldman.NewYork:OxfordUniversityPress.
Gilbert,Eric,TonyBergstrom,andKarrieKarahalios.2009.Blogsareechochambers:Blogsareechochambers.InSystemSciences,2009.HICSS'09.42ndHawaiiInternationalConferenceon.
Goldberg,Sanford.2010.RelyingonOthers:AnEssayinEpistemology.OxfordUniversityPress.
———.2011.IfthatweretrueIwouldhaveheardaboutitbynow.InTheOxfordHandbookofSocialEpistemology.Oxford:OxfordUniversityPress.
———.2013.Epistemicdependenceintestimonialbelief,intheclassroomandbeyond.JournalofPhilosophyofEducation47(2):168-186.
Goldman,AlvinI.2001.Experts:Whichonesshouldyoutrust?PhilosophyandPhenomenologicalResearch63(1):85-110.http://www.jstor.org/stable/3071090.
Hardwig,John.1985.Epistemicdependence.TheJournalofPhilosophy82(7):335-349.———.1991.Theroleoftrustinknowledge.TheJournalofPhilosophy88(12):693-708.Jamieson,KathleenHallandJosephNCappella.2008.EchoChamber:RushLimbaughand
theConservativeMediaEstablishment.Oxford:OxfordUniversityPress.Kahan,DanMandDonaldBraman.2006.Culturalcognitionandpublicpolicy.YaleLaw&
PolicyReview24(1):149-172.Kelly,Thomas.2008.Disagreement,dogmatism,andbeliefpolarization.TheJournalof
Philosophy105(10):611-633.Kitcher,Philip.1993.TheAdvancementofScience.NewYork:OxfordUniversityPress.Lackey,Jennifer.2013.Disagreementandbeliefdependence:Whynumbersmatter.In
TheEpistemologyofDisagreement:NewEssays.Ed.JenniferLackeyandDavidChristensen.Oxford:OxfordUniversityPress.
Langone,MD.ReflectionsonPost-CultRecovery.http://www.icsahome.com/articles/reflections-on-post-cult-recovery-langone(accessedMarch15,2017).
Lifton,Robert.1991.Cultformation.CulticStudiesJournal8(1):1-6.Mengus,Bryan.2016.PizzagatersAren'tGivingThisShitUp.Gizmodo,Millgram,Elijah.2015.TheGreatEndarkenment:PhilosophyforAnAgeof
Hyperspecialization.Oxford:OxfordUniversityPress.Miller,BoazandIsaacRecord.2013.Justifiedbeliefinadigitalage:Ontheepistemic
implicationsofsecretinternettechnologies.Episteme10(02):117-134.Mills,Charles.2007.Whiteignorance.InRaceandEpistemologiesofIgnorance.Albany:
StateUniversityofNewYorkPress.Nelson,JacobL.andJamesG.Webster.2017.Themythofpartisanselectiveexposure:A
portraitoftheonlinepoliticalnewsaudience.SocialMedia+SocietyJuly-September:1-13.Nguyen,C.Thi.2010.Autonomy,understanding,andmoraldisagreement.Philosophical
Topics.38(2):111-129.--.2011.Anethicsofuncertainty:moraldisagreementandmoralhumility(PhD
dissertation).UniversityofCaliforniaatLosAngeles.RetrievedfromProQuestDissertationsandThesesdatabase.(UMINo.3532448)
--.2018a.Cognitiveislandsandrunawayechochambers:Problemsforexpertdependence.Synthese.https://doi.org/10.1007/s11229-018-1692-0
--.2018b.Escapetheechochamber.AeonMagazine.https://aeon.co/essays/why-its-as-hard-to-escape-an-echo-chamber-as-it-is-to-flee-a-cult
36
--.Forthcoming.Hyper-specializationandthefragmentationofintellectualautonomy.PhilosophicalInquiries.
Orwell,George.1968.Politicsandtheenglishlanguage.InTheCollectedEssays,Journalism,andLettersofGeorgeOrwell.1ed.Ed.SoniaOrwellandIanAngos.NewYork:Harcourt,Brace,Javanovich.
Pariser,Eli.2011.TheFilterBubble:WhattheInternetIsHidingFromYou.PenguinUK.Pollak,Joel.2017.#FakeNews:MainstreamMediaContinuetoSlanderBreitbart.
Breitbart,January30.http://www.breitbart.com/big-journalism/2017/01/30/fakenews-mainstream-media-continue-slander-breitbart/.
Robbins,ThomasandDickAnthony.1982.Deprogramming,brainwashingandthemedicalizationofdeviantreligiousgroups.SocialProblems29(3):283-297.
Robson,Jon.2014.Asocialepistemologyofaesthetics:Beliefpolarization,echochambersandaestheticjudgement.Synthese191(11):2513-2528.
Saez-Trumper,Diego,CarlosCastillo,andMouniaLalmas.2013.Socialmedianewscommunities:Gatekeeping,coverage,andstatementbias.InProceedingsofthe22ndACMInternationalConferenceonConferenceonInformation&KnowledgeManagement.https://dl.acm.org/citation.cfm?id=2505623
Saslow,Eli.2016.ThewhiteflightofDerekBlack.TheWashingtonPost,October15.https://www.washingtonpost.com/national/the-white-flight-of-derek-black/2016/10/15/ed5f906a-8f3b-11e6-a6a3-d50061aa9fae_story.html?utm_term=.67f00ca25ce9.
Singer,MargaretT.1979.Comingoutofthecults.PsychologyToday12(8):72-82.Smart,PaulR.2017.Mandevillianintelligence.Synthese.https://doi.org/10.1007/s11229-017-1414-z
Sunstein,CassR.2001.Republic.Com.InRepublic.com.Princeton:PrincteonUniversityPress.
———.2009a.GoingtoExtremes:HowLikeMindsUniteandDivide.OxfordUniversityPress.
———.2009b.Republic.com2.0.PrincetonUniversityPress.Trouche,Emmanuel,PetterJohansson,LarsHall,andHugoMercier.2016.Theselective
lazinessofreasoning.CognSci40(8):2122-2136.Vogt,PJandAlexGoldman.2016.VoyageIntoPizzagate.
https://gimletmedia.com/episode/83-voyage-into-pizzagate/(accessedFebruary20,2017).Watson,JamieCarlin.2015.Filterbubblesandthepublicuseofreason:Applying
epistemologytothenewsfeed.InSocialEpistemologyandTechnology:TowardPublicSelf-AwarenessRegardingTechnologicalMediation.Ed.FrankScalambrino.London:Rowman&Littlefield.
Weathers,Cliff.2014.CrossFitIsaCult:WhySoManyofItsDefendersAreSoDefensive.Salon.https://www.salon.com/2014/10/22/crossfit_is_a_cult_why_so_many_of_its_defenders_are_so_defensive_partner/
Wittgenstein,Ludwig.2010.PhilosophicalInvestigations.Hoboken:JohnWiley&Sons.Zagzebski,Linda.2012.EpistemicAuthority.NewYork:OxfordUniversityPress.