AAAI/CCCSymposiumonAIforSocialGood

TalkSessions1:HealthcareSessionChair:Dr.EricHorvitz

FeiFang: It'sagreathonortohaveEricHorvitzfromMicrosoftResearchtochairthesession.ErichasgothisPhDfromStanford,sowelcomebacktothecampus.He'scurrentlythetechnicalfellowofMicrosoftandisalsothemanagingdirectorofMicrosoftResearchDevelopment.I'llgiveittoEricfortheopeningtalk.

EricHorvitz: Beinginvolvedinresearchdoesn'tmeanthatyoucangetPowerPointtorun.Therewego,great.IthoughtIwouldjustshareafewthoughtsabouthealthcaretoday.It'sapassionofmine-

Speaker3: Couldyouuseamicbecause[inaudible00:00:51]-

EricHorvitz: You'rerecordinghere.

Speaker3: Yeah.

EricHorvitz: Okay.

Speaker3: [inaudible00:00:54]

EricHorvitz: Hello,hello.

Speaker3: It'snowonthespeakers.

EricHorvitz: Good,that'sgood,soIcanjustlooklikearockstarandnotreallybeusingthemiclocallyhere.Whenyou'reinhealthcare,youseelotsoflowhangingfruitandpossibilityforapplyingstatistics,thedatathat'sbecomingavailable,utilitytheory,decisionmaking,models,toreallydeliverincrediblevalue.ThewayIlikethinkingabouthealthcareiswehavelotsofsensitivedatabecomingavailable,wetendtobuildpredictivemodels,andthosepredictivemodelsforspecificpatients,distributionsoveroutcomes,orhiddendiagnosesthatwecan'tseewithoutinvasivetests.Mostmedicaldecisionmakingtendstostophereinmedicalwork,likelet'sseewhatwecandowiththedistributions,buttherubbermeetstheroadwithdecisionmodels,typically.

Really,howdoyouconsidercostsandbenefits.Likeinotherfields,wehaveadatatopredictiontodecisionpipelineandonceyouhavethat,youcanofcoursethinktheotherwayandwhatkindofdatamightwecollecttomakeourdatasetsmorecompleteinthecontextofdeliveringvaluetopatients.Ofcourseinhospitalsandinthefield,andinpublichealth,weoftenhavetoreallythinkabout,it'sinsightbuilding,notjustautonomousactionsorrecommendations.Andunderstandtheroleofasystemwhenitcomestopeopleanddecision

Talk Session 1 - AI in Healthcare Page 2 of 30

support.Whichalsobringstotheforeallsortsofinterestingculturalissuesandergonomicsofthefield.

Sowhatareopportunities?Well,there'sincredibledatalockedupinelectronichealthrecords.ItendtolookatEHRdataasgoldmineswheretherearelotsofdiamondsbeneaththesurfaceandyouhavetojustgetaccesstoit,andthat'snotoftenaneasythingwhenitcomestoworkingwithhospitals,aspeopleknow.There'sincredibleamountsofonlinebehavioraldata,Henry[Couch00:03:14]haslookedatsomeofthisthroughTwitterfeedsandhisstudents.Ourteamhaslookedatquiteabitofthisdatawhenitcomestoouraccesstoanonymizedlogsofsearchactivity.Youcanimaginenewcombinationsofthesethings.

Theclassworkthathasbeendoneinthisfieldandwedidthisworkourselvesasoneofthecontributorsinthisrealm.Asagreatexample,isreadmissions.A2004datasetpublishedin2009,NewEnglandJournalarticle,showedthat20%patients,Medicarereimbursedpatients,werebouncingbackwithin30days,35%within90days,anestimatedcostof$17.5billionofavoidablecosts,itwasthoughtinthattime.

Wetookalargedatabaseof20yearsofdata,30,000[binarized00:04:02]variables.Thisisworkingwith[inaudible00:04:06]andMark[Braverman00:04:06],andothers.Andcreatedstandardsetof...Apredictivemodelthathadcharacteristicscharacterizedbythisreceiveroperatorcharacteristiccurvebasedonatrainandtest.

Ofcoursethesedatasetsalso,whenyoudotheseanalysis,giveyouasenseforsomeinterestingobservationsthatarediscriminatory.Someareobviously,otherslikeacomplaintsentenceanywhereinthetextthatsaysfluidisabadsignforstayingoutofthehospital.Veryinterestingtothephysicians.

WebuiltsomethingcalledReadmissionsManager,whichshippedworldwideasaproductinthosedays.Whatdoctorswouldseeisprobabilitiesofreadmissionatdischargetimeandlittleexplanationscomingoffthelogisticregression,inthiscase,tomakeitunderstandable.

WhatI'lltalkabouthereisthefactthatwediscoveredthathospitalssystems,likeMidwestunnamedhospitalsystemintheMidwest,weremakingdecisionsbasedonwhattheycalledtheMicrosoftscore.Soifthescorewasover...Itwasaprobability,buttheycalledittheMicrosoftscore.Itwasover25,theysaid,"Wedoaspecialprogram,wehaveaspecialpolicyonkeepingpatientsoutofthehospital,wegivethemextrasupport,weschedulethemforoutpatientvisits."Theyhadaprogramanditsaiddoingthisiscostly,butitturnsouttosavethem$1.5millionayearandalsoit'sverygoodpatients.

Wethoughtabout,waitaminute?Howdoyoureallydothis?Youdon'tjustpickanumberoutofahatandthentrytouseandlookattheoutcome.You

Talk Session 1 - AI in Healthcare Page 3 of 30

wanttothinkaboutthedecisionproblemhere.Sowelookedatcongestiveheartfailure,I'llmakeaquickframingcasestudyheretotalkaboutbecauseIthinkitframesotherwork[inaudible00:05:56]mightdo.It'sa$35billionayearmanagementproblem.Thesepatientsareoftenrevolvingdoorpatients,they'reonmedications,specialdiets.Iftheytaketoomuchsaltintheirmeals,theytendtogetfluidoverloadedandcomeintotheemergencyroom,what'scalledatune-up,it'saboutsevento10daysanditcostsabout$18,000orso,andit'sdangerous.Soweaskedthequestion,canweactuallypredict,notjustpredict,whichpatientswillcomeback,butalso,ifwecouldinterveneandlowertheprobabilityofthemcomingback,what'stheidealdecisionthreshold?

Sowebuildamodel.Actually,inthedatasetwehadfromtheWashingtonHospitalCenterinWashingtonDC,weactuallyhadthecostofeachreadmission,sowecanactuallydoalittledecisionanalysis.Verysimpledecisionanalysishere.Atthex-axisyouseetheprobabilityofreadmissiongivenevidence,zerotoone,andhere'sthefullpriceofhandlingapatientwhohadbeenreadmitted.Ifit'sa.5probability,it'stheexpectedvalueofhalfthatamount,andthat'swhatthatlittledecisioncurvelookshere,oraline.

WhatifIactuallyhadamagicaltreatmentthatcouldreducetheprobabilitythatthatpatientwouldcomebackandhavesomesortofacost?Well,lookattheefficacyhere,we'llcallitbeta,it'sgoingtoreducetheprobabilityofbeingreadmitted,givenevidence.Whathappensis,youputthatinthedecisionmodelhere,yousayyoustartatzero,you'repayingapriceforthatspecialtreatment,andyouhaveacurvethat'sgoingupalittlebitmoreslowlywithaggressivefollowupthanthestandarddischargecurve.Thesetwolinescossatadecisionthreshold.Theideais,ifyouhaveapredictivemodellikewehadbuilt,youwanttoreasonsuchthatyouchooseyourtreatmentbasedonwhetheryou'reaboveorbelowthatdecisionthreshold.That'sthewaythatworks.Ifyoudothat,decisiontheorytellsusyouwillbeideallyallocatingresourcestominimizingreadmissions.

Wehaveabasic,niceideahereIthinkthatwearequitefondofis,wecanactuallynowinsteadofjustdoingastandard[ROSC00:08:19]curvewithatestandtrain,wecannowsay,let'srunatestandtrainonthedecisionmodel,thesimulator,toseewhatwouldhappenwithholdoutdata,forexample.

Youcanactuallydostudies.Youcansay,evenifwedon'tknowinadvancethecostofsomenewprogramandhowmuchitreducestheprobabilityofadmission,wedon'tknowthat,wecanactuallyrunasimulator.Foranycombinationofdollarsandreturn,reductioninreadmissioncost,wecancomputewhatwouldhappen.

Here'sanexample.WiththathospitalinDC,youseehereatthebottomhere,theproposedcostofanintervention,there'sspecialkindsofprograms,there'snursevisits,there'sspecialkindsofdevicesyoucanwear,smartscalestocapturefluidoverloadandsoon.There'ssomeassumedefficacy,howmuchit'll

Talk Session 1 - AI in Healthcare Page 4 of 30

reduceprobabilityofreadmissionandwecanrunasimulatorandittellsus,ifyouhavean$800interventionandit's35%efficacy,youwillreducereadmissionsby31%andyou'llbereducingcostby13%.Whyisthatthecase?Becausenotallthetimeitworkedoutnicely,sometimesyou'dpaymoneyandtheywouldn'thavecomebackanyway,sotheydon'talignperfectly.

Whatifsomeonetellsyoutheyhavean$1,800interventionand20%efficacy,well,youlosemoneythatway.ThenI'lljustendwiththis,becausewe'reat10minutesnow,endwiththischartherewhichIthinkisreallyimportantforustothinkabout.Thisactuallycameinatourlunchconversationaboutsimplerulesversusmorecomplicateddecisionanalysis.WhatyouseehereiswhatIwouldcalllikeacrystallographyofintelligence.What'sthevalueoftakinganexistinghospitalsystemandwithsomesortofaninterventionprogram,withanefficacyandacostthatexists,andsaying,whatcoulddoIbeyondafixedpolicy?

Here,everything'sexpensiveandit'snotveryefficacious,youwouldneverusethisprogram,youwouldneverintervene.Overhere,everybodyyoucancomputeinadvance,wouldgetthistreatment.It'sinexpensiveandefficacious.Butwhatdecisiontheoryandmachinelearninggivesusis,itopensupthisregionofreflectionanddeliberation,patientbypatient,personalizingthetreatment,anditjustexpands,opensthisareahere,andthat'swherethevalueisdelivered.Soyouhavetoaskthequestion,wheredowewanttouseourAIsystems,ourdecisiontheoryoffline,tojustdesignidealpolicies,boom,wehaveit.Proof.Proof.Wheredowewantalivesysteminthehospitalthat'sbuzzing,andaware,andworking,patientbypatient.Now,wehavetheabilitytobuildsimulatorstounderstandallthat.SoI'llstopthere,thanks.

Sonow,Igettochairthesessionandourfirstspeakertoday...Oh,shouldItakeaquickcouplequestionsorjustkeepongoing?Noquestions?AnyquestionsatallaboutanythingIsaid?Yeah,please?

Speaker4: Whataresomeofthedynamicapproaches[inaudible00:11:46].Youjustmentionedthat[inaudible00:11:49]staticcase,insteadofdoingitoffline.

EricHorvitz: Oh,Isee.

Speaker4: [inaudible00:11:57]

EricHorvitz: Tobehonest,togettotheofflineassessment,youneedtosortofcomeatthefromthe[fullbore00:12:04]modelapproach,soyoucansortofmaketheseproofsandunderstandwhatitmeansbesidesgoingwithabackoftheenvelopeapproach.Almosteverythingwe'vedoneinhealthcare,andtheothersignificantareaI'veworkedinwith[JennaWeans00:12:22]aformerintern,withlargescaleelectrotonichealthrecordsispredictinginadvanceofanadverseoutcome,whichpatientsaregoingtobecomeinfectedinthehospital.Bylookingatmanyvariablesincludingsomethatareproxiesforexposureandsomethatareproxiesforsusceptibilityofpatients.

Talk Session 1 - AI in Healthcare Page 5 of 30

Inthatwork,wehaveactuallyanicepaper,Ithinkit'snice,ongettingittowardsyourquestion,whichis,howmuchbetterdoyoudowithavery,veryheavydutytopofthelinemachinelearning,opaquestylemachinelearningalgorithm,lookingat1,000variables,versus10clinicallyknownvariablesthatareknownintheliteratureforbeingassociatedwithriskofhospitalacquiredinfection?Inthiscase,C.[Deficile00:13:13],anastydiarrhealinfection.Justusingthatonvariousamountsofdataandlookingattheboostsyougetfrombeingdeeperandsmarter,andusingvariablesthatnooneknewhadaninfluenceonthisoutcome.I'llstopthere,butwecantakethisofflineaboutthesimpleversusnotsimple.

It'sinteresting,thepaperonreadmissionsthatwepublished,it'sonline,comparedtheresultsandpowerofthismethodtobackofthe...Icallthembackoftheenvelope,butthey'restandardscoresusedintheclinicsrightnow.There'svariousApachescoresfortraumacare,therearescoresbeingused,theTorontoscoreforreadmissions,andoftenthey'redesignedtobebackoftheenvelope,thekindsoffunctionsthat...Alistofwellknownobservationsthatadoctorcouldmakeandtheirweightedscores,basically.They'retypically,frommyunderstanding,isthey'renotdesignedwithmachinelearningalgorithms,they'reareconjuredbyexpertsperintuitionandthentheytestthesescoreswithdata.Say,"Thisishowwellthescoreworks."

Soweactually,inourpaper,lookedatthescoreversusourapproaches,andofcoursetoevenvalidatethesemethods,youhavetotakethescoreanconvertittoaprobabilityofsomekind,whichisinterestingworkinitself,andthenshowhowwellitworksversusyourprobabilisticmethods.[Melind00:14:44],yeah.

Speaker3: Thisconeofintelligence,IguessI'mwonderingifthatconeoverlapswiththeconeofinterpretabilityorconeofunderstanding.Thatconemightbealittlebitsharperinthesensethatitcouldbetherearesomedecisionsthatareeasytoexplainandthoseareoutsidethatcone,buttherearesomedecisionsthatareintheconethataregoingtobeharderexplain.

EricHorvitz: Well,it'sagreatquestion,andbeforeIgettothatanswer,Iwouldfirstsay...WejusthadameetingwiththeBerkeleyLawInstituteonFridayaboutexplainabilityandnewEuropeanlaws,it'scalledGDPRcomingtothefore,whichisrequiringsystemsthatdoanykindofautomateddecisionmakingorrecommendationstoinformthepeopleinfluencedbythedecisionsaboutthelogic...Meaningfullyinformthemaboutthelogicoftheinference.

Soallthisdiscussion,"WellhowdoyouexplainthistothousandsofEuropeanswhenyoumakeadecision?"It'sthelawnow.Sothere'slotsofdiscussionabout,doyoubackoffanduseasimplelinearmodel,doyouusenewkindsoflinearmodelsthathavewhatarecalledthesegeneralizedadditivefunctionsthathaveshapingfunctionsinthem?Recentworkhasshownyoucangetthefullpowerofthatmachinelearning,almostthefullpower,withexplainable

Talk Session 1 - AI in Healthcare Page 6 of 30

additivemodelsthathavemostlysingletermswithsomepairwiseandsometripletinteractionsthatcapturesomeoftheerror.

Thebiggerquestionwouldbe,foranymachinelearnedmodel,canwegofromthelesserperforminglogisticregressionstylemodels,theclassicsimplifiedmodelsthatwethinkareexplainablebecausetheytendtoletyouseewhat'smoving...Whatsinglepiecesofevidencewouldbeholdingeverythingelsethesame.Sopeopleconsiderthatagoodversionofexplainability.Asyoutakethemforwardtomorecomplex,richmodelsthatstillhavethatpower,yetstillretainsomeoftheexplainabilityproperties.

I'mnotsureifIcanmapthatanswer,whichIknowprettywellwiththeworkrightnow,ontothecone.Certainly,youcanexplainthedecisiontheoreticcostbenefitargumentsinthecornersbyjustsayingit'snotworthitandshowthecost.Iwilltakeyourquestionasaninterestingconjectureforstudy.

It'sanhonortohaveLanboSheheretalkingaboutteachingandcheckingofconstraintsforsurgicaltraylayout.

LanboShe: Yes.

EricHorvitz: Lanbo,you'reatMichiganStateandthisiswithJonathan[Connell00:17:48]whoisatIBMTJWatsonResearch.Thisoneseemshookedtoamicrophone,toaspeaker,no?I'mjusthearingechoes.I'llsitoverhere[inaudible00:18:00].

LanboShe: Goodafternooneveryone,myname'sLanboandourworkisaboutteachingandcheckingofconstraintsforsurgicaltraylayout.Ouroriginalgoalwastobuildacomputeragentthatcanononesidelearnfromhumanexpertsthroughlanguageinteraction,andontheotherhand,itcanacquireknowledgetoteachorsupervisethosenon-experthumanstoaccomplishatask.Itookthisideawithmymentor,JonathanConnell,andithappenedtoberelatedwithoneofhisprojects,whichgoalwastobuildacognitiveoperatingroom.

Basically,foroperations,oneveryimportantstepistohaveagoodpreparationoftheseinstruments.Basically,beforeanyoperations,awelltrainedsurgicalassistantneedstogetoutthedifferentkindsofinstrumentsthatwillbeusedduringtheoperationandplacedonthetrayaccordingtocertainorders,orregulations.Here'sacomparisonbetweenawellorganizedtrayandadisorganizedtray.Basically,aswecanseewiththiswellorganizedtray,thedoctorscanquicklyfindtheinstrumenttheywantandquicklycatchthem.Thiswillsavealotoftimeforthesurgery,whichwillbeveryimportant.

Then,whydoweneedanautomatedagenttohelpwiththepreparationandsupervision?Firstly,evenforawelltrainedsurgeryassistant,itmayrequirealotofmemorizationaboutdifferenttypesofinstrumentsthatwillbeusedfordifferentoperations.Sometimesevenhumancanmakemistakes.Thistime,theautomatedagentcancheckthe[inaudible00:20:31]environmentthatalready

Talk Session 1 - AI in Healthcare Page 7 of 30

setupbytheassistantandthengivesuggestionsifidentifiesanymistakesorviolations.Secondly,insomesituationsthewelltrainedassistantscanbeshorthanded.Forexample,inemergencysituationsoreventhebattlefield.Inthiscase,thedoctormayjustneedsomeone,maybenotwelltrainedoranexpert,tohelpwiththepreparationprocess,buttheymaynotknowhowtodoitorcannotdoitproperly.Inthatsituation,suchanautomatedagentcangivesuggestionsortellthemwhataretheinstrumentsthatareneededandhowtoplacethemproperly,andmaybeevengivestepbystepinstructions.Last,theseautomatedagentscanalsobeusedinschoolstohelptrainthestudents.

Here'soneexampleofourintegratedsystemthatcheckedtheenvironment.Specifically,wehaveacameramountedoverheadtolookatthistrayareaandthenanalyzethesetup,andcomparehisknowledgetoseewhetherthere'sanyviolationsandgivefeedbacktothehuman.Thisisoneexamplehere.Inthiscase,theagentidentifiesmultipleviolations.Firstly,ittellsthehumanthatthisscalpelatthetopleftistooclosetothescalpelatthetopright.Atthesametime,therelatedobjectswillbehighlightedonthescreen.Basedonthesesuggestion,thehumancanfixthissetup.Wherewecanseethesetwoobjectsarealittlebitfarawayfromeachother.Thenthistime,theagentwillchecktheenvironmentagain,andnowitidentifiedanotherproblemwherethesetwoscissorsarealsotooclosetoeachother.Wecanseetheyarekindofoverlapped.Thentheagentwilltellthehuman,"Thesetwoscissorsarenowtooclosetoeachother."Afterthehumanfixthisstuff,theagentwillsay,"Now,thetraylooksgood,Idon'thaveanysuggestionsfornow."

Here'sanexampleofhowtheagentslearnnewconstraintsetupsfromthehumanexpert.Inthiscase,thewelltrainedassistantteachestwoconstraints.Oneis,weneedthreescissorsinthissurgery.Anotheris,theseshouldbegroupstogether.Theagentwillanalyzethesetwosentencesandthenformulatethecorrespondingconstraintknowledgewhicharerepresentedby[inaudible00:23:54]sentence.Thenwiththisnewlyacquiredknowledge,itcandirectlyapplyittocheckthecurrentenvironment.Inthiscase,itidentifiestwoscissorsonthetrayanditsays,"Thereshouldbethreescissorsonthetray."Thehumanputanotherscissor,whichisfarawayfromtheotherscissors,andthentheagentwillsay,"Thescissorsatbottomrightshouldbegroupedwiththeotherscissors."Thisishowtheagentlearnsnewconstraintsandapplyitinthenewsituations.

Theimplementedsystemistheintegrationofdifferentmodules,whichincludelanguageunderstanding,[inaudible00:24:40],knowledgerecognition,dialoguemanagement,andlanguagegeneration.Actually,eachofthesemodulesbythemselvesisabigresearchtopicandherewemerelyfocusontheintegrationofeverythingandthenhavea[inaudible00:24:54]systemtorealizethefunctionalityoflearningandcheckingthelayout.

Atlast,weintroduceacognitiveagentthatcangiveadvicesandassistantnon-experthumantopreparethetraysetup.Secondly,theagentcanlearnnew

Talk Session 1 - AI in Healthcare Page 8 of 30

setupconstraintsthroughdialoguewithanexpert.Thirdly,thesystemcanintegratemultipletypesofobjectpropertiesinlanguageliketheobjecttypesandspatialdescriptions.Atlast,thisisjustoneexampleofamoregeneralclassofproblemswhereacognitivesystemcansupervisenonexperthumantoaccomplishcertaintasks.

Sothat'sallformypresentation,thankyou.

EricHorvitz: Acoupleminutesforquestions.Yes?

Speaker6: Whatistherespondingtimeofthissystem?

LanboShe: Currently,wejusthavea[inaudible00:26:08],sotheresponseisveryfast,evenrealtime.

Speaker6: Okay.

LanboShe: Yeah.

EricHorvitz: I'mcurioushowyouchosethis[inaudible00:26:28].

LanboShe: That'sagoodquestionbecausewhatIdidbackinuniversityisabouthumanrapidinteractionandthat'sthesortofdialogue.Wealsofocusedonlearningoracquiredknowledge,andwhenIdidmysummaryinternatIBM,whatIdidbackinuniversityandthenwethought,canwemoveitalittlebitforward?Insteadofjustlearningfromhuman,wecanalsoteachnon-expertandthenwecameoutwiththisidea.

EricHorvitz: That'sfabulous.Fabulousdirection[crosstalk00:27:03]-

LanboShe: Yeah.

EricHorvitz: ...lotsofimplications.Allright,thankyouverymuch.

LanboShe: Yeah.[inaudible00:27:09]

SujoyC.: Goodafternooneveryone.Today,mytalkisoninsearchofhealthdoubles.Thisisnotmywork,itisworkby[inaudible00:27:23].He'sanassistantprofessorofdepartmentofIT,[inaudible00:27:29],Iampresentingonbehalfofhim.

Inthiswork,[Arturo00:27:37]wantsshowhowthehealthparameter,howtheproblemoffindingsimilarpersonswhosehealthparameterspredominantlymatcheswithotherpersonsintheworld.Sohealthdoublesbecomesachallengingproblembecausethehealthparametersaredynamicallychanging.

Tobetterexplainthisingreatdetails,let'ssupposedinonefinemorningthedoctortellssomeonethathehasbeendiagnosedwithararedisease.Healso

Talk Session 1 - AI in Healthcare Page 9 of 30

admitsthatwedon'thavemuchknowledgeaboutthisdisease.Itcannotbecuredunlessthediseasecanbestudiedingreaterdepth.So,hewantedtofindthesimilarpersonswhohavesamedisease.Inthiswork,outsourcingcanhelpustofindoutthesimilarpersonwhosehealthparameterpredominantlymatcheswithsimilarpersonssothatthiscanbehelpfultofindhealthdoubles.

Thesearesomeschematicviewofthisworkorproblem.Therearethreetypesoffeatures,thereisstaticfeatures,dynamicfeatures,andmedicinalfeatures.Sostaticfeaturesisbasicallytheage,sex,andbloodgroup,etcetera.Medicinalfeaturesarebasicallythehistoryofvaccinations,medicinehistoryofpersons.Dynamicfeaturesdealswithsomeoflevelsofbloodglucoseandpressure.Thesearebasicallydynamicallychanging.Itisveryhardtofindoutthesimilarpersonwhosehealthparameteraredynamicallychanging.

Thesearesomequantifyingformulatofindouttherelationbetweenhumanandobjects,humanandfeatureswithrespecttoaparticulartimeinstant.Becauseasthehealthparameterarechangingintimetotime,sotheperson,similarperson,arealsochangingforaparticularperson,foraspecificfeatureorforaspecificdisease.Thisisthe[inaudible00:29:53]denotestheassociation,strengthofassociation,ofaparticularhumanbeing,"i",andfeatures,"j".Thisistheassociation,thethisis[inaudible00:30:03]vectorforallofthesefeatures.Thatis,[inaudible00:30:08],thatisHKmeanswordvector[inaudible00:30:11]humanbeings.Sothesearetheallfeaturevectors.

Tofindtheassociationofdifferentfeaturevector,thisaSHS,[HUA00:30:23],thisisthecosinesimilarity.Here,youcanseetherearetwotypesoffeaturesvector,thereispositivefeaturesandnegativefeatures.Hereyoucanseethatsomevaccinationsisapositivefeaturesandnegativefeaturescanbeadisease.Humandiseasecanbeanegativefeatures.Sobeforecomparingthissimilarity,weneedtocareaboutwhatarethepositiveandnegativefeaturestocombinethissimilarity.

Now,thisisquantifyingthesimilarityscoreof,finalscore,bywhichwecanfindtheassociationoftwohumanbeingsofaparticularfeatureataparticulartimeinstant.

Toviewconsiderthisprobleminaglobalview,[Arthur00:31:13]showsthatthisproblemcanbetreatedasan[inaudible00:31:17].Inthis[inaudible00:31:19],thenotesarebasicallythecrowdworkerwhoareprocessingtheinformationfromonepersontoanotherpersons.Sothechallengeisthatastheparameters,dynamicparameters,aredynamic,sothat'swhyit'sverychallengingtofindoutwhatwillbethe...Howtheinformationwillflowfromonelayertoanotherlayer.

Sotherearethreesteps.Thereisaknowledgeextraction,learning,andsearch.Thisisthesearchforhealthdoubleshere.Youcanseethatifweconsiderthisnodeasacrowdworkerandifeachofthesecrowdworker...The[inaudible

Talk Session 1 - AI in Healthcare Page 10 of 30

00:32:02]basicallytoflowtheinformationinfirst.Soeveryoneshouldcommunicationwithother.Inthatcase,ifthereare"M"numberofcrowdworkers,itwillbeMC2numberofcombinations,soitwillbeorderofMsquared.

Butinthiswork,Arthurwantstoshowthatifthehierarchicalnatureofgraphcanbeconsideredandcrowdworkerscanbelayersasahierarchicalway,andthentheinformationcanbeflowedthroughhierarchicalway,andthiscanreducethecomplexityinorderofM[inaudible00:32:38].Thatisheightofthistree.

Themajorchallengesisthatbecauseofthediversityofparameterandhealthconditionofdifferentpeople,itisveryhardtofindthehealthdoubles.Werealizethatthesuccessofthisproposedmethodentirelydependsontheinvolvementofcrowdworkers,soweneedmoreinvolvement,moreinputfromthiscrowdworker,toflowthisinformationveryfast.

Thesearetheconclusionthatit'svery,reallychallengingtofindoutthehealthdoublesbecauseitcansolve,sayforexample,foraparticulararea,ifweneedthebloodgroupofapersonstoknowtheinformationofthisbloodgroup.Soitwillbeveryhelpfulifthehealthdoublescanbedecidedinadvance.Thechallengeisthattherearestillsomeproblemsthatweneedtoaddressbecausetheinformationflowrequiresmorenumberofinput.Thesearethe[inaudible00:33:44]references.Thankyou.

EricHorvitz: Anycomments?Iknowtherearesomehealthdoubledatasets,right?Because-

SujoyC.: Yeah.Forthistimebeing,astheworkisinongoingstage,soIthinkthatotherisworkingonit,soitwillbebetterto[inaudible00:34:13]dataset,butmyknowledgeisthatthedatasetisnotpubliclyavailableuntilnowandhe's[inaudible00:34:21].

EricHorvitz: Right,butImeantlikeWashingtonStatehasacredibletwindatasetthattheyputtogetherovertheyears.

SujoyC.: Yeah.

EricHorvitz: Therearesomerisingsetsofdatasetsoftwinsforstudiesofinfluenceofenvironmentonpeople,byunderstandingpeoplewhohaveatleastthesamegenomicfoundation.It'sinterestingtothinkaboutevenmaybetherelevanceofthatkindofgoldstandardintrainingupyourmodels,forexample.Anyothercommentsorquestions?Okay,great.Thankyouverymuch.

SujoyC.: Thankyou.

Talk Session 1 - AI in Healthcare Page 11 of 30

EricHorvitz: Sorryforthewireshere,becareful.NextwehaveArnaudDelaunayworkingwithJeanGuérin,wanderingdetectionwithinanembeddedsystemforAlzheimer'ssufferingpatients.

ArnaudDelaunay: Hieveryone,mynameisArnaud,I'mreallygladtobeheretopresentourworkonwanderingdetectionwithinanembeddedsystemforAlzheimer'ssufferingpatients.I'veworkedonthistopicwithJeanGuérinwhoistheCTOofCo-Assist.Co-AssististhestartupdevelopingconnectedwatchesforAlzheimer'ssufferingpatients.

Formypart,I'mstudentatEcolePolytechnicand[inaudible00:36:01]inacompanycalled[inaudible00:36:03]inParis.Today,I'mgoingtopresentyouwhatiswanderingwhyisitadangerforAlzheimer'ssufferingpatients.Thestateoftheartonthistopicofdetectingwanderingbehaviors,presentCo-Assist,thewatch,andthenthesolutions,ourresolutes.

ThewanderingproblemforAlzheimer'ssufferingpatients,whatiswandering?Well,it'sjustwhenyourloseyoursenseofdirectionandit'sduetodementiapathologiesandAlzheimer'sisoneofdementiapathologies.It'sreally,reallyfrighteningforthepatientswhoislosingitselfoutside,outdoor,andfortherelatives,aswell.It'smainlyduetotheinabilitytorecognizeyoursurroundingsandwhenyouhavememoryloss.Sosometimesitcanleadthepatientstoreallyseriousrisks,tobelostoutdoor.

Whatsolutionexiststodaytocounterthiskindofwanderingbehaviors?Well,Iwouldjustspeakaboutthemostfamousone,whichisgeo-fencing.Theideaofgeo-fencingisjusttotriggerandalerteachtimeyougobeyondaboarderofapredeterminedzone.Sotheprosis,reallyeasytoimplement,itworkswell,it'sprecise,it'ssecure,butoneofthelimitofthissolutionisthatitaffectsthefreedomofthepeoplebecauseit'slikeavirtualleashoravirtualjailforthepatientssufferingfromAlzheimer's.

ManypeoplehasworkedonthetopicofdetectingwanderingbehaviorsandIjustwanttostresstwomaingroupsofpeoplewhohaveworkedonit.[inaudible00:37:49]intheearly90'sand[inaudible00:37:51]in2007,haveworkedontheclassificationofthedifferentpatternsofwandering.Soherearethedifferentpatterns,it'slikepacingbetweenAandB,doingreturns.HavingarandomtripbetweenAandBwithalotofchangeofdirections.Orlappingaroundthepoints.

Thesearethekeynotionstheyfigureoutandanothergroupofresearchersin2012builtanalgorithmtodetectwanderingbehaviorbasedonGPStraces.SotheyusedthatassetfromMicrosoftAsiacalledGeoLife,withafrequencyofonepointeverysecond.What'sthedefinitionofsharppoints?It'swhenyouchangeyourdirectionsandcountingthenumberofsharppointswithinadistancerange.Theywereabletodetectawanderingbehaviorslikeyouseeonthesekindoftrips.

Talk Session 1 - AI in Healthcare Page 12 of 30

ForCo-Assist,thesolutionwewantedtobuildwas...Theaimofthesystemwastobeembeddedinawatch,inaconnectedwatch.YouseethewatchtheyarebuildingforAlzheimer'ssufferingpatients,[inaudible00:38:58]andwithalsosomefulldetectionsysteminit.

Thequestionisnowhowwe'veimplementedadetectionsolutionthatrunsintheembeddedsystem.Well,justletmerecallthegoals,it'sreallyimportanttounderstandthatwewantedtohave[inaudible00:39:15]forthedevicebecausewecannotimaginesellingadevicetopeoplesufferingfrommemoryloss,adevicethatyouhavetoremembertorechargeeverynight.Well,it'snotpossibletosellthiskindofproducttoAlzheimer'ssufferingpatients,sotheautonomywasthekeypointsofourgoals.SotheapproachistoanswerthatwastolowertheGPScorefrequency.Weusedtheaccelerationtosetdifferentmodes,whetherweareinthehouseoroutdoor.Whenweareoutdoor,weusethemaximumfrequencyofone[inaudible00:39:49]every30secondstokeepalongerautonomyforthewatch.

Anothergoalwastohavethecomputationembedded,sousingalightcomputationpipes,we[inaudible00:40:02]orusereally[inaudible00:40:04]modelinsidethewatchandstillhaveapreciseandsensibledetectionsolution.

Ourwork,it'slikeanapproximationofa[inaudible00:40:15]model.Meaningthatweusethestateofpreviouspositionstocomputethestateofthenewoneandwecheckthestates.Ifthestateisbeyondthethresholds,wetriggerthealert.Basically,youcanseenanexampleonthistripwhere,forexamplehere,thereisachangeofdirections,butthesepeoplehasbeenworkingforalongtime,soit'snottriggerthestatestoincreasingthestats.Here,forexample,youhavealotofchangeofdirectionsinasmalldistancerange,soitwillincreasethestates,andwhenthestatespassbeyondathreshold,thealertistriggered.Withthiskindofalgorithm,youhavetodefinefourmainparameters,andthat'swhenweuseatrainingsetandatestingsettodefinetheseparametersinordertohaveanoptimizedmodel,precise,andsensible.Sooptimizedfor[AUCT00:41:12]score.

Theresults,wehavetounderstandthatfortheprocess,forthetestprocess,wehavein[parallela00:41:19].Thetestforthescores,themetrics,ofourmodel,Mbatterlifetesttobeabletohavealongerlifeexpectancyforthebatteryinthewatch.Sowebuilda[inaudible00:41:36]usingthealgorithmof[inaudible00:41:39]researchergroup.Wesplititintrainingandtestingtests.Welowerthefrequencytohaveafrequencyforourmodelembeddedinthewatchand[inaudible00:41:49]implementationofoursolution.Wehadgoodmetricscomparedtotheworkdonebytheresearchersin2012andin[parallela00:42:01]wecanseethattheestimatedbatterylife[inaudible00:42:03]21daysforaconnectedwatchforpeoplesufferingfromAlzheimer's.It'swellenoughforthemtowearthiskindofwatch.Here,wecanseeanexampleofhowwetriedtooptimizethethresholds,theparametersyousawbefore.

Talk Session 1 - AI in Healthcare Page 13 of 30

So,what'snext?Well,wethoughtabouttodayinthealgorithm,wejustincreasedbyonethestates,butwecanuseacontinuousincrementofthestates,meaningmorelikepredictingthelevelofthestatescouldbelikeanincreasedimprovementinourmodel.Wewanttotryto...It'salwaystheproblemtohave[inaudible00:42:53]datasets,sousingacustom[inaudible00:42:56]trainingsetswithreallydatacomingfromAlzheimer'ssufferingpatientswearingthewatchwebuilt.Atleastattheend,releasethebeta,includingthedetectionsystemwithinthewatch.Sostaytunedandthankyouforyourattention.

EricHorvitz: [inaudible00:43:18]audience.

Speaker9: Howwillyoubeusingthesignalfromthesystem?Whatinterventionwillitdrive?

ArnaudDelaunay: Fromthesystem,wehaveGPSantenna,whichisgivingustheGPSpoints-

Speaker9: Imeantthedecision,ifittellsyouthepatient'swandering[inaudible00:43:40].

ArnaudDelaunay: Yeah.Whathappenswhenwetriggerthealert?

Speaker9: Yeah.

ArnaudDelaunay: Yeah,okay.Thereisachainofrelatives...Well,thechaincanbedecidebythepatientandhisrelativeandit'sachainofpeoplewhoarealertedbythealert.Somaybe,forexample,thechainbeatfirstyouhavetocallmyson,andthenifthesondoesn'tanswer,youcancallthe,Idon'tknow,theclosesthousehelping[inaudible00:44:14]patients,andifhedoesn'tanswer,wecancallanemergencysolution.Sothereisachainofpeopletriggeredbythealert.

Speaker10: Afollowup,akeywhoisfalsepositives.

ArnaudDelaunay: Exactly.Wecannothaveanunprecisemodelforthat.

Speaker10: [crosstalk00:44:36]

ArnaudDelaunay: Yeah,butwestillwantasensiblemodelbecausewecannotsellaproductsaying,"Okay,don'tworry,we'lldetectwhenthereiswandering,"sothemodelstillhastobesensibletonotletpassanyrealtruepositivewanderingcase.

EricHorvitz: Wayintheback,first,andthenbehindyou,thenyou,andthenthethirdpersonhere.Thanks.

Speaker11: Thatwasanicetalk.Intheapproachesfordetectionofwandering,oneofthedrawbacksofthefirstapproachyoumentionedisitlimitsthefreedomoftheperson.SoIwaswonderinginthesecondapproach,theonethatyoudetecttheturns,thesharpturns,aren'tyoustilllimitingtheperson'sfreedom?

Talk Session 1 - AI in Healthcare Page 14 of 30

ArnaudDelaunay: That'sagoodquestion.Whatisthefreedomofmovementforthepeopleandwherearethelimitsoffreedomofthismovement?Becausewearetargetingpeoplesufferingfromadiseaseandthesymptomsofthisdiseasecanhurtthesepeople,soattheleveloratanother,youhavetowatchthesepeopletohelpthemifthereisaproblemwithhiscurrentposition.SomepatientssufferingfromAlzheimerwilltellyou,"Okay,Idon'twantpeopletowatchmeateverysecond,"sothealgorithmwherejusttherelativeshaveaccesstotheGPSpointsisnotagoodsolutionforthem.Otherpeoplejustwillsay,"Oh,IwillbemoreassuredifIknowthatsomeoneiswatchingonmeeverytimeandIwantthiskindofsolutions."Youalwayshavelimitsonfreedoms,buttoansweryourquestion,comparedtogeo-fencingsolutions,enablethepeopletomovewherevertheywantaroundtheworld,anditwillbestilldetectingwanderingsituations.

Speaker12: So,thereseemstobeabitofdichotomybetweenglobalsolutionstothisproblem,whichisthegeo-fencing,versusthelocalsolutionwhichisthepersonaldevicethat'sonthewrist.Haveyouputanythoughtintoactuallytryingtohybridizethiswhereit'snotjustoneortheother,butsomecombinationofsuch?Whereifapersoniswanderingoutsideoftheirtypicalarea,itisanindicatorofamoreextremeexample.[inaudible00:47:15]inthelastquestion,buttherewasalotoflimitingfactors,eitherone,you'remakingtrade-offs,isthereawaywheremarryingbothsystemsgivesyoumoreflexibilityandmorefreedom?

ArnaudDelaunay: Yeah,Ithinkit'sagreatremarkandwehaven'tthoughtofthiskindofcombination.Ithink,yeah,thereissomeplacetoworkonthat.Yeah,forsure.Thankyou.

EricHorvitz: Wait,wehaveanotherquestion.Don'tgettooexcitedjustyet.

Speaker13: Whataboutprivacy,indangerofhackingbycriminalswhothenknowthattheoldpeopleisnotathomeandallthatkindofstuff?

ArnaudDelaunay: Actually,thereisnocommunicationbetweenthewatchandtheinternational,allthecomputationaredonelocally,anditonlytriggerthealertswheneverthestatesishigherthanthethresholds,sothereisnocommunicationbetweenthewatchandthecloud,ifit'snotawanderingbehavior.Yes?

EricHorvitz: Ihaveaquickquestion,afollowupontwoquestionsago.I'dbecurioustounderstandifyou'vedonesomefailureanalysis.Sowhenyouhaveafalsepositive,tounderstandwhatwasgoingon,maybeit'ssomethingaboutlandmarksthatwouldhelpyoutoreducethatfalsepositiverate?Thereactuallyissomethinginterestingthatyouconsideredtobewandering,butactuallywasgoaldirected,haveyoulookedatthat?

Talk Session 1 - AI in Healthcare Page 15 of 30

ArnaudDelaunay: Yeah,it'saquitedifficultquestionbecausethedatasetsweusedforourtrainingsetandtestingsets,isnotonlywanderingpeople,andithasbeenhandlelabeledbydoctorsandresearchers.Sowe-

EricHorvitz: Can'tanswerit,yeah.

ArnaudDelaunay: ...wouldn'tbesureofbeingreallyafalsepositiveaboutwanderingornot,butitwasjustlikewewereusingthesekindofpatterns-

EricHorvitz: Right.

ArnaudDelaunay: ...tosay,"Okay,thisisthewanderingpatterns."

EricHorvitz: It'dbeinterestingto-

ArnaudDelaunay: Yeah.

EricHorvitz: ...ifyouactuallydidhaveactualgroundtruth-

ArnaudDelaunay: Yeah.

EricHorvitz: ...andtolookatfailurestheretoseewhattheyactuallyare.It'dbeinterestingtoknow-

ArnaudDelaunay: Yeah.

EricHorvitz: ...tohelpyououtwiththat.

ArnaudDelaunay: Yeah,forsure,butthisisa[inaudible00:49:22]problem,likehavingthegroundtruthsonthesets.

EricHorvitz: Right.

ArnaudDelaunay: Thankyouverymuch.

EricHorvitz: Thankyou.Nowwehaveapaperby[Bolee00:49:34],Yevgeniy[inaudible00:49:34],[inaudible00:49:36]Lee,andBradley[Malin00:49:37].Eugene'sgoingtopresentonsanitizinglargescalemedicalrecordsbeforepublishing,whichaddressesareallyinterestingproblemwithHIPPAandsoon.Thanks.

YevgeniyV.: Okay,thankyouforhavingmehere.Yes,thisisaboutAIforsocialgood.Oneoftheargumentsisthattheincreasingamountsofdatapromotesatremendousamountofsocialgood,thechallengeistheprivacychallenge.Alotofthisdata,especiallyinmedicaldomains,hassensitiveinformation.Thingslikepeople'sname,theiraddress,socialsecuritynumbers,andsoon.Thistalkreallyisbroaderthanthat,butthefocusisonmedicalandclinicaldomains.Specifically,thingslikeclinicalnotes,whichareinherentlyhardtosharebecausethe

Talk Session 1 - AI in Healthcare Page 16 of 30

informationinthoseclinicalnotesisnotstructuresoit'snotdirectlylabeled.Instructuredata,youcanjustremovethenamecolumnandwhatnot.Inclinicalnotesorinanytextdata,thatinitselfidentifyingwhatarenamesorwhatissensitiveinformationitself,isacomplicatedtask.That'swhatthistalkisabout.

Thereareahostofapproachesfordoingthisatscaleusingmachinelearning.Thesetupisfairlystraightforward.Soyouhavesomehumanlabelersthatidentify,here'saname,let'scallthisplusone,here'snotaname,let'scallthisminusone,andyoucreatethesedichotomiesoflabels.Youhavethislabeleddataandyouuseyourfavoritemachinelearningtool,commonlyit'ssomethinglikeaCRForNLPdomains,andtrytopredictwhatissensitiveinformationthatyoucansubsequentlyextractfromthisdata.

Learningisgreat,butjustlikeanytoolswedevelop,itmakesmistakes.Twokindsofmistakes,tobeprecise.Falsenegatives,orunderreduction,isthethingsthatareactualnamesorsensitiveinformationthatyoudon'tcatch,andsubsequentlywhenyousharethatdata,youleakittothepublicortowhoeveryoushareitwith.Falsepositivesaretheoverreduction,thingsthatyousuppress,eventhoughithasnosensitivecontent.Thereasonthat'sbadisbecauseyou'resuppressinginformationthatcouldpotentiallybeusefulinanalytics.Soyouwanttosuppressaslittleaspossible,butofcourseyouwanttosuppressthesensitivestuff.

Sothat'sreallythechallenge.Theprivacypartisthefirstone.Youwanttobesuretosuppresstoanacceptablelevelofrisk,thestuffthat'sactuallysensitiveandnotleakittothepublic.Sothat'sthechallenge.

Inpriorwork,implicitlythethreat[inaudible00:52:06]priorworkorthesemachinelearningapproaches,implicitlythethreatmodelhasbeenthatthereissomehumanadversarywhoistryingtoactuallyreadingthesedocumentsandidentifyingwhatisthesensitiveinformation.Now,inreality,humanadversariescanalsousemachinelearningandthat'swhatthistalkisabout,howcanwequantifythiskindofadversarialbehaviorinaprecisewayandbuildathreatmodeloutofthis?

Here'sathreatmodelforanattackerthatwouldalsousemachinelearningtofacilitatediscoveryofsensitiveentities.Solet'sjustsaythatyou'vereleasedsomeamountofdataafteryou'veappliedmachinelearningtoit.We'regoingtoassumethattheadversaryhassomebudgetformanualinspection.Whatdoesthatmean?Theytake,theydosomepre-processingusingmachinelearning,andthentheymanuallyverifywhethersomeentitiesthatarehighlyprioritizedareactuallysensitiveornot.

Theapproachisasfollowing,thatwasposittheattackerwoulduse.Sotheyruntheirmachinelearningmodel,let'scallit[laronic00:53:01]classifierH,topredictsensitiveentitiesnowinadatathat'sactuallybeenpublished,thattheyhaveaccessto.Theyrankthepredictedpositivespriortopredictednegatives,

Talk Session 1 - AI in Healthcare Page 17 of 30

whichispresumablywhyyouwouldusemachinelearningtobeginwith.Thentheygoupuntiltheirbudget,andgrabandinspectmanuallythoseinstancesthatarerankedinthefirstBspots.

Now,weendowadversarywithtwosuperpowers.Thefirstsuperpowerisperfectverification.Iftheyseesomething,theyknowifit'sanidentifierornot,ifit'sasensitiveentityornot,that'sone.Thesecondoneisoptimalityinthemachinelearningsense.Weassumeherethat,atleastforpurposesofanalysis,thattheadversarycanactuallylearnthebestaccuracylearningmodel.Inpractice,youcankindofsimulatethisveryeasilybyjust,again,havingtheadversarylabelthedatamanuallyoraportionofthedatathattheyreceive,andsortoflearnthemodelthisway.Butforanalysispurposes,assumethattheyareoptimalinaccuracysense.

Solet'snowrevisitthetypicalframeworkthatpeopleusetoapplymachinelearningtosanitizeddata,inthiscontext.Sostepone,topublish[inaudible00:54:06]asaclassifier,let'scallthisH.Theyremoveallpredictedpositivesandthenreleasethedata.Nowcomestheadversary,theyactaccordingtothethreatmodelIjustdescribed,whichI'mnotgoingtodescribeagain.Now,here'sthechallenge,andalsotheopportunity.Thechallengeisthatnowyoupotentiallyhavegivenupsomesensitiveentitiesadversarycanidentify.Theopportunityisthesameway,youcanactually,asapublisher,simulatewhattheadversarywoulddo,andthenusethattojudgewhetheryoushoulddosomethingelsebeforeyoureleasethedata,orjustletthedatago.

That'sthekeyinsight.Ineffect,wecandothisinaniterativefashionwhichgivesrisetoafairlystraightforward,[inaudible00:54:47]algorithm,orwecallit[greedy00:54:49]sanitize.Theideaisasfollows.Youstartwithsomedataset,youlearnaclassifierH,youremovethepredictedpositives,thepredictedsensitiveentities,youlearnagainaclassifieranditsrestricteddata.Youhavesomenewpredictedpositives,youremovethose,youkeepgoing.Now,obviouslyifyoukeepgoinglikethisforever,you'regoingtoremoveeverything,soyou'vegottostopatsomepointandthekeyquestionis,whendoyoustop?

Inordertoanswerthisquestion,weneedtomorepreciselydefinewhatitisthatyouaretryingtoactuallyaccomplish,whichistosavepublisherutility,aswellasthelosses.Soweassumethatwheneveryoupublishthedata,anytimeyoureleasesomething,thepublishergainsCforeverynon-sensitiveentitythat'sreleased,andlosesL,whichissomequantitywespecifyapriori,forsensitiveentitythat'sactuallybeingleaked.That'sidentifiedbytheadversary.

Sopublisher'sgoalisgoingtobenowtradingofthesetwothings.Youwanttosharemoredata,that'stheCpart,butatthesametime,youwanttolimitidentifiableentitiesthatyouleak,that'stheLpart.Whatdowedo?We[inaudible00:55:57]this[greedy00:55:58]iterativealgorithm,essentiallyoneofthemarginalbenefits,nolongeroutweighthemarginalcosts.WhichIwillcall

Talk Session 1 - AI in Healthcare Page 18 of 30

thelocaloptimumbecausewe'redoingthisgreedilyandwestopatthefirstchanceweget.Soit'slocaloptimum.

Now,itturnsoutthatwecancharacterizeintermsoffalsepositivesandfalsenegativesofthesimulatedadversarymodel,hencethesubscriptA,whendowestop?It'sbasically,falsepositivesaretoomanyrelativetothetruepositivehere,andtherelationshiphereisLoverC,whichissortofintuitive,therelativevaluerelativetocostoverdata.Now,somethingyoumightbewondering,willGreedyiterate,willactuallydothisforeverwithoutstopping?It'sactuallynothardtoseethatthat'snotthecase,itwillstopatmost"N"iterations."N"is,ingeneral,thesedomain'sfairlylarge.Inourexperimentsweshowthatweactuallydomuchbetterthan"N",butwe'llcomebacktothat.

Arguably,thekeyresultisthatwheneverGreedysanitizeterminates,wecanactually[inaudible00:56:58]thenumberoftruepositivesweleak.Truepositivesinthemachinelearningsenseintermsofsimulatedadversaryandisboundasafunctionessentiallyofourcostbenefittradeoff,theCoverL,whichiskey.Sothemorewecareabouttheleakedsensitiveentities,themoretightlywe'regoingtoboundthis,whichmeansbasicallythemoreiterationsyou'regoingtorunthisalgorithmfor,beforeessentiallythenoiseisgoingtooverpowerthesignal.

Now,youcanimagineasafunctionofadversary'sbudget,theotheraspectthatyoucanconsideristhatthisisn'tenough.Theadversary,iftheyhaveinfinitebudget,they'llactuallyinspecteverything,truepositivesaswellaseverythingelse.Soyouneedsomethingelsetomeaningfullyboundexactlywhattheadversarywoulddo.Ameaningfulbaselineis,whatiftheadversaryjustpicksarandomsubsample,essentiallyignoringthefactthatthey'reusingamachinelearningalgorithm.Let'scallthisthebaseline.Soweshowedthatessentiallywhenbudgetislargerthantheminimumamountofbudgetoncetheystartgoingbeyondsortofthepositivecount.Atthatpoint,theycandoessentiallynobetter,andnotmuchbetterthanjustrandomlysub-sampling.Sotheuseofmachinelearning,thevalueofmachinelearning,fortheadversary,isprettyminimalatthispoint.

Evaluation,weusedfourdatasets,twomedical,twonon-medical,andourgoalhereistosuppressnames,whichisoneofthereasonswecouldusethingsliken-rundatasetandnewsgroupdatasetbecausetheyactuallycontainnames,whichwecanlabel.Iwon'tspecificallygetintotheexactclassifiersweuse.CRFisoneofthebestonesforthistask,sothat'soneofthemainonesweused,butafewothersweusedaswell.

Goingbacktothequestionoftruepositives.Wehaveaboundthat'snice,howdoesitworkinpractice?Itworksmuchbetterthantheboundevenwouldsuggest,thisiswhattheleftpartshowsyou,LoverC.Basically,itsaysthatyoudon'tneedtocareaboutsensitiveentitiesthatmuchbeforeyou'renotleakingmuchinthewayoftruepositives.Sothat'sastrongresult.Thebaseline,the

Talk Session 1 - AI in Healthcare Page 19 of 30

dottedlineshere,youseethehorizontal,that'sifyouonlydooneiteration,whichiswhatpeoplewouldtypicallydonow.Sohavingmultipleiterationsisenormouslyhelpful,butasyou'llseeinasecond,youdon'tneedmanyofthose,youonlyneedafew.

So,here'sthenumberofiterationsontheleft.Again,thisisafunctionofLoverC.Ifyounotice,thenumbersonthehorizontalaxis,whichisthenumberofiterations,aresmall.Thedatasetsareinthousands,butwe'reonlyrunningitforfiveiterationsatthemost,usuallyalotlessthanthat,onaverage.That'sverygoodnews.Essentiallywhatthissaysisthat,itdoesn'ttakeawholelotofiteratingbeforebasicallyyougetsomuchnoisethatthere'snotmuchvalueinlearning.That'swhatthisistellingyou.

Whatyouseeontherightisthefractionofthedatayouactuallyrelease.Sothat'sthepositiveside.Youwanttoshareasmuchofthisdataaspossible,andhorizontalaxis,again,isthisLoverCfactor.Asyougototheright,youcaremoreaboutnotleakingsensitiveentities.Sohereiswhereyouseethebaselinesarestartingtodoreallybadlyandtheinterestingthingistherightmostapproachisourapproach,andit'sinterestingthatyou'resuppressingawholelot.Soessentiallywhatit'ssayingisthatyou'rebalancingthisprettywell,you'resuppressingverylittle,butwhatyou'resuppressingisactuallyreallyimportantstuff.You'renotreallyover-redactingtoomuch.

Tosummarize,we'vedevelopedaiterativemethodforsanitizingdataatscale.Itusesformalmachinelearningbasedattackmodelintheloop,andallowedustoimproveapproachofbettermachinelearningapproachesforentityresolutionaskindsoftaskslikethiscomeup.ThisisactuallysomethingIdidn'treallyspendalotoftimewith,butbecausewe'reusingessentiallytheattacksthataresimulatedintheloop,ifsomebodycomesuptomorrowandsays,"Ihaveabetterattackmodelthatisabletore-identifythesethingsmuchbetter,"wecanactuallythrowthatintotheloopandthisalgorithmgetsimprovedasthenewattacksgetdeveloped.Whichisreallyniceproperty.

Itenablesformalguaranteesaboutprivacy.Ofcourse,theguaranteesarewithintheframeworkweuse,they'renotbroaderthanthat,butourframeworkatleastallowsustostatethesethingsprecisely.It'sextremelyeffectiveinpractices,asI'veshowed.Thankyou.

EricHorvitz: Sojusttostartthingsoff,I'mcurious,twothingscametomind.Actually,severalthings,butI'llstartwiththefirsttwo.IfyoulookatHIPPAlawandHIPPAregulatoryactivity,andthewaytheyspecifydifferentlevelsofaccess,therearedifferentpatternsofwhatisallowedtodifferentkindsofresearchers,giventodifferentkindsofIRBapproval,andsoon.Notthatwehavethelevelofnamesversusnonames,it'slikename,zipcode,age,youhavethesesets,constellations,offeaturesandit'dbekindofinterestingtotakeyourmethodtooneofthesestandard,templatizedpatternsoflevelsofaccessandseewhat

Talk Session 1 - AI in Healthcare Page 20 of 30

protectionwetakeatlevelone,leveltwo.Theyactuallyhavenames,buthaveyoulookedatthatatall?

YevgeniyV.: Afewthings.NotaspecialistinHIPPAspecifically,butperyourquestion,there'ssortofseveralkindsofthings-

EricHorvitz: Right.

YevgeniyV.: ...thatyoucango...KindsofapproachesyoucangetatHIPPA.Oneisessentiallyarulebasedapproach,whichiswhatIthinkyouarealludingto,thisisthesafeharborapproachwhereyousuppress,atthislevel,theseparticularattributes,atthislevel,theseotherparticularattributes.Thisisreallydesignedforstructuredatabecauseit'spresumed,tobeginwith,youknowexactlywhatthoseattributesare.ThisisalreadyabitstepremovedfromHIPPAbecauseit'sunstructuredandsoevendiscoveringthoseattributesisinherentlyhard.So,inHIPPA,thoseattributesthatwe'retalkingabout,thingslikequasi-identifiers,forexample,andsafeharbor,theseareactualidentifiers.Evenknowingwherethenamesareofthepatientshereisapriori[inaudible01:02:56].InHIPPA,thisisalreadyactuallyachallenge.[crosstalk01:03:01]

EricHorvitz: IguesswhatIwassayingwas,youtakethosewillbasedpolicies,whichapplytostructureddata,andyouhaveframeslikenames.Soyousay,"Okay,I'mgoingtodealwithnamespermydiscoverymethodologyhereandattackandprotect."Youalsohavethingslikeaddresses,andzipcodes,andages,youcantakesomeofthesespecificationsandmapittothiskindofworkinunstructureddata,andthencallitproxiesforprotectingatlevelone,leveltwo,levelthree.

YevgeniyV.: Yeah.WhatIwantedtosayis,oncethisproblemissolved,sothenextstepwouldbeexactlywhatyou'resuggesting,right?

EricHorvitz: Isee.

YevgeniyV.: Sothisfirststepisjustidentifyingwherethosethingsliveinthedata.Justanotherparttoit,sothere'ssafeharborisoneoption,theotheroptionisbasicallyriskbased.Youhavetheexpertcertificationthatthedataislowenoughrisktobeshared.Thisreallyismuchmoreinthespiritoftheriskbasedbecauseifyougobysomethinglikesafeharbor,[inaudible01:04:00]clearthatyouwouldlegallybeabletodothisbecauseagain,youcan'tguaranteethatyou'veremovedallthenames.

EricHorvitz: Right,interesting.[crosstalk01:04:08]questionwas,there'sabunchofmachinelearninggoingonthesedayswithunstructuredmedicaldata.It'dbeinterestingtoknowwhatkindofhitsyoutakeinaparticulardiagnosticchallenge,forexample,apredictivechallenge,atsomelevelofprotection.[inaudible01:04:25]tradesintermsof,forexample,AUCcurvegoingdownasyougetbetterandbetteratprotectingagainstX,Y,andZ.Soasyourriskgoesdown,whatdoestheAUCdo?

Talk Session 1 - AI in Healthcare Page 21 of 30

YevgeniyV.: Yeah.

EricHorvitz: Toactuallydoastudymaybeforaspecificentityordiagnosticchallenge.

YevgeniyV.: Yeah,it'sagoodquestion.Thisisjustfocusedonnames,buttherehavebeenotherstudiesthatarerelatedtothislookingatthedifferentthings.Also,thisisjustaverylimitedsnapshotoftheresearch.We'vedonethingslikealsocostsensitiveclassification,aswell,thatallowustokindmorefinelymapoutthesetradeoffs.

EricHorvitz: Nowwehavesomequestions.Yeah?

Speaker15: [inaudible01:05:06]small.Justtoclarifythat.

EricHorvitz: What'ssmall?

Speaker15: Thehityoutakeindetectionofothertypesofentities,issmall.SotherewasastudythatcameoutofCincinnatiChildren'swheretheywerere-runningthemedicationextractionprogramfromTexas,the[Medex01:05:21]method.Theyshowedthatjustafterasingleiteration,whichiswhereyougetalotofthis,theAUCforthedetectionofmedications,theactualdrugsandthedosing,actuallydidn'tchange.So,it'susuallynot...Youdon'tgetalotofconfusionbetweenclinicaltermsandpotentialidentifiersinthemedicalrecords.

EricHorvitz: Yeah,that'sactuallyaveryinterestingcomment.It'srelatedtoit,butnotexactlywhatIwasgettingat.Iwasgettingatthisidea,ifyouhaveanaspirationaltargetwhichisnotinthedataanywhere,liketheprobabilitythispatientwillcomebackandbereadmittedforaconditionofheartfailurein30days,andyouhaveaccesstoallthisdataandyouhavethealgorithmschugalongandthey'reconsideringeffectivelythousandsofvariable.Let'ssayeverysinglewordbeingpresentorabsentinthedocument.Thenyousay,"Wellandthat'sprotectinthisway,"it'dbeinterestingtoseeifthere'saneffectandtodiscoverwhatthetermswerethatwereactuallyusefulingivingyouthatextraliftforwhichyouremovethem,giveyoukindofalossofaccuracyinadiagnostictaskorpredictivetask,whichisnotnecessarilybasedinentities.

YevgeniyV.: IthinkIunderstandwhatyou'resayingbetternowandtheshortansweris[crosstalk01:06:39]-

EricHorvitz: Sorrytobesogarbledbefore.

YevgeniyV.: No,well,it'sokay,Ijustmisunderstood.

EricHorvitz: Yeah.

YevgeniyV.: Theshortansweris,wejusthaven'tlookedatthat-

Talk Session 1 - AI in Healthcare Page 22 of 30

EricHorvitz: Yeah.

YevgeniyV.: ...specificallyforthiskindofdata.

EricHorvitz: Yeah.

YevgeniyV.: Thisisdefinitelysomethingthat'sonourplate.

EricHorvitz: Yeah?Wehaveaquestion-

Speaker16: Hi,soI'mtryingtounderstandhowtointerpretyourprivacyguarantee.Forexample,differentialprivacybasicallysaysthatyourexpectedutilitychangesverylittledependingonwhetherornotyou'reinthedataset.Whatdoesyourprivacyguaranteemeanandhowisthatrelatedtosomeotherapproachesthatpeopleuse?

YevgeniyV.: It'sdifferent.Theprivacyguarantee,whatI'mtalkingaboutisthenumber...Isspecificallywithinthemachinelearningcontext.It'sessentiallyhowmuchtheadversarycangainasafunctionofbudget,ifyouwill.Sowehavethisdichotomyasafunctionofbudgethere.Sohowmuchdotheygain,howmuchaddedvaluedoesmachinelearninghaveforthem?Thosearethekindofpropertyguarantees,they'reorthogonaltothethingsthat,forexample,indifferentialprivacyyouhearabout.Differentialprivacyguaranteesaremuchstrongerthanwhatwe'reclaiming,forexample.

Speaker16: Okay.

YevgeniyV.: Butreallydifferent.

Speaker16: Okay.

EricHorvitz: [inaudible01:07:44]doyouhaveacomment?Question?

Speaker3: Isthereawayinwhichyoucanrankorratethelossofdifferentthingsthatyoumayloseinprivacy?Orareallofthelossesofequalvalue?Forexample,somemightbemore,somemightbeless,andsomightbeabletokindofplayagameinordertotrytoonlylimitlossestothemoreimportanttermsorsomethinglikethat.

YevgeniyV.: That'sagoodpoint.Rightnow,we'rebasicallythinkofallsensitiveentitieshomogenous.Inpractice,whatwewoulddoiswe'dtakejustnamesfirst,usingthisapproach,andthatwouldbesomeLoverC.ThenyoutakesomethingelselikeSSNanditwouldbeadifferentLoverC.Sowe'djust,inthislanguage,wouldbeplayingdifferentgamesforthedifferentsensitiveentitiestocapturethefactthattheyofcoursehavedifferenttradeoffs.Theremightbeabetter,cleanerwaytodoitwithinoneframework,butIdon'tknow.

Talk Session 1 - AI in Healthcare Page 23 of 30

EricHorvitz: Yeah,theinterestingthingforme,[inaudible01:08:51]medicaliscompanieslikeMicrosoft,andIassumeGoogle,andAmazon,wouldlovetosharetheirinternaldatasetsundercompliancewiththeirenduserlicensingagreementswithresearchersandscientists,butit'softenchallengingtodosoinawaythatprotectstheconsumerbase.Havingsomemethodologyforcleaningthedatainawaythatreducesriskwouldmakethatmorepossible.We'rehopingtoseesomecrossindustryinitiativesonclosingthedatadividebetweencompanies,andacademia,andNGOsbycomingupwithmethodslikethat.Onemorequestion,yeah?

Speaker15: [crosstalk01:09:33]Eric,thisone'sforyou.

EricHorvitz: Oh,okay.

Speaker15: That'sfirsttimeI'veheardyousaysomethinglikethis.

EricHorvitz: Wellwe'retransitioningtoopendiscussion,somaybewe'llbothtakethisquestionasadual-

Speaker15: Okay,sowhatwouldittakeforamethodlikethis,oranytypeofasanitizationmethod,toconvincetheMicrosoftlawyerstoallowthedatatogooutthedoor?BecauseIwouldlovetoworkonthattechnology.

EricHorvitz: Youmeantoenablethatorgettheaccesstothedata?

Speaker15: Both.

YevgeniyV.: [inaudible01:09:58]collaboratorinthiswork,bytheway.[crosstalk01:10:00]

EricHorvitz: Oh,okay.Youare...Letmeseeinthepaperhere-

Speaker15: I'mthelastone,I'mtheonethataskedyouforthePokemonGodataset.

EricHorvitz: Oh,okay.We'vehadseveralrequestslikethat.Forone,we'dhavetoprobablychangeourEULAsbecausetheEULAssaywedonotgiveoutdataofanykindtooutsideofMicrosoft.SoifyoutrustMicrosoft,feelfreetousetheservice,andit'ssimilartowhatothercompaniesdo.Youcanimagineussaying,ifyouoptin,yourdatawillbe...Yournameandagewillbedetectablewithsomeprobability.Somepeoplesay,"Oh,that'sokaywithme,Iwanttodonatetoscienceandbehavioralstudies."TheotherapproachtothatMicrosofthasdone,andIthinkwe'vebeenleading,atleastintheindustry,inthis,becauseIdon'tknowofothercompaniesthathavedonethesamething.

ThreetimeswehadRFPswhereweinvitedtheacademiccommunitytoapplytobepartofaprojectandaprogramtocomeintothetentunderlicenseandusesanitizedlogs,andthenit'dbeayearofworkshops,andgatherings,andpeopleshowedtheirstudiesandpublishedtheirresults.Butitwasinaccordancewith

Talk Session 1 - AI in Healthcare Page 24 of 30

theenduserlicensingagreementbecausethey'renow"Microsoftinternal"bysigningadocumentbecausetheEULAssay,"Microsoftoritscontractorsandcoworkers,"andsoon.

Speaker15: [inaudible01:11:38]identifyinformation[inaudible01:11:40]?IsitsubjecttoEULA?

EricHorvitz: Ibelieveso,butyoucanimaginethatifitwasreally,trulysoeasychangetomake.Becauseourlawyersareverybrightpeopleandtheysay,"Oh,wellinthatcase,we'lljustchangethesentencehereand[inaudible01:11:58]everybodyrunawayfrom,morethantheyarealready,fromBingtoGoogle."Thatwassupposedtobeajokethere.Actually,Bingisdoingverywell,bytheway,asIsayforthelivestreamthesedays.Okay.Well,thankyouverymuch.[crosstalk01:12:14]

Sonowwehaveabout10minutesofopendiscussionbeforewebreak,onhealthcareingeneral.Toanyoftheauthorsorco-authorsinpresence,oronanytopictodaythatwediscussed.SoIthoughtsomeinterestingdiscussionsforthisgroupwouldbethequestionIaskedoneofthespeakerstoday,howwechoosetopicstoworkon,whatmakesatopicinterestingandvaluable.Wherevalueisparticularlyafocusofthisworkshop.IwaslookingatthingsthatIworkedoninthepastandIjustnoticedthatItendtogetreallyexcitedabout,evenifthey'renotsosophisticated,iftheupsideisreallybigintermsof...Ioftenlikelookingatthevalueorthecostofacertainhandlingormishandlingofadiseaseentity.

SoIhad$35billionayearforCHF,congestiveheartfailurecare,andthefactthatcongestiveheartfailurewillaffectabout10%ofusifthingscontinuethewaytheydo,pernobreakthroughsandsoon,over65yearsofage.Andthefactthatwhenyouhavediagnosedcongestiveheartfailure,youhavea10%peryearmortalityrate.Or,hospitalacquiredinfections,they'reinthetopseven,IbelieveofallcausesofdeathintheUnitedStates,believeitornot.Ihavethestatsonthetipofmytongue,butsomelargefractionofpatientsgetsickinthehospitalwithsomethingtheydidn'tbargainfor.

IfIevenaskedthisaudience,howmanypeoplehereknowpeople,orfamilymembers,orfriends,thatwenttothehospitalforcomplicatedprocedureXanditwentwell,andthewholefamily'sexcited,andthenyouhearaboutthisotherthinggoingon,thishospitalacquiredinfection.Ibetyouinthisroomsomepeoplehavelostfamilyandfriendsthatway,notfromtheinitialentity,butthesideeffectortheinadvertentinfection.

Inthatdepartment,thereare,ifyoubelievethevarious,several,studies,closeto750peopledyingperdayintheUSbecauseofpreventablemedicalerrors.That'shumancognitionandhospitalworkflowrelatedchallengesandopportunitiesforustosolvewithvariouskindsofcomputationalsafetynets.Ofcourse,thebigoneweallknowabout,about100deathsintheUSperday,peoplejusttryingtodriveacarfromAtoB,andabout1,000lifetimedisabling

Talk Session 1 - AI in Healthcare Page 25 of 30

injuriesperday.Inahealthcaresession,youdon'tthinkaboutautomaticbrakingsystemsandtheattentionofdrivers,butit'sahugepublichealthdisaster,butIthinkit'sabout4milliondeathstodateintheUSfromautomobiles,cars,sincewestarteddriving.

IoftensaywhenIthinkaboutpolitics,andDonaldTrump,andhiscampaign,therearevariouswaystomakeAmericasaferandtherearemoreobviouswaysthanwhatarebeingtalkedaboutinthepress.Withthat,I'llopenituptoanydiscussionorconversation.Yeah?

Speaker12: Ihadageneralquestion,andthiskindoftiesbackintotheinvitedtalkaboutthesocialjustice,orthesocialwork,andincarcerationrates,wherepeoplecouldbeletoutbyjudges.Therewasinterestinmakingthemachinelearning,theveryblackboxyalgorithmicdesign...Notopaque,makeittransparent,andmakeitavailableforlaymenpeopletounderstand.Iwouldthinkiflookingatthefutureofmachinelearningandartificialintelligenceinhealthcareinthenextfiveor10years,that'sgoingtobeamajorfundamentalchallengetotrytomakethesebigdata,deeplearningmodelstransparentandtrustworthy.

Therealquestionis,howdoyoubridgethatgapofprovidingsystemslikeWatsonthatarescanningmillions,andmillions,andmillionsofhealthrecords,andprovingdiagnosesorrecommendations,andhoweverthatworksisacompletemysteryofmagic.Howdoyoutranslatethatbarrierandhowdoestheindustryandtheprofessiontranslatethatbarriertomaketheseartificialintelligencesolutionslessopaque?Anyauthorscan...

EricHorvitz: Right.Ihavemyreactiontothat,butlet'sopenituptotheaudience.

Speaker3: Iwantto-

EricHorvitz: Here.

Speaker3: Interpretabilityofdecisionsiscertainlyoneimportantaspectofdeployingthesesystemsinsociety,butIwantedtoaskallthepeoplewhogave...Speakersinthissession,ormaybeeventhe[inaudible01:17:20],arethereothersuchprinciplesthatwearetobethinkingaboutforthisarea?Forexample,earliertherewassomediscussionofautonomyinthesenseofwheredoctorsorsoforth,experts,giveupsomelevelofautonomybecauseyouhaveaAIagentsaying,"Thisiswhatyoushoulddo,"andwhethertheywouldwillinglyacceptthatorwhetheryouwanttoplayalongwiththemtosomedegreebecauseyouwanttorespecttheiropinionsaswell.

There'sclearlyalsoasenseofwhethersomeofthesedecisionsareperhapscausingmoreharmthangood.So,arethereasetofprinciplesthatneedtobeobeyedaswestartdeployingthesesystemsinsociety?Inadditiontointerpretability.IknowthatourpanelshavebeenputtogethertothinkaboutaseriesofprinciplesbywhichAIsystemscouldbedeployedinsociety,butthis

Talk Session 1 - AI in Healthcare Page 26 of 30

might...Theotherthoughtthen,isthislikewhenwedohumansubjectexperimentandwehavetogothroughourinstitutionalreviewboardandsoforth,isthereanAIreviewboardthatwouldneedtosay,"Wecertifythisisinterpretable,itdoesallthesethings,"andsonowwecandeploythem?

EricHorvitz: I'lladdathirdone,whichis,whenyouhaveasystemthat'sinhealthcare,butinotherareasaswell,safetycriticalareas,whenyouhaveasystemthat'sdoingreasoninganddecisionmaking,it'sactuallyinfluencingtheworldthatit'sstudying,thatitwasbuilttomakedecisionsabout.Itsinfluenceisinripplesofitsowneffectsbeingunderstoodandingestedbythemodelsinawaythatmakessense.Icangiveyouexamplesofwhereitisnot.That'sathirdkindofprinciple.You'regettingattheideaofwhatIwouldcalllooselyorsummarizedasbestpracticesfordifferentsectorswhenitcomestomachinelearningandintelligence.Thisissomethingthatanumberofpeoplearethinkingabout.There'saneffortcalledthePartnershipinAItobenefitpeopleinsociety.

ThePartnershipinAI,PAIforshort,was,Iwouldsayco-authoredbyresearchersatthemajorITcompaniesstartingwith...ItwasMicrosoft,Facebook,Amazon,Google,andIBM,andthenApplejoinedupwiththisgroup.It'llbeabiggergroupsoon,buttheideawas,companiesgettogetherwithnon-profits,withcivillibertiesgroups,tocreateanon-profitthatwouldbearrayedaroundbestpracticesforthefieldmoreglobally,butalsobysector.ThesearethekindsofquestionsthatIthinkwillbeaskedforwhichpartners,andcolleagues,andacademicswillbereliedupontohelpanswerthroughworkshopsandwhitepapers,andsoon.SoIthinkthisisgoingtobehappeningasearlyasthiscomingyearintermsofthisbeingstoodup.Soit'sareallygoodquestionastowhatarebestpracticeswhenitcomestoAIenteringanarealikehealthcareandwhatitmeans.

Myothercommentwasonthetrustworthinessandsoon,isthatinmanycases,humandecisionmakingissopoorinhealthcare,evenbyexperts,thatthere'sreasontotrustevenaclassicallyvalidatedtestandtrainvalidationthatshowshowpowerfuladata-centricmodelcanbeinmakingaprediction.It'softenthecasethattheydon'thavetoexplainthingsnecessarilytoendusersorpatients,orevensingledoctors,butthatexpertslookingatthemodelshouldbeabletoinspectandunderstandwhat'shappening.There'sadramaticcasethatRich[inaudible01:21:32]talksabout,about...Itwaspublished1996AIjournal,GregCooperleadthestudyatPittsburgh,ofpredictingdeath,patientsathighriskofdeath,whohavebeendiagnosedwithpneumonia.

Itturnsoutthatmostpatientsdofinewithpneumonia,theygetoverit,butwehearaboutpeopleevenintheir50's,and40's,andyoungpeople,dyingafteraboutofpneumonia.SoaclassifierwasbuilttotrytopredictwhichpatientsshouldgototheICUandgetspecialcareimmediatelyversusbesenthomewithsomeantibiotics,whichisastandardprocedure.Whenthemodelwaslookedat,ithadagreatAUC,itperformedwell,itwasfabulous,everyonewasexcited.Buttheywerelookingatthismodelandsomeonenoticedlookingatthis,the

Talk Session 1 - AI in Healthcare Page 27 of 30

rulesthatcomeoutofthemodel,thatifapatienthasasthma,they'relowriskfordyingfrompneumonia.Anddoctorssaid,"Well,that'skindoffunny,Idon'tbuythat.WhywouldthatbeinthelargedatafromPittsburghhospitals?"

Itturnedoutthatwasonlyseenbecauseitwasatransparent,scrutinizable,linearmodel,orelsehewouldn'thaveseenthatrightaway,poppingoutofthedataset.Itturnedoutthatthereasonthatwasinthemodel,andit'sascaryexample,andit'swellworthusthinkingdeeplyaboutthis.Thereasonthatthaterroneousrule,whichwouldhavekilledpatientsifitwasinpracticeinthatmodelwasbecausethosepatientsthathadasthmawereconsideredsocriticalandsohighrisk,theywereremovedbeforethedataanalyticpipelinecouldcapturetheminthisdataset.Sothedatasetitselfendsupwithablindspotinaveryimportantway,thatwasonlyseenthroughexpertscrutinizingandtransparencyofthelevelthatithad.Yeah.

Speaker17: [inaudible01:23:38]

EricHorvitz: Oh,sorry.Yourhandwasn'tveryhigh,butyouhavethemicrophonethough,soyouhavethepower.

Speaker13: [inaudible01:23:45]afewthingsanswering[inaudible01:23:48]ontheprinciplesthatweshouldlookat.InaEuropeanprojectwhereI'minvolved,wearetalkingabouttheartprinciplesofAI,ART,accountability,responsibility,andtransparency.Inthatresponsibilityrefersmostlytoourselves,whatisourresponsibilityonnotonlywhatwecando,butwhywearedoingitandwhat'stheimpactonit.Accountabilityandtransparencygokindoftogetherasmakingitmoretransparent,you'llgetmore.Ifittranslatedtothe[inaudible01:24:23]thatissomethingwhichwehavebeendiscussinginIEEEinitiativeforethicallyaligneddesign,whichIbelieveseveralofyouareinvolvedaswell.

Therewehavelookedaswell,it'snotonlythataswearedevelopingmechanismsforbetterdiagnosticsorautomaticdiagnostics,andsoon,thatshouldbegoingtogetherwithalsoachangeonthewayweperformmedicine.Also,arewetrainourmedicalpractitionerssoasmachinesaremoreandmoredoingbetterdiagnosisthanpeople,theeducationofourdoctorsshouldshiftnotsomuchonthe,let'ssay,thedetailsofthediagnostic,butmuchmoreonthecommunicating.Soitismuchmorethanjustthealgorithmicapproach,wehavetolookatitinamuchbroadersocietalsetup.[inaudible01:25:31],butontheotherhand,alsoalltheresponsibilitywehaveastheoneswhoaredevelopingthosealgorithmstocreatethesocietyinwhichthosealgorithmsarereallyappliedforgoodandnotjusttoupsetthesetup.

EricHorvitz: That'sfabulous.TheART,isthatacronym,thatinitiative,amongAIresearchersandscientists?OristhatpartoftheEuropeanParliamentaryguidelinesandsoon?

Talk Session 1 - AI in Healthcare Page 28 of 30

Speaker13: [inaudible01:26:05]theParliament,butwearestarting[inaudible01:26:10].That'swherewearetalkingabouttheART.

EricHorvitz: Yeah.

Speaker13: [inaudible01:26:21]acronym.

EricHorvitz: Partofthis...It'sabeautifulacronym,ittranslateswellacrossthelanguages.It'sveryinterestingtothinkaboutevenjustgettingsensitivetothestories.Clinicalmedicineisallaboutbeingexposedtothesestories,tothesesituations,versusthefirsttwoyearswelookatdatasets,andsymptomtables,andsoon.Theactualexperienceswiththesestories,likethepneumoniastory,thestoryaboutevenbeingsensitivetoaskthequestion,ifthewholeUnitedStates,forexample,isarrayedatminimizingpenaltiesfor30dayreadmissionrates,whathappenstopatientswhocomeinat45daysevensicker?

Whywasn'tthatonthemapthatifyouhaveasimplerulethatyou'reoptimizing,youmightbepushingthepoorlyfilledballoononthissideandhaveitpoponthisside.Weneedtobethinkinggloballyaboutevenhowthesesystemscanbegamed,howtheycancreatinggamingsituations,howadversarialscanattackthem.Forexample,thisisasimpleadversarialmodel,toreduceyourpenaltiesinahospital,I'llhaveyoucomebackonthe33rdday.Justkeeponusingtheoxygenathome.Yeah?Oh,I'msorry,you'renext,yeah.Thenwe'llpassituphere.Yeah.

YevgeniyV.: Okay.Ihaveacoupleofquickcomments.Thefirstcommentisactuallytiedintoanumberofexamplesthatyouhavemade,whichisinmodelingthecommonassumptionyoumakeasaclosedworldassumption.Onceyoudeployamodeloranythingthat'sbasedonthemodelintheworld,theworldisnolongerclosedfromaperspectiveofthismodel.Soalotoftheseexamplesthatyoumentionedarereallyviolationsoftheclosedworldassumption.Anotheroneinsecurity,averysortofintuitiveexample.Let'ssayyoudeployareallygoodsecurityapproachinoneplace,sotheattackerscouldattacksomeotherplacesasaconsequencebecausenowtheotherplaceiseasiertoattack.Thesearejustviolationsoftheclosedworldassumption.

EricHorvitz: Yeah.

YevgeniyV.: Anotherquickcommentisabouttransparency.Sothere'ssortofmultiplemodesoftransparencyandIthinkit'sdistinctfrominterpretability.Sointerpretabilityis,firstofall,interpretabletowhom,atwhatlevelofeducation?Buttransparencyisabroaderissue.Forexample,opensource,creatingthingsthatareactuallyopensource,makesitpotentiallytransparentbecauseitcanbeinspectedandevaluatedbyotherpeople,evenifit'snotnecessarilyinterpretable.

Talk Session 1 - AI in Healthcare Page 29 of 30

EricHorvitz: OnecommentI'lljustmakeisthatyousaidthatallthese...[crosstalk01:29:10]Thatwastheethicalthingtodo.[inaudible01:29:14]justthiscommentforasecondhere.Ilovethatyousaidthatalltheseexamplesviolatetheclosedworldassumption.Iwasgoingtosay,therealityoftheopenworldviolatestheclosedworldassumption.I'msorry,goahead.

Speaker18: AnotherexamplethatIthink[inaudible01:29:33],gettingbacktoyourpresentationonthewandering,Alzheimer'spatients,theyforgetalot,sotheideaistohavethiswatchlastingfor21daysbeforeithastobecharged.Why?Iwouldsay,actually,myexperience,mymotherhasAlzheimer,isifthere'ssomethingthatshedoeseveryday,shewillsticktoit.Ifitlastsmorethanafewdays,shewillnothaveacluewhattodoanymore.Sohavingawatchthatyouhavetorechargeeverydayisbetterthanonethatyoudo21days.

You'resolvingaproblemwhichIthink,"Well,isitreallyaproblem?"Andwe'redoingthatfartoooften.Wehaveanideaabout,what'sthesolution?Withoutactuallylookingat,whatistherealworldproblem?Notwhat'sourproblem,butdowethinkisaproblem,butwhat'stheactualproblem?Thenlookingatthatandseewhatwehavetodotosolvethatproblem.Thisisoneofthosetinythingsandtherearemany,manyofusthinkwherewethinkwe'redoingverycleverthings,andthenwe'reactuallynot.Gettingbackintointerpretability,wecanactuallydosomeveryclevermachinelearning,wedon'thavetoexplainthat.Anexpertdoctorisnotgoingtoexplainexactlywhathedidtoapatient,he[inaudible01:30:58]givesomereasonsthatapatientwillaccept,thatshouldbeacceptable,nottrue,itshouldbesomethingwhichissomethingthatactuallyisclosetowhathedid,butnotwhathedid.Soit'sadifferenttypeofproblem.

EricHorvitz: Yeah.Youcouldimaginethoughthatthereareapproachesto...Backtothisquestionabouttrustworthinessandsoon,whereinmedicineorcriminaljustice,perthismorning'slecture,thatatleastadatasetisavailabletobescrutinizedbyexpertsuponrequest,andtheyshouldhammeronthistosortoffigurethingsout.Evenifitcan'tbeavailabletoeverybodyperadversariesandsoon.Ourlastcommentbecausewehaveacoffeebreakinprocessaswespeak.

Speaker15: Justtofollowuponthatdiscussion,Iactuallydon'tthinkyoucandothisbecauseforthepurposesofmalpracticeandhavingdocumentationofwhatthephysicianactuallydid,youneedtohavethatdecisionmakingprocessandyouneedtohaveappropriatedocumentationofwhatactuallyhappened.Ifyoudon'thavethatdocumentation,thenit'sactuallygoingtobeproblematicforthehealthcareinstitutionsandthephysicianthatdidthisbecausethey'regoingtobesubjecttoallsortsoflawsuitsatthispointbecausethey'renotdocumentingappropriately.

EricHorvitz: Permyunderstandingofmalpractice,itoftengetsdowntobestpractices.Didthisdoctorfollowbestpractices?Ifnot,oftengoestodecisionanalysis,actualcostbenefitanalysisunderuncertainly,soyoucanimaginethataprobabilistic

Talk Session 1 - AI in Healthcare Page 30 of 30

modelbeingusedwouldbeadmitted,anditscharacterization,aseitherabestpracticeorit'scharacterizable.

Speaker15: Yeah.Itwillplayonbestpractice,butMillenniumPharmaceuticalslostthisbattlewhentheytriedtohidesomeoftheirdecisionmakingmethodswhentheyweredoinggeneticbaseddecisionmaking.Whentheytriedtohideit,peoplesaid,"Whatexactlyareyoumakingrecommendationsfor?"Theysaid,"That'sproprietary,we'renotgoingtotellyou,"andthentheyendeduplosingincourt,andthatinformationhadtobemadepublicatthatpoint.

EricHorvitz: Well,thissoundslikeagreatcoffeebreakstorytocontinue.Thankstoallthespeakersandthediscussions.