gamificaon and analy’cs in educaon - uab
TRANSCRIPT
![Page 1: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/1.jpg)
Gamifica'onandAnaly'csinEduca'on
JamesH.Willig,MD,MSPH
AssociateProfessorDepartmentofMedicine,DivisionofInfec'ousDiseases
AssistantDeanofClinicalEduca'onUniversityofAlabamaatBirminghamSchoolofMedicine
![Page 2: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/2.jpg)
Mapfortoday…
• Twothreadscoalesce– ACGMEchanges– Gamifica'on
• Firstversion,firstgame
• Toolsforbuildingbears…andthenow.
![Page 3: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/3.jpg)
2003&2011ACGMEchanges:Effectoneduca'on
• Somestudieshavesuggestedthatsomeeduca'onaloutcomeshavebeennega'velyimpactedbythedutyhourstandards.
• Othershavesuggestedthatthedutyhourstandardsnega'velyimpactresidenteduca'oninthefollowingways:– Fewerandmorenega'veclinicalexperiences– Decreasedinstandardizedtestscores– Decreasedpublica'onandpar'cipa'oninacademicmedicine– DecreasedteachingbyaSendingphysicians.
![Page 4: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/4.jpg)
Onaday,notunlikethisoneinTHservice….
• Conversa'onaboutteachingundernewACGMEworkhours
• Wonderedabout– Hadthelimitedworkhoursalteredourteaching?– Howhadweadapted?Werethereadverseeffects?– Whotrainedus,totraininless'me?
• Howdoyoutacklethe“stackedtomorrows”problem?
![Page 5: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/5.jpg)
2011ACGMEchanges:Residentfocusgroups
• “Imean…youhavethesameamountofworkwithless'metoputintoit.Something’sgottogo,andforthemostpartIthinkitistheteachingaspectofthingsjustsortofgetslostintheshuffle.”
• Ifindthatthethirdyearresidentshavemuchless'metoteachthemedicalstudentsandtoteachtheinternsthantheyusedtohave.
• “Attheendofourinternyear,Imean,minewasaliSledifferent,butprobablyeveryonehadbetween20and40lines,youknow,centrallines.Andnowwe'retalkingaboutinternswhohaven'tdonefive–”
![Page 6: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/6.jpg)
SecondthreadàThemorethingschange…
Rounds 2001 Rounds Now
![Page 7: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/7.jpg)
Gamifica'on
• Theuseofgameelementsinnon-gameapplica'onstoimproveuserengagementandexperience.
• Todate,gamifiedapplica'onshavebeenusedinavarietyofsocial,economic,andeduca'onalenvironments.
![Page 8: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/8.jpg)
• CouldweapplytheelementsofGamifica'ontoGraduateMedicalEduca'on?
![Page 9: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/9.jpg)
Mapfortoday…
• Twothreadscoalesce– ACGMEchanges– Gamifica'on
• Firstversion,firstgame
• Toolsforbuildingbears…andthenow
![Page 10: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/10.jpg)
Kaizen
• Kaizen:Japanesemanagementconceptforincremental(gradual,con'nuous)change(improvement)
• Kai:change;Zen:tobecomegood
• Lifephilosophy,underpinsmanymanagementconcepts
• Lifelonglearning!
![Page 11: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/11.jpg)
Somecharacteris'csofgames
• Goal– Clearlydefinedoutcomeplayerscanworktoachieve
• Rules– Rulesarelimita'onsonhowplayerscanachievethegoal
• Feedbacksystem– Playerscantrackprogress;iden'fyhowclosetheyaretogoal
• Voluntarypar'cipa'on– Playersknowandwillinglyacceptgamegoal,rulesandfeedbacksystem
![Page 12: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/12.jpg)
Gamifica'onandrewardsystems
• Commonlyused,mosteffec'vewhencombinesbothintrinsicandextrinsicrewardstoincreasemo'va'onandengagement.
• Intrinsicrewards– Self-efficacy:increasedknowledge,acquisi'onofanewskill– Personalchallenge:problem-solving– Socializa'on:collabora'onwithcolleagues,socialinterac'on
• Extrinsicrewards– Achievements:points,badges,levels– Reputa'on:statuswithinthecommunityofusers
![Page 13: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/13.jpg)
Content
• Conciseques'ons– Clearteachingpoint,focused– Keypointsthatyoubelievemustbetaught– Humor,mnemonics–anythingthathelpsusremember!
• ImagesupportfromtheVisualDx– www.visualdx.com
• InternalMedicineResidencyProgram– Integratedinto2011-2012academicyear;endofyearawards– Threeseasons(8,6,6weeks)eachincludesIMplusthree
subspecial'es
![Page 14: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/14.jpg)
Badgesandscoring
• Correctques'ons– Eachcorrectansweraddstoyourscore
• Timeques'ons– Doublepointswithin'melimit
• Hotstreaks– Numberofoverallques'onscorrectinarow
• Marathons– Numberofconsecu'vedaysdoingques'ons
![Page 15: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/15.jpg)
• Web-based
• Alias
• Rank
![Page 16: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/16.jpg)
RankSystem:Increasedscore,increasedrank
Yourknowledgefesters,mul'pliesexponen'ally,disseminateitonthewards!
Hot?Cold?Acognoscen'caresnot!
![Page 17: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/17.jpg)
• Iden'fy&enlistgreateducators
• Shortfocusedques'ons,followedbyconciseexplana'onkeypoint
• Humorousifpossible
• Twoperday–aimfor5minutes
• Availabletodowhenever,accumulate
![Page 18: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/18.jpg)
• Teams
• No'fica'ons
• Scoringsystems
![Page 19: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/19.jpg)
Scoring:HotstreakandMarathonSystems
HotstreakBadge:Likeavola'lemixtureofmardigras,alcohol,andaurethra,youareonfire!
MarathonBadge:Likeanovernightinternlookingatbreakfastbagels,nothingcanstopyou!
![Page 20: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/20.jpg)
FirstgameinIMGMEprogram:Howdidwedo?
• Round1:– 92residents;71%allques'onsanswered
• Round2:– 90residents;69%allques'onsanswered
• Round3:– 55residents;66%allques'onsanswered
• Total16,427ques'onsanswered
![Page 21: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/21.jpg)
53.1%answered5:01pmand7:59am
28.8%doneonSaturdaysandSundays
![Page 22: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/22.jpg)
No'fica'onEffect:“Fanningtheflames”
![Page 23: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/23.jpg)
Univariate Mul'variablePlayerClass
PGY1
PGY2
PGY3
1.0
1.99(0.80,4.97)
2.88(1.04,8.02)
1.0
1.54(0.49,4.79)
2.17(0.59,7.94)DebutSegment
UAB1.1
UAB1.21
Huntsville1.2
1.0
21.65(2.62,179.25)
0.60(0.16,2.30)
1.0
12.69(1.34,120.15)
1.33(0.29,6.15)Averageperformance2
1.39(1.04,1.86) 1.31(0.84,2.05)
Percentcompleted2
0.74(0.65,0.84) 0.81(0.69,0.95)
Predominantplaystyle3
Daily
Other
1.0
3.01(1.26,7.16)
1.0
2.82(0.86,9.28)
PlayerASri'onaverdebutsegment
![Page 24: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/24.jpg)
Reten'onofKnowledge
Total TotalPlayers;AnswerPairs
Ini'alpercentcorrect1
Subsequentpercentcorrect1
MeanChange P-value2
Overall 50;1,046 40.5 52.3 +11.9 <0.0001Round1
8/20-10/13/1239;546 37.8 47.9 +10.2 0.0008
Round2
1/14–2/26/201232;500 46.2 63.6 +17.4 <0.0001
![Page 25: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/25.jpg)
Mapfortoday…
• Twothreadscoalesce– ACGMEchanges– Gamifica'on
• Firstversion,firstgame
• Toolsforbuildingbears…andthenow
![Page 26: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/26.jpg)
Sothenofcourse,wesetouttobuildabear…
![Page 27: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/27.jpg)
BuildingBears&TheNow
• Techinneedofoverhaul(ThekindnessofstrangersITmodel)– Acquiredinternalfunding
• The“Build-a-Bear”– Or,“Whatwouldhappenifwecouldputthesetoolsinthehandsof
othereducators”
![Page 28: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/28.jpg)
Kaizen-ME
NevinCR,WesyallAO,RodriguezJM,DempseyDM,CherringtonA,RoyB,PatelM,WilligJH.Gamifica'onasatoolforenhancinggraduatemedicaleduca'on.PostgradMedJ.2014
![Page 29: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/29.jpg)
• Enhancedservers,redonefrontend
• Teamlanding&communica'onspage
![Page 30: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/30.jpg)
GameManager:The“Build-a-Bear”
• Gamemanagersetstheirgameparameters:#players,#teams,badges,scoring,ques'ons,topics,etc.
![Page 31: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/31.jpg)
• Sharedques'ons
• Timedornot
![Page 32: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/32.jpg)
• Selectownbadges(level,marathon,hotstreaks)fromsharedpool
![Page 33: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/33.jpg)
Growthensues…
• 87educa'onalgamesacross15disciplines
• Collec'vely5,816dis'nctKaizenusershavebeenregistered
• Intotal,204,722ques'onresponses
![Page 34: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/34.jpg)
Variedpartnerships
• OtherGMEprograms– OBGYN,ED,GeneralSurgery,ENT
• Schoolofbusiness– Connectwithalumniaverevents
• UMEforintroduc'ontoclinicalmedicine– Inlieuofformerweeklyquizzes
• Den'stry– Toiden'fyareasneedingaSen'onduringboardreview
![Page 35: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/35.jpg)
PresidentandCEODr.RickValachovicAmericanDentalEduca'onAssocia'on
MonthlyNewsleSerJune2016
• MichelleRobinson,D.M.D.,M.A.,isalsoenthusias'caboutthepoten'alofdigitalgamestoenhancedentaleduca'on.TheAssociateProfessorandAssociateDeanforHealthInforma'onandBusinessSystemsattheUniversityofAlabamaatBirmingham(UAB)SchoolofDen'stryrecentlydevelopedagameforresidentsinperiodon'cs.UsingtheKaizen(Japanesefor“con'nuousimprovement”)playormdevelopedattheUABmedicalschool,Michellecreatedanonlineknowledge-reviewgametopreparetheresidentsforamockstandardizedexam.Newcontentwasreleasedeachmorning,andstudentswhoplayedthegamedailywererewardedwithextrapoints.Thestudents'regularprac'ceappearstohavepaidoff.Michelletoldmetheprogramdirectorwas“floored”bytheimprovementshesawinexamscoresandplanstousethegameagainnextyear.
![Page 36: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/36.jpg)
Variedpartnerships
• Interna'onaleduca'on– UABEDInterna'onaltrackfordistancetrainingofclinicians
• SchoolofPublicHealth– Pilottoprovecompetenciesacquiredviacurriculumtoaccredi'ng
body
• Educa'onofunderservedpopula'onswithchronicillness– DiabetesClinicPilot;approachtoques'ons
• SchoolofNursing– Usingin6graduateandundergraduateprograms
![Page 37: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/37.jpg)
Reten'onofknowledgeinsecondroundamong151pairedresponsesamongNursingstudents
IniMalaNemptincorrectTotal(%)
SubsequentaNemptcorrect
Total(%)
GLIMMIXModel1
OR(95%CI)
49of151(32%) 25of49(51%) 1.8(1.0-3.4)
![Page 38: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/38.jpg)
FinalExamPerformance–Educa'onalAnaly'cs?
ExamscoreMean±SD
p-value
PlayStyle
Engagedthroughout 52.6±2.3 0.045
Answeredinbursts 51.5±4.0
Justslightlyengaged 45.8±16.2
PERTerMle
Highest 52.4±3.2 0.017
Middle 51.1±3.9
Lowest 44.6±17.6
• PlayStyle– ArrivedatviaK-meanscluster
analysesofdaysplayedandques'onsanswered
• PlayerEfficiencyRa'ng(PER)– Globalreflec'onofplayer
effec'veness– Combinesaccuracy(correct);
consistency(howdeepintoseasonplayed)and'meliness('mefromques'onposttoanswer)
![Page 39: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/39.jpg)
InConclusion
• WehavedonethisforTraininginRigorReproducibilityandTransparencytrainingattheUABCTSAwithgreatengagement
• Gamifica'onprovidesamo'va'onalframeworkwithmanyapproachestoenhanceengagement
• Tomaximizethebenefitsofgamifica'onforourstudents,wemustusetheresul'ngdatatopowereduca'onalanaly'cstouncoverthemosteffec'vestrategiestoenhanceengagement,reten'onofknowledgeandmaximizeperformance
![Page 40: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/40.jpg)
ComingASrac'onsFall2017
![Page 41: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/41.jpg)
NewNaviga'onforGameManagers
![Page 42: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/42.jpg)
FeaturedandShare-ableGames
![Page 43: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/43.jpg)
NewTeamBadges
![Page 44: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/44.jpg)
BossBaSles!
![Page 45: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/45.jpg)
• SuperMarioBrothers
• SpaceInvaders
• MegaMan
• Galaga
• DuckHunt
• Mrs.Pacman
• LegendofZelda
• DigDug Metagame
Kirby
Sonic
Piyall
VirtualFighter
![Page 46: Gamificaon and Analy’cs in Educaon - UAB](https://reader030.vdocuments.mx/reader030/viewer/2022012614/619cfe877930571e2d17b1f4/html5/thumbnails/46.jpg)
Thankyouforyour'meandaSen'on!PleasefeelfreetocontactmeortheFollowingColleaguesformoreinfo:1. JamesWillig:[email protected]
2. DonaldDempsey:[email protected]
3. NancyWingo:[email protected]
4. CathyRoche:[email protected]
5. RyanOutman:[email protected]
6. DavidRedden:[email protected]