Games for health: do they actually

work? Baltasar Fernandez-Manjon

balta@fdi.ucm.es , @BaltaFMe-UCM Research Group , www.e-ucm.es

Festival for Digital Health: Fun and GamesUniversity College London, 22/02/2016

Realising an Applied Gaming Eco-System

Do games for health actually Works?In many cases, we are not fully sure about it

- Usually, no full formal evaluation has been carried out- Limited number of users

- Formal evaluation could be as expensive as creating the game (or even more expensive)

Example: First Aid - CPR game

• Collaboration with Centro de Tecnologias Educativas de Aragon, Spain• Identify a cardiac arrest and teach how to do a cardiopulmonary

resuscitation• Oriented to middle and high school students• Includes how to use a automatic defribilator• Tested in 4 schools with 340 students

Marchiori EJ, Ferrer G, Fernández-Manjón B, Povar Marco J, Suberviola González JF, Giménez Valverde A. Video-game instruction in basic life support maneuvers. Emergencias. 2012;24:433-7.

Available at http://first-aid-game.e-ucm.es

Evaluation Simultaneous sessions with same timeOne with gameOther with 2 emergency medical doctors with a medical mannequin and an automatic defibrillator

First aid - CPR game ResultsPre-test and post-test for evaluating knowlege

game

Example: Surgical Safety Checklist

8http://www.who.int/patientsafety/safesurgery/en/index.html

9

10

When implementation leaders did not explain why or show how the checklist should be used, staff neither understood the rationale behind implementation nor were they adequately prepared to use the checklist, leading to frustration, disinterest, and eventual abandonment despite a hospital-wide mandate

Conley, et at (2011). Effective surgical safety checklist implementation. Journal of the American College of Surgeons, 212(5), 873–9

We created the Checklist Game• Raise awareness about the checklist• Learn how to apply it properly and consequences of not applying it • Let practice it application in a free-risk environmentIn cooperation with UCM (Surgical Department), Hospital Doce de Octubre, LCS-

MGH

http://sourceforge.net/projects/e-adventure/files/games/checklist/

Checklist game• Formative evaluation at UCM and MGH/Harvard• Final evaluation at 5 hospitals in Madrid (Clinico, 12 de Octubre, Santa

Cristina, Puerta de Hierro)• Good results (but no formal results yet…)

Methodologies for game development

Torrente et at (2014) Development of Game-Like Simulations for Procedural Knowledge in Heathcare Education. IEEE Transactions on Learning Tecnologies. 7(1), 69-82

USERS

Formal evaluation pre-post• Formal evaluation of games is very

complex and expensive• Pre-test• Post-test

• Very few games have been formally probed to be effective

• Similar results with Learning Analytics than with pre-post test?

14

Can we use Learning Analytics for formal evaluation of games?

Formal evaluation of games from the analysis of the user (interaction) data?

RAGE: creating the Learning Analytics infrastructure H2020 RAGE project will simplify the process of SG creation with ready to use assets• Game trackers• LA server infrastructure• Standards support (e.g. xAPI)

Using interaction data for assessment • Evaluating the game• Discovering user problems

Realising an Applied Gaming Eco-System

H2020 Beaconing project• BEACONING stands for ‘Breaking Educational Barriers with

Contextualised, Pervasive and Gameful Learning’

• Global goal is learning ‘anytime anywhere’• Exploitation of technologies for contextual pervasive games and use of

gamification techniques• Problem based approach to learning• Enriching the Learning Analytics data model with the contextual,

geolocalized and accessibility information

Educational (medical) games challenges• Identification of more large success cases • including scientific and formal evaluation• taking into account current technical infrastructure

• Scalability and maintainability • reducing initial cost and TCO

• Taking into account curriculum and involving educators

• Sharing the actual game (open code)• Including data-based evaluation into games

• Learning analytics

19

Thank You!Gracias!

¿Questions?• Mail: balta@fdi.ucm.es• Twitter: @BaltaFM

• GScholar:  https://scholar.google.es/citations?user=eNJxjcwAAAAJ&hl=en&oi=ao

• ResearchGate:  www.researchgate.net/profile/Baltasar_Fernandez-Manjon• Slideshare: http://www.slideshare.net/BaltasarFernandezManjon