using an assessment engine for creating flexible educational games
TRANSCRIPT
8th European Conference on Games Based Learning An Assessment Engine Educators as Editors of their Serious Games Assessment
UWS Research Student Conference 2015
Using an Assessment Engine for Creating Flexible Educational GamesYaelle [email protected]
1
OutlineEngAGe: An Engine for Assessment in Games MethodologyPreliminary results on Usability and UsefulnessConclusion
29 April 20152/14UWS Annual Research Student ConferenceEngAGeMethodologyResultsConclusion
2
Engage An Engine for Assessment in Games:29 April 2015UWS Annual Research Student Conference
3/14EngAGeMethodologyResultsConclusion
3
Story of a serious game29 April 2015UWS Annual Research Student Conference4/14EngAGeMethodologyResultsConclusion
4
29 April 2015UWS Annual Research Student Conference
In-game Assessment and FeedbackTime consumingCommunication (Teacher - Developer)Black Box
Problems to addressTeacherLittle trust in a games assessmentNeed to feel in controlImprovisation and AdaptationLearning analyticsLittle documentationBeyond reach of most teachers
5/14EngAGeMethodologyResultsConclusion
5
An Assessment Engine: EngAGe29 April 2015Domain Specific Language for configuration of the assessment
Learning Analytics dashboard
Authoring Tool for teachers
UWS Annual Research Student Conference
6/14EngAGeMethodologyResultsConclusion
6
EVALUATIONMethodology and results29 April 2015UWS Annual Research Student Conference7/14
EngAGeMethodologyResultsConclusion
7
MethodologyTarget users:
Aspects to evaluate:
Generalisation UsabilityUsefulness
29 April 2015UWS Annual Research Student Conference8/14
DevelopersEducators(Players)
EngAGeMethodologyResultsConclusion
8
Methodology29 April 2015UWS Annual Research Student Conference9/14TutorialConfiguration file of the gameCalls to web services
Post questionnaireUsefulness (Likert scales)Usability (SUS)
EngAGeMethodologyResultsConclusion
9
Preliminary resultsUsefulness29 April 2015UWS Annual Research Student Conference10/14
EngAGeMethodologyResultsConclusion
10
Preliminary resultsUsability: System Usability Scale (SUS) by Brooke (1996)10 questions to be graded between Strongly Agree and Strongly disagree.
29 April 2015UWS Annual Research Student Conference11/14pilot study tutorial study overallI think I would like to use this system in the future when developing serious games71.410085.7I found the system unnecessarily complex28.6014.3I thought the system was easy to use14.371.442.9I think that I would need support to be able to use this system57.1028.6I found the various functions in this system were well integrated28.671.450I thought there was too much inconsistency in this system000I would imagine that most serious game developers would learn to use this system very quickly42.971.457.1I found the system very cumbersome to use28.6014.3I felt very confident using the system14.385.750I needed to learn a lot of things before I could get going with this system57.1028.6
Percentage of participants agreeing or strongly agreeing with the statementsEngAGeMethodologyResultsConclusion
11
Preliminary resultsPilot study (n=7) OKMean = 55Median = 50
Tutorial study (n=7) GoodMean = 73.5Median = 72.5
Total score (n=14) GoodMean : 64.2Median: 7029 April 2015UWS Annual Research Student Conference12/14
Bangor, A., Kortum, P. T., and Miller, J. T. (2008) An empirical evaluation of the system usability scale. Intl. Journal of HumanComputer Interaction. Vol.24(6), pp.574-594.EngAGeMethodologyResultsConclusion
12
ConclusionsEngAGe for developers:Guidelines for assessment in gamesEfficient and Effective toolFairly easy to use
EngAGe for educators:No more Black Box issueAccess to gameplay data (Learning Analytics)
Future workMore participants for tutorial Include EngAGe in various on-going projectsExperiment with educators
29 April 2015UWS Annual Research Student Conference13/14EngAGeMethodologyResultsConclusion
13
Any Question?Thank you!29 April 2015UWS Annual Research Student Conference14
Interested?Email me: [email protected]
14