1

Designing, Using and Evaluating Educational

Games: Challenges, Some Solutions and Future Research

First International Workshop on Story-Telling and Educational Games STEG'08The European Conference on Technology Enhanced Learning

ECTEL 2008, 16- 19 September 2008

Maastricht School of ManagementMaastricht, The Netherlands.Dr. Nalin Sharda, Victoria University, Australia Nalin.Sharda@vu.edu.au http://sci.vu.edu.au/~nalin/

2

Abstract• Educational games can use storytelling as the underlying

model for designing their content and narrative to enhance learning outcomes.

• We need to evaluate their efficacy to maximise the user benefits. However, the field of user experience (UX) evaluation is full of challenges in itself; and evaluating the UX for digital educational games adds many other complexities. This paper presents various aspects involved in developing digital educational games and in evaluating the user experience derived from these.

• Felder Silverman learning style mode is presented to incorporate it into educational games and cater for a range of learners. A story design model called Movement Oriented Design (MOD) is presented to guide the process of creating effective educational narratives.

• A User Experience Research (UXeR) meta-model is presented as the pathway to advance research in this multidisciplinary field.

3

Outline• Introduction

• Content, Story and Game design• Learning Style Models

• Felder Silverman Model• Digital Educational Games

• Three main challenges• Storytelling as a Pedagogical Model

• Movement Oriented Design (MOD)• User Experience

• User Experience Research Model (UXeR)• Conclusions

4

Introduction• Three aspects of Educational Game Design

• Content Design• Story Design• Game Design

Educational Content Design

Game Design

Story Design

User Experience Space

Creative Design Space

Emotions

Plot &

Narrative

Learning outcomes

5

Learning Style Models

• Main Objective: Enhance Learning Outcomes• Many learning style models

• Overlap with personality types given by• Carl Jung, Myer Briggs, Kolb, and Howard

Gardner• A simple learning style model

• Visual learners prefer to use visual aids such as diagrams, photos, and videos.

• Auditory learners prefer to learn by listening to lectures and discussions.

• Kinaesthetic learners prefer to learn by building physical systems or models.

6

Learning Styles: Felder Silverman Model

Input

Processing

5- Learning Dimensions

Meta Learning Style-A: Physical

Meta Learning Style-B: Intellectual

1.

Perception

Sensing: Concrete and practical, considers facts and procedures.

Intuitive: Conceptual and innovative, considers theories and meanings.

2. Input

Visual: Prefer visual representations – pictures, diagrams, flow charts.

Verbal: Prefer written and spoken explanations.

3. Organisation

Deductive: Prefer to proceed from the general to the specific.

Inductive: Prefer to proceed from the specific to the general.

4. Processing

Active: Learn by trying things out, and working with others.

Reflective: Learn by thinking things through, generally working alone.

5. Understanding

Sequential: Linear and orderly, learn in small incremental steps.

Global: Holistic and system wide thinkers, learn in large leaps.

Understanding

Perception

7

Learning Style Models

• Aim for Educational Game Design • Cater for different learning styles• Encourage learners to use alternative

learning styles

• Games provide • Physical learning experience.

• Important to include• Intellectual activities to• Balance and reinforce the learning

• Without interrupting the game experience.

8

Digital Educational Games

• Digital Educational Games (DEGs)• offer exciting and dynamic environments • engage gamers in motivating learning

activities, • inspire them to explore a variety of topics

and tasks.

• Law and Kickmeier-Rust state: characteristics of DEGs include• contribute to knowledge construction • enhance collaborative learning skills

9

Digital Educational Games

• Three main challenges related to developing, using and evaluating DEGs• Personalising learning experience, i.e.

developing DEGs that can provide content and experience to match different learning styles.

• Reusing learning objects created for the same or other DEGs.

• Evaluating DEGs vis-à-vis1. learning outcomes: formal models exist2. user experience: formal models needed

10

Storytelling as a Pedagogical Model• Storytelling

• A universal cultural activity• Can enhance learning outcomes • An effective model for developing DEGs

• As per Levi-Strauss, an anthropologists• our brain uses a story oriented structure

• Storytelling is powerful. • Because it links with a basic human need: • Creates emotional engagement and

movement

11

Movement Oriented Design

• Movement Oriented Design (MOD) • a framework for creating contextualized stories • connect with and impact learners’ cognitive

absorption• MOD methodology is underpinned by the

concept that• all multimedia systems are story telling

artifacts• MOD bridges the semantic gap between

• multimedia author’s thought process and • programming metaphors for multimedia design

12

Movement Oriented Design

•Based on the concept that •every story must have a beginning, middle and an end.

•The core - Movement: •a micro story with its own beginning, middle and end.

Begin Middle

End

Begin

Middle

End

Story or Story Unit

Movement

Begin Middle

End

Begin Middle

End

Begin Middle

End

Begin Middle

End

13

Movement Oriented Design

Negative Emotional Charge

Knowledge

Positive Emotional Charge

Moving Emotional Charge for effective knowledge transmission

• McKee suggests that we move emotional charge: • +ve to –ve • -ve to +ve• To keep the viewer emotionally engaged

14

User Experience

• Hassenzahl separated attributes into • Pragmatic Attributes (Usability ) -of practical

matters • Hedonic Attributes (UX) -devoted to pleasure

• Hassenzahl and Tractinsky: • UX is a strange phenomenon: • readily adopted by (HCI) community • at the same time critiqued for being vague,

elusive, ephemeral. • Consider three aspects

• The experiential: dynamic, complex, unique, situated, temporally-bounded.

• Beyond the instrumental: holistic, aesthetic, and hedonic.

• Emotional: subjective, positive, antecedents & consequences.

15

User Experience

• Mutlu states: • “Current theories of emotions fail to explain

the relationship between experience and emotions”

• User Experience Research (UXeR) model • Aims for UXeR model

• Define cognitive and behavioral science terms in the UX context

• As psychologist seem to give different interpretations

• Link ephemeral concepts to physical actions in a layered manner.

16

User Experience Research ModelExperience

Emotions

Feelings (Thoughts)

Behaviour (Actions)

Systems (Components)

Level 5

Level 4

Level 3

Level 2

Level 1

Mind

Interfaces

17

User Experience Research Model

L5: Experience : most ephemeral concept. L4: Emotions: play a central role in any user

experience. L3: Feelings and thoughts produce emotional

responses; emotional feelings (happy, sad), and not physical feeling (hot, cold).

Mind provides a link between thoughts and behaviours.* Hromek : “…mind becomes body through a multidirectional flow of information throughout the whole organism”* Conscious and the subconscious mind

L2: Behaviours are collections of actions. Even toys and computer games have behaviours.

Interfaces link to:L1: System and components, including user

interfaces (physical or screen based) needed to perform actions and tasks.

Experience

Emotions

Feelings (Thoughts)

Behaviour (Actions)

Systems (Components)

Level 5

Level 4

Level 3

Level 2

Level 1

Mind

Interfaces

18

User Experience

• Generic UXeR equations• Provide a starting point

• Further work required• Fully define UX equations using

appropriate mathematical techniques.

• Validate the UX equations using empirical data.

• Verify the results obtained from UX equations.

• Develop measurement techniques based on usability research.

• Adapt the UXeR model for specific application domains.

)(1 SyfBe , where ,...),( 21 CCSy (1)

)(2 BefFe , where ,...),( 21 AABe (2)

)(3 FefEm , where ,...),( 21 TTFe (3)

)(4 EmfUX (4)

))))(((( 1234 SyffffUX (5)

19

Conclusions • To design effective digital educational games

(DEGs), we need models for: • Content, Story and Game design.

• To check the effectiveness of the game, we need to measure:• Learning outcomes• User Expereince

• Felder Silverman Model provides • understanding of different learning styles.

• Movement Oriented Design (MOD) • provided a model for designing the narrative of

story-based DEGs. • User Experience Research (UXeR) model

• is a seminal model for formal treatment of UX. • Therefore, further research is required

• to evaluate UX for educational games.

20

• Thanks.

• Any questions please.

21

References1. Desmet, P.M.A: Measuring emotion; development and application of an

instrument to measure emotional responses to products. In M.A. Blythe, A.F. Monk, K. Overbeeke, & P.C. Wright (Eds.), Funology: from usability to enjoyment, pp. 111-123, Kluwer Aca-demic Publishers. (2003)

2. Mutlu, B.D.: The Chaotic Nature of Human Experience: Insights on the Subject Matter of Design towards Establishing a Science of Design. M Des Thesis. Carnegie Mellon Univer-sity. USA. (2004)

3. Law, E.L.C., Hornbaek, K.: Measures of Usability and User Expereince (UX): Correla-tiona and Confusion, Proceeedings of the COST294 Workshop: Towards a UX Manifesto, Lancaster, UK, 3 September. (2007)

4. Sharda, N.: Authoring Educational Multimedia Content Using Learning Styles and Story Telling Principles, ACM Workshop on Educational Multimedia and Multimedia Educa-tion (in conjunction with ACM Multimedia 2007), 28 September, Augsburg, Germany. (2007)

5. Schroeder, C.: New Students – New Learning Styles. http://www.virtualschool.edu /mon/Academia/KierseyLearningStyles.html. Accessed May 2007.

6. Learning Styles. http://www.ldpride.net/learningstyles.MI.htm Accessed May 2007.

7. Felder, R. M. and Silverman, L. K. Learning and Teaching Styles in Engineering Educa-tion, Engineering Education, 78(7), pp674-681. (1988).

22

References8. Law, E.L.C., Kickmeier-Rust, M.D.: Addressing Challenges for Immersive

Digital Educa-tional Games, Proceedings of the CHI-2008 Conference, Florence, Italy, April 5-10. (2008)

9. Lévi-Strauss, C.: Myth and Meaning: Cracking the Code of Culture, Schocken Books, New York. (1995)

10. Heo, H.: Story telling and retelling as narrative inquiry in cyber learning environments, In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 374-378). Perth, 5-8 December. (2004)

11. Sharda, N. Movement Oriented Design: A New Paradigm for Multimedia Design, Interna-tional Journal of Lateral Computing, Vol.1, No. 1, May 2005, pp 7-14. (2005)

12. McKee, R.: Story: Substance, Structure, Style and the Principles of Screenwriting, Methuen. (1998)

13. Sharda, N.: Applying Movement Oriented Design to Create Educational Stories, Interna-tional Journal of Learning, Vol.13, Issue 12, pp.177-184. (2006)

14. Anderson, L. W. et al: A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon. (2000).

23

References15. Capota, K., van Hout, M. & van der Geest. T.: Measuring the emotional

impact of web-sites: a study on combining a dimensional and discrete emotion approach in measuring visual appeal of university websites, August 2007, Proceedings of DPPI '07. (2007)

16. Hassenzahl, M. & Tractinsky, N.: User experience - a research agenda. Behaviour & Information Technology, No 25, pp. 91-97. (2006)

17. Ben-Bassat, T., Meyer, J. & Tractinsky, N.: Economic and Subjective Measures of the Perceived Value of Aesthetics and Usability ACM Transactions on Computer-Human In-teraction, Vol. 13, No. 2, June 2006, pp. 210–234. (2006)

18. Hassenzahl, M.: Hedonic and Ergonomic Quality Aspects Determine a Software’s Appeal. Presented at ACM CHI conference, April 2000. (2000)

19. Hromek, R.: Emotional coaching: a practical programme to support young people, Paul Chapman Publishing, London. (2007)

20. Murphy, J.: The Power of your subconscious mind, Simon & Schuster, London. (1988)

21. Jsselsteijn, W.I. et. al.: Characterising and Measuring User Experiences in Digital Games, Proceedings of the ACE Conference, June 13-15, 2007, Salzburg, Austria. (2007).