experimental game prototyping and play testing using iterative design
DESCRIPTION
Lecture "Experimental Game Prototyping and Play Testing using Iterative Design" by Mirjam P Eladhari, given 2010 at the course International Game Production Studies at Gotland University.TRANSCRIPT
experimental prototyping and play testing using iterative game
design
experimental prototyping and play testing using iterative game
design
2010
Lecture in the course International Game Production Studies I
Mirjam Palosaari EladhariGotland University, Sweden
2010
Lecture in the course International Game Production Studies I
Mirjam Palosaari EladhariGotland University, Sweden
Overview
- Wicked problems and the importance of building
- Types of Prototypes
- Types of Play Tests
- Obtaining Data
- Trade-offs
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Focus of this talk:
experimental prototyping and play testing using iterative game design
how to plan for obtaining data from play tests that can yield answers to specific research questions.
Read more: Design for Results: Considerations for experimental prototyping and play testing using iterative game design by Mirjam P Eladhari and Elina M I Ollila. Article to be published in a special issue of the Journal Simulation and Gaming focusing on game research methods. (Uploaded to course website)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Prototype
A prototype is “played, evaluated, adjusted and played again, allowing the designer or design team to base decisions on the successive iterations or versions of the game.
Iterative design is a cyclic process that alternates between prototyping, play-testing, evaluation, and refinement.”
Salen and Zimmerman (2001)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Early questions
• What types of game play dynamics and game play experiences can a certain, mechanic, feature, approach or method result in?1
• What are the qualities of the play experience, and how can these qualities be documented, analysed, interpreted and explained comprehensively?
• How can it be decided if a certain game play feature results in something valuable, such as a new type of experience, a meaningful experience, or a ‘better experience’ in some other way, and if so, compared to what? Researchers may find themselves in situations where innovative results do not necessarily lend themselves to comparisons.
• What type of data can be useful to obtain in order to explore the research question?
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Beware• Researchers need to, in the design
process, take into account what questions they aim to explore and stay focussed on these throughout the design and implementation work.
• It is easy to fall into a frame of mind where one aims to produce a good game, losing focus of obtaining research material.
• In the process (especially when implementing) take a step back regularly to think about the research question.
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Wicked Problems
‘wicked problem’ is used in social planning to describe problems where every attempt at producing a solution changes the understanding of the problems
(Rittel & Webber, 1973)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Wicked problems in game design
“For a wicked problem such as game design, exploring design space consists of navigating the complex relationships and constraints among individual design features, while at the same [time] discovering or inventing new features and approaches that expand the design space.
All existing games form tiny islands of partially understood regions of design space; all around these islands lays a vast ocean of unexplored potential design space waiting to be brought into existence through the invention of new features and approaches, and mapped out through the hard empirical work of exploring a variety of designs.”
Mateas and Stern (2005)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Build it to understand it
“[...] if game studies is limited to analysing existing games and design spaces, it can be problematic to imagine or theorise about potential game features outside of these design spaces. Models about the nature of games and their features run the risk of being incomplete or wrong, simply because certain design spaces have not yet been explored.”
(Mateas and Stern 2005)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Navigational Aid in the wicked problem space of
game design
- Types of Prototypes
- Types of Play Tests
- Obtaining Data
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
TYPES OF PROTOTYPESTYPES OF PROTOTYPES
Prototyping by acting and showing
• body storming - participants imagine the game and act as though it would exist, in either a real or imaginary place where it could be played. (Burns, Dishman, Verplank, & Lassiter, 1994)
• polished video scenarios, the purpose of which is to show how a concept would work in its natural settings.
FIGURE The player is taking a picture of a person smiling, 2. The smile is mapped to a smiley in a grid that the player needs to complete (i.e., take
pictures with people having similar expressions on their faces as the smileys do)
Early
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Paper (or physical) prototypes
• A physical prototype can at an early stage give pointers to whether a designed game mechanic results in the intended game dynamics when played. A physical prototype is often made of paper mock-ups of the intended game, but can also include physical ‘bits’ such as figurines and tokens.
game-mastering method, where the player was brought to various game situations with the help of a game master, like in pen-and-paper role-playing games.
Middle
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Computationally aided physical prototypes
• In cases where the game mechanics demand higher degrees of computation of significant values it can be useful to add aids for calculation to a paper prototype.
• A common and very simple method is to use excel sheets
Middle
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
The developers of Raptor found that using a table-top was superior for collaborative sketching compared to the interface of traditional PCs. Designers were presented with a sand-box environment where they could shape the geography using hand-gestures. They could also ‘stamp’ game bits into the environment such as cars for a racing game, and attach a ‘camera’ to the object representing the player, giving the user, who had access to a PC client inter- face, their point of view in the geography.
Raptor, a tool for sketching and
prototyping games using a table-top surface.
Raptor: Sketching Games with a Tabletop Computer, by JD Smith, FuturePlay 2010Mirjam P. Eladhari, [email protected] Gotland University, Department of
Game Design, Technology and learning
Software Prototypes
• Software prototypes are quite often thought to be something that is created later in the project, when there is already game design documentation available. However, plenty of fast prototyping tools are available, ranging from general pur- pose tools like Flash to more specialised frameworks.
Late
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Production cycle, example
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
TYPES OF PLAY TESTSTYPES OF PLAY TESTS
ad-hoc test
An ad-hoc test is a quick informal test which requires minimal organisation. It can be as simple as finding a colleague not working on the same project to quickly test some aspect of the prototype on them.
(grab someone in the corridoor- test)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
focus testIn a focus test a group of potential players are probed about their perceptions, opinions, beliefs and attitudes towards the prototype. Questions are asked in an interactive group setting.
(put a group of people from your target group in the same room and ask them your questions)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
expert evaluation
An expert evaluation is the appraisal of a prototype by someone who has the professional training or experience to make an informed judgement on the design.
(Show what you have to an expert, get their advice)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
QA TestThe functional test, or quality
assurance (QA), can be conducted by the production team or by QA-experts. QA is conducted in order capture errors in the functionality of the prototype as well as for balancing game play according to the intended game play experience
(Find the bugs and fix them)Mirjam P. Eladhari, [email protected] Gotland University, Department of
Game Design, Technology and learning
Participatory designParticipatory design workshops are conducted by
the production team and a small number of invited guests, potentially experts in the field or potential players of the game.
The workshops are often intended to aid in balancing the game and to eliminating dysfunctional elements or features. They can also be useful for gathering new ideas for further iterations or for enhancing the quality the prototype according to the goals by specific investigations.
(Have people your trust go through the test with you, and get their advice on your design as you go along)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Guided scenariosIn guided scenarios a test-leader may
use the Wizard of Oz - method to simulate user-interaction. The scenarios designed for the particular prototype are played individually by participants(Wizard of Oz experiment is a research experiment in which subjects interact with a computer system that subjects believe to be autonomous, but which is actually being operated or partially operated by an unseen human being.)
(Game master what you haven’t implemented. If it doesn’t work you save lots of time knowing that.)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Example (4 roles)
The Player, the Game master, the Engine, and the Manual (taking the pictures)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
game-mastered play sessions
In game-mastered play sessions several participants may interact with a prototype, or scenarios designed for the particular prototype, guided by a game master.
Similar to guided scenarios, but later, when the prototype is functional. Ie, no wizard-of-ozzing for functionality.
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
free-form play sessions
In free-form play sessions participants are interacting with the prototype unaided by guiding test-leaders or game-masters.(common end-phase play test. Put the user in front of the screen and see what happens. ...Or send it off to a testing consultant along with your testing script.)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Practical Considerations
Type of play test Who testsTiming in production-cycle
Ad-hoc test
TeamColleaguesConsultants
very early
Participatory design workshops early
Quality Assurance (function-test) medium
Expert evaluations early-medium
Focus test
Players fromtarget group(s)
very early
Guided scenarios early
Game mastered play sessions medium
Free-form play sessions late
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
OBTAINING DATAOBTAINING DATA
Surveys?Surveys can be used to capture
players’ attitudes, get ideas for development and can also be used for a ‘memory check’ - in those cases a player remembers or have forgotten something about the design
The effort required for this method is less than for many other methods for obtaining data.
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Example – Surveys as stand-alone tool
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Interviews?
Interviews can allow for obtaining data that might not arise in surveys.
In some cases it may be useful to add a survey in addition to the interview in order to verify the results and to see if some results were received for ‘pleasing’ the interviewer, that is, give answers based on their interpretation of what they think the test leader wants to hear
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Teach back tasks
‘teach-back tasks’ – players are asked to describe the system they tested to someone who has no prior knowledge of it.
The player ‘does the job’
Can be used both in surveys and interviews.Teach-back tasks are often used in order to assess users’ mental models of
systems (Veer, Wijk, & Felt, 1990; Puerta-Melguizo, Chisalita,& Veer, 2002).
For example, in a test of a prototype where the players’ avatars had different ‘moods’ given by a psychological model, players were asked to explain to a friend how the mood affected what they could do or not do in the game world prototype (Eladhari, 2009).
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Video tape it?
Is it is useful to videotape the interaction?
Possible to later on make verbatim transcriptions of what players said in the test.
Allows for detailed study of body-language, actions and utterances.
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Bio feedback?Is it useful to obtain physical data
from the participants of the test, and if so, what data is most likely to be useful?
Potential physical data include:
• eye-tracking,
• heart-rate variability,
• galvanism and temperature in fingers,
• electroencephalography (EEG) or f MRI,
• posture (posture sensors may be placed on participants’ chairs, see (Plass, Perlin, Nordlinger, & Isbister, 2010)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Data mining?
If the test involves a software prototype, it can be useful to also develop or use a back end-system for capturing events in the form of player-input to the system.
Consider:
• to what extent it is useful to log the player's interactions with the system,
• which types of interactions to log.
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Data mining exampleThe team at Microsoft Labs who conducted the testing of Halo 3 maps showing recurring events mapped to the geography of the game. This was useful for the designers of the game when they iteratively redesigned the environment and the placement of items in it relevant to the game play.
Thompson, C. (2007). Halo 3: How Microsoft Labs invented a new science of play. Wired Magazine (15.09).
Map used at Microsoft Game Labs showing a top down view of the fourth mission of Halo 3. The dots show the locations of player deaths. (thanks to John Hopson and MS game labs for permission to use
picture)
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Process the data
• Is it useful to make a rough coding scheme, for example counting certain types of utterances, behaviours, and/ or expressions or reactions?
• Is it useful to make verbatim transcriptions of the videotaped play tests? If so, is it useful to not only transcribe the utterances but also (or in- stead) make notes of facial expression, body language, laugher etc. to the transcripts?
• Is it useful to trace and describe the actions a player performs in relation to the paper prototype? For example it can be useful to observe and describe in what order players do something, and if there are patterns in how they approach different elements of the prototype.
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Obtaining Data
Data to obtain
Asking players via: Observing players’:
Surveys Interviews
Interaction - Choice of actions- Patterns of actions
Expressions - Facial expressions- Body language- Voice (pitch and tone)- Verbal expressions (think-aloud, conversation, etc)
Physiological responses - Heart rate variability- Gaze (eye tracking)- EEG- Temperature and galvanism in fingers
Capturing of data
(documentation inherent in the actual survey forms)
- Note taking- tape recording- video taping
- Video tapingIn case of a software prototype:- Screen capture of video sequences- Logging of significant actions
- Video-taping- sound recording
- Observation ofmeasured values in real-time and note-taking- Log-reports from devices measuring physiological responses.
Processingof obtained data
- Identification of relevant parts of documentation.- Verbatim transcriptions of relevant parts, and annotation of significant expressions.- Synchronisation of used reports of documentation from different sources, such as software event-logs, verbatim transcripts and log reports of physiological responses. After this the data may be analyzed and interpreted with methods appropriate for the individual project
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
The trade-off of research freedom vs resources
Game Industry Game Hobbyists
Game Research
Success criteria
Make good games. Sell them.
Pursue own passionate interest.
New knowledge. Understand games and
understand how to make better games.
Resources Large Small Medium
Testing Yes – rigorous Seldom Yes – rigorous
Freedom to share knowledge No – constrained YES YES
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Navigation
• 1. Question.
• 2. What kind of prototype can answer it?
• 3. What data is needed to proove/refute?
• What tests to make?
• How process the data?
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Mirjam P. Eladhari, [email protected] Gotland University, Department of Game Design, Technology and learning
Thank you for listening
Thank you for listening