two justifications for game-based assessement : an introduction to gba
DESCRIPTION
Two Justifications for Game-Based Assessement : An Introduction to GBA. Thomas E Heinzen William Paterson University ASSESS 2014 KDD 2014 Bloomberg, NYC August, 2014. This presentation is like speed dating; I use images and bold print so you remember only what I want you to remember. - PowerPoint PPT PresentationTRANSCRIPT
Two Justifications forGame-Based
Assessement:An Introduction to GBA
Thomas E HeinzenWilliam Paterson University
ASSESS 2014 KDD 2014Bloomberg, NYC
August, 2014
This presentation is like speed dating; I use images and bold print so you remember only
what I want you to remember.
Two justifications for GBA in higher eduation that have been hiding in plain
sight:
1. People love to be assessed.We develop our sense of self by comparing what we have to what others have: Social comparisons.
2. College is already a game.Points = G.P.A.
Badges = Diploma
Leaderboard = Relative feedback
Positive emotions (joy) are denied until graduation day.
Two Observations 1. GBA requires us to design more difficult courses. 2. GBA uses chance as motivation (not error).
6
Four Critical Definitions
1. Fun is any intrinsically motivated, engaging activity.
2. Play is the exuberant expenditure of energy in aimless activities.
3. A game is a situation in which participants voluntarily try to overcome unnecessary obstacles (i.e. rules and goals).
4. Gamification is the application of game-based principles to non-game situations. (Players may not know that they are in a game.)
Elements of Game-Based Assessment
1. The assessment must be fun (including “hard fun” and “serious games”) so that we can
a. measure peak performance;
b. conduct comprehensive assessments.
2. The experience of failure must be motivating.
3. The assessment data must be psychometrically sound (reliable, valid, and actionable).
Games are inclusive, cross-cultural, social activities.
Notice the level of spectator engagement in this game of hop scotch.
11
Games are a big part of our present…
In 2011, Cityville went from 0 to 100 million users in 41 days.
Video gaming is a 66 billion dollar industry (double Hollywood); online sales will surpass retail sales this coming year (in 2013). China is the biggest online market.
$8 billion was spent on supportive goods such as cyber seeds for Farmville.
Games are NOT just about boys blowing stuff up:
Sandbox (Minecraft) Building (Civilization, Sim City)
Social Building (Farmville) MMOGs (World of Warcraft)
Puzzles (Portal 2, Angry Birds) Vocabulary (Words With Friends)
12
Games also are a big part of our future because…
“We’re witnessing what amounts to no less than a mass exodus to virtual worlds and online game environments” (Economist Edward Castronova).
The 10,000 Hour Principle has been at work for several years (Ericsson, Krampe, & Tesch-Romer, 1993).
There are 500 million gamers who are “extraordinarily good at something” (Jane McGonigal’s TED talk).
13
Why Now?
• We have a growing base of students for whom playing games is the social norm.
• Games are already connecting people through social media.
• Game design is already being applied in non-game settings.
Drivers who obey the speedenter a lottery funded by drivers who did not.
The scientific foundation is growing.
Ifenthaler, D., Eseryel, D., & Ge X. (2012). Assessment in game-based learning. New York: Springer.Kaya, T. (Nov. 7, 2010). A ‘Stealth Assessment’ Turns to Video Games to Measure Thinking Skills, Chronicle of Higher Education, https://chronicle.com/article/A-Stealth-Assessment-Turns/125276/ Shute, V. & Ventura, M. (2010). Stealth assessment: Measuring and supporting learning in video games. Cambridge, MA: MIT Press.
Game-based research in psychology appears to be
accelerating. Game-related terms: “game,” “game-based,” “gamer,”
“gamification,” but not “gaming” or “game theory.”
1975 2012 Increase factor
Total # articles in PsychInfo
29,538 173,184 5.9
# articles with game-related terms
127 1527 12.3
Increase factor in PsychInfo entries between 1975 and 2012:
(Total: 29,538 to 173,184 vs. Game-Related: 127 to 1,527)
Total Entries Game-Related Entries
5.9
12.3
# References with Game-Related Terms
in Two Databases (1975-2012)
ERIC PsychInfo
516
591
127
1567
1975
2012
More DVs: The TAPAS Guide to Automated Assessments
TAPAS
Time to respond.
Accuracy of responses.
Points earned (weighted based on data).
Attempts (to measure persistence in some games)
Send it all to a spread sheet (automatically)
Item Analysis
Individual Analysis
Group Analysis
A more informative leaderboard strengthens the feedback loop and can
provide a rich information trail.
100 150 200 250 300 350 400 450 5000
10
20
30
40
50
60
70
80
Alexa
Janita
Rosa
Tomas
Henri
Keishan
Ernie
Becco,
Rosario
Bert
Ann
Gus
Leaderboard as of Week 3
Total Points Earned So Far
Total HoursSpent in the
Digital Textbook
Keishan (Week 2)
Keishan (Week 3)
Keishan’s Information Trail
Keishan (Week 1)
Converting to GBA will be like switching from a manual to an automated inventory assessment
(less work; more analysis).
Some candid student assessments after a gamified statistics course through
“fat” points, leveling up, and allowing choice.
(I probably should have been offended.)
Trial and error were good. There was no real help along the way, but there were no consequences.
I like that we got to leave early if we learned all we needed to know for that week.
I liked having someone other than the professor teach me; it was much better. (!)
When, if ever, did you experience something like an “epic win”?
When I began to be able to do the material and solve problems alone.
Whenever I finished a new level I felt that I accomplished it.
Completing the practice tests and doing them correctly.
I inadvertently did several things well because ancillary benefits seem to flow out of good design.
Ancillary benefits flow out of good design.
Allowing high achieving others to teach other students
Rewarding immediate achievement with immediate rewards (leaving early)
Giving them self-paced practiceKeeping their attention for four hoursBecoming more confident in the fairness of the
tests, including when some students still deserved to fail
Help Donny!(Testing Beliefs in Myths
About Psychology)
Designed by Sam Richman (User Experience Strategist)
FairLawn, NJ
As Donny’s therapist, indicate what you believe is the best science-based answer.
Note the inappropriate use of theFreud couch by the game designer.
“I’m getting such low grades that I don’t know how I made it this far. My father says I need to work on my self-esteem. Will building my self-esteem help me earn higher grades?”
A) Sorry Donny. Building self-esteem has little to do with improving academic performance; study more.
B) Your father is right, Donny. Raising your self-esteem will usually improve academic achievement.
A B
“While I’m taking tests sometimes I’m unsure of the answers. Should I go with my gut or would it be better to go back over the answers?”
A) Stick with your first hunch; you can almost always trust your intuition.
B) Go over the questions you are worried about and change them if you think another answer might be right.
A B
“Could I be doing so badly in my math classes because I’m left handed? If I am right brained then it would make math courses more difficult, right?”
A) It’s true, people who are right brained usually lean more towards the arts, while you may not be great at math you are probably much more creative than other people if you are right brained.
B) Actually, the brain works in an integrated fashion and which hemisphere is dominant mostly doesn’t matter. You probably just need to study a little more.
A B
A) Try punching a pillow. It will help you blow off steam and redirect your anger so that you are less likely to actually harm anyone.
B) Try listening to music or going for a walk. Distraction and self-control are the best ways to manage your anger.
A B
Profile 1: Donny is not doing well. His Quality of Life Score = 0.
Profile 2: Donny is doing better.His Quality of Life Score = 5.
Two possible profiles of Donny based on the player’s advice:
The Job Interview(One of seven critical thinking assessments using
everyday examples and classic experiments)Story Board 1.
Stinkin’ Thinkin’Before trying to start the car again, he asked a friend to use her mighty psychic powers…
Stinkin’ Thinkin’When you are ready, click on the stop watch and drag each box into the appropriate column on the next screen until you have earned 100%!
Screaming Using psychic powers
Hitting the steering wheel
Slamming the hood
Ways to get the car to
start
The car starts or does not
start
One of these influences caused the car to start
None of these influences caused the car to start
One of these influences may have
caused the car to start.
Being worried about the interview
Having a dead cell phone
We cannot logically
determine why the car
started
Putting the key in very
slowly
START
Screaming Using psychic powers
Hitting the steering wheel
Slamming the hood
Ways to get the car to
start
The car starts or does not
start
One of these influences caused the car to start
None of these influences caused the car to start
One of these influences may have
caused the car to start
Being worried
about the interview
Having a dead cell
phone
We cannot logically
determine why the car
started
Putting the key in very
slowly
Independent Variable(s)
Dependent Variable(s)
Alternative Explanation(s)
Extraneous influence(s)
A Logical Conclusion
An Illogical Conclusion
A more informative leaderboard strengthens the feedback loop and can
provide a rich information trail.
100 150 200 250 300 350 400 450 5000
10
20
30
40
50
60
70
80
Alexa
Janita
Rosa
Tomas
Henri
Keishan
Ernie
Becco,
Rosario
Bert
Ann
Gus
Leaderboard as of Week 3
Total Points Earned So Far
Total HoursSpent in the
Digital Textbook
Keishan (Week 2)
Keishan (Week 3)
Keishan’s Information Trail
Keishan (Week 1)
ReferencesDaniels, V. (1967). Communication, Incentive, and structural variables in interpersonal exchange and negotiation. Journal of Experimental Social Psychology, 3, 47-74.
Day, J. (1838). An inquiry into the self-determining power of the will; or, contingent volition. New Haven, CT: Herrick & Noyes.
Everett, C. C. (1891). Ethics for young people. Boston: Ginn & Company.
Hard, R. (1995). The discourses of Epictetus. ed. With introduction and notes by Christopher Gill. London: Everyman/Dent.
de Holstein, S. (1798). A treatise on the influence of the passions unpon th happiness of individuals and of nations: Illustrated by striking references t the principal events and characters that have distinguished the French revolution. London: George Cawthorne.
Ifenthaler, D., Eseryel, D., & Ge X. (2012). Assessment in game-based learning. New York: Springer.
Ifenthaler, D., Eseryel, D., & Ge X. (2012). Assessment in game-based learning. New York: Springer.
Kennedy, J. L. (1962). The system approach: Organizational Development, 4, 25-52.
Lesley, J. P. (1881). Lecture XII. The destiny of man. In Man’s origin and destiny, sketched from the platform of the physical sciences. London: Trübner & Co.
Lubbock, J., (1894). The use of life. London: Macmillan and Co.
Parkhurst, J. L. (1832). Elements of moral philosophy (2nd ed., rev. and improved). Boston, MA: Perkins & Marvin.
Seddon, K. H. (2005). Epictetus (55-135 CE). Internet Encyclopedia of Philosophy. Retrieved on October 24, 2013, from http://www.iep.utm.edu/epictetu/.
Riis, J. (1890). How the other half lives: Studies among the tenements of New York. New York: Charles Scribner’s Sons.
Schell, J. (2008). The art of game design. Boca Raton, FL: CRC Press.