a large-scale, longitudinal study of user profiles in world of warcraft

@_jon_bell_WiC May 14, 2013

A Large-Scale, Longitudinal Study of User Profiles in

World of WarcraftJonathan Bell, Swapneel Sheth, Gail Kaiser

Columbia University, New York, NY USA

PROGRAMMING SYSTEMS LAB

COLUMBIA UNIVERSITY

http://www.psl.cs.columbia.edu/

enable(vt):to make possible, practical, or easy

1


Motivation

2


Motivation

• We’re a group of Software Engineers.

2


Motivation


• Recently studying Gamification of Software Development

2


Motivation



• What principles can we carry over from game design to apply it to non-game contexts?

2


Motivation



• What principles can we carry over from game design to apply it to non-game contexts?

• Why not look to real games?

2


Game Studies

3


Game Studies

• Qualitative Research:

• 100’s-1000’s of samples [Debeuvais; Yee]

• Personalities, demographics, etc

3


Game Studies

• Qualitative Research:

• 100’s-1000’s of samples [Debeuvais; Yee]

• Personalities, demographics, etc

• Quantitative Research:

• 10,000’s-100,000’s of samples [Duchenaut; Lewis]

• In-game demographics, etc

3


WoW: Quantitative Data

5



• In-game “Demographics”

• Guild, Class, Race, Level

5





• Quests

5





• Quests

• Achievements

5





• Quests

• Achievements

• Professions

5





• Quests

• Achievements

• Professions

• Raids

5





• Quests

• Achievements

• Professions

• Raids

• Companions

5





• Quests

• Achievements

• Professions

• Raids

• Companions

• Mounts

5





• Quests

• Achievements

• Professions

• Raids

• Companions

• Mounts

• Titles

5





• Quests

• Achievements

• Professions

• Raids

• Companions

• Mounts

• Titles

• Reputation5


Methodology

7


Accessing the Data

• Blizzard’s API allows you to retrieve this per-character

• No option to list all characters

• How do we discover the characters?

8


Character Discovery

9


Character Discovery

• Blizzard does list characters by guild

9


Character Discovery


• Several popular websites have listings of guilds and characters [e.g. WarcraftRealms]

9


Character Discovery


• Several popular websites have listings of guilds and characters [e.g. WarcraftRealms]

• Solution: Find all guilds belonging to these characters. And find all members of all identified guilds

9


Fetching the Data

• Blizzard API is rate-limited per-IP

• Solution: use a cluster of 60+ IPs to crawl the API

10


Fetching the Data

• Blizzard API is rate-limited per-IP

• Solution: use a cluster of 60+ IPs to crawl the API

MySQL

Crawler Nodes (60+)

ManagementConsole (Normal PC)

Retrieve "todo-list"

Store resultsStarts Nodes

Runs Analysis

10


0"

200000"

400000"

600000"

800000"

1000000"

1200000"

1400000"

9/29/2011"

9/30/2011"

10/1/2011"

10/2/2011"

10/3/2011"

10/4/2011"

10/5/2011"

10/6/2011"

10/7/2011"

10/8/2011"

10/9/2011"

10/10/2011"

10/11/2011"

10/12/2011"

10/13/2011"

10/14/2011"

Characters)

Date)

Crawler)Performance)

11


Raw Data

• Found approximately 12 million characters with basic information

• Complete data for half of these

Basic information

12


Other MMORPG Study Sizes

1"

10"

100"

1,000"

10,000"

100,000"

1,000,000"

10,000,000"

100,000,000"

WoW:"Debeauvais"

2011"

EverQuest:"Williams"[2008]"

Everquest,"Ul<ma,"

Camelot,"SW"Galaxies:"Yee"

[2006]"

WoW:"Harrison"[2011]"

WoW:"Lewis"[2010]"

WoW:"Bell"[2013]"(1)"

WoW:"Bell"[2013]"(2)"

Sample'Size'(Log'Scale)'

Sample'Size'of'MMORPG'Studies'

13



1"

10"

100"

1,000"

10,000"

100,000"

1,000,000"

10,000,000"

100,000,000"

WoW:"Debeauvais"

2011"


Everquest,"Ul<ma,"


[2006]"


WoW:"Lewis"[2010]"

WoW:"Bell"[2013]"(1)"

WoW:"Bell"[2013]"(2)"



Closest: 136,047

13



1"

10"

100"

1,000"

10,000"

100,000"

1,000,000"

10,000,000"

100,000,000"

WoW:"Debeauvais"

2011"


Everquest,"Ul<ma,"


[2006]"


WoW:"Lewis"[2010]"

WoW:"Bell"[2013]"(1)"

WoW:"Bell"[2013]"(2)"



Closest: 136,047

Us: 6 (12) Million

13


Research Questions

14


Research Questions1. Can we cluster characters into profiles

based on the type of goals that they complete in game?

14




2. Do characters' play profile evolve as they level?

14




2. Do characters' play profile evolve as they level?

3. Do characters with different play profiles play through the game content at different rates?

14


RQ1: Creating Player Profiles

15



• Within WoW, players complete Achievements

15




• Our dataset contains each achievement each player earns, with a timestamp

15




• Our dataset contains each achievement each player earns, with a timestamp

• Profile players based on the types of achievements they complete

15


Categorizing Achievements

• Draw on motivation categorizations described by Yee

Achievement Social ImmersionAdvancement Socializing Discovery

Mechanics Relationship Role-Playing

Competition Teamwork Customization

Escapism

Categorization of play motivations, Yee

16



17



• 1000 Conquest Points

17




• Earn 1000 conquest points -> Achievement

17





• Archavon the Stone Watcher

17






• Defeat Archavon the Stone Watcher in 10 player mode -> Social

17







• It’s Happy Hour Somewhere

17








• Drink 25 different types of beverages -> Immersion

17









• Arrested Development

17









• Arrested Development

• Allow all three of Corla’s zealots to evolve, then defeat Corla after slaying the evolved zealots in Blackrock Caverns on Heroic Difficulty -> All three

17


Four Distinct Player Profiles

Generalists Achievement Loners Socialists

Size (Millions of players)

Social

Immersion

Achievement

1.6 1.4 1 1

12% ± 6 7% ± 8 1% ± 3 28% ± 6

38% ± 6 20% ± 7 38% ± 29 31% ± 5

66% ± 4 87% ± 6 35% ± 24 61% ± 4

18


RQ2: Achievements per level

0"

20"

40"

60"

80"

100"

120"

140"

160"

180"

0(9"

10(19"

20(29"

30(39"

40(49"

50(59"

60(69"

70(79"

80(84"

85"

Number'of'Achievements2

Level2

Achievement"

Immersion"

Social"

19



0"

20"

40"

60"

80"

100"

120"

140"

160"

180"

0(9"

10(19"

20(29"

30(39"

40(49"

50(59"

60(69"

70(79"

80(84"

85"


Level2

Achievement"

Immersion"

Social"

Mostly get achievements after finishing most game content

19



0"

20"

40"

60"

80"

100"

120"

140"

160"

180"

0(9"

10(19"

20(29"

30(39"

40(49"

50(59"

60(69"

70(79"

80(84"

85"


Level2

Achievement"

Immersion"

Social"

Very few do social achievements early on

19



0"

20"

40"

60"

80"

100"

120"

140"

160"

180"

0(9"

10(19"

20(29"

30(39"

40(49"

50(59"

60(69"

70(79"

80(84"

85"


Level2

Achievement"

Immersion"

Social"

Had to be social to progress?

19


RQ3: Leveling by Primary Motivation

• “Primary” motivation - whatever the player did most

• Look purely at time to reach level 85 (maximum level)

20



0!1000!2000!3000!4000!5000!6000!7000!8000!9000!10000!

Achievement! Immersion! Social! Overall!

Tota

l Hou

rs to

Lev

el!

Character Classification!

21



0!1000!2000!3000!4000!5000!6000!7000!8000!9000!10000!

Achievement! Immersion! Social! Overall!

Tota

l Hou

rs to

Lev

el!

Character Classification!

Players that “stop to smell the roses”

21


Bonus RQ: Has Blizzard made the game easier?

22


Studying Game Difficulty

• Modeled based on how long it takes to level

• Measured in real-world time

• Compare time-to-level from before Nov 2011 to post Nov 2011

23


Leveling by Start Date

0!500!1000!1500!2000!2500!3000!3500!

20! 30! 40! 50! 60! 70! 80! 90*!

Hou

rs b

etw

een

leve

ls!

Level!

Pre-Cataclysm!Post-Cataclysm!

24


Leveling by Start Date

0!500!1000!1500!2000!2500!3000!3500!

20! 30! 40! 50! 60! 70! 80! 90*!

Hou

rs b

etw

een

leve

ls!

Level!

Pre-Cataclysm!Post-Cataclysm!

Abandoned characters

24


And there’s so much more.

25


What’s the most popular character name?

Name Count (Normalized)Shadow 922Nightmare 839Nemesis 831Loki 772Hades 722Pandora 707Lilith 672Lucian 660Wolverine 657Holycow 656Luna 644Kratos 642

26


WoW: Social Data

• Guild

• Achievements (surprise!)

• Co-completions of group achievements

• Huge network of players

• Over 100 million edges

27


Contributions

28


Contributions

• Gathered the first massive dataset from the MMORPG, World of Warcraft

28


Contributions


• Defined player profiles in WoW

28


Contributions



• Showed that these profiles are not static, and players evolve over time

28


Contributions




• Showed that these players advance through content at different paces

28


Contributions




• Showed that these players advance through content at different paces

• API Scraper: http://code.psl.cs.columbia.edu/

28

http://code.psl.cs.columbia.edu

http://code.psl.cs.columbia.edu


A Large-Scale, Longitudinal Study of User Profiles in

World of WarcraftJonathan Bell, Swapneel Sheth, Gail Kaiser

Columbia University, New York, NY USA

PROGRAMMING SYSTEMS LAB

COLUMBIA UNIVERSITY

http://www.psl.cs.columbia.edu/

enable(vt):to make possible, practical, or easy

29

a large-scale, longitudinal study of user profiles in world of warcraft

Documents

game demographics guild

game studies3

level quests achievements5

level quests5

group of software engineers

s of samplesduchenaut

quantitative research

fromgame design