identifying mmorpg bots: a traffic analysis approach
DESCRIPTION
TRANSCRIPT
Identifying MMORPG Bots:Identifying MMORPG Bots:A Traffic Analysis ApproachA Traffic Analysis Approach
(MMORPG: Massively Multiplayer Online Role Playing Game)(MMORPG: Massively Multiplayer Online Role Playing Game)
Kuan-Ta Chen
National Taiwan University
Jhih-Wei JiangPolly HuangHao-Hua ChuChin-Laung LeiWen-Chin Chen
Collaborators:
2Identifying MMORPG Bots: A Traffic Analysis Approach
Talk OutlineTalk Outline
Motivation
Trace collection
Traffic analysis and bot identification schemes
Performance evaluation
Scheme Robustness
Conclusion
3Identifying MMORPG Bots: A Traffic Analysis Approach
Game BotsGame Bots
AI programs that can perform many tasks in place of gamers
Can reap rewards efficiently in 24 hours a day break the balance of power and economies in the game world
Therefore bots are forbidden in most games
4Identifying MMORPG Bots: A Traffic Analysis Approach
Bot DetectionBot Detection
Detecting whether a character is controlled by a bot is difficult since a bot obeys the game rules perfectly
No general detection methods are available today
The state of practice is identifying via human intelligence (as bots cannot talk like humans)
Labor-intensive and may annoy innocent players
This work is dedicated to automaticdetection of game bots
(without intrusion in players’ gaming experience)
5Identifying MMORPG Bots: A Traffic Analysis Approach
Key ContributionsKey Contributions
We proposed to detect bots with a traffic analysisapproach
We proposed four strategies to distinguish bots from human players based on their traffic characteristics
6Identifying MMORPG Bots: A Traffic Analysis Approach
Bot Detection: A Decision ProblemBot Detection: A Decision Problem
Game client Game server
Traffic stream
Q: Whether a bot is controlling a game client given the traffic stream it generates?
A: Yes or No
7Identifying MMORPG Bots: A Traffic Analysis Approach
Ragnarok Online Ragnarok Online ---- a screen shota screen shot
Figure courtesy of www.Ragnarok.co.kr
Ragnarok Online
One of the most popular MMORPGs (they claimed 17 million subscribers worldwide recently)
Notorious for the prevalence of the use of game bots
8Identifying MMORPG Bots: A Traffic Analysis Approach
Game Bots in Ragnarok OnlineGame Bots in Ragnarok Online
Two mainstream bot series:
Kore -- KoreC, X-Kore, modKore, Solos, Kore, wasu, Erok, iKore, and VisualKore
DreamRO (popular in China and Taiwan)
Both bots are standalone (game clients not needed), fully-automated, script-based, and interactive
9Identifying MMORPG Bots: A Traffic Analysis Approach
DreamRODreamRO ---- A Screen ShotA Screen Shot
World Map
View Scope
Character Status
Character is
here
10Identifying MMORPG Bots: A Traffic Analysis Approach
Trace CollectionTrace Collection
Category Trace # Participants
8 traces 2 rookies2 experts
2 bots11 traces
Average Length Network
2.6 hours
Bots 17 hours
ADSL, Cable Modem,Campus Network
Human players
Player skills
Character levels / equipments
Network connections
Network conditions (RTT, loss rate, etc)
Heterogeneity was preserved
206 hours and 3.8 million packets were traced in total
11Identifying MMORPG Bots: A Traffic Analysis Approach
Traffic Analysis of Collected Game TracesTraffic Analysis of Collected Game Traces
Traffic is analyzed in terms of Command timing
Traffic burstiness
Reaction to network conditions
Four bot identification strategies are proposed
12Identifying MMORPG Bots: A Traffic Analysis Approach
Command TimingCommand Timing
Observation
Bots often issue their commands based on arrivals of server packets, which carry the latest status of the character and environment
game client game server
time
Client response time (response time)
Time difference between the release of a client packet and the arrival of the most recent server packet
State update
t1
Client commandt2
Response time T = t2 – t1
13Identifying MMORPG Bots: A Traffic Analysis Approach
CDF of Response TimesCDF of Response Times
DreamRO> 50% response times are extremely small
KoreZigzag pattern (multiples of a certain value)
14Identifying MMORPG Bots: A Traffic Analysis Approach
Histograms of Response Times Histograms of Response Times (DreamRO traces)(DreamRO traces)
1 ms
multiple peaks
1 ms multiple peaks
Many client packets are sent in response to server packets
15Identifying MMORPG Bots: A Traffic Analysis Approach
Histograms of Response TimesHistograms of Response Times
Regularity in the distribution of bots’response times
Scheme #1: Command Timing
A traffic stream is considered from a bot if it has …
Quick response times (< 10 ms) clustered
Regularity in the distribution of response times, i.e., if any frequency component exists
16Identifying MMORPG Bots: A Traffic Analysis Approach
Traffic BurstinessTraffic Burstiness
Traffic burstinessAn indicator of how traffic fluctuates over time
The variability of packet/byte counts observed in successive periods
Index of Dispersion for Counts (IDC)
The IDC at time scale t is defined as
It =Var(Nt)
E(Nt),
where Nt indicates the number of arrivals in intervalsof time t.
17Identifying MMORPG Bots: A Traffic Analysis Approach
Example: Wine Sales and IDCExample: Wine Sales and IDC
The period is approximately 12 months
The IDC at 12 months is the lowest
18Identifying MMORPG Bots: A Traffic Analysis Approach
The Trend of Traffic BurstinessThe Trend of Traffic Burstiness
Traffic generated by human players, of course, has no reason to exhibit such property
Conjecture for Bot Traffic
1. Each iteration of the bot program’s main loop takes roughly the same amount of time
2. Each iteration of the main loop sends out roughly the same number of packets
3. Bot traffic burstiness will be the lowest in the time scale around the time needed to complete each iteration
19Identifying MMORPG Bots: A Traffic Analysis Approach
Examining the Trend of Traffic BurstinessExamining the Trend of Traffic Burstiness
Regularity in the distribution of bots’response times
Scheme #2: Trend of Traffic Burstiness
A traffic stream is considered from a bot if …
the IDC curve has a falling trend at first and after that a rising trend, and
both trends are detected at time scales < 10 sec
20Identifying MMORPG Bots: A Traffic Analysis Approach
The Magnitude of Traffic BurstinessThe Magnitude of Traffic Burstiness
Difficultyno “typical” burstiness of human player traffic
Solutioncompare the burstiness of client traffic with that of the corresponding server traffic (as servers treat all game clients equally)
Scheme #3: Burstiness MagnitudeA traffic stream is considered to be generated by a bot if the client traffic burstiness is much lower than the corresponding server traffic burstiness
Conjecture
Bot traffic is relatively smooth than human player traffic
21Identifying MMORPG Bots: A Traffic Analysis Approach
Human Reaction to Network ConditionsHuman Reaction to Network Conditions
Conjecture for Human Player Traces
1. The network delay of packets will influence the pace of game playing (the rate of screen updates, character movement)
2. Human players will unconsciously adapt to the game pace (the faster the game pace is, the faster the player acts)
server
Traffic jam!!
Is there any relationship between network delay andthe pace of user actions?
22Identifying MMORPG Bots: A Traffic Analysis Approach
Packet Rate vs. Network DelayPacket Rate vs. Network Delay
Scheme #4: Pacing
A traffic stream is considered from a bot if …correlation between pkt rate vs. network delay is non-negative
Human player traces: downward trend
23Identifying MMORPG Bots: A Traffic Analysis Approach
Performance EvaluationPerformance Evaluation
Evaluate the sensitivity of input size by dividing traces into segments, and computing the above metrics on a segment basis
Metrics
Correct rate the ratio the client type of a trace is correctly determined
False positive rate the ratio a player is misjudged as a bot
False negative rate the ratio a bot is misjudged as a human player
24Identifying MMORPG Bots: A Traffic Analysis Approach
Performance Evaluation ResultsPerformance Evaluation Results
[Burstiness magnitude]always achieves low false positive rates (< 5%) and yields a moderate correct rate (≈ 75%)
[Command timing and Burstiness trend]Correct rates higher than 95% and false negative rates lower than 5% given an input size > 2,000 packets
25Identifying MMORPG Bots: A Traffic Analysis Approach
An Integrated ApproachAn Integrated Approach
In practice, we can carry out multiple schemes simultaneously and combine their results according to preference
Conservative approach:command timing AND burstiness trend
Aggressive approach:command timing OR burstiness trend
26Identifying MMORPG Bots: A Traffic Analysis Approach
An Integrated Approach An Integrated Approach ---- ResultsResults
Aggressive approach (2,000 packets): false negative rate < 1% and 95% correct rateConservative approach (10,000 packets): ≈ 0% false positive rate and > 90% correct rate
Aggressive
27Identifying MMORPG Bots: A Traffic Analysis Approach
Robustness against CounterRobustness against Counter--AttacksAttacks
Just like anti-virus software vs. virus writers
Our schemes only rely on packet timings
An obvious attack is adding random delays to the release time of client packets
Command timing scheme will be ineffective
Schemes based on traffic burstiness are robust
Adding random delays will not eliminate the bot signatureunless the added delay is longer than the iteration time by orders of magnitude or heavy-tailed
However, adding such long delays will make the bots incompetent as this will slowdown the character’s actions by orders of magnitude
28Identifying MMORPG Bots: A Traffic Analysis Approach
Simulating the Effect of Random Delays on IDCSimulating the Effect of Random Delays on IDC
29Identifying MMORPG Bots: A Traffic Analysis Approach
SummarySummary
Traffic analysis is effective to identify game bots
Proposed four bot decision strategies and two integrated schemes for practical use
The proposed schemes (except the one based on command timing) are robust under counter-attacks
Thank You!Thank You!
Kuan-Ta Chen