institutionen för datavetenskap - diva...

Institutionen för datavetenskap Department of Computer and Information Science

Final thesis

Monkey Gamer: Automatic Profiling of

Android Games

by

Javier Marián Santos

LiTH-IDA/ERASMUS-A--13/002--SE

2013-08-29

Linköpings universitet

SE-581 83 Linköping, Sweden

Linköpings universitet

581 83 Linköping

Linköping University

Department of Computer and Information Science

Final Thesis

Monkey Gamer: Automatic Profiling of

Android Games

by

Javier Marián Santos

LiTH-IDA/ERASMUS-A--13/002--SE

2013-08-29

Supervisor: Aruna Prem Bianzino

Examiner: Simin Nadjm-Tehrani

Abstract

Due to the huge amount of smart-phone applications available nowadays,there is a need to check them in order to know if they are trustworthy, effi-cient and reliable. Furthermore, research on smart-phones is really intensive,so it is important to be able to easily profile applications and collect datathat researchers can use. In this thesis we have focused on Android gamesas they represent a big portion of Android applications and Android is themost used mobile operative system nowadays.

The thesis work can be divided in two main tasks. The first task con-sists of research on Android games in order to know how they are developed,which game design engines are used nowadays and how can we automatisethe execution of Android applications and, in particular, games. The secondstep consists of development of a program called Monkey Gamer that canplay Android games automatically and collect execution traces of the game.The Monkey Gamer analyses the screen shown on a device, recognises theplaces where the user should touch, and interacts with them. Then, consid-ering each screen as a state of a state machine, the program tries to coverthe whole game, generating the highest possible number of execution traces.

In order to test our solution we have compared the traces generated bythe Monkey Gamer with the ones obtained by some real players, achievinga significant similarity. The tests were executed on three games, coveringdifferent categories and implementation solutions.

iii

Acknowledgements

First, I would like to thank Prof. Simin Nadjm-Tehrani and Dr. Aruna PremBianzino for their support, help and guidance through this thesis work.

I would also like to thank all the members of RTSLab, it was great towork with you. Thank you for your help and for making that good workingenvironment everyday.

I would like to thank my family for their unconditional support, that letme come to Sweden to do this thesis.

Last but not least, I would like to thank all my friends, specially thepeople I have met in Linkoping. Enyoing my free time with you was thebest vitamin to do this work.

Javier Marian SantosLinkoping,May 2013

v

Contents

1 Introduction 11.1 Context and motivation . . . . . . . . . . . . . . . . . . . . . 11.2 Problem definition . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Evaluation criteria . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.6 Intended audience . . . . . . . . . . . . . . . . . . . . . . . . 61.7 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Background 82.1 Related scientific work . . . . . . . . . . . . . . . . . . . . . . 82.2 Android applications . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.1 Elements of an Android application . . . . . . . . . . 92.2.2 GUI design . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3 Game engines . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.4 Description of used tools . . . . . . . . . . . . . . . . . . . . . 12

2.4.1 Real APK Leecher . . . . . . . . . . . . . . . . . . . . 122.4.2 Android apktool . . . . . . . . . . . . . . . . . . . . . 122.4.3 Hierarchy Viewer . . . . . . . . . . . . . . . . . . . . . 122.4.4 monkeyrunner . . . . . . . . . . . . . . . . . . . . . . 132.4.5 Android View Client . . . . . . . . . . . . . . . . . . . 132.4.6 OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4.7 Python Image Library . . . . . . . . . . . . . . . . . . 142.4.8 strace . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4.9 tcpdump . . . . . . . . . . . . . . . . . . . . . . . . . 142.4.10 Android Debug Bridge . . . . . . . . . . . . . . . . . . 14

2.5 Object detection . . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Android games developmenttechnologies 163.1 Classification by engines . . . . . . . . . . . . . . . . . . . . . 163.2 Classification by user inputs . . . . . . . . . . . . . . . . . . . 183.3 Percentage of games covered . . . . . . . . . . . . . . . . . . . 20

vii

CONTENTS CONTENTS

4 Monkey Gamer 214.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . 214.2 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2.1 Main Controller . . . . . . . . . . . . . . . . . . . . . 244.2.2 Disassembler . . . . . . . . . . . . . . . . . . . . . . . 254.2.3 Installer . . . . . . . . . . . . . . . . . . . . . . . . . . 254.2.4 Screens Controller . . . . . . . . . . . . . . . . . . . . 264.2.5 Screen Analyser . . . . . . . . . . . . . . . . . . . . . 344.2.6 Object Detector . . . . . . . . . . . . . . . . . . . . . 374.2.7 Input Generator . . . . . . . . . . . . . . . . . . . . . 404.2.8 Trace Collector . . . . . . . . . . . . . . . . . . . . . . 40

5 Prototype evaluation 425.1 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.1.1 Testing hardware . . . . . . . . . . . . . . . . . . . . . 425.1.2 Game selection . . . . . . . . . . . . . . . . . . . . . . 425.1.3 Testing procedure . . . . . . . . . . . . . . . . . . . . 43

5.2 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445.2.1 System calls comparison . . . . . . . . . . . . . . . . . 455.2.2 IP addresses comparison . . . . . . . . . . . . . . . . . 495.2.3 Screens comparison . . . . . . . . . . . . . . . . . . . . 50

5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6 Conclusions 546.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556.3 Personal comments . . . . . . . . . . . . . . . . . . . . . . . . 57

A Android games analysis 58

B Screenshots used in comparison testing 61

viii

List of Figures

1.1 Simplified data flow of the Monkey Gamer . . . . . . . . . . . 31.2 Different screens in the Tic Tac Toe Free game . . . . . . . . 41.3 Example of screens flow when Monkey Gamer playing . . . . 5

2.1 View tree and screenshot from the Tic Tac Toe Free mainscreen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1 Views tree of two engine games got with Hierarchy Viewer . . 173.2 Distribution of games by GUI implementation . . . . . . . . . 183.3 Distribution of the input requirements in games . . . . . . . . 19

4.1 Monkey Gamer blocks diagram . . . . . . . . . . . . . . . . . 224.2 Monkey Gamer flow chart . . . . . . . . . . . . . . . . . . . . 234.3 Main Controller flow chart . . . . . . . . . . . . . . . . . . . . 244.4 Disassembler flow chart . . . . . . . . . . . . . . . . . . . . . 254.5 Installer flow chart . . . . . . . . . . . . . . . . . . . . . . . . 264.6 Screens Controller flow chart . . . . . . . . . . . . . . . . . . 274.7 IsThisANewScreen algorithm flow chart . . . . . . . . . . . . 294.8 Tic Tac Toe Free board screen differences among steps . . . . 304.9 Histogram comparison algorithm . . . . . . . . . . . . . . . . 314.10 Pixel by pixel comparison algorithm . . . . . . . . . . . . . . 324.11 Screen Analyser flow chart . . . . . . . . . . . . . . . . . . . . 354.12 GetNextElement algorithm . . . . . . . . . . . . . . . . . . . 364.13 Object Detector flow chart . . . . . . . . . . . . . . . . . . . . 384.14 Object detection in Angry Birds . . . . . . . . . . . . . . . . 394.15 Object detection in Bubble Worlds . . . . . . . . . . . . . . . 394.16 Input Generator flow chart . . . . . . . . . . . . . . . . . . . 404.17 Trace Collector flow chart . . . . . . . . . . . . . . . . . . . . 41

5.1 Detecting patterns in system calls sequences (step 1) . . . . . 465.2 Detecting patterns in system calls sequences (step 2) . . . . . 46

6.1 Superuser permission request in Angry Birds . . . . . . . . . 56

ix

LIST OF FIGURES LIST OF FIGURES

B.1 Screen 1 from Angry Birds . . . . . . . . . . . . . . . . . . . . 61B.2 Screen 2 from Angry Birds . . . . . . . . . . . . . . . . . . . . 61B.3 Screen 3 from Angry Birds . . . . . . . . . . . . . . . . . . . . 62B.4 Screen 4 from Angry Birds . . . . . . . . . . . . . . . . . . . . 62B.5 Screen 1 from Bubble Worlds . . . . . . . . . . . . . . . . . . 62B.6 Screen 2 from Bubble Worlds . . . . . . . . . . . . . . . . . . 63B.7 Screen 1 from Tic Tac Toe Free . . . . . . . . . . . . . . . . . 63B.8 Screen 2 from Tic Tac Toe Free . . . . . . . . . . . . . . . . . 64B.9 Screen 3 from Tic Tac Toe Free (TT3.1) . . . . . . . . . . . . 64

x

List of Tables

3.1 Games covered by Monkey Gamer . . . . . . . . . . . . . . . 20

4.1 Comparisons of screenshots of the same screen . . . . . . . . 344.2 Comparisons of screenshots of different screens . . . . . . . . 34

5.1 Execution times of Monkey Gamer . . . . . . . . . . . . . . . 445.2 Comparisons per game, in brackets the system calls compared 455.3 Tic Tac Toe Free system calls comparison . . . . . . . . . . . 485.4 Bubble Worlds system calls comparison . . . . . . . . . . . . 485.5 Angry Birds system calls comparison . . . . . . . . . . . . . . 495.6 IP’s reached in Tic Tac Toe Free . . . . . . . . . . . . . . . . 495.7 IP’s reached in Bubble Worlds . . . . . . . . . . . . . . . . . 505.8 IP’s reached in Angry Birds . . . . . . . . . . . . . . . . . . . 505.9 Screens comparison for Tic Tac Toe Free . . . . . . . . . . . . 515.10 Screens comparison for Bubble Worlds . . . . . . . . . . . . . 515.11 Screens comparison for Angry Birds . . . . . . . . . . . . . . 52

A.1 Full results of the Android games analysis . . . . . . . . . . . 58

xi

LIST OF TABLES LIST OF TABLES

xii

Chapter 1

Introduction

This report is the result of a 30 credit points Master project performedat the Real-time Systems Laboratory within Department of Computer andInformation Science at Linkoping University.

1.1 Context and motivation

During the last years, the use of smart-phones has hugely increased. Differ-ent Operative Systems (OS) have been developed for these devices, such asAndroid (by Google) [2] or iOS (by Apple) [14]. The developers of these op-erative systems give the possibility to everybody (including software devel-oping companies or independent developers) to develop applications (apps)and upload them to a virtual shop where users of that specific operative sys-tem can download them (for free or paying a fee). The high level of freedomin the system results in thousands of applications for mobile phones used bymillions of people.

As apps developing and using are very common, there is the need toeasily test them and check them in order to know if they are functional,trustworthy, efficient and reliable. Apple examines every app uploaded toits App Store [6]. However, there is less control on the security or efficiencyof the applications that are uploaded to the Google Play Store (previouslyAndroid Market) [20] [3].

To check the apps there is the need for collecting traces about the systemstatus, system calls, user inputs, accesses to sensitive data, network usage,etc. Then, these traces are analysed by a human supported by a program todetect security threats, unknown packets sent or received and developmentor system failures.

Furthermore, research is really intensive on Android smart-phones be-cause of their diffusion and open nature. Researchers usually need executiontraces from applications as test cases for their work. With games the situ-ation becomes worse because this kind of application has more traces than

1

1.2. PROBLEM DEFINITION CHAPTER 1. INTRODUCTION

other apps. This is due to the fact that games have more screens, they aremore complicated, have a lot of user interaction and can be played againstthe computer or against other persons (on the same device or online).

There is no information about the percentage of games over the totalnumber of applications available on the Google Play Store, but their shareis big enough to consider games as a different category and let this thesiswork focus on them. Finally the focus on games is justified as there are somesolutions for automatic testing and profiling normal Android applicationsbut not for the games category (as we will see later in Section 2.1).

1.2 Problem definition

This thesis work solves two problems. The first one is the lack of informa-tion about the current state of the developing of Android games. Usuallygames are developed using game engines. In this thesis we briefly surveywhich are the most used engines at the moment. Moreover we make a clas-sification of the type of user input needed to play the games. Thus, we arecreating an overview of games based on the engines and type of inputs notavailable in the literature. These classifications are useful for us to definethe applicability field of our solution to the second problem.

The second problem is lack of tools for efficient generation of inputs inorder to automatically create program traces. This problem is solved by de-veloping and testing a program called Monkey Gamer, which is able to playautomatically a wide selection of Android games. In this way, this solutionwill give a tool that can reduce the human effort in the process of profilingan Android game. Manual testing, profiling or execution trace generationdemand human time, so the use of a monkey program can reduce the costs.The Monkey Gamer ideally would be able to cover all execution traces of thegame, but in practice this is not possible due to the path explosion problem.This ability to create a wide set of inputs is necessary to detect unexpectedbehaviour, like a security risk. It is our intention to keep Monkey Gamer’sbehaviour as similar as possible to a human player so it can create human-like execution traces. The Monkey Gamer is also responsible for collectingexecution traces and screenshots during its execution. There are tools pre-ceding this project that can perform inputs to an Android device, but notautomatically. There are also some automatic profilers, but they only workwith normal apps and not with games, that require different kind of inputs.Monkey Gamer will be an automatic profiler and an automatic unexpectedbehaviour tester for Android games.

1.3 Approach

In order to analyse the characteristics of Android games we will select the topone hundred free games on the Google Play Store. We will install them on

2

1.3. APPROACH CHAPTER 1. INTRODUCTION

Figure 1.1: Simplified data flow of the Monkey Gamer

a device and with the tool Hierarchy Viewer [12], we will be able to analysethe Graphic User Interface (GUI) of the game running on the device andknow which engine is used for developing it. After that, we will play thegame for a while to classify the user input.

Monkey Gamer will be a Python [21] program executed on a Linux com-puter and it will control a real Android device. To develop Monkey Gamerwe will use monkeyrunner [18]. This tool is able to generate user inputs toan Android device or emulator. Monkey Gamer will take the installable file(.apk package) of the game we want to profile and will install and launchit on a real device. Monkey Gamer controls the device in order to play thegame automatically. Feedback from the device is given to Monkey Gamer toknow where it has to click or in which part of the game it is. This feedbackconsists of screenshots, information about the present GUI, etc. At the be-ginning of the execution Monkey Gamer starts to collect execution tracesfrom the device. These traces represent the result of the Monkey Gamerprogram. The traces that Monkey Gamer has to collect are system calls,network traffic and a screenshot after every automatic input. The traceshave also to be mapped to the input sequence. This process is shown inFigure 1.1.

In detail, after launching the game, we record in which screen we are.We consider a screen as an image shown by the game that is considerablydifferent from other images shown by the game. For example, in the Tic

Tac Toe Free game we have, among others, the main screen, the optionsselection screen and the playing screen as it is shown in Figure 1.2.

Games GUI can be defined using an xml file (as normal applications) or agame engine. If the GUI of the game is designed using an xml file, we analysethe shown screen getting the elements that are defined as clickable. On thexml file, interactive elements on the GUI have the clickability property setas true. This is done with the help of the Android View Client library [17].

For those games made using games engines getting the clickable elements

3

1.3. APPROACH CHAPTER 1. INTRODUCTION

Figure 1.2: Different screens in the Tic Tac Toe Free game

with Android View Client library is not possible, so an object detection isdone to know where sharp shapes and contours are in the screen. Objectdetection is done using the Hough Transform algorithm implemented in theOpenCV library [19]. Usually clickable places are drawn as sharp shapes orcontours, as the human eye must recognize them easily.

Once we have the clickable elements list, the monkey program imitates a“touch” on the first element. Afterwards we analyse the present image on thedevice display. We compare it with the screenshot we have of the previousscreenshots (if any). If it is similar we consider it as the same screen, ifnot we consider it as a new screen. Obviously, for the first screenshot, wewill consider it directly a new screen. Then, if the present screen is an oldone, we touch on the following clickable element, while if it is new we lookfor clickable elements as before. Once we have touched on every clickableelement on a screen, we start touching them again in a different order.

Monkey Gamer theoretically ends when it has touched on every clickableelement in every order and for all the screens detected in the game. Dueto the path explosion problem a complete coverage of all the screens is notpossible, so Monkey Gamer can be stopped on request.

An example of the Monkey Gamer flow is shown in Figure 1.3. Thegame is launched and the main screen is shown (screen 1). Buttons 1 and2 are detected and the first one is clicked. Screen 2 is presented, but notouchable elements are detected. Then a network answer (e.g., there arenot opponent players available at the moment) makes the game return tothe first screen. As the screen is not new, the Monkey Gamer proceeds toclick on button number 2 that takes the game to screen 3. Some objectsare detected and interaction is done. We reach screen number 4 wherethere are no clickable elements. Nothing happens in a pre-defined timeout

4

1.4. EVALUATION CRITERIA CHAPTER 1. INTRODUCTION

Figure 1.3: Example of screens flow when Monkey Gamer playing

so back button is pressed automatically and we get back screen 3. As wehave clicked on every element (and in every order) on screen 3, the MonkeyGamer presses the back physical key taking the game to screen 1. Then theMonkey Gamer would press on button 2 and later on button number 1, toachieve the ”touch on every element on every order” requirement.

We can consider this flow as a run in a state machine, where the differentscreens are the states and the user inputs and the game answers are thetransition triggers.

1.4 Evaluation criteria

In order to evaluate our solution we will collect automatic execution tracesof a selection of games, which must cover different categories, input kindsand game engines. Then, we will ask to at least four people to play the samegames. These persons will play a while so they create in that time the samenumber of inputs as Monkey Gamer will create during its execution. Wewill compare the automatically generated traces with the human generatedtraces. These traces will contain at least the system calls done by the appli-cation and the connected IP addresses while the game was being executed.We will also see how many and which screens have Monkey Gamer and thehumans covered and if they overlap.

We will obtain similarity rates between human’s and Monkey Gamer’straces. This will be done applying some trace comparison techniques andcomparing one to one the system calls, IP addresses and screens shown. Wewill consider that our traces are similar enough to the ones collected byhumans and that our solution is valid if:

• the percentage of similarity between traces is above the 50% for thesystem calls

• the IP addresses are common in the 50% of the cases

• the coverage of screens is above the 80%, comparing the amount ofscreens checked by humans with the screens checked by Monkey Gamer

5

1.5. LIMITATION CHAPTER 1. INTRODUCTION

This test will serve as an evidence that Monkey Gamer is functional asexpected and show its ability to run a game like a human, so the generatedtraces can be used as human patterns by researchers.

As a by product we may get interesting traces with Monkey Gamer thatwe do not get with human players, such as a connection to a particularserver or the launch of another application by the game. This will show theeffectivity of using Monkey Gamer as an unexpected behaviour tester.

1.5 Limitation

The analysis of the Google Play Store is done based on the top 100 free gamesof the Swedish store, because Google just lets the users visit the nationalstore from where they are connected. The Swedish market is not big enoughto be representative for the world, so the analysis and classification of gameswould be more interesting if done on the largest store. Unfortunately thereis not a world-wide official Google store and there is not information aboutit, like top one hundred most downloaded games in the world during oneyear.

Monkey Gamer is only intended to play games that do not need the in-puts at a given moment, such as driving cars games (where you are supposedto turn when it is required) or platform games (where the character has tojump at the right moment). This is due to the fact that Monkey Gamerneeds time to analyse the screen in which it is located, so the inputs aredone after some time and we can not control it. This is explained in detailin section 3.2. Also Monkey Gamer can not play to games that need somekind of intelligence such as quiz games. The estimated percentage of gamescovered by our solution is calculated in section 3.3.

1.6 Intended audience

The reader of this thesis is supposed to have basic knowledge in computerscience and mobile applications. The Android games overwiew is suitable forthose who want to know which are the most used engines at the moment, likeresearchers interested in games or the game engine development companies.

The Monkey Gamer automatic profiler is intended for developers thatwant to collect traces of their games more systematically or at a lower cost.They can use it for some functional testing too. Researchers can use it to col-lect traces from games for systematic and repeatable research experiments.For example it is useful for those who want to perform security analysis ofsuspicious games, to know which network connections they are establishing,which system calls they are using, which applications they are launching,etc. Finally it may be useful for Android, to perform an automatic checkon every game that is uploaded to their Google Play Store.

6

1.7. THESIS OUTLINE CHAPTER 1. INTRODUCTION

1.7 Thesis outline

This thesis is structured as follows. Chapter 2 presents background infor-mation as related scientific work, Android applications basics and the usedtools. Chapter 3 presents an overview of Android games based on the gameengines and the type of inputs. The chapter is concluded with a discussionabout the percentage of games that can be profiled using Monkey Gamer.Chapter 4 explains in detail how the Monkey Gamer is implemented andtested. Chapter 5 explains how Monkey Gamer is validated and presentsand discusses the results. Finally, chapter 6 concludes the thesis and presentssome possible future work directions.

7

Chapter 2

Background

In this chapter we are going to present the literature related to this thesis,some basic knowledge about Android applications and the tools used toachieve the results.

2.1 Related scientific work

In this section we are going to analyse some scientific papers about auto-matic Android applications testing, Android application profiling and execu-tion traces analysis. Hu and Neamtiu present techniques for the automatictesting of the GUI of Android applications [13]. They feed random inputevents to the application in order to discover sequences of events that arenot supported by the GUI, so a bug is detected. Takala et al. automatethe GUI testing by defining some keywords that appear in the applicationbuttons [26]. Then, with the help of Hierarchy Viewer they get informationabout in which elements these keywords appear (on the element ID, on theelement text, etc.) and they can automatically touch on them with mon-keyrunner. In these two papers, they test the GUI automatically but eitherin a random way or using some previous information about the application.There is no automatic detection of the elements that should be clicked.

Anand et al. enhance the process of automatic testing by analysing theGUI and touching on every element on the screen [1]. In this way they ensurethey are touching the buttons that make the application do something.

Dai et al. go one step further and perform automatic (random anddirected) user inputs in order to profile the network traffic [10]. The directeduser inputs are generated by recording the actions of a human using theapplication. Then they repeat these inputs along with inputs on similarplaces. For example, if a human pressed on an image in a given screen, thenit should be proper to click on other images in that screen. Finally theydetect the elements on the GUI that are buttons (marked by Android asclickable) and they automatically click on them.

8

2.2. ANDROID APPLICATIONS CHAPTER 2. BACKGROUND

We considered the very last procedure as the best way to achieve ourgoal, since we want the whole process of profiling completely automaticwithout intervention of any human. Random user input are not interestingin our case since we want a human-like behaviour.

Wei et al. profile applications analysing the permissions the app has,registering the user inputs, system calls and network traffic [28]. Anyway,they do not have automatic inputs. This paper is interesting for us becauseit explains how to properly profile an Android application.

Once the execution traces are automatically generated, we have to com-pare them with the human-generated traces. An example of comparisonof execution traces is given by Cornelissen and Moonen, where they use amatrix having the two traces under comparison as axes [8]. The entriesfor which column matches the one in the row are colored. For example, ifrow X and column Y are the same system call, then, the entry located incoordinates (X,Y) will be coloured. Thus we can identify execution tracespatterns appearing in both execution traces. We will follow this approachto perform our trace comparison, explained in chapter 5.

2.2 Android applications

In this section we explain the basics of Android applications developmentfocusing on the areas that are more interesting to make this thesis workunderstandable. First we give an overview of the elements that define anAndroid app and then we explain in detail how to define the GUI. All thecontent of this section is based on Android Application Fundamentals sectionof the Android Developers web site [4].

2.2.1 Elements of an Android application

An Android application is a program written in the Java language. Thisis compiled to Dalvik code, which is the natural language of the virtualmachine running on the phone, and assembled into a package that can beinstalled on a device. This package has an extension .apk.

Application components are the essential building blocks of an Androidapplication. There are four different blocks: activities, services, contentproviders and broadcast receivers. Activities represent a single screen of anapplication with a defined user interface. A service is a component runningin background to perform tasks when the user is not using the applica-tion. Content providers manage the shared data of the application. Finally,broadcast receivers are responsible of responding when the system makes abroadcast announcement, like low battery.

Among these four types of blocks, activities are the most importantones for us since our approach to build Monkey Gamer is based on a statemachine and the screens are the different states. In normal applications theactivity-state mapping works. For example on an email application there

9

2.2. ANDROID APPLICATIONS CHAPTER 2. BACKGROUND

are three activities, corresponding to three GUIs: inbox, compose and read.They have different elements and every time you go to the compose activityyou get the same GUI. Another example is the Tic Tac Toe Free, for whichsome screens were showed before in Figure 1.2. Here each screen is a differentactivity. These GUIs are defined using an .xml file as it will be explainedlater in 2.2.2.

However, in most games the activity-state mapping is not obvious. Fewgames have their GUI designed as an .xml file, like the Tic Tac Toe Free.Normally they are implemented using game engines that allow dynamicinterfaces so that the whole game (with its multiple screens) can fit into asingle activity. As we said before we consider a screen an image displayed bythe game that is significantly different from the others shown before. Thisdifference turns into different interactive elements, which are the triggers togo to another state. When two screenshots are that similar, they must havethe same interactive elements on the same places, so we can consider thatthey correspond to the same screen. Of course, two screens with differentactivities will be considered immediately as different screens.

Another element that every Android app must have is the Android Man-ifest. All the components present in the application must be declared in this.xml file. Focusing again on activities, in the Manifest file we find the ac-tivity list with their properties (like the orientation). There is always anactivity that is labelled as main activity, meaning that is the one whichshould be shown first. Monkey Gamer will launch the game by the mainactivity. Every app must also have a package name. This identifies theapplication uniquely among all the apps available for Android. This pack-age name is also specified in the Android Manifest. The package name isimportant because it is needed to install the game on the device. It alsolets us know if we are inside the game or not if the current package ID isdifferent to the game one. We find also the permissions that the applicationhas, meaning which tasks the app is allowed to do (like reading the contactslist or sending sms). This is interesting for security analysis.

2.2.2 GUI design

In some cases the GUI is defined by a .xml file. This file, called layout,contains all the elements that are shown in the screen like text labels, images,text input fields or buttons. There is one layout file per activity on theapplication. Every element is defined by an xml tag in the layout file statingthe type of view, the ID and other parameters. Each one of this elements iscalled “view”. They define a tree, where there is parent view and childrenviews that can in turn have more children views. An example obtained withHierarchy Viewer from Tic Tac Toe Free game is shown in Figure 2.1 .

We can see the tree with the views. These views have a collection ofproperties. One of these properties is the clickability, which indicates if aview is interactive or not. The view “id/logo” and the views conforming the

10

2.3. GAME ENGINES CHAPTER 2. BACKGROUND

Figure 2.1: View tree and screenshot from the Tic Tac Toe Free main screen

menu buttons (“id/oneplayer”, “id/twoplayer”, “id/moregames”, “id/op-tions”), that are bounded by a grey rectangle in the screenshot below thetree are of special interest. Looking in detail at the view properties we willbe able to see that the menu button views have their clickable property setas true. This lets us know that these views are interactive.

The games that do not define their GUI in the described way, use thegames engines. Usually these games have just one activity. The layoutfor this activity has only one view, for which the content is dynamic andcontrolled by the engine. The images shown in this view are hard coded.So, by analysing the layout file we can not know anything about the GUI.

In this thesis the difference between a game whose GUI is defined by alayout file and a game created with a game engine is very important becauseMonkey Gamer needs to know how it can detect the clickable elements inboth cases.

2.3 Game engines

Game engines are systems that allow the games developers to abstract thedetails of doing common game-related tasks, like rendering, physics, andinput, so that developers can focus on the details [27].

These systems allow developers to use other languages than Java andcode with an environment different than the Android SDK to increase flex-

11

2.4. DESCRIPTION OF USED TOOLS CHAPTER 2. BACKGROUND

ibility. This makes much harder to get any information about the screensfrom the compiled code as we can do with the .xml defined games. A studyof the most used engines for Android is shown in section 3.1.

2.4 Description of used tools

In this section we are going to list and explain the tools used in this thesiswork. Some of them have been partially already introduced in previoussections. In cases where no alternative is presented there are no other toolsthat can perform the same tasks.

2.4.1 Real APK Leecher

Real APK Leecher is an open source Java application that is able to down-load .apk files to a computer from the Google Play Store [23]. Officially the.apk files can only be downloaded from the Android app of the Google PlayStore straight to the phone. Using this tool we are able to download theapplications installable .apk file to our computer.

This is needed because monkeyrunner requires an .apk file to installapplications on a device, but also since we need to decompile the packagebefore installing it. The decompilation is done with Android apktool.

2.4.2 Android apktool

Android apktool is an open source tool for reverse engineering Android apps[5]. It disassembles the .apk file into the smali code, resources, assets and theAndroid Manifest. The smali code is the result of decompiling the Dalvikcode, which is executed on the phone. The resources include files like images,sounds or the layout .xml files. In assets we find things like the fonts used inthe program. In our work, the most important file is the Android Manifest,which contains the information about the activities, so we will need to parseit.

2.4.3 Hierarchy Viewer

As we introduced before Hierarchy Viewer is a tool available in the AndroidSDK that is able to show the hierarchy of views of the current activitydisplayed on the display of the device [12]. In other words, using HierarchyViewer, we can see the views tree of the present activity.

Hierarchy Viewer is intended for debugging and optimizing GUI whendeveloping apps. In our case we will use it to analyse GUI of existing gamesand classify them.

Hierarchy Viewer requires a developer built of Android to be used ona real device, as Android indicates on the Hierarchy Viewer specifications.This tool is only intended for developers who want to analyse the GUI

12


of their own applications, and not to let the public analyse everybody’sapps. An Android developer built can be obtained recompiling the operativesystem and setting the built as developer or downloading and installing somenon official Android versions, like Cyanogenmod [9]. In this thesis we haveworked with both solutions and they are perfectly usable. We realized thatthe Cyanogenmod version was less stable than an official one so for longexecutions we used an official Android distribution.

2.4.4 monkeyrunner

Monkeyrunner is also found in the Android SDK. It is the most importanttool used in this work. It implements an API for writing a Python programthat can automatically perform some tasks on a device or on an emulator.It can install an application, launch an activity, touch somewhere in thedisplay, swipe from one place to another, take a screenshot or press a physicalkey among others.

The limitations of this tool are that it can not perform multi touchgestures (like pinch or rotate) and that is only able to touch the screen whenreceiving the pixel coordinates. So, if we want to touch on a determinateview, we need to know in which position of the screen it is while a directaccess is not possible. Also, to do swipe gestures a real device is required,in other words, the swipe command is not working with an emulator.

The API implementation is available in the monkeyrunner web site. Oncethe Python code is written, when monkeyrunner is executed it converts thePython code into Java, as monkeyrunner is actually not Python but Jython[15]. This is somehow problematic because some external Python libraries(like OpenCV) are not working on Jython, so we have to use them separately.

2.4.5 Android View Client

Android View Client is as an open source project started by D.T. Milano [17],Android View Client extends monkeyrunner and simplifies script creationproviding higher level operations, like being able to touch on the display bygiving the ID of a view instead of the coordinates. Also, it gives the abilityof obtaining the tree of views present at any given moment on the devicedisplay with all the views’ properties.

Using this two abilities, we analyse the current screen on the display, getthe views, select the clickable ones and touch them giving as parameter theview ID. The disadvantage of using this tool is that the process of analysingthe screen takes several seconds. This is not due to this tool, but to theconnection between the ViewServer on the phone and the computer, so itwould not be solved by using another solution.

As Hierarchy Viewer, Android View Client requires a developers built ofAndroid to be executed.

13


2.4.6 OpenCV

OpenCV is an open source library for image processing that implementsan API for Python language among others [19]. It implements methodsto calculate the Hough Transform of pictures, and detect lines, circles andcontours.

This is used to detect shapes in those screens fow which the GUI is builtby a game engine, so the elements are not described as views.

2.4.7 Python Image Library

Python Imaging Library (PIL) is a free library distributed by F. Lundhand Secret Labs AB [22]. It implements another API to easily manage andprocess images in Python. Its main advantage is its capability to processimages pixel by pixel, which will be very useful to compare the screenshotswith the previous ones in order to know if they are repetitions or new.

Compared to OpenCV, PIL is less powerful when processing full images,but it is better when comparing pixels. Due to this fact, PIL is used in thescreenshots comparisons and OpenCV is used to detect objects, since thisfeature is not available on PIL.

Every image processed by PIL will be a 16 bits Red-Green-Blue (RGB)image. This means that every colour is codified by 16 bits. This gives 256possible values for brightness with each color.

2.4.8 strace

In order to collect traces from the games that we want to profile we utilizestrace a Linux tool to collect system calls and signals [24]. strace is includedin Android, as it is Linux based.

2.4.9 tcpdump

Another tool used to get traces is tcpdump, a packet analyser. It capturesall the network traffic recording, source and destination IP addresses, sizeand a time stamp [25].

2.4.10 Android Debug Bridge

Android Debug Bridge, more known as ADB, is a command line tool thatallows to execute commands on an Android phone from the computer [7].It is formed by a daemon running on the device, a server running on thecomputer and a client where you execute the commands. The client isconnected to the daemon using the server.

With this shell we can execute normal Linux commands on the phonefrom the computer and also the specific Android commands. Using ADB wecan launch strace, or more specific Android tasks to know which packages

14

2.5. OBJECT DETECTION CHAPTER 2. BACKGROUND

are running on the device or which activity is being shown on the screen atany moment.

Hierarchy Viewer, monkeyrunner and some other tools included in theAndroid Development Kit also create ADB clients that use the ADB serverto communicate with the device.

2.5 Object detection

As it was said before, when the GUI is not defined by the views, we performan object detection using the Hough Transform. This transform is appliedin [11], where the Duda and Hart describe an algorithm for detecting linesand circles in images.

In order to detect lines, the Hough transfrom parametrizes using theslope-intercept space. Duda and Hart proposed to use ρ-θ (radius and angle)parameters. Thus, collinear points will have the same ρ-θ values so theproblem to detect lines gets reduced to finding points with the same ρ-θ as other points. In the same way, we can detect circles with a properparametrization, like a-b-c, (a,b) being the centre and c the radius. Forevery point we define circles around it, and the intersection of some of themin one point, will define a real circle in the image. These algorithms areimplemented by the OpenCV library and enhanced. There are methodsthat detect not only lines and circles, but contours.

When the screen is defined using a game engine, we can not use AndroidView Client, so we analyse a screenshot of the image displayed using thesemethods. They give us the information about where rectangles, circles andother contours are. Then we directly touch on their coordinates. In thisthesis the difference between a clickable view, which is defined as clickableon a XML layout, and an object, which is detected using this procedure, isreally important. We will consider both as candidates to be touched, butthe procedures to get and touch them are completely different.

15

Chapter 3

Android games

development

technologies

The main goal of this thesis work is to develop Monkey Gamer. In orderto know its scope of application, we decided to analyse Android games andin this chapter we are going to present the current development technolo-gies. This analysis is motivated by the fact that Monkey Gamer is onlyable to play with those games that do not require time-constrained inputs.Analysing some games we have an idea of how many games will be coveredby our solution.

Also we want to know how many games use XML layouts to definethe GUI and how many are developed using game engines. This secondclassification is motivated by the fact that we will know how much effortwe should put on developing the object detection or if just analysing XMLscreens with Android View Client would be enough to cover most of games.

The classification has been made using the top one hundred most down-loaded free games in the Swedish Google Play Store. As we commentedbefore, Google only allows to access the national Play Store of the countryof connection, so the Swedish Store was the only market we have had accessto.

The selection of the one hundred games was done on November 28th2012, and all the details about the analysis and the complete results can beconsulted in appendix A.

3.1 Classification by engines

In order to do this classification, we downloaded and installed the top onehundred games on a device. Then, using Hierarchy Viewer, we could identify

16

3.1. CLASSIFICATION BY ENGINES

CHAPTER 3. ANDROID GAMES DEVELOPMENT

TECHNOLOGIES

(a) AirplayView in Tetris Free

(b) Cocos2dxGLSurfaceView in Unblock Me Free

Figure 3.1: Views tree of two engine games got with Hierarchy Viewer

the views present in the game. When developing a game using an engine,there is usually only one activity, with only one significant view. This viewis often identifiable because it covers the whole space on the playing screenof the game.

In order to know which engine has been used to create this dynamic viewwe can just check the ID of the view, which is often named as the used gameengine. For example, in the game Tetris Free, if we analyse the views tree onFigure 3.1a, we can see that a view (which corresponds to a view coveringthe entire display) has the ID ”AirplayView”. This lets us know that thisgame has been developed using the Airplay game engine. Another exampleis the Unblock Me Free game, where the Cocos2D engine used is mentionedon the ID of the main view, as you can see on figure 3.1b. This is true inmost of the observed cases.

Anyway, in a few cases the ID value of a view does not give us cluesabout the used engine. In these cases we have to analyse the smali code gotfrom disassembling the .apk file of the game with apktools. The smali codeis organised in folders where it is possible to clearly see the packages thatform the application. Every game is usually made by the packages built bythe developers and some packages that work as a base for the application,like for example, the game engine. Checking here we are able to find thepackage that belongs to some game engine, so we know which game enginehas been used.

Finally, after studying the one hundred games, we got the results shown

17

3.2. CLASSIFICATION BY USER INPUTS


TECHNOLOGIES

Figure 3.2: Distribution of games by GUI implementation

in Figure 3.2. This graph shows the top five most used game engines and thenumber of games that use XML layouts to define their GUI. Those enginesthat are only used in one game are grouped in the others column. Thereare few games (7 out of 100) for which we were not able to identify the usedengine. Those are placed in “others” column as well.

The chart shows that the number of games using XML is below 10%.After getting this result, we considered really important to implement theobject detection feature in Monkey Gamer, so we are able to profile gamesimplemented using engines.

Considering the big number of different engines, we believe that objectdetection on the screenshots of the games is the best approach because weare able to use it independently of the engine used. Other approaches, likegetting information from the smali code, would be harder because everyengine codes the game in a completely different way, so we would have toimplement a specific solution for every engine.

3.2 Classification by user inputs

This classification gives an idea on how many games will Monkey Gamer beable to play with. As we said before, it will be only able to play with thosegames that do not need inputs in a time sensitive fashion In other words,they work as a state machine: they change the state only when an input is

18

3.2. CLASSIFICATION BY USER INPUTS


TECHNOLOGIES

Figure 3.3: Distribution of the input requirements in games

done, and not because we did not touch the display when it was required.For this classification we considered as “time-dependent” games those

games like driving games or those that you have to control a character.“Time-independent” are those like puzzle, board or gunnery games. On adriving game you have to turn when it is required (when there is a turn onthe circuit), on a platform game you have to jump when the evil charactersare close to your character. On the other hand, in a board game you canput your tab whenever you want, or in an archery game, you can throw thearrow when you desire.

It must be said that this classification is not precise as it is really hardto say if a game is like a state machine or not. Sometimes some parts ofthe game are like a state machine and others are not. Anyway, we did ourbest and we played to each game in order to be able to say something aboutthem.

Apart from the time-dependent and time-independent classes we definethe “Intelligence” one. Intelligent games are time-independent but they needsome human knowledge to be played, like quiz games. These games requiretyped text as inputs (usually to answer the questions), so Monkey Gamerwill not be able to play such games either. An example of this category canbe Logo Quiz [16].

The result of this analysis is shown on Figure 3.3. In our sample, thenumber of time dependent and independent is balanced around the 50% andthe games that require human intelligence are not so many (less than 5%).

19

3.3. PERCENTAGE OF GAMES COVERED


TECHNOLOGIES

3.3 Percentage of games covered by Monkey

Gamer

After this analysis we can estimate the proportion of games that MonkeyGamer can play.

If we did not implement the object detection feature, we could only playto the XML defined games that are time independent. XML games are 9%of the games. The time independents are 45% of the games. Multiplyingthe two percentages, we get a coverage of about 4% which is really low andMonkey Gamer would not be an interesting and generic solution.

Thus, the implementation of the object detection is really important inMonkey Gamer. With this feature, we are able to play all time independentgames, which amounts to 45% of the games in our sample. This results areshown in table 3.1.

without object detection with object detectiongames covered 4 % 45 %

Table 3.1: Games covered by Monkey Gamer

Of course we have to consider that this is just an estimation based ona overview analysis that only covered the most downloaded one hundredgames in the Swedish market. It would be desirable to test Monkey Gameron a selection of more than one hundred games and check if it can properlyplay to them or not, in order to have exact results. This was not possible inthis thesis work due to time constraints. Anyway this estimation gives usthe idea that Monkey Gamer will be a good approach to potentially profilenearly half of the Android games.

20

Chapter 4

Monkey Gamer

In this chapter explain in detail how Monkey Gamer works. Monkey Gameris able to play games automatically and collects execution traces. MonkeyGamer is implemented using Python and designed to run on a Linux ma-chine. The game is executed on a real Android device connected by a USBport to the computer. The Android phone can be connected to the internetusing any available Wi-Fi or 3G connection.

4.1 Architecture overview

Monkey Gamer is structured in eight blocks or modules that are related asshown in Figure 4.1. The <Main Controller> is responsible for bootingand proper ending of the program, and it launches the <Disassembler> ,the <Installer> and the <Screens Controller> modules. It takes as param-eter the .apk game installable file. The <Disassembler> is responsible fordisassembling the .apk package. The <Installer> installs the game into thedevice. The <Screens Controller> is in charge of the state machine thatcontrols Monkey Gamer, as we consider every screen as a different state.The <Trace Collector> is launched at the beginning and execution tracesare collected throughout the process. The <Screen Analyser> is able todetect if a screen is new or not. If it is not, it detects which of the previousscreens it corresponds to. <Screen Analyser> also detects the views thatare clickable with the help of Android View Client. For those games codedwith engines, the <Object Detector> analyses a screenshot and gives the co-ordinates of possible interactive objects. The <Input Generator> throughmonkeyrunner, touches or swipes on the clickable elements obtained before.

This is a summary of the main functions of each module. They will beexplained in detail in their own subsection. To see how all this process iscarried on by the different modules, we can see Monkey Gamer flow chartin Figure 4.2.

Monkey Gamer receives as input the .apk file of the game that we want to

21

4.1. ARCHITECTURE OVERVIEW CHAPTER 4. MONKEY GAMER

Figure 4.1: Monkey Gamer blocks diagram

profile. Disassembling and installation are done. Then, with the informationfrom the Android Manifest we know which activity is the main one. Themain activity is the one that is shown at the beginning of the execution ofan application. Once it is launched, we start to collect execution traces.This cannot be done before because the process ID (PID) is required to getthe game system calls, and this is assigned when the application boots.

Then, once we have the first screen displayed, we get the current activityID and current package name and we take a screenshot. This step is thebeginning of every iteration on the main loop of Monkey Gamer. With thisdata, we check if (for any reason) we are outside of the game. For example,we could end on the web browser, seeing the game developers web page. Inthis case Monkey Gamer forces to get back to the game. Then we analyze ifthat screen is new or not, if we continue inside the game. When the screenis new, we have to load the view tree from the device, get the clickableviews and if necessary, detect objects. We save all this information to use itlater, because whether the screen was previously shown, then we can loadthe saved information instead of gathering them again.

After that, the next element to touch is calculated, in order to touch onevery element in every order. When this is achieved, we mark that screen ascompleted. After that, the element obtained is touched or a swipe gestureis done over it. If all the screens we have discovered are completed, thenwe save the results (traces, screenshots, etc.) and Monkey Runner stops. Ifnot, we start the loop again.

As it is very hard to complete all the screens due to the path explo-sion problem, Monkey Gamer can be stopped at every moment In this case

22

4.2. COMPONENTS CHAPTER 4. MONKEY GAMER

Figure 4.2: Monkey Gamer flow chart

Monkey Gamer also finishes properly, as if all the screens were completed.

4.2 Components of the solution

In this section we explain in detail the tasks performed by each module thatforms Monkey Gamer. All the flow charts shown in this section fulfill thefollowing pattern:

• when a task is written in capital letters means that is a call to thatMonkey Gamer module.

• MR in brackets stands for monkeyrunner, AVC for Android ViewClient; meaning that these tasks are done using some method im-plemented by these tools. Other names in brackets have a similarmeaning.

• blue boxes are tasks, yellow rhombuses are decisions and green par-alellograms are inputs and outputs.

23


4.2.1 Main Controller

This module is the one to be executed by the monkeyrunner tool. It receivesthe location of the .apk file of the game in the file system of the computer.So the execution command is

monkeyrunner Main.py <apk file location>

Note that Monkey Gamer is executed as a monkeyrunner script and not asa normal Python script. Its flow chart is shown in Figure 4.3, so we explainthe tasks that it does.

Figure 4.3: Main Controller flow chart

The initialization of the program includes the check of the parametersgiven and the creation of the temporary folders where the files are goingto be placed. After that, the <Disassembler> module is called. Then,monkeyrunner is connected to the device (that should be plugged into thecomputer via USB) and the Android View Client is created. The view clientis responsible of communicating with the view server on the device to retrievethe information about the tree of views shown at any moment. The following

24


step is to install the game, so the <Installer> module is called. After that,all the processes of Monkey Gamer lay on the <Screens Controller> module.When all the game is covered or the stop signal (Control + C) is given, allthe files are copied to definitive folders, the temporary ones are deleted andMonkey Gamer stops.

Files returned as results include the full disassembled .apk file, screen-shots after every automatic input, detected objects on engine defined screens(both an image with the screenshot and the objects marked over it, and atext file with the coordinates), views trees for every screen, activity ID ofeach screen, system call and network traffic events record.

4.2.2 Disassembler

This small module is in charge of disassembling the .apk file into all itsfiles. Its simple flow chart is shown in Figure 4.4. It receives the locationof the .apk file and starts a subprocess that calls the apktools tool, whichdisassembles the package. All the files are saved in a temporary foldercreated before, which is also given as parameter to apktools.

Figure 4.4: Disassembler flow chart

4.2.3 Installer

The <Installer> module is also a simple one. It installs the game into theAndroid device connected to the phone. The installation is done althoughthe application is already installed. This guarantees a fresh installation everytime Monkey Gamer is executed. The flow chart can be seen in Figure 4.5.

The monkeyrunner method to install an application requires the .apk fileand the package name of that application. The package name is obtainedfrom the Android Manifest extracted in the disassembling process. Then, thegame is installed on the device. This process takes few seconds, dependingon the size of the application package.

25


Figure 4.5: Installer flow chart

Installing the game does not mean launching it.

4.2.4 Screens Controller

This module is responsible for the state machine that controls MonkeyGamer and the game, along with <Screen Analyser> module. It is themost important module of Monkey Gamer. Its flow chart is shown in Figure4.6.

Screen Controller set up process

Once the game is installed on the phone, <Screens Controller> is called andits state machine launched. The first task is taking the main activity ID,which is the activity that must be launched at the beginning of the game.The ID is obtained from the Android Manifest file.

Then, a screenshot of the initial status screen of the phone is taken. Thisis useful later to know if for some reason, we are back to it, like for example,after pressing the ”exit” button on the main menu. Usually that screen isthe Android home screen, unless there is another application being executedbefore the game.

After this, the main activity is launched using a monkeyrunner method.To call this method the package name and the activity we want to launch areneeded. A sleeping time is set in order to let the game load before startingany automatic input. This time can be changed, but a good value is aroundten seconds. This value has been obtained after checking the loading timeof different games.

After the game is loaded the <Trace Collector> module is started. Thiscannot be done before since the application must be running to be able to

26


Figure 4.6: Screens Controller flow chart

start collecting system calls traces from strace. All this procedure is thefirst phase of the Screen Controller process. Now Monkey Gamer is readyto start processing the first (or whichever) screen.

Screens Controller main loop

Every iteration of this loop is a single process of analysing the present screen,knowing which element must be touched and touching it.

First of all, some seconds are spent waiting in order to get the presentscreen completely loaded. From experimental observations, two seconds isa good value for this timer. This sleeping time is needed because usuallythere are animations when switching between screens and these should beinvisible for Monkey Gamer.

When the screen is stable, a screenshot is taken with the monkeyrunnermethod. This screenshot will work as trace (Monkey Gamer keeps a screen-shot of the device after every automatic input) and to compare the presentscreen to previous ones, to know if it is repeated or not.

The screenshot is compared to the one taken before launching the game.With this we know if we are back into the initial screen. We will intialy

27


analyse the case in which this is not true.Then we get the package name of the displayed application, to know if

we are outside the game in study. Also we take the activity ID. This is donethrough the ADB shell, where a command returns the focused activity andpackage. If the present package name is not the package name of the game,we know that we are outside the game. Again, we will initially analyse thecase in which this is not true. Then, we ascertain if we are in a new screenor if this screen was showed before. This algorithm is explained later insection 4.2.4.

If the screen is new we assign it a progressive screen number. Then,with the Android View Client method we retrieve the information aboutthe views tree and we save it into a file. This retrieving takes some seconds,normally between five and ten, depending on the amount of elements theviews tree has.

Also we have to get the orientation looking at the Android Manifest file,where this value is given for every activity in the application. The orientationis needed since monkeyrunner swaps the X and Y axes depending on if thescreen is in landscape or portrait. All this information is stored in files orin memory.

Then, in the case that the screen in not new (in other words, was shownpreviously) we just have to load all the information about it.

When all the information about the screen has been obtained, a call to<Screen Analyser> module is done. It will analyse all the data and generatean automatic input.

A new screen will appear and the loop will start again. If after theinput the screen does not change at all, we will proceed in the same manneranyway. The main loop is repeated always after any automatic input.

In the case that we were back into the initial state screen, we will deletethe last element touched from the clickable elements list of the previousshown screen. This clickable element was responsible for exiting the game.Doing this we avoid to came back to the initial state screen again, becausethe button that takes the game to that screen will never be touched again.Then, the main activity is launched again. Unfortunately, there is no wayof recovering the game where it was before going to this state.

If the game goes out the application to another application, we just pressthe back key and we will be back again into the game, in the same placewhere we were before. In this case we also delete the last clicked element.

IsThisANewScreen algorithm

This algorithm is responsible for detecting if a screen was shown before ornot. In the case that the screen was shown before, the corresponding screennumber is returned. The flow chart of this algorithm is shown in Figure 4.7.

We take the present activity ID. If this activity ID was the ID of previousscreens we consider this as a potential old screen. Remember that oneactivity can hold multiple screens. If we cannot find this ID among the

28


Figure 4.7: IsThisANewScreen algorithm flow chart

previous ID’s, then this screen must be new. An activity can hold multiplescreens, but different activities must be different screens.

In the case we find the activity ID as the ID of some previous screens,we will take all the screenshots of these screens. We will have a screenshotof every screen after every automatic input. So, if we have been three timesin a given screen, there will be three different screenshots of that screen,presenting potential minor differences.

Then, applying an algorithm that we will explain later in section 4.2.4,we will compare this selection of screenshots with the one displayed at thatmoment. If some matching is above a threshold, we consider the screen isthe same as the one that matched. Then the screen is old and we can giveits screen number. If there are no matchings, then the screen is new.

Consider that the comparisons are done with every copy of the screen-shots for those screens that have the same activity ID as the shown one.This can take time, but as screens can vary little by little after inputs, it isbetter to compare with all of them, since after a long time, the same screencan look really different. This is shown in Figure 4.8, where it is possibleto see the differences among steps in the same screen in the Tic Tac Toe

Free game. These screenshots represent the same screen, but it is obvious

29


that the first capture is considerably different to the last one. Comparingthe present screenshot with all the copies we will ensure to detect properlythe screens, not duplicating or missing them.

Figure 4.8: Tic Tac Toe Free board screen differences among steps

Screenshots comparison algorithm

The comparison is done using two algorithms: histogram and pixel by pixelcomparisons. The use of these two algorithms is justified by the fact that thepixel by pixel comparison gives excellent results when the images are static.But when there are animations on the screens (for example, the backgroundis moving), then it fails. Hence we need the histogram algorithm, whichgives the similarity of the colours, light and tones of the image, withoutconsidering the position of them, so it works perfectly with animations.The use of the pixel by pixel comparison is justified because the interactiveelements should be the same and they have to be in the same place toconsider two screens as the same screen. By applying both of them weachieve a really good performance as will be shown in section 4.2.4.

The image processing library PIL, can calculate the histogram of animage. Each pixel is represented by 3 values corresponding to the brightvalues of the red, green and blue color components in the [0,255] range. ThePIL library is producing a histogram which is the composition of 3 differenthistograms, each one reflecting the distribution of the brigthness values ofone of the 3 colour components in the pixels of the screenshot under analysis.This makes a 768 elements histogram. This gives us an idea about whichcolour is predominant in the image or if the image is dark or bright. Considerthat every pixel will be counted three times, as there will be a bright valuefor the red component, another for the green one and other for the blue one.Each pixel of an image can be accessed just giving the coordinates. PILreturns an array of three elements, each giving the bright for each of thethree colours.

The histograms are compared taking each of the 768 elements from oneimage and from the other. The elements are compared one by one to deter-mine if the two screen are actually the same screen. A proper threshold is

30


selected for the comparison. The histogram comparison algorithm is shownin Figure 4.9.

Figure 4.9: Histogram comparison algorithm

In this example, histogram 1 has value 1000 in the position 0. Thismeans there are 1000 pixels with no red colour. In the histogram 2, thereare 3000 pixels on the first position, thus 3000 pixels with no red. As thedifference 3000 minus 1000 is over the threshold (threshold = 1000 in thiscase), the position 0 of the histograms is considered different. Nevertheless,the position 1 of the histograms is considered equal (257 and 300 pixels re-spectively). After comparing the 768 elements, a percentage of similarity isgiven.

A similar procedure is done with the pixel-by-pixel comparison. For bothimages, every pixel is taken. Consider that the intensity values for each pixelcan vary from 0 to 255. If the red, green and blue values of a pixel are close,then we consider that the pixel is equal. If just one of them is far, then thepixel is considered different. This closeness is given by a proper threshold,now set to <10,10,10>, for the three colour components. This threshold wasobtained after testing some images, similarly to the histogram one. Aftercomparing every colour for every pixel, we give a percentage of similarity.

In the example in Figure 4.10, we can see that in the purple comparison,the green value differs by 100, so pixel in coordinates (0,0) is considereddifferent. On the other hand, yellow comparison differs only by 5 in the

31


Figure 4.10: Pixel by pixel comparison algorithm

blue value, so the pixel in coordinates (1,1) is considered equal.After applying both algorithms, the similarity rate from the histogram

comparison is added to the percentage of the pixel by pixel comparison.This gives a value of equality of the images that varies from 0 to 2 (rangeℜ ∈ [0,2], 0 stands for completely different, 2 for completely equal), as boththe pixel by pixel and the histogram from 0 to 1. After testing with somescreenshots from different games, we set the threshold to 1.2. Those screen-shots comparisons above this threshold will be considered the same screen,and the ones that are below, different screens. Once that two screenshotsmatch, we stop comparing, as we have detected which screen is the presentone. If any comparison is below the threshold, then we know that the screenis new.

To clarify both algorithms, to explain better how we compute the sim-ilarity rate and how we decide if a screenshot is new or not, we show thepseudocode used to calculate it:

# histogram1 and histogram2 are two uni−dimens iona l 768e lements ar rays conta in ing the histograms f o r the twocompared images

# pixe l s Image1 and pixe l s Image2 are two ar rays conta in ingf o r every p i x e l on the images , the three co l ou rcomponents ; c oo rd ina t e s are g iven with the two f i r s tindexes and c o l o r with the th i rd

32


# xSize and yS ize i n d i c a t e the s i z e o f the images in p i x e l s# histogramThreshold i s the th r e sho ld to compare histogram

va lues (now s e t to 100)# pixThreshold i s the th r e sho ld s e t to compare p i x e l s ( not

s e t to 10)# f i na lThr e sho ld i s the th r e sho ld used to say i f a s c r e en

i s new or not (now s e t to 1 . 2 )

equalElements = 0f o r ( i =0, i <768 , i++) :

i f ( abs ( histogram1 [ i ]−histogram2 [ i ] )<histogramThreshold ) :equalElements++

s im i l a r i t yH i s t og ram = equalElements / 768

equa lP i x e l s = 0f o r ( i =0, i<xSize , i++) :

f o r ( j =0, i<ySize , j++) :i f ( ( abs ( p ixe l s Image1 [ i ] [ j ] [ 0 ] − pixe l s Image2 [ i ] [ j ] [ 0 ] )<

pixThreshold )&(abs ( p ixe l s Image1 [ i ] [ j ] [ 1 ] −pixe l s Image2 [ i ] [ j ] [ 1 ] )<pixThreshold )&(abs (p ixe l s Image1 [ i ] [ j ] [ 2 ] − pixe l s Image2 [ i ] [ j ] [ 2 ] )<pixThreshold ) ) :e qua lP i x e l s++

s im i l a r i t yP i x e lByP i x e l = equa lP i x e l s / ( xS i ze ∗ yS ize )

s im i l a r i t y = s im i l a r i t yH i s t og ram + s im i l a r i t yP i x e lByP i x e l

i f ( s im i l a r i t y < f i n a lThr e sho ld ) :r e turn Old Screen

Screenshots comparison algorithm parameters calculation

Now we will show some results of screenshots comparisons, indicating theaverage difference in the histogram comparison, in the pixel by pixel com-parison and the final equality result (the value between 0 and 2). This willjustify the selected parameters (100 for the histogram difference, 10 for thepixel by pixel difference, and 1.2 for the screens equality threshold). Alsoit will show how adequate is this method for comparing Android gamesscreens.

The screenshots for the images in test are included in appendix B. Theimage codes are set as follows: the letters stand for the game (AB for AngryBirds, BW for Bubble Worlds and TT for Tic Tac Toe Free), the first numberfor the screen and the second number for the screenshot copy of that screen.

The histogram diff. av. column shows the average of the 768 histogramdifferences for that comparison. The pixelbypixel diff. av. column showsthe average the differences for all the pixels for the three colours. TheEquality column is the final equality value obtained (the value between 0and 2) when using as thresholds 100 and 10 respectively. Remember that

33


Table 4.1: Comparisons of screenshots of the same screen

Im. 1 Im. 2 histogram diff. av. pixelbypixel diff. av. EqualityAB1.1 AB1.2 112,8 15,7 1,4AB2.1 AB2.2 40,5 1,6 1,9BW1.1 BW1.2 71,8 3,2 1,7BW2.1 BW2.2 135,3 6,9 1,5TT1.1 TT1.2 138,9 18,3 1,4TT2.1 TT2.2 16,6 3,9 1,9

Average 86,0 8,3 1,6

Table 4.2: Comparisons of screenshots of different screens

Im. 1 Im. 2 histogram diff. av. pixelbypixel diff. av. EqualityAB1.1 AB2.1 2707,7 79,5 0,2AB3.1 AB4.1 484,5 23,6 1,0BW1.1 BW2.1 1925,5 62,7 0,1TT1.1 TT3.1 2762,9 78,0 0,0

Average 1970,2 61,0 0,3

histogram diff. av. goes from 0 to the total number of pixels (around500.000) and pixelbypixel diff. av. goes from [0,255].

For the same screen comparisons, we got values of 86 for the histogramdifference and 8 for the pixel by pixel difference. Letting the system beingmore flexible, and considering the higher values for the different screens(1970 and 61 respectively), we think it is appropriate to select 100 and 10respectively.

Following the same approach, we select the Equality threshold as 1.2,considering the minimum value for the same screen comparisons (1.4) andthe maximum for the different screens comparisons (1.0).

4.2.5 Screen Analyser

This module is responsible for getting the views that are clickable for thepresent screen. If the screen is defined with an engine, it has to get thepossible clickable elements from the screenshot. Also it gets the editablefields, just to let Monkey Gamer fill them with some random text. Its flowchart is shown in Figure 4.11.

First, we should know if the screen is defined by an XML layout or anengine view is rendering the screen. This is known by analysing the viewstree. If there are views that are no Android views (like images or buttons),then we know that there is a view drawing the display that is controlled byan engine. In other words, if all the views are normal buttons, text labels,images, etc. we know that the screen is defined by an XML file.

34


Figure 4.11: Screen Analyser flow chart

Once we know this information, if the screen is defined by a XML layout,we proceed to check which views have the property “isClickable()=true”.This is done reading the properties of the views tree. For those clickableviews, we store their view ID since Android View Client needs the viewID to touch on them. If the screen is defined using an engine, we call the<Object Detector> module which is responsible for detecting the clickableplaces in the screenshot.

The clickable views ID or the coordinates of the objects are stored intoan array. Then the next view/object to click is obtained following the Get-NextElement algorithm which will be explained below. If there are editableviews (set with the property “isEditable()=true”), all of them will be takenand filled by the <Input Generator> module. Finally, the next elementto click obtained previously is given to <Input Generator> module to betouched or swiped.

35


GetNextElement algorithm

Let C be the number of clickable elements on a screen (can be either views orobjects detected in the screenshot) and P = C! the number of permutationsof these elements.

For every screen detected in the game, two parameters are stored: lastClickedand permutation. The first one specifies the position in the array of clickableelements of the last one that was touched. The second is the permutationnumber that is being used now. This parameter goes from 0 to P-1. Thealgorithm is shown in Figure 4.12.

Figure 4.12: GetNextElement algorithm

If the present screen is new, then the first element of the clickable el-ements array is touched. The parameters for that screen are stored aslastScreen=0, permutation=0, meaning the first element was already clickedand that we are in the first permutation.

When the screen is not new, we load the stored parameters for thatscreen, let us say they are lastClicked=K and permutation=Q. If K+1 isbelow C, meaning that the new elements to click exist, we calculate per-mutation number Q and we touch the element stored in the K+1 positionfor that permutation. Then values are updated to lastClicked=K+1 andpermutation=Q.

If K+1 is equal to C, meaning that an unclicked element does not exist,we look at the permutation value. If Q+1 is below P, meaning that untried

36


permutation exists, we calculate the permutation Q+1 and touch its firstelement. The values that are stored are lastClicked=0, permutation=Q+1.

Finally, if Q+1 is equal to P, all the permutations have been tried. Thus,the screen has been fully covered (all the elements in every order havebeen touched). We save that fact and the back button is pressed by the<Input Generator> module. The values saved are, like at the beginninglastClicked=0, permutation=0. This is due to the fact that, possibly wewill came back later to this screen and we will need to start a second lap.This is needed because, although this screen is complete, it can give accessto another screen that is not.

The method that calculates the permutations is implemented in Python,and it calculates all the permutations preceeding the required one. Forexample, if the Qth permutation is needed, then permutations from 0 toQ-1 have to be calculated. Once a permutation is calculated, it is storeduntil the next one is needed for every screen. Considering that the numberof permutations is P = C!, this calculation can take considerable time forthe latest permutations when the number of clickable elements (C) is high.

4.2.6 Object Detector

This module performs the analysis of screenshots in order to detect sharpedges, contours or shapes that could be a button or a touchable element onthe screen. It is used when on a screen there is a view that is controlledby an engine, so from the views tree itself we can not get any informationabout the GUI. The flow chart of this module is shown in Figure 4.13.

At the beginning, the coordinates of the view controlled by the engineare gathered. Usually it covers the entire display, but not always. We needto adapt these coordinates to the orientation of the screen, so we read thisvalue from the information we stored about the screen.

Sometimes, there are also some engine views overlapped. Since we arelooking for objects on the screenshot, we do not care about in which viewthey are. Thus we can remove overlapped engine views. Then, we crop thescreenshot to those areas that are rendered by an engine view.

Using the OpenCVmethod to detect edges and then contours, we retrieveall the contours on the image. Once we got them, we remove the smallones. After some tests, we decided to consider as small contours those withan area smaller than 500 square pixels. They represent objects which aretoo small to be interacted with for a human user. For each contour, itscentral coordinates are calculated, again adapted to landscape or portraitview depending on the screen set up and stored. Following this process weget the list of object coordinates for the screenshot.

For further information, an image file containing the screenshot and overit the detected contours, is saved for every screen. Also we save on a file thecoordinates list in text format.

Now we are going to show some screenshots with the objects that we

37


Figure 4.13: Object Detector flow chart

detect on them, in order to show the capability of Monkey Gamer. Theresults are shown in Figures 4.14 and 4.15. These results are obtained fromthe execution of Monkey Gamer in two games: Angry Birds and Bubble

Worlds. Both games are implemented using engines.Green rectangles bound the contours detected and blue dots are the

central point for each contour. The blue points coordinates are the onesthat are considered the coordinates of the object and the ones that will besent to the <Input Generator> . As we can see, the object detection isworking properly.

• In Figure 4.14a we can see how the buttons for level selection aredetected correctly, as well as the back button on the bottom-left corner.

• In Figure 4.14b some of the buttons are detected and also the bird onthe slingshot, but some are not, like the pause button.

38


(a) Screen 1 (b) Screen 2

Figure 4.14: Object detection in Angry Birds

(a) Screen 1 (b) Screen 2

Figure 4.15: Object detection in Bubble Worlds

• In Figure 4.15a all the buttons are detected although there is a non-touchable element detected on the bottom-right corner and in the topleft corner.

• In Figure 4.15b the bubbles on the top are detected. In this case,the letters in the bottom are also detected but they are not touchableelements.

As we commented before, this method is not the most precise, but clearlywe are detecting almost all the touchable places on screens. There is theproblem that some non-touchable elements are also detected, but the conse-quences of this are just that Monkey Gamer will touch on these places andnothing will happen.

39


4.2.7 Input Generator

<Input Generator> is the module in charge of sending inputs to the device.These inputs can be pressing the back key, typing some text, touching aview or touching or swiping on a detected object coordinates. Dependingon the needs of Monkey Gamer it works as shown in Figure 4.16.

Figure 4.16: Input Generator flow chart

• back: monkeyrunner directly implements pressing physical back key.

• editable field: if a form must be filled, first we touch it to get the focusand then a random chars string of random length is generated andtyped into the form by the monkeyrunner typing method.

• view: when we need to touch on a view (a clickable view defined assuch in a XML layout), Android View Client can touch it just givingthe view ID.

• object: if we need to touch a detected object, for which we havethe coordinates, first we randomize the selection of input to make.The inputs can be swipes (in multiple directions) or simple touching.Then, with the coordinates, the monkeyrunner methods (either touchor drag) generate this input on the device.

Over objects, swipes can be in 16 different directions, one every 22.5degrees (360/16 = 22.5). The direction is also selected randomly. We con-sidered that 16 directions are enough since it is hard for a human to be moreprecise than a difference of 22.5 degrees. Swipes are not done over viewssince a normal Android view will never respond to a swipe gesture. Thisonly has sense if we are doing it over an object detected on an engine definedscreen.

4.2.8 Trace Collector

Trace collector is the module responsible of collecting traces from the exe-cution of the game. The traces that this module collects are system callsand network traffic. These are collected once the game has been launched.The flow chart is shown in Figure 4.17.

40


Figure 4.17: Trace Collector flow chart

Once the application is loaded, the PID of the app is got by executingthe ”ps” command in the ADB shell. The results are filtered by the packagename of the game, so the current PID is obtained.

Since we want to capture traces along all the execution of Monkey Gamer,we start a new thread of Monkey Gamer which launches strace. This toolis also launched through the ADB shell and it collects the system calls doneby a given process, in this case, the game process. Another thread is createdto launch tcpdump, which is responsible of monitoring the network traffic.The outputs of these two tools are stored into files. The main thread thencontinues the normal execution of Monkey Gamer.

The screenshots that are collected as traces are taken from the ScreensControl module, after every automatic input. The output of Monkey Gamercan also be considered as trace. Here we print the view ID or coordinateswhere we touch, if the application takes us to another package and its pack-age name, the detected screens and the lastClicked and permutation param-eters for each one, etc.

41

Chapter 5

Prototype evaluation

In this chapter we present the results of profiling games using MonkeyGamer. We explain how we tested Monkey Gamer, how we evaluated its per-formance and the similarities between its behaviour and a human behaviourwhen playing a game. We also show the evidences that make Monkey Gamera good unexpected behaviour tester.

5.1 Testing environment

5.1.1 Testing hardware

Once Monkey Gamer was developed, we set up a testing environment con-sisting of the following hardware:

• Dell Latitude E6320 laptop, Intel Core i7-2620M CPU at 2.70GHz x4, 8GB memory, running Linux Ubuntu 12.04 64-bit.

• HTC Sensation Z710e smart-phone, 1.2GHz QualcommMSM8660 Dual-Core, running Android Gingerbread 2.3.5 (developers built).

• LG-P990 smart-phone, ARMv7 Processor rev0, running Android JellyBean 4.2.2 (CyanogenMod).

5.1.2 Game selection

One of the objectives of the work was to have a tool that could be used onreal games, so we discarded the option of developing our own games andthen profile them with Monkey Gamer. We looked at the Google Play Storefor real games, with millions downloads. The games we looked for had tomeet some requirements like they must cover all the key points in MonkeyGamer.

42

5.1. ENVIRONMENT CHAPTER 5. PROTOTYPE EVALUATION

Since our evaluation criteria was set to test Monkey Gamer in a selectionof games, we decided to choose three that cover different categories, inputkinds and GUI implementations. These games are:

• Tic Tac Toe Free

• Bubble Worlds

• Angry Birds

Tic Tac Toe Free is the most downloaded tic-tac-toe Android game withmore than 5 million downloads. We decided to take a tic tac toe game sinceit is the simplest game we could find. It has a much reduced number ofscreens, the game final results are limited, and the required user inputs aresimple, then it is perfect to be used on a first evaluation. The Tic Tac ToeFree GUI is also an example of a game defined using a XML layout.

Bubble Worlds is the best rated game in the bubble shooting subcategory,and widely played, with more than 10 million downloads. Its complexity isone step above Tic Tac Toe Free, with more screens, uncontable final resultsbut simple user inputs (taps). It defines its GUI using a game engine,OpenGL, the most used game engine according to our analysis.

Angry Birds has become a classic in smart-phone games. Its more that100 million downloads just for Android endorse this fact, probably makingit the most downloaded smart-phone game ever. It is a complex game, witha wide variety of screens, uncontable final results and needs swipe gesturesto be played. It defines the GUI using a game engine, in this case, Cocos2d.

For us it was really important to cover both XML and engine games,including different game engines. With this selection we also cover the twotypes of inputs that monkeyrunner (and thereby Monkey Gamer) is able tomanage (i.e., tap and swipe). Furthermore, these games require internetconnection since they are free. Free games display ads or they have linksto buy extras for the game. Therefore we are testing the most importantfeatures in Monkey Gamer, like the clickable elements detection dependingon the GUI definition, different types of inputs, simple games with fewscreens and complex games with hundreds of them.

5.1.3 Testing procedure

Since Monkey Gamer takes several seconds to perform each input (first testsgive a time around 20 seconds), we need to execute it some hours in orderto generate the same number of inputs that a human generates in minutes(a human waits less than a second between inputs).

We executed these three games, once each, during the time shown intable 5.1 on the HTC phone while Monkey Gamer was running in the Delllaptop. The phone was connected to the internet via Wi-Fi to a general-purpose network (eduroam).

43

5.2. VALIDATION CHAPTER 5. PROTOTYPE EVALUATION

Table 5.1: Execution times of Monkey Gamer

Tic Tac Toe Free Bubble Worlds Angry BirdsTime (hours) 21 16 14

Short Monkey Gamer tests (less than 2 hours) have been done in theLG phone that runs a different Android version than the HTC. MonkeyGamer required minor changes to be able to work with Android Jelly Bean,but the results were the same as working with Gingerbeard on the HTC.These changes were mostly related with the way of parsing the informationobtained from the execution of ABD shell commands. The output of theADB shell commands varies from one Android version to another. Thisproves that Monkey Gamer is independent of the Android version runningon the phone.

Once all the execution traces and screenshots for the three games werecollected, we wanted to compare them with traces and screenshots obtainedby the execution of the game by humans. Therefore, we asked nine personsto play our selection of games, three persons for each game.

These people were required to play the game normally, as they woulddo it if they were playing for fun, during a time between 15 minutes and 30minutes. Monkey Gamer produced in the time detailed before in table 5.1the same number of inputs (i.e., taps and swipes) as the humans produced in15-30 minutes. When the humans were playing we collected the same tracesand in the same way as we did when Monkey Gamer was the player. Ac-tually, the traces are collected using the Monkey Gamer human mode, thatonly uses the <Main Controller>, <Installer>, <Disassembler>, <ScreensController> and <Trace Collector> modules. <Screens Controller> is justused to launch the main activity.

Observing the previous values for the execution times that Monkey Gamerrequires it is obvious that our solution is not very profitable in terms of time.Nevertheless we have to consider that once Monkey Gamer is developed, thecost of executing it during some hours (or even days), is really low in termsof money compared to the cost of a test person playing a few hours.

5.2 Validation

So far we got, per game, three human execution traces and one MonkeyGamer trace. These traces include the record of system calls made, theTCP packets sent and received (including IP addresses) and a screenshotafter every automatic input in the case of Monkey Gamer and a screenshotevery second in the case of human players.

We are going to compare the system calls made by the three humanswith the ones made by Monkey Gamer for each game. Then, we will com-pare the IP addresses that the phone connected to when the humans were

44


playing and when Monkey Gamer was playing. Finally, we will compare thescreens shown when then humans were playing and when Monkey Gamerwas playing.

These comparisons will turn out into figures that will show the similaritybetween Monkey Gamer’s and human’s playes and the viability of usingMonkey Gamer as an automatic profiler.

5.2.1 System calls comparison

To compare the system calls records we are going to follow the approachproposed in [8], as we explained before in section 2.1.

As we collected records of more than 100.000 system calls, per execution(both Monkey Gamer and human), we are going to divide each one intothree chunks. The first one will have the system calls from 1 to 10.000,the second one from 10.001 to 20.000 and the third one from 100.001 to110.000. Using these selections we compare the traces in an early phase, ina middle phase and in the late phase. These are selected to cover all therange of system calls (from the first ones to the last ones) and always havingthe same number of system calls on every chunk (in this case, 10.000, forcomputational reasons).

This splitting turns into having three chunks per player (both for humanand Monkey Gamer). Since we want to compare the three human traces wegot per game versus the single trace we got with Monkey Gamer, we haveto do nine comparisons per game, as shown in table 5.2.

Table 5.2: Comparisons per game, in brackets the system calls compared

Human 1 (1-10.000) Monkey Gamer (1-10.000)Human 1 (10.001-20.000) Monkey Gamer (10.001-20.000)Human 1 (100.001-110.000) Monkey Gamer (100.001-110.000)Human 2 (1-10.000) Monkey Gamer (1-10.000)Human 2 (10.001-20.000) Monkey Gamer (10.001-20.000)Human 2 (100.001-110.000) Monkey Gamer (100.001-110.000)Human 3 (1-10.000) Monkey Gamer (1-10.000)Human 3 (10.001-20.000) Monkey Gamer (10.001-20.000)Human 3 (100.001-110.000) Monkey Gamer (100.001-110.000)

Then, for each of these comparisons, we draw the matrix of similarity.Once we have it drawn, we search on it for diagonals. Diagonals are theresult of consecutive system calls in both sets, namely, “patterns” that arepresent in the Monkey Gamer and in the human trace. We consider that wecan draw a diagonal when in both sets there are at least two system callsin a row, or two diagonals split by a single system call that does not belongto any diagonal. This is better explained by the example reported in Figure5.1.

45


Figure 5.1: Detecting patterns in system calls sequences (step 1)

Figure 5.2: Detecting patterns in system calls sequences (step 2)

Let A-H be different system calls. We place the two system call sequences(trace 1 and trace 2) that we want to compare in the two axis of a table.Following the introduced approach, when on a column there is the same

46


system call as on a row, we colour the corresponding cell (in our case wecolour cells in green in figure 5.1).

Then, we define the diagonals over every diagonal of green cells on whichthere are at least two green cells in a row. If we find a single cell which isnot green, but will allow to connect a green cell to a diagonal, we colour itin yellow, and we let the diagonal go over it.

Thus, patterns like AABC*B and BC*B can be considered. The reasonfor considering as patterns sets of traces like these is that system calls canbe synchronous or asynchronous. This fact results in different executions ofthe same game in the same way turning into different traces because oneasynchronous system call can be called among some synchronous ones.

The diagonals we draw are marked with red. They represent patternsappearing in both sets of traces. Now we draw squares that cover all thediagonals, as seen in Figure 5.2. In this case, rectangles are coloured in deepred over the same red diagonals on the previous Figure. Doing this we donot count double sub-patterns, because they get enclosed into the biggestrectangle that bounds the parent pattern. For example, the sub-pattern ABis contained in the parent pattern AABC*B.

Finally, we just need to calculate the area covered by the rectangles outof the total area to efficiently compute how similar the two sets of tracesare. For example, in the case shown in the Figures, there are 71 colouredcells out of 121. This turns into a 59% similarity measure.

We applied this procedure to all the comparisons between human andMonkey Gamer system calls records, getting the results shown in tables 5.3,5.4 and 5.5. Column ”pattern (%)” indicates the percentage of area cov-ered by rectangles, as explained above. Therefore, this column indicates thepercentage of similarity between system calls sets. ”Common SC” containsthe number of unique system calls in common between the human and theMonkey Gamer traces. The other two, ”human SC” and ”MG SC” showthe total number of unique system calls made by the human and by MonkeyGamer respectively.

The results indicate that the similarity is really high, reaching patternrepetitions above 80% in Bubble Worlds. The results are above the 50%that we defined in section 1.4, so the system calls comparison is successful.

The common system calls result is not really high, but this is due to thebig amount of different system calls called during the first phases of the gameexecution. If we look at the values when offset is 0 (i.e., the first chunk),there are few common system calls out of the total. This changes whenthe game is going forward, like on the third chunk (offset 100000, meaningtraces from 100000 to 110000), when nearly all the system calls are common.Actually Monkey Gamer generates more system calls than a human player.

Due to the high value of similarity, although the number of commonsystem calls is not very high (compared to the human or Monkey Gamersystem calls), we can conclude that the system calls in common are used

47


Table 5.3: Tic Tac Toe Free system calls comparison

offset limit human SC MG SC common SC pattern (%)H1 0 10000 980 812 560 41,5

10000 10000 407 458 105 69,4100000 10000 323 293 31 75,5

H2 0 10000 888 812 554 44,210000 10000 413 458 158 68,7100000 10000 276 293 30 72,4

H3 0 10000 868 812 489 43,810000 10000 380 458 201 73,0100000 10000 293 293 270 80,1

average 536 521 266 63,2

Table 5.4: Bubble Worlds system calls comparison


10000 10000 64 201 25 89,5100000 10000 18 86 15 89,6

H2 0 10000 252 292 46 72,010000 10000 197 201 33 81,5100000 10000 39 86 21 87,9

H3 0 10000 219 292 32 73,510000 10000 48 201 29 86,5100000 10000 39 86 19 88,1

average 132 193 28 81,9

48


Table 5.5: Angry Birds system calls comparison


10000 10000 332 142 34 76,0100000 10000 52 115 35 88,8

H2 0 10000 217 395 34 72,710000 10000 21 142 18 85,4100000 10000 76 115 16 88,7

H3 0 10000 515 395 138 63,010000 10000 303 142 33 77,2100000 10000 63 115 17 89,1

average 227 217 47 79,0

more often than the ones not in common.

5.2.2 IP addresses comparison

Now we are going to compare the number of IP addresses to which bothhumans and Monkey Gamer connected. From the execution traces obtainedfrom tcpdump, we got the TCP packets record. Then, we parsed the file totake just the IP addresses. We decided this analysis was the most interestingone in order to compare the Monkey Gamer and human players behaviour.The connection to a server is done when some point of the game is reached,so comparing IP addresses we know how many different Internet connectionare done by Monkey Gamer and real players.

After that, we compare the IP’s to which the human connected withthe ones to which Monkey Gamer connected. This leads to some commonIP addresses and the total number reached by both of them. Results arereported in tables 5.6, 5.7 and 5.8.

Table 5.6: IP’s reached in Tic Tac Toe Free

common by human by MG common/human (%)H1 7 10 79 70,0H2 6 18 79 33,3H3 10 25 79 40,0

In these tables we can see that common IPs values and the percentageover the human ones presents high variance, going from low values as 16,7%to 100%. Thus, we can only conclude from these data that the IPs con-nected depend widely on the way the game is played. But we can observethat Monkey Gamer is connecting to widely more servers than a human doesif we focus on the IPs connected by humans over the IPs connected by Mon-key Gamer. This makes Monkey Gamer a really good network connections

49


Table 5.7: IP’s reached in Bubble Worlds


Table 5.8: IP’s reached in Angry Birds


explorer, suitable to do unexpected behaviour analysis.If we calculate the average of common IPs, we get a result of 52.0%, above

the 50% defined in section 1.4, so the IPs comparison between humans andMonkey Gamer is also successful as profiler.

5.2.3 Screens comparison

Finally, our last test is about the screens that a human and Monkey Gamerreach when playing. Reaching a screen means that the screen is showed atleast once during the playing time. This time we are doing a hand compari-son using the screenshots captured during the execution. Hand comparisonmeans that we analysed the screenshots and we classify them into screenslooking at them and checking if they were similar or not.

The results of this test are shown in tables 5.9, 5.10 and 5.11. The Xona cell means that the screen on that row has been visited by the player onthe column, empty cell means the opposite. The value in row MG/humans(%) is calculated dividing the number of screens visited by Monkey Gamerby the average of the number of screens visited by the three humans.

As we can see, in all the three games in test we reach high percentages(as 150%), so Monkey Gamer covers more screens than humans do. Theresults are above the 80% set in section 1.4, so the screens comparison isalso successful.

Game by game, in Tic Tac Toe Free we can see how Monkey Gamer iscovering all the screens on the game, what human players do not do. Thisresult indicates that in simple games, Monkey Gamer may be a much betterunexpected behaviour tester than a human.

In Bubble Worlds, Monkey Gamer is not covering all the screens reachedby humans, but in general it is covering on average three screens more thanhumans.

Finally, in Angry Birds, the screens played by Monkey Gamer differconsiderably from those played by humans, but also the human screens

50


Table 5.9: Screens comparison for Tic Tac Toe Free

Screens Monkey Gamer Human 1 Human 2 Human 3hangman promo X X X

main menu X X X X

one player options X X X X

two player options X X X

more games X

options X

board X X X X

want to quit? X X X

win/lost screen X X X X

number of screens 9 6 7 5MG/humans (%) 150,0

Table 5.10: Screens comparison for Bubble Worlds

Screens Monkey Gamer Human 1 Human 2 Human 3main menu X X X

help X

world selection X X X X

level selection X X X X

level options X X X X

level playing screen X X X X

pause X X

complete condition X X X X

rate game X

level completed X X X X

level failed X X X

basketball shot promo X


51


Table 5.11: Screens comparison for Angry Birds

Screens Monkey Gamer Human 1 Human 2 Human 3main menu X X X X

ad X X X

mighty eagle ad X

world selection X X X X

level selection world 1 X X X X

video intro 1 X X X X

Level 1-1 X X X X

level failed X X X

level selection world 2 X

level selection world 3 X X


video intro 4 X

level 4-1 X

pause X X X

purchase extras X

level selection world 5 X X

video intro 5 X X

Level 5-1 X X

purchase mighty eagle X


Level 6-1 X

how to play X X X X

level cleared X X X

Level 1-2 X X X

Level 1-3 X X X

Level 1-4 X X

Level 1-5 X X

Level 1-6 X

Level 1-7 X

Level 1-8 X

Level 1-9 X

Level 1-10 X

Level 3-1 X

Level 3-2 X

Level 3-3 X

Level 3-4 X

Level 3-5 X

Level 5-2 X

Level 5-3 X

Level 5-4 X

Level 5-5 X


52

5.3. DISCUSSION CHAPTER 5. PROTOTYPE EVALUATION

differ among themselves. This can be explained by the complexity of thegame. Anyway, on average, Monkey Gamer is covering again more screensthan humans.

To sum up we can say that on simple games, Monkey Gamer and humansvisit approximately the same screens, so Monkey Gamer is a good profiler.As on every case Monkey Gamer is visiting more screens than humans, wecan say that our program is a good unexpected behaviour tester.

5.3 Results discussion

After analysing the results for the three selected games we can say that:

• the system calls generated by Monkey Gamer are quite similar to theones generated by humans while playing the games

• Monkey Gamer is connecting to much more servers than a human does

• Monkey Gamer is exploring a higher number of screens in these ex-periments

We can conclude that Monkey Gamer is an adequate solution to auto-matically profile Android games and to do automatic unexpected behaviourtesting.

53

Chapter 6

Conclusions

In this chapter we are going to give our perspective about the achievementsaccomplished in this thesis, the future work that can be done in order toenhance the results and some personal comments.

6.1 Contributions

This thesis work has led to four major achievements. First, we presented thestate of art of Android automatic profiling, based on different approachesadopted in the papers presented in section 2.1. We learned about Android,how applications and games are developed, and how the GUIs for Androidare defined. Furthermore, we have identified a range of tools available toautomatize inputs for Android. This knowledge about Android, testing,automatic testing, profiling, and execution traces analysis has been docu-mented in the thesis and is hopefully useful for others. All this acquiredknowledge is described in chapter 2.

Second, we did an analysis about popular Android games, giving twouseful classifications: one about how games are developed nowadays andwhich kind of user inputs are needed to play. The result of this study isinteresting itself, but it also helped to estimate the percentage of games thatMonkey Gamer is able to cover. We found that 45% of the most downloadedgames would be candidates to be profiled by Monkey Gamer. This part ofour study is explained in detail in chapter 3.

Third, we developed Monkey Gamer, an automatic profiler of Androidgames. We planned its architecture and the tasks it should perform. Thenwe designed the different modules that would form the program and wespecified the tasks that each module will carry. Then we implemented theprogram and tested it. Hence, we had developed a program that was ableto play some Android games automatically and capture execution tracesmeanwhile. The implementation of Monkey Gamer is detailed in chapter 4.

Fourth and last, we validated the profiling abilities of Monkey Gamer,

54

6.2. FUTURE WORK CHAPTER 6. CONCLUSIONS

using approaches given by existing literature and we adapted them withour own trace comparator algorithm, suiting our requirements. The resultsof these comparisons were really good, reaching the requirements set be-fore the development. In general, these comparisons reached values of 80%for system calls, 50% for IP addresses with respect to human players andMonkey Gamer covers more screens (over 130% more). This makes MonkeyGamer a good candidate when there is the need of profiling a game or doingunexpected behaviour testing. This validation is described in chapter 5.

Overall the thesis results are innovative, since, as far as we know, thereare no published analyses like the one we did and there was no automaticprofiler of Android games before. The existing solutions for automatic profil-ing of Android (non-game) applications were really simple, just consideringtap events and only working for XML designed user interfaces. This is asimplification because, on these applications every activity is a single screenand clickable elements are properly defined.

We introduced a new concept, the use of screenshots to compare andknow in which screen of the game we are, since on games, an activity canhold multiple screens. Also we used object detection in screenshots, to detectclickable elements and touch them, since they are not defined on a XMLfile. We have demonstrated that this procedure is really effective to detectclickable places in Android screenshots, and this is something completelynew. It was more complicated to develop a games profiler than a normalapplications profiler.

The development of Monkey Gamer may be of interest for game devel-opers, game testers, computer security companies, researchers and Google.After a few independent tests we would be confident that others are ableto automatically generate execution traces, similar to human ones, in orderto use them as test cases, patterns of normal behaviour, look for securitythreats, etc.

A good example we would like to present before the end of this thesisis shown in Figure 6.1. We can see there an Angry Birds game screenshotrequesting Superuser permissions, something that is not supposed to beneeded and it may represent a security threat. This screenshot was capturedusing Monkey Gamer and it is just an example of the capabilities of the toolwe have developed.

6.2 Future work

Although we are really satisfied with Monkey Gamer results, there are someaspects that can be enhanced. These are some of the tasks that could bedone in the future:

• Redo the game analysis, considering a bigger market, like the one inUnited States. Then analyse the absolute number of downloads and

55

6.2. FUTURE WORK CHAPTER 6. CONCLUSIONS

Figure 6.1: Superuser permission request in Angry Birds

make a list according to that.

• Improve the state machine work, avoiding wasting time in not usefulscreens and making more inputs in interesting ones. This may re-quire adding some game specific knowledge and the implementationof heuristics to let Monkey Gamer learn in which screens it shouldspend less time (for example, in menus). This can be done addingtester-interaction trough some new APIs to implement.

• Improve the object detection, to better detect the objects, since nowthere are some non-clickable objects detected and some clickable ob-jects not detected.

• Enhance the performance of Monkey Gamer in general, to let it per-form more inputs at the same time. Maybe if the performance isreally enhanced, it would be able to profile games which require timedependent inputs.

• A new version of Monkey Gamer could be implemented for games thatcan not be considered as a state machine. For example, the screencould be analysed to know where the buttons are and then use alwaysthese buttons and press them according to how the game is going.

• Implement more trace collectors that obtain a wider set of executiontraces, like the record of inputs from the phone sensors.

• Test Monkey Gamer in more games, to really attest its coverage.

56

6.3. PERSONAL COMMENTS CHAPTER 6. CONCLUSIONS

6.3 Personal comments

From a personal development perspective, I learned how to do research tasksand how to plan and carry a 25 weeks project. Also I learned how to designa big piece of software, how to plan the development and the tests, how toimplement it and finally test it. It is also remarkable the communicationskills acquired as I presented the thesis multiple times, on different stagesof the work.

57

Appendix A

Android games analysis

Table A.1: Full results of the Android games analysis

Top Name GUI type Input type1 Hill Climb Racing Cocos2d Time-Depend.2 Subway Surfers Unity Time-Depend.3 Quizkampen * Intelligence4 Flow Free Open GL Time-Independ.5 Wordfreud Free Open GL Time-Independ.6 Angry Gran Run Unity Time-Depend.7 Bike Race Free Bike Race Free Time-Depend.8 Bus Parking 3D Unity Time-Independ.9 Ice Cream Jump Open GL Time-Depend.10 Ruzzle Free Render Time-Independ.11 Angry Birds Cocos2d Time-Independ.12 Bubble Mania Open GL Time-Independ.13 Air Hockey Deluxe Open GL Time-Depend.14 Jetpack Joyride Mortar Time-Depend.15 Temple Run Unity Time-Depend.16 Bad Piggies Unity Time-Independ.17 Fruit Ninja Free Fruit Ninja Time-Depend.18 3D City Runner Unity Time-Depend.19 Happy Poo Fall Open GL Time-Depend.20 Lie detector XML Time-Independ.21 Shoot Bubble Shot Bubble Time-Independ.22 Robbery Bob Free Open GL Time-Depend.23 SongPop Free Adobe AIR Time-Independ.24 Ice Cream Drop Open GL Time-Depend.25 Drag Racing Racing Time-Depend.26 Angry Birds Star Wars Legacy Time-Independ.

58

APPENDIX A. ANDROID GAMES ANALYSIS

27 Transporters Unity Time-Depend.28 Virtual Dental Surgery * Time-Independ.29 Subway Surfers Play Tips&Tricks XML Time-Independ.30 Curious Unity Time-Depend.31 Extreme Road Trip 2 Unity Time-Depend.32 Contract Killer 2 Unity Time-Depend.33 Minecraft Pocket Ed. Demo Minecraft Time-Depend.34 NinJump Deluxe Open GL Time-Depend.35 Angry Birds Seasons Legacy Time-Independ.36 Draw Something Free Burstly Time-Independ.37 Burguer Burger Time-Independ.38 Survivalcraft Demo Survivalcraft Time-Depend.39 Coin Dozer Unity Time-Independ.40 3D Bowling Unity Time-Independ.41 Monster Pet Shop Open GL Time-Depend.42 Ant Smasher Ant Smasher Time-Independ.43 Subway Surfers Top Tips&Tricks XML Time-Independ.44 Angry Birds Space Legacy Time-Independ.45 Global Outbreak Unity Time-Depend.46 Logo Quiz XML Intelligence47 Unblock Me Free Cocos 2d Time-Independ.48 Lie Detector HD XML Time-Independ.49 SlenderMan Slenderman Time-Depend.50 Dead Trigger Unity Time-Depend.51 My Horse Open GL Time-Depend.52 Jewels Star Open GL Time-Independ.53 Angry Birds Rio Angry Birds Time-Independ.54 Pet Shop Story XML Time-Independ.55 Cut the Rope Cut the Rope Time-Independ.56 Starry Nuts * Time-Independ.57 Moto X Mayhem Free Open GL Time-Depend.58 Plague Inc. Open GL Time-Independ.59 Happy Poo Jump Open GL Time-Depend.60 Perfect Piano Piano Time-Independ.61 Clouds & Sheep Open GL Time-Independ.62 Kick the Boss Unity Time-Independ.63 Slots XML Time-Independ.64 QuizMe XML Intelligence65 PapiMelt PapiMelt Time-Depend.66 Restaurant Story XML Time-Independ.67 Where’s My Water? Free WML Time-Independ.68 Mega Jump Open GL Time-Depend.69 Yatzy Unity Time-Independ.

59

APPENDIX A. ANDROID GAMES ANALYSIS

70 Prize Claw Unity Time-Depend.71 WorldCraft (with multiplayer) * Time-Depend.72 Bubble Blast Boxes BubbleBastBoxes Time-Independ.73 Beach Buggy Blitz BeachBuggyBlitz Time-Depend.74 Zombiewood * Time-Depend.75 Lego App4+ Unity Time-Depend.76 Strip Holdem - Sexy Nurses * Time-Independ.77 MF3 JetSki Unity Time-Depend.78 Sudoku Free Sudoku Time-Independ.79 Blood Brothers BloodBrothers Time-Depend.80 Pumpkins vs. Monsters PumpkinsVsMonsters Time-Depend.81 SpeedCarll SpeedCarl Time-Depend.82 Tetris Free Airplay Time-Independ.83 Angry Grandma AngryGrandma Time-Depend.84 Heavy truck parking HeavyTruckPArking Time-Depend.85 MovieStarPlanet * Intelligence86 Monpla Smash Airplay Time-Depend.87 Dragin, Fly! Free Open GL Time-Depend.88 Teeter Teeter Time-Independ.89 101-in1 Games Hundred Time-Depend.90 250+ Solitaries Collection Solitaries Time-Independ.91 Hobbit: King of Moddle-earth Unity Time-Depend.92 Zombie Frontier Open GL Time-Depend.93 Pou Pou Time-Independ.94 Skater Boy SkaterBoy Time-Depend.95 Rail Rush Unity Time-Depend.96 Shooting Bubble ShoothingBubble Time-Independ.97 Lep’s World Cocos 2d Time-Depend.98 Uno Free Open GL Time-Independ.99 Virtual Table Tennis 3D Open GL Time-Depend.100 Pool Master Pro Open GL Time-Depend.

* Not possible to retrieve the used game engine; the name is not specified.

60

Appendix B

Screenshots used in

comparison testing

(a) AB1.1 (b) AB1.2

Figure B.1: Screen 1 from Angry Birds

(a) AB2.1 (b) AB1.2


61

APPENDIX B. SCREENSHOTS USED IN COMPARISON TESTING



(a) BW1.1 (b) BW1.2

Figure B.5: Screen 1 from Bubble Worlds

62


(a) BW2.1 (b) BW2.2

Figure B.6: Screen 2 from Bubble Worlds

(a) TT1.1 (b) TT1.2

Figure B.7: Screen 1 from Tic Tac Toe Free

63


(a) TT2.1 (b) TT2.2

Figure B.8: Screen 2 from Tic Tac Toe Free

Figure B.9: Screen 3 from Tic Tac Toe Free (TT3.1)

64

Bibliography

[1] S. Anand, M. Naik, M.J. Harrold, & H. Yang. Automated concolic test-ing of smartphone apps. In Proceedings of the ACM SIGSOFT 20thInternational Symposium on the Foundations of Software Engineering,2012, p. 59.

[2] Android, http://www.android.com, retrieved 2013-05-30

[3] Android App Publishing Checklisthttp://developer.android.com/distribute/googleplay/publish/preparing.html,retrieved 2013-05-30

[4] Android Application Fundamentals,http://developer.android.com/guide/components/fundamentals.html,retrieved 2013-05-30

[5] Android apktool, http://code.google.com/p/android-apktool, retrieved2013-05-30

[6] Apple App Review, https://developer.apple.com/appstore/guidelines.html,retrieved 2013-05-30

[7] Android Debug Bridge (ADB), http://developer.android.com/tools/help/adb.html,retrieved 2013-05-30

[8] B. Cornelissen, & L. Moonen. Visualizing similarities in execution traces.In Proceedings of the 3rd Workshop on Program Comprehension throughDynamic Analysis (PCODA), 2007, pp. 6-10.

[9] Cyanogenmod, http://www.cyanogenmod.org, retrieved 2013-05-30

[10] S. Dai, A. Tongaonkar, X. Wang, A. Nucci, & D. Song. Networkprofiler:Towards automatic fingerprinting of android apps. In Proceedings ofthe 32nd IEEE International Conference on Computer Communications,INFOCOM, 2013.

[11] R.O. Duda, & P.E. Hart. Use of the Hough transformation to detectlines and curves in pictures. Communications of the ACM, vol. 15(1),pp. 11-15, January 1972.

65

BIBLIOGRAPHY BIBLIOGRAPHY

[12] Hierarchy Viewer, http://developer.android.com/tools/help/hierarchy-viewer.html, retrieved 2013-05-30

[13] C. Hu, & I. Neamtiu. Automating gui testing for android applications.In Proceedings of the 6th International Workshop on Automation ofSoftware Test, 2011, pp. 77-83.

[14] iOS, http://www.apple.com/ios, retrieved 2013-05-30

[15] Jython, http://www.jython.org, retrieved 2013-05-30

[16] Logo Quiz,https://play.google.com/store/apps/details?id=logos.quiz.companies.gameretrieved 2013-08-27

[17] D.T. Milano, ”Android View Client”,https://github.com/dtmilano/AndroidViewClien, retrieved 2013-05-30

[18] MonkeyRunner,http://developer.android.com/tools/help/monkeyrunner concepts.html,retrieved 2013-05-30

[19] Open Source Computer Vision Library, http://www.opencv.org, re-trieved 2013-05-30

[20] Play Store (previously Android Market), http://play.google.com/store,retrieved 2013-05-30

[21] Python, http://www.python.org, retrieved 2013-05-30

[22] Python Imaging Library (PIL), www.pythonware.com/library/pil, re-trieved 2013-05-30

[23] Real APK Leecher, https://code.google.com/p/real-apk-leecher, re-trieved 2013-05-30

[24] strace, http://linux.die.net/man/1/strace, retrieved 2013-05-30

[25] tcpdump, http://www.tcpdump.org, retrieved 2013-05-30

[26] T. Takala, M. Katara, & J. Harty. Experiences of system-level model-based GUI testing of an Android application. In Software Testing, Veri-fication and Validation (ICST), IEEE Fourth International Conference,March 2011, pp. 377-386.

[27] J. Ward. What is a Game Engine?. GameCarreerGuide.http://www.gamecareerguide.com/features/529/what is a game .php,retrieved 2013-05-30

[28] X. Wei, L. Gomez, I. Neamtiu, & M. Faloutsos. Profiledroid: Multi-layer profiling of android applications. In Proceedings of the 18th annualinternational conference on Mobile computing and networking, August2012, pp. 137-148.

66

Linköping University Electronic Press

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare –från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår.

Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för icke-kommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och administrativ art.

Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan be-skrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart.

För ytterligare information om Linköping University Electronic Press se för-lagets hemsida http://www.ep.liu.se/

Copyright

The publishers will keep this document online on the Internet – or its possible replacement –from the date of publication barring exceptional circumstances.

The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.

According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement.

For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/. © Javier Marián Santos

institutionen för datavetenskap - diva...

Documents