revisiting prior empirical findings for mobile apps: an ...mdsyer/wp-content/... · in an app store...

15
Revisiting Prior Empirical Findings For Mobile Apps: An Empirical Case Study on the 15 Most Popular Open-Source Android Apps Mark D. Syer, Meiyappan Nagappan, Ahmed E. Hassan Software Analysis and Intelligence Lab School of Computing, Queens University {mdsyer, mei, ahmed}@cs.queensu.ca Bram Adams Lab on Maintenance, Construction and Intelligence of Software enie Informatique et G´ enie Logiciel, ´ Ecole Polytechnique de Montr´ eal [email protected] Abstract Our increasing reliance on mobile devices has led to the explosive development of millions of mo- bile apps across multiple platforms that are used by millions of people around the world every day. However, most software engineering research is performed on large desktop or server-side software applications (e.g., Eclipse and Apache). Unlike the software applications that we typically study, mo- bile apps are 1) designed to run on devices with limited, but diverse, resources (e.g., limited screen space and touch interfaces with diverse gestures) and 2) distributed through centralized “app stores,” where there is a low barrier to entry and heavy com- petition. Hence, mobile apps may differ from tradi- tionally studied desktop or server side applications, the extent that existing software development “best practices” may not apply to mobile apps. There- fore, we perform an exploratory study, comparing mobile apps to commonly studied large applica- tions and smaller applications along two dimen- sions: the size of the code base and the time to fix defects. Finally, we discuss the impact of our findings by identifying a set of unique software en- gineering challenges posed by mobile apps. Copyright c 2013 Mark D. Syer. Permission to copy is hereby granted provided the original copyright notice is repro- duced in copies made. 1 Introduction App stores (e.g., Google Play and Apple App Store) have changed the software development world by providing a platform for the rapid growth of mobile apps (i.e., software applications for smartphones and tablets). Since 2007, mobile apps have exploded into a multi-billion dollar market [1, 2] and become hugely popular among con- sumers and developers. Mobile app downloads have risen from 7 billion in 2009 to 15 billion in 2010 [2] and are projected to reach 74 billion in 2016 [1]. Simultaneously, the number of mobile apps has also increased: Google Play now hosts over 1 million mobile apps [3]. Despite the ubiquity of mobile devices and the popularity of mobile apps, few software engi- neering researchers have studied them [4, 5, 6, 7]. During the thirty-five years following Fred Brooks’ seminal text, The Mythical Man Month [8], the software engineering community has pro- posed and evaluated several ideas of how high qual- ity, successful software is developed and main- tained. These “software engineering concepts” aim to tie aspects of software artifacts (e.g., size and complexity) [9], their development (e.g., number of changes) [10] and their developers (e.g., devel- oper experience) [11] to definitions of quality (e.g., post-release defects). However, such software en- gineering concepts have primarily been evaluated against large-scale projects [12].

Upload: others

Post on 04-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

Revisiting Prior Empirical Findings For Mobile Apps:An Empirical Case Study on the 15 Most Popular

Open-Source Android Apps

Mark D. Syer, Meiyappan Nagappan, Ahmed E. HassanSoftware Analysis and Intelligence Lab

School of Computing, Queens University{mdsyer, mei, ahmed}@cs.queensu.ca

Bram AdamsLab on Maintenance, Construction and Intelligence of Software

Genie Informatique et Genie Logiciel, Ecole Polytechnique de [email protected]

AbstractOur increasing reliance on mobile devices has

led to the explosive development of millions of mo-bile apps across multiple platforms that are usedby millions of people around the world every day.However, most software engineering research isperformed on large desktop or server-side softwareapplications (e.g., Eclipse and Apache). Unlike thesoftware applications that we typically study, mo-bile apps are 1) designed to run on devices withlimited, but diverse, resources (e.g., limited screenspace and touch interfaces with diverse gestures)and 2) distributed through centralized “app stores,”where there is a low barrier to entry and heavy com-petition. Hence, mobile apps may differ from tradi-tionally studied desktop or server side applications,the extent that existing software development “bestpractices” may not apply to mobile apps. There-fore, we perform an exploratory study, comparingmobile apps to commonly studied large applica-tions and smaller applications along two dimen-sions: the size of the code base and the time tofix defects. Finally, we discuss the impact of ourfindings by identifying a set of unique software en-gineering challenges posed by mobile apps.

Copyright c© 2013 Mark D. Syer. Permission to copy ishereby granted provided the original copyright notice is repro-duced in copies made.

1 IntroductionApp stores (e.g., Google Play and Apple App

Store) have changed the software developmentworld by providing a platform for the rapid growthof mobile apps (i.e., software applications forsmartphones and tablets). Since 2007, mobile appshave exploded into a multi-billion dollar market[1, 2] and become hugely popular among con-sumers and developers. Mobile app downloadshave risen from 7 billion in 2009 to 15 billion in2010 [2] and are projected to reach 74 billion in2016 [1]. Simultaneously, the number of mobileapps has also increased: Google Play now hostsover 1 million mobile apps [3].

Despite the ubiquity of mobile devices and thepopularity of mobile apps, few software engi-neering researchers have studied them [4, 5, 6,7]. During the thirty-five years following FredBrooks’ seminal text, The Mythical Man Month[8], the software engineering community has pro-posed and evaluated several ideas of how high qual-ity, successful software is developed and main-tained. These “software engineering concepts” aimto tie aspects of software artifacts (e.g., size andcomplexity) [9], their development (e.g., numberof changes) [10] and their developers (e.g., devel-oper experience) [11] to definitions of quality (e.g.,post-release defects). However, such software en-gineering concepts have primarily been evaluatedagainst large-scale projects [12].

Page 2: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

Understanding how established software engi-neering concepts apply to mobile apps is an im-portant research question. Mobile apps distinguishthemselves from desktop and server applicationsby their limited functionality, low memory andCPU footprint, limited screen sizes, touch inter-faces and access to diverse hardware (e.g., GPS andaccelerometers). Hence, mobile apps are expectedto become a major challenge for software devel-opers and maintainers in the near future becausethe underlying hardware and software is constantlyand rapidly evolving. In addition, centralized “appstores” (e.g., Google Play) provide consumers witheasy access to tens of thousands of apps, and devel-opers with access to millions of consumers. Thisecosystem has become characterized by low bar-riers to entry for prospective developers, but withheavy competition.

As a first step towards understanding how estab-lished software engineering concepts apply to mo-bile apps, we perform an exploratory study to com-pare mobile apps and desktop/server applications.We study fifteen open-source Android apps and fivedesktop/ server applications (two large, commonlystudied applications and three smaller unix utili-ties). Our study addresses the following two re-search questions:

• RQ1: How does the size of the code base com-pare? – mobile apps and unix utilities tendto have smaller code bases than larger desk-top and server applications. The developmentof mobile apps and unix utilities tends to bedriven by one or two core developers. Further,mobile apps tend to depend heavily on an un-derlying platform.

• RQ2: How does the defect fix time compare?– mobile app developers typically fix defectsfaster than desktop/server developers, regard-less of the size of the project. Defects inmobile apps tend to be concentrated in fewersource code files.

The paper is organized as follows: Section 2 mo-tivates our case study and presents related work.Section 3 describes the setup of our case study andSection 4 describes the results. In Section 5, wepresent potential directions for future research ofmobile apps. Section 6 outlines the threats to thevalidity of our case study. Finally, Section 7 con-cludes the paper.

2 Motivation and RelatedWork

Mobile app downloads have seen explosivegrowth in the past few years, which is expectedto continue well into the future [1]. Despite thisgrowth, few researchers have studied mobile appsfrom a software engineering perspective. Softwareengineering concepts like “high churn leads to poorquality” [10] and “high cohesion and low couplingleads to high quality” [9], have primarily been eval-uated against large-scale projects such as Apacheand Eclipse [12]. Determining how these conceptsapply to mobile apps may reduce the effort in de-veloping and maintaining high quality mobile apps.

Our existing software engineering knowledgemay not apply to mobile apps due to differencesbetween the mobile app and desktop/server ecosys-tems. Two potentially influential differences are1) the hardware limitations and diversity of mobiledevices and 2) the distribution channel provided byapp stores.

2.1 Hardware Limitations andDiversity

The hardware limitations of mobile devices hasled to mobile apps with small memory and CPUfootprints that are intended for mobile devices withsmall screen sizes. These hardware limitations maylimit the scope of mobile apps. Indeed, even thelatest generation of mobile devices fails to meetthe minimum system requirements for best-sellinggames (e.g., World of Warcraft) and applications(e.g, Adobe Photoshop CS5). Further, the shift inusage from desktop and server systems to mobiledevices (mobile devices are intended to be used“on-the-go”) may also limit the scope of mobileapps.

While the capabilities of mobile devices havebeen rapidly increasing (e.g., many new devicesboast dual-core processors) some of these limita-tions are permanent. This is particularly true of thescreen size and resolution, which may limit the in-formation and functionality displayed at one time.

Further, the diversity of hardware (e.g., ac-celerometers and touch interfaces with gesturerecognition) may complicate development as de-velopers experiment with these features.

Page 3: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

Therefore, the diversity and limitations of mo-bile device hardware may limit the scope of mobileapps. Although all software faces hardware limita-tions, the limitations of mobile devices are gener-ally much greater than those of desktop or serversystems.

2.2 Distribution ChannelMajor mobile platforms (e.g., the Android plat-

form) provide centralized app stores for users todownload mobile apps. Centralized app stores areeasily browsed and directly available from a user’sdevice. Therefore, the effort to install mobile appsis minimal. Further, the cost of listing a mobile appin an app store is very low (occasionally there is nocost) while the potential to reach millions of con-sumers is high.

The low cost to enter the mobile market andthe potential to generate revenue has attracted alarge number of developers [1, 2]. In 2010, morethan 50,000 developers were creating mobile appsfor the Android or iOS platforms [13]. Further,development tools have been released to the gen-eral public so that anyone can develop a mobileapp, even without prior development experience.Nearly half of mobile app developers have less thantwo years of experience [14]. The large and con-stantly expanding development community leads tohigh competition between developers. For exam-ple, Google Play contains hundreds of competingweather, media and instant messaging apps.

The high competition between developers mayaffect the quality of mobile apps for fear of losingusers to competing apps. Therefore, the emphasison defect fixing may be high and hence the defectfix times may be low.

2.3 Related WorkSoftware engineering researchers are now be-

ginning to explore the challenges and issues sur-rounding mobile apps and platforms [15]. Re-searchers are also studying mobile apps from otherperspectives, including app ecosystems [4, 16],cross platform development [17, 18] and security[19, 20, 21]. However, there are only a few studiesof mobile apps from a software engineering per-spective.

Mojica et al. studied code reuse in 4,323 Androidapps and found that, on average, 61% in the classesin a mobile app are reused by other apps in the samedomain [6] (e.g., social networking). The authorsalso found that 23% of the 534,057 classes in theircase study inherit from a class in the Android plat-form. In this paper, we are interested in how suchmeasures compare across mobile apps and desktopand server applications.

Maji et al. studied defect reports in the Androidand Symbian platforms to understand where de-fects occur in these platforms and how defects arefixed [22]. The authors determine that developmenttools, web browsers and multimedia modules arethe most defect-prone and that most defects requireminor code changes. The authors also determinethat despite the high cyclomatic complexity of theAndroid and Symbian platforms, defect densitiesare surprisingly low. In this paper, we study An-droid apps, not the platform itself.

Minelli and Lanza have developed SAMOA, anew tool that can gather and visualize basic sourcecode metrics (e.g., size and complexity) from mo-bile apps [5]. SAMOA is intended to help devel-opers better understand the development and evo-lution of their app, whereas the purpose of ourwork is to empirically establish typical propertiesof mobile apps and to contrast these propertieswith traditionally-studied desktop and server appli-cations. Further, we extract and analyze metricsfrom both the source code and issue tracking sys-tems of both mobile apps and desktop and serverapplications.

In our previous work, we performed a study ofthree pairs of functionally equivalent mobile appsfrom two popular mobile platforms (i.e., the An-droid and BlackBerry platforms), as a first step to-wards understanding the development and mainte-nance process of mobile apps [7]. We found thatBlackBerry apps are much larger and rely more onthird party libraries. However, they are less sus-ceptible to platform changes since they rely less onthe underlying platform. On the other hand, An-droid apps tend to concentrate code into fewer largefiles and rely heavily on the Android platform. Onboth platforms, we found code churn to be high.However, we are unaware of how these findings onmobile apps compare to desktop or server applica-tions.

Page 4: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

3 Case Study Setup3.1 Mobile App Selection

In this paper, we studied mobile apps written forthe Android platform. The Android platform is thelargest (by user base) and fastest growing mobileplatform. In addition, the Android platform hasmore free mobile apps than any other major mobileplatform [23].

Mobile apps for Android devices are primarilyhosted in Google Play [3]. Google Play classifiesmobile apps into two groups (i.e., free and paid)and records details such as cost and the number oftimes each app has been installed. We used GooglePlay to list the top 2,000 free apps and the top 2,000paid apps (2,000 is the maximum number of appsthat Google Play ranks). However, we limit ourstudy to free Android mobile apps, because 1) themajority (63%) of Android apps are free [23], 2)free apps are downloaded significantly more thanpaid mobile apps [24] and 3) the source code repos-itories and issue tracking systems are not availablefor paid apps.

Google Play uses the term “free” to describeAndroid apps that are may be downloaded at nocost. Free apps are not necessarily open source.One reason is that many of these apps are de-veloped internally by organizations as mobile in-terfaces to their online services (e.g., Facebook,Google Maps and Twitter). Another reason is thatmany paid apps have versions that are available forfree. This is because a common revenue model isto use a multi-tiered structure (e.g., a free app withads and a paid app without ads, or a demo versionwith a limited feature set and a paid app). How-ever, we are unable to study these apps becausethey are only available as bytecode files (i.e., wedo not have access to their source code repositoriesor issue tracking systems). Therefore, we must de-termine which apps in the top 2,000 free app listare open source.

We selected apps for our case study by cross-referencing the list of the top 2,000 free apps inGoogle Play, with the list of apps in the FDroidrepository. The FDroid repository is an alterna-tive app store that exclusively lists free and open-source (FOSS) Android apps that are also listed inGoogle Play [25]. It contains links to the home-page, source code repository and issue trackingsystem (if available). Since it does not contain any

Table 1: Selected Mobile Apps

ID Name DescriptionM1 3 Music PlayerM2 Apps Organizer UtilityM3 Barcode Scanner UtilityM4 ConnectBot SSH ClientM5 Cool Reader E-Book ReaderM6 Frozen Bubble GameM7 K-9 Mail Email ClientM8 KeePassDroid Password VaultM9 Quick Settings UtilityM10 Reddit is fun Social NetworkingM11 Scrambled Net GameM12 Sipdroid VOIP clientM13 Solitaire GameM14 Tricorder GameM15 Wordpress CMS Client

information regarding the user base (e.g., the num-ber of downloads), we must retrieve this informa-tion from Google Play.

Mobile apps in the resulting list of filtered appsare 1) popular amongst users (allowing us to study“successful” apps) and 2) open source (allowing usto access source code repositories and issue track-ing systems). This list contains twenty mobileapps. However, five were excluded from our casestudy: three were excluded because they did nothave an issue tracking system, one used a differ-ent source control system (other than SVN/GIT)and one was a mobile port of a desktop applica-tion. We excluded the latter app because the samedevelopment team produced the mobile and desk-top versions within the same source code reposi-tory, meaning that the app was developed largelyfollowing a desktop mind set, with minor changestowards the mobile end. This is not typical for amobile app. Therefore, our case study includes fif-teen mobile apps.

Table 1 contains the list of mobile apps that wereincluded in our case study. We have assigned anID to each mobile app for brevity. Table 1 con-tains mobile apps from a number of different do-mains, including utilities, networking, multimediaand games.

Page 5: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

Table 2: Selected Desktop and Server Applications

ID Name Description First Commit Last CommitD1 Apache HTTP Server HTTP Server 07/03/1996 07/02/2011D2 Eclipse UI Component User Interface 05/23/2001 09/12/2011D3 aspell Spell Checker 01/01/2000 01/01/2004D4 joe Text Editor 04/19/2001 04/19/2005D5 wget File Retriever 02/12/1999 01/11/2003

3.2 Desktop/Server ApplicationSelection

In this paper, we use two types of desktop/serverapplications.

First, we study two large, commonly studieddesktop/server applications. In particular, we studythe User Interface (UI) component of the Eclipseplatform and the Apache HTTP server projects.These applications are two of the most commonlystudied desktop/server applications in software en-gineering literature [12], used by many researchersto derive general findings about the software en-gineering process. We do not claim that theseprojects are the baseline against which all softwareengineering research should be measured. How-ever, we wish to determine the similarities and dif-ferences between these two projects and our mo-bile apps as a first step towards understanding howthe software engineering concepts developed fromstudying desktop/server applications apply to mo-bile apps.

Second, we also study three smaller unix util-ities. In particular, we study the aspell, joe andwget unix utilities. These applications are rarelystudied by software engineering researchers, how-ever, they may be more similar to mobile apps thanthe larger, more commonly studied desktop/serverapplications. Similar to mobile apps, these applica-tions have small feature sets and are readily avail-able to large user bases (typically pre-installed onall unix-like operating systems and hence availableto millions of users). To enhance the similarity be-tween these applications and mobile apps, we con-sider only the first four years of development (i.e.,four years from the date of the initial commit tothe source code repository). Therefore, the mod-

ified aspell, joe and wget data sets represent rela-tively young projects with small feature sets thatare available to a large user base (these characteris-tics are very similar to the mobile apps in our casestudy).

Table 2 contains the list of desktop/server appli-cations that were included in our case study. Wehave assigned an ID to each desktop/server appli-cation for brevity. For each of the selected desk-top/server applications, Table 2 contains 1) a briefdescription and 2) the date of the first and last com-mits included within our analysis.

3.3 Research QuestionsAs a first step toward determining how exist-

ing software engineering knowledge may be ap-plied to mobile apps, we perform an exploratorystudy comparing mobile apps and desktop/serverapplications. We study fifteen open-source An-droid apps, two large desktop/server applications(Apache HTTP server and Eclipse UI component)and three smaller unix utilities (aspell, joe andwget) to address the following two research ques-tions:

• RQ1: How does the size of the code base com-pare between mobile apps and desktop/serverapplications?

• RQ2: How does the defect fix time comparebetween mobile apps and desktop/server ap-plications?

The purpose of our research questions is to em-pirically establish properties of mobile apps and tocontrast these properties with traditionally-studiedsoftware.

Page 6: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

4 Case Study ResultsRQ1: How does the size of thecode base compare?Motivation

Many problems typically addressed by softwareengineering researchers and faced by developersare caused by large code bases and developmentteams. For example, the difficulty of code naviga-tion increases as the code base grows. In addition,the size of the code base (e.g., lines of code) hasbeen shown to be highly correlated to the complex-ity of a software application [26, 27]. The difficultyof understanding, evolving and maintaining a soft-ware application is strongly tied to the complexityof the application. Therefore, we measure the sizeof the code base and of mobile apps and compare itto desktop/server applications.

Approach

We explore the size of the code base by extract-ing the total number of lines of code and numberof source code files. We used the Understand toolsby Scitools [28] to extract these metrics. Under-stand is a toolset of static source code analysis toolsfor measuring and analyzing software projects. Ta-ble 3 presents the total number of source code filesand lines of code (LOC) in the code base of eachmobile app and desktop/server application. Thesemeasurements were made on the latest version ofthe projects, either 1) August 8, 2011 for the mo-bile apps or 2) the date of the last commit for thedesktop/server applications in Table 2.

ResultsSize of the Code Base

From Table 3, we find that the size of our mobileapps ranges between 2,332 and 47,927 LOC with amedian value of 14,014. This is the same order ofmagnitude as aspell, joe and wget. This value is ap-proximately 9 times smaller than the Apache HTTPserver project and 20 times smaller than the EclipseUI component (note that the Eclipse UI componentis only one component of the Eclipse project).

From Table 3, we find that many games tend tobe small. For example, M6, M11, M13 and M14are less than 6,000 LOC. These apps offer simple

Table 3: Size of the Code Base

ID #Files LOCM1 69 20,050M2 131 12,282M3 237 22,785M4 229 32,692M5 39 14,014M6 15 2,846M7 157 47,927M8 306 23,808M9 63 5,288M10 43 10,524M11 6 2,332M12 250 24,259M13 14 4,196M14 32 5,470M15 64 17,287D1 386 123,293D2 2,360 276,980D3 129 12,311D4 91 27,419D5 61 17,376

board games, puzzles and card games. Conversely,the largest mobile app, M7, is a full-fledged emailclient designed to replace the default email clientthat ships with an Android device. M7 has anextensive feature set, including support for IMAP,POP3 and Exchange, multiple identities, customiz-able viewing preferences and alternate themes.

Since mobile apps tend to have small code bases,we further explore two factors that may influencethe size of the code base: 1) the size of the devel-opment team and 2) platform usage.

Size of the Development Team

One factor that may influence the size of the codebase is the number of developers. If few developerscontribute to a project, then the size of the projectis constrained by the developers combined effort.

We explored the size of the development team byextracting the number of developers and the num-ber of source code commits that each developermakes to a project. We extracted the commit logsfrom the source code repository and isolated thecommitter field for each source code commit (i.e.,

Page 7: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

a commit that included at least one source codefile (e.g., “.java,” or “.c” files)). To measure thenumber of unique developers, we extract the lo-cal part of each email address (i.e., the charactersbefore the @ symbol), and count the number ofunique local parts across all source code commits.We then count the number of source code com-mits made by each developer. We perform manualchecks to verify that the extracted list of develop-ers contains only unique entries and we merge anyuncaught cases (e.g., john [email protected] [email protected]). Similar to Mockus et al.[29], we then calculate the minimum number ofdevelopers who, when combined, are responsiblefor at least 80% of the commits. These “core” de-velopers are responsible for the majority of the de-velopment and maintenance effort. Although thisdefinition of core developer and may miss devel-opers that contribute substantially through othermeans, it is commonly used in practice [29]. Ta-ble 4 presents the number of developers and coredevelopers for each mobile app and desktop/serverapplication and the average number of lines of codeper developer and core developer.

From Table 4, we find that the number of coredevelopers participating in the development andmaintenance of a mobile app is approximately thesame as the number of developers participatingin aspell, joe and wget, but much smaller thanthe number of core developers participating in theApache HTTP server and Eclipse UI component.We also find that, despite the large variability inthe number of mobile app developers, from 1 to 22developers, the number of core developers is one inthree quarters of the mobile apps. Therefore, thedevelopment of mobile apps tends to be driven bya single developer.

The size of the development team was further ex-plored by comparing the size of the code base andthe size of the development team. Figure 1 showsthe number of developers plotted against the num-ber of lines of code in the code base. Figure 1 alsoshows a trend line generated by fitting a straightline to the data from our mobile apps.

From Figure 1, we find that the number of de-velopers increases as the number of lines of codeincreases. However, this trend line is not a perfectfit to the data, indicating that the relationship be-tween the size of the code base and developmentteam varies between projects. This is supportedby Table 4, where we find that the average number

Figure 1: Size of the code base and developmentteam.

of lines of code per developer varies significantly,from 501 to 12,282. Interestingly, the median num-ber of lines of code per developer across the mobileapps (10,524) and the Apache HTTP server andEclipse UI component (9,977) are very similar.

There are many reasons why mobile apps havevarying numbers of developers. Some developersrequest that the user community contribute towardsthe project. For example, the core developer ofM10 informed users that he was unable to continuemaintaining the project and asked the user commu-nity to contribute. Conversely, some developers actas gate keepers to their source code repository andare responsible for each commit. For example, thedeveloper of M2 asked that translations be submit-ted via email, whereas translators were given com-mit privileges in M11 (many of whom would latercontribute defect fixes).

Platform Usage

Another factor that may influence the size ofthe code base is the reuse of existing functional-ity through libraries. The Android and Java APIs(Android apps are written in Java) provide basicand commonly required functionalities, thereby re-ducing the size of the code base and the amount offunctionality that the development team must im-plement themselves.

Page 8: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

Table 4: Size of the Development Team

ID #Devs LOC/ #Core Devs LOC/#Devs #Core Devs

M1 4 5,013 1 20,050M2 1 12,282 1 12,282M3 11 2,071 2 11,393M4 12 2,724 1 32,692M5 12 1,168 2 7,007M6 1 2,846 1 2,846M7 16 2,995 4 11,982M8 4 5,952 1 23,808M9 1 5,288 1 5,288M10 21 501 1 10,524M11 1 2,332 1 2,332M12 22 1,103 5 4,852M13 2 2,098 1 4,196M14 2 2,735 1 5,470M15 2 8,644 1 17,287D1 96 1,284 27 4,566D2 71 3,901 18 15,388D3 3 4,104 1 12,311D4 6 4,570 2 13,710D5 6 2,896 1 17,376

Table 5: Java and Android Depen-dencies

ID Java AndroidM1 20% 39%M2 13% 16%M3 19% 13%M4 21% 14%M5 33% 11%M6 19% 20%M7 14% 17%M8 38% 7%M9 8% 42%M10 64% 17%M11 10% 30%M12 13% 11%M13 4% 12%M14 7% 25%M15 68% 18%

We explore the extent of platform usage by ex-tracting the number of references (e.g., calls to theAndroid library). We used the Understand tool toextract, for each source code file in the subject mo-bile apps, a list of classes on which the file depends.These dependencies were classified into one of thefollowing categories based on the class name:

• Android - dependency on a class thatis part of the Android platform (e.g.,android.app.Activity).

• Java - dependency on a class thatis part of the Java platform (e.g.,java.io.IOException).

We then determined platform usage (i.e., the ra-tio of the number of platform dependencies to thetotal number of dependencies). Table 5 presents thePlatform and Java dependencies as a percentage ofthe total dependencies.

From Table 5, we find that Java and Android de-pendence is relatively high in most mobile apps.Further, the median value of the Java and Androiddependencies combined is 39% and exceeds 80%in M10 and M15.

From Table 5, we also find that Android de-pendence is particularly high (42%) in M9 (QuickSettings). The app is designed to interface withthe Android platform and give users control overdevice settings (e.g., screen brightness). In con-trast, platform dependence is low (7% and 11% re-spectively) in M8 (KeePassDroid) and M5 (CoolReader). KeePassDroid is a port of the KeePassPassword safe and Cool Reader is a cross-platforme-reader.

Uncovering the rationale for the relatively smallcode bases in mobile apps requires more analysison other systems and consumer usage trends (areconsumer demands satisfied by mobile apps withlimited functionality?).

We find that mobile apps and unix utilitiestend to have smaller code bases than largerdesktop/server applications. The developmentof mobile apps and unix utilities tends to bedriven by a one or two core developers. Fur-ther, mobile apps tend to rely heavily on anunderlying platform.

Page 9: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

RQ2: How does the defect fixtime compare?Motivation

Software quality is a major draw for users, andwith access to thousands of mobile apps throughapp stores, users demand the highest quality. If auser installs a low quality app from the app store,he or she can typically find and install a replace-ment from the app store within minutes. Competi-tion may force developers to place a greater empha-sis on fixing defects, or risk losing users to compet-ing apps. Thus, the emphasis on defect fixing, andhence defect fix times, may differ between mobileapps and desktop/server applications. Therefore,we must compare the defect fix times of mobileapps and desktop/server applications.

Approach

We explored the time it takes to fix defects by ex-tracting the list of issues that have been resolved as“closed” with a “fixed” status from the issue track-ing system. We did not include issues that wereclassified as “not a bug,” “duplicates” or “not repro-ducible”, because these issues did not involve anytime to fix. The resulting issues describe uniquedefects that have been fixed. We then calculatethe number of days between the date the issue wasopened and the date it was closed. For issues withmultiple close dates (i.e., issues that have been re-opened) we take the last close date. We then calcu-late the percentage of defects that have been fixedwithin one week, one month and one year of beingreported. Figure 2 presents these measures for eachmobile app and desktop/server application and Ta-ble 6 presents the median value of these measuresacross all 1) mobile apps, 2) large desktop/serverapplications and 3) unix utilities.

Table 6: Median Percentage of Defects Fixed inOne Week/Month/Year

Week Month YearMobile Apps 36 68 100Apache HTTP & 33 69 92EclipseUnix Utilities 21 36 80

Figure 2: Percentage of Defects Fixed in OneWeek, One Month and One Year.

ResultsDefect Fix Times

From Figure 2, we find that developers of fourmobile apps fix 50% of reported defects in oneweek and developers of eleven mobile apps fix 33%of reported defects in one week. This is greaterthan the aspell, joe and wget utilities, which fix24%, 21% and 17% of the reported defects in oneweek respectively. This is also greater than theEclipse UI component, where 19% of the reporteddefects are fixed in one week. However, it is lessthan the Apache HTTP server, where 46% of thereported defects are fixed in one week. The num-ber of defects fixed within one week in M6 is zerobecause only a single defect was reported in total(it was fixed in 26 days).

From Table 6, we find that the defect fix timesin mobile apps are more similar to large desk-top/server applications than to unix utilities. Forexample, the median percentage of defects fixedwithin one week is 36% in mobile apps and 33%in large desktop/ server applications compared toonly 21% in unix utilities.

From Table 6 and Figure 2, we also find that 20%of defects in unix utilities take longer than one yearto fix, whereas 100% of defects in mobile apps arefixed within one year.

From Figure 2, the percentage of reported de-fects fixed in one year in D4 is 61%, compared to80% in D3 and 80% in D5. However, the develop-ers of D4 close groups of defects at the same time.

Page 10: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

For example, between May 5, 2003 and February 1,2006, the developers did not close any issues, how-ever, on February 2, 2006, ten issues were closed(on average, these ten issues were open for threeyears). Therefore, it is possible that developers fixdefects within a short time span, but fail to updatethe issue tracking system until a later date.

As the defect fix times for mobile apps tend to beless than the defect fix times of large desktop/serverapplications, we further explore two factors thatmay influence the time it takes to fix defects. First,we explore the number of defects reported in theissue tracking system. Second, we explore the dis-tribution of defects across source code files.

Number of Defects Reported

One factor that may influence the time it takes tofix defects is the number of defects reported in theissue tracking system. If few defects are reported,then fewer defects need to be fixed and developerscan focus more of their attention on these defects.

We explore the number of defects reported bycounting the number of issues in the issue track-ing system that have been marked as defects. Sim-ilar to our analysis of defect fix times, we did notinclude issues that were classified as “not a bug,”“duplicates” or “not reproducible.” However, un-like our analysis of defect fix times, we do not limitour analysis to issues that have been closed. Thisis because we are including defects that have beenreported, but have yet to be fixed.

We also explore the number of reporters whohave reported at least one defect in the issue track-ing system. This analysis is similar to our identifi-cation of unique developers, except that we use thelist of people who have reported issues, instead ofthe list of people who have committed source code.

Table 7 presents the number of defects reportedand the number of users reporting defects in eachmobile app and desktop/server application. We findthat few defects are reported and few users reportdefects in both mobile apps and unix utilities com-pared to large desktop/server applications.

Android users primarily download apps throughGoogle Play, where they can also rate apps and pro-vide comments. However, user ratings and com-ments do not provide the same structure as is-sue tracking systems. The developers of M7 havespecifically asked users to report defects in their is-sue tracking system.

Table 7: Number of Defects Reported and UniqueDefect Reporters

ID Reporters Reported DefectsM1 4 21M2 13 15M3 267 342M4 315 425M5 8 20M6 6 7M7 1,293 2,518M8 178 192M9 99 120M10 18 62M11 6 7M12 591 803M13 77 100M14 33 38M15 15 98D1 3,425 5,104D2 1,206 6,287D3 35 268D4 26 64D5 25 55

From Table 7, we find that the greatest numberof defects are reported in M7 (email client), M12(VOIP client), M4 (SSH client) and M3 (barcodescanner), whereas, few defects are reported in mo-bile apps related to entertainment, M1 and M5 (me-dia players), M6, M11 and M14 (games) and M10(social networking).

Distribution of Defects

Another factor that may influence the time ittakes to fix defects is the distribution of defectsacross source code files. If defects tend to be con-centrated in a few files and developers are aware ofthese files, then they may be able to locate thesedefects with less effort. Ostrand et al. found that,in large software systems, most defects are foundwithin a small subset of the source code files [30].The authors found that 80% of the defects are foundwithin 20% of the source code files (this is often re-ferred to as the 80-20 rule). Hence, developers canprioritize their code reviews and test cases to fo-cus on these files and reduce the effort required tolocate most defects.

Page 11: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

We explore the distribution of defects acrosssource code files by extracting the number of de-fect fixing changes made to each source code file.We assume that each defect fixing change corre-sponds to a defect in the source code file. We finddefect fixing changes by mining the commit logmessages for a set of keywords [31, 32]. The key-words (i.e., “bug(s),” “fix(es,ed,ing),” “issue(s),”“defect(s)” and “patch(s)”) were developed basedon manual analysis of commit log messages. Wefind the total number of defects in a source code fileby counting the number of times the file is changedby the set of defect fixing changes.

We were unable to use the issues reported inthe issue tracking system because these tend not toinclude information regarding how the defect wasfixed (e.g., the location of the defect). We were alsounable to trace defect fixing changes to specific is-sues in the issue tracking system because mobileapp developers tend not to reference specific issuesin their commit messages. Hence, heuristics basedon the commit message need to be used to identifydefect fixing changes despite the lack of a connec-tion between the source code repository and issuetracking system.

Once we have extracted the number of defects ineach source code file, we then calculate the num-ber of defects in the top 20% most defect-pronefiles by sorting the list of files by the number ofdefects in descending order and summing the num-ber of defects in the first 20% of the list. Figure 3presents the percentage of defects in the top 20%most defect-prone files.

We find that the concentrations of defects in themost defect-prone files is the highest across mobileapps, followed by large desktop/server applicationsand finally unix utilities.

In our first research question, we found that mo-bile apps are typically small. Combined with thedistribution of defects, mobile app developers canfind the majority of defects in a very small numberof files, typically only 10s of files.

From Figure 3, we find that a higher percent-age of defects are concentrated in the top 20%most defect-prone files in mobile apps, comparedto desktop/server applications. At least 80% ofthe defects are concentrated in the top 20% mostdefect-prone files in nine of our mobile apps andat least 70% of the defects are concentrated in thetop 20% most defect-prone files in thirteen of ourmobile apps.

Figure 3: Percentage of All Defects in the Top 20%Most Defect-Prone Files.

Finding the rationale for the relatively quick de-fect fix times in mobile apps requires more analysison other systems.

We find that mobile app developers typi-cally fix defects faster than desktop/server de-velopers, regardless of the size of the project.Defects in mobile apps tend to be concentratedin fewer source code files.

5 DiscussionWe identified several potential areas for future

research in mobile apps during our current and pre-vious case studies [7]. We believe that the potentialimpact of such research may be significant giventhe ubiquity of mobile devices and growing im-portance of mobile apps. We present two such ar-eas based on observations made during our manualanalysis of the source code and development his-tory of mobile apps.

5.1 Platform UsageWe observed that many mobile apps depend

highly on their underlying platform (i.e., the An-droid platform). In our previous work, we definedthe “platform dependency ratio” as the ratio of plat-form API calls to all calls [7]. A low platformdependency ratio indicates that developers do notrely significantly on the platform APIs. For exam-

Page 12: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

ple, their app may be simple or self-contained, orthe platform may be too difficult to use. Such mo-bile apps may be easily ported to other platforms.Conversely, a high platform dependency ratio in-dicates that mobile app developers heavily exploitplatform APIs. However, this leads to platform“lock-in”, which may complicate porting to otherplatforms and potentially introduces instability dueto the rapid evolution of mobile platforms. Whilethese issues are relevant to all software that is builton an underlying platform or framework, it is par-ticularly acute in mobile apps. This is because theAndroid platform averages one major release everyyear. Hence, researchers should look at the impactof platform dependence on quality and how back-ward compatibility issues could affect quality.

5.2 Development Processes

We observed that many mobile apps have a veryhigh frequency of releases. For example, K9Mailtypically has two internal releases every week andone release to Google Play every month. Quick re-lease cycles may be required to remain competitivewithin the marketplace.

Currently, there is evidence that some mobileapps do not follow a formal development or main-tenance process. These apps are developed in anad hoc manner to get to the market as quickly aspossible. For example, as we discussed in Subsec-tion 3.1, three mobile apps were excluded from ourcase study because they did not have a public issuetracking system. These apps had been downloadedhundreds of thousands of times, and yet they didnot have a system where users could report defects.In addition, the source code repositories of elevenmobile apps in our case study do not contain anytest cases. Such ad hoc development and mainte-nance processes may adversely affect the qualityor maintainability of mobile apps.

Researchers should also investigate the relation-ship between these two factors (frequent releasesand lack of testing) and the quality of code. Khomhet al. have studied the Mozilla Firefox project andfound that a shorter release cycle 1) allows defectsto be fixed faster and 2) does not introduce signif-icantly more defects [33]. However, several openquestions remain. Does such a high frequency ofreleases mitigate the lack of testing? If there arefrequent releases for the mobile app, then does

quality matter as much? Is the project in a con-stant beta testing state? Does the platform providesufficient support for building high quality appsquickly? Is the frequent release only influenced bythe demand factor in the app store? Are the devel-opers of mobile apps more skilled or do they havemore resources at hand? Or, are mobile apps them-selves less complex to develop?

6 Threats to ValidityThis section outlines the threats to the validity of

our case study.

6.1 Construct ValidityWget was first released in January 1996 and As-

pell was first released in September 1998, however,the development history of this time is not avail-able. Therefore, the first three years of the publiclyavailable source code repository and issue trackerdo not correspond to the first three years of devel-opment.

The number of downloads is not an ideal mea-sure of success (unlike user retention or engage-ment), however, it is the best measure currentlyavailable [4]. Therefore, it is possible that we havemistakenly included mobile apps with small userbases or excluded mobile apps with large user basesfrom our analysis.

The number of unique developers was foundby counting the number of unique local parts(i.e., the characters before the @ symbol) foreach developer who made at least one committo a source code file in the repository. Simi-larly, the number of unique reporters was foundby counting the number of unique local parts foreach reporter who submitted at least one defectreport. While we did perform a manual veri-fication of this analysis, it is possible that wemisidentified two local parts as either unique ordistinct. For example, john [email protected] andadmin@my project.com were counted as two dis-tinct developers/reporters, although they may bea single developer/reporter with multiple (distinct)email address. Conversely, j [email protected] andj [email protected] were counted as one distinct de-veloper/reporter, although they may be two distinctdevelopers/reporters (e.g., John and James).

Page 13: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

The number of defects in each source code filewas measured by identifying the files that werechanged in a defect fixing change. Although thistechnique has been found to be effective [31, 32],it is not without flaws. We identified defect fixingchanges by mining the commit logs for a set of key-words. Therefore, we are unable to identify defectfixing changes (and therefore defects) if we failedto search for a specific keyword, if the committermisspelled the keyword or if the committer failedto include any commit message. We are also unableto determine which source code files have defectswhen defect fixing modifications and non-defectfixing modifications are made in the same commit.However, such problems are common when miningsoftware repositories [34].

6.2 Internal ValidityThe time to fix a defect was calculated by count-

ing the number of days between the date an issuewas opened and the date it was closed. It is pos-sible that the defect was fixed quickly, but the is-sue tracking system was not immediately updatedto reflect the fix. In addition, this analysis does notconsider defects that were not reported within theissue tracking system.

The number of unique developers is based on thelist of people who commit to the source code repos-itory and the number of unique reporters is basedon the list of people who submit issues to the issuetracking system. This does not capture people whosubmit code or issues using other mediums (e.g.,email or forums).

6.3 External ValidityThe studied mobile apps and desktop/server ap-

plications represent a small subset of the total num-ber of mobile apps and desktop/server applicationsavailable. We have limited our study to open-source mobile apps and desktop/server applica-tions. In addition, we have only studied the mo-bile apps of a single mobile platform (i.e., the An-droid Platform). Further, some mobile apps (actingmerely as a client) rely on data or services frombackend servers, however our results are limited tothe apps themselves. Therefore, it is unclear howour results will generalize to 1) closed source mo-bile apps and desktop/server applications and 2)other mobile platforms.

6.4 Conclusion ValidityOur results may have been affected by confound-

ing factors. One such confounding factor is that thedefect reporting mechanisms of Android apps af-fects the defect fixing process [35]. This may haveaffected the time to fix defects and the number ofdefects reported in each mobile app. For example,the authors found that Google Code’s bug tracker,which is used by most open-source Android apps,offers less support for management (i.e., triaging)than other widely-used issue tracking systems (e.g,Bugzilla or Jira). This may increase the time ittakes to fix defects in mobile apps.

The conclusions of empirical software engineer-ing research may be mistaken as “obvious.” How-ever, the goal of empirical research is to scientif-ically validate whether the “obvious” conclusionsare true. We would like to point out that therehave been no empirical studies to prove/disprovethe claims in this study. Mobile app developerscannot make decisions based on “gut-feeling” in-stincts. Data-driven empirical studies such as thiswill provide the necessary scientific foundation fordevelopers to make informed decisions regardingtheir software.

7 ConclusionsThis paper presented an exploratory study to

compare mobile apps and desktop/server applica-tions, as a first step toward understanding how thesoftware engineering concepts developed by study-ing desktop/server will apply to mobile apps. Westudied fifteen open-source Android apps and fivedesktop/server applications (two large, commonlystudied systems and three smaller unix utilities).

We find that, in some respects, mobile apps aresimilar to unix utilities and differ from large desk-top and server applications. Mobile apps and unixutilities are smaller than traditionally studied desk-top and server applications (e.g., Apache HTTPserver and Eclipse). This is true in terms of thesize of the code base and the development team.Further, we find that the number of core develop-ers (i.e., those responsible for at least 80% of thecommits), is very small in both mobile apps andunix utilities, typically only one or two. We alsofind that few users report defects and few defectsare reported in both mobile apps and unix utilities.

Page 14: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

We also find that, in other respects, mobileapps differ from both unix utilities and largedesktop/server applications. We find that mobileapp developers place a great deal of emphasison rapidly responding to quality issues and mostprojects fix over a third of reported defects withinone week and two thirds of reported defects withinone month. This is greater for the Eclipse UI com-ponent and the aspell, joe and wget utilities, whichtypically fix only 20% of reported defects in oneweek and 40% of reported defects in one month.However, developers of the Apache HTTP serverproject fix 46% of all reported defects in one weekand 96% of all reported defects in one month. Wealso find that the concentrations of defects in themost defect-prone files is the highest in mobileapps, followed by large desktop and server appli-cations and finally unix utilities. Most mobile appshave more than 80% of the defects in 20% of themost defect-prone files. This compares to two thirdand half for large desktop and server applicationsand unix utilities respectively.

In conclusion, our findings suggest that mobileapps may be facing unique challenges. In order tosupport the 50,000 developers creating mobile apps[13], researchers should begin to study mobile appsalongside traditionally studied desktop and serverapplications.

References[1] L. Columbus, “Roundup of mobile apps

and app store forecasts,” Jun 2013.[Online]. Available: www.forbes.com/sites/louiscolumbus/2013/06/09/roundup-of-mobile-apps-app-store-forecasts-2013/

[2] C. Sharma, “Sizing up the global appsmarket,” www.chetansharma.com/mobileappseconomy.htm, Chetan SharmaConsulting, Mar 2013.

[3] “Google play,” play.google.com.

[4] M. Harman, Y. Jia, and Y. Z. Test, “App StoreMining and Analysis: MSR for App Stores,”in Proceedings of the International WorkingConference on Mining Software Repositories,Jun 2012.

[5] R. Minelli and M. Lanza, “Software ana-lytics for mobile applications - insights &lessons learned,” Mar 2013, pp. 144–153.

[6] I. J. M. Ruiz, M. Nagappan, B. Adams, andA. E. Hassan, “Understanding reuse in the an-droid market,” in Proceedings of the Interna-tional Conference on Program Comprehen-sion, Jun 2012, pp. 113–122.

[7] M. D. Syer, B. Adams, A. E. Hassan, andY. Zou, “Exploring the development of micro-apps: A case study on the blackberry and an-droid platforms,” in Proceedings of the Inter-national Working Conference on Source CodeAnalysis and Manipulation, Sep 2011.

[8] F. P. Brooks, The mythical man-month – Es-says on Software-Engineering. Addison-Wesley, 1975.

[9] S. Chidamber and C. Kemerer, “A metricssuite for object oriented design,” Transactionson Software Engineering, vol. 20, no. 6, pp.476 –493, Jun 1994.

[10] N. Nagappan and T. Ball, “Use of relativecode churn measures to predict system defectdensity,” in Proceedings of the InternationalConference on Software Engineering, 2005,pp. 284–292.

[11] C. Bird, N. Nagappan, B. Murphy, H. Gall,and P. Devanbu, “Don’t touch my code!: ex-amining the effects of ownership on softwarequality,” in Proceedings of the Symposiumand the European Conference on Foundationsof Software Engineering, 2011, pp. 4–14.

[12] B. Robinson and P. Francis, “Improving in-dustrial adoption of software engineering re-search: a comparison of open and closedsource software,” in Proceedings of the Inter-national Symposium on Empirical SoftwareEngineering and Measurement, Sep 2010, pp.21:1–21:10.

[13] N. O’Neill, “10 surprising app platformfacts,” http://allfacebook.com/app-platform-facts b18514, All Facebook, Sep 2010.

[14] A. Cravens, “A demographic and businessmodel analysis of today’s app developer,”http://pro.gigaom.com/archives/research-briefings/, GigaOM, Sep 2012.

[15] “Mobile software engineering,” mobilese-workshop.org, Apr 2013.

Page 15: Revisiting Prior Empirical Findings For Mobile Apps: An ...mdsyer/wp-content/... · in an app store is very low (occasionally there is no cost) while the potential to reach millions

[16] H.-W. Kim, H. L. Lee, and J. E. Son, “An ex-ploratory study on the determinants of smart-phone app purchase,” in Proceedings of theInternational DSI and the APDSI Joint Meet-ing, Jul 2011.

[17] Y. Wu, J. Luo, and L. Luo, “Porting mobileweb application engine to the android plat-form,” in Proceedings of the InternationalConference on Computer and InformationTechnology, Jul 2010, pp. 2157–2161.

[18] C. Xin, “Cross-platform mobile phone gamedevelopment environment,” in Proceedingsof the International Conference on Industrialand Information Systems, Apr 2009, pp. 182–184.

[19] W. Enck, M. Ongtang, and P. McDaniel, “Un-derstanding android security,” Security andPrivacy, vol. 7, no. 1, pp. 50–57, Jan 2009.

[20] A. Shabtai, Y. Fledel, U. Kanonov, Y. Elovici,S. Dolev, and C. Glezer, “Google android: Acomprehensive security assessment,” Securityand Privacy, vol. 8, no. 2, pp. 35–44, Mar2010.

[21] M. C. Grace, Y. Zhou, Q. Zhang, S. Zou, andX. Jiang, “Riskranker: scalable and accuratezero-day android malware detection,” in Pro-ceedings of the International Conference onMobile Systems, Applications, and Services,Jun 2012, pp. 281–294.

[22] A. K. Maji, K. Hao, S. Sultana, and S. Bagchi,“Characterizing failures in mobile oses: Acase study with android and symbian,” in Pro-ceedings of the International Conference onSoftware Reliability Engineering, Nov 2010,pp. 249–258.

[23] “Comparisons and Contrasts: WindowsPhone 7 Marketplace and Google AndroidMarket,” www.distimo.com/publications,Distimo, Jan 2011.

[24] “In-depth view on download vol-umes in the google android market,”www.distimo.com/publications, Distimo,May 2011.

[25] “F-droid,” https://f-droid.org/.

[26] R. Lind and K. Vairavan, “An experimen-tal investigation of software metrics andtheir relationship to software development ef-fort,” Transactions on Software Engineering,vol. 15, no. 5, pp. 649–653, May 1989.

[27] I. Herraiz, J. M. Gonzalez-Barahona, andG. Robles, “Towards a theoretical model forsoftware growth,” in Proceedings of the Work-shop on Mining Software Repositories, 2007,pp. 21–28.

[28] “Understand Your Code,” scitools.com.

[29] A. Mockus, R. T. Fielding, and J. Herbsleb,“A case study of open source software devel-opment: the apache server,” in Proceedingsof the International Conference on SoftwareEngineering, Jun 2000, pp. 263–272.

[30] T. Ostrand, E. Weyuker, and R. Bell, “Predict-ing the location and number of faults in largesoftware systems,” Transactions on SoftwareEngineering, vol. 31, no. 4, pp. 340–355, apr2005.

[31] A. E. Hassan, “Automated classification ofchange messages in open source projects,”in Proceedings of the Symposium on AppliedComputing, Mar 2008, pp. 837–841.

[32] A. Mockus and L. G. Votta, “Identifyingreasons for software changes using historicdatabases,” in Proceedings of the Interna-tional Conference on Software Maintenance,Oct 2000, pp. 120–130.

[33] F. Khomh, T. Dhaliwal, Y. Zou, andB. Adams, “Do faster releases improve soft-ware quality? an empirical case study ofmozilla firefox,” in Proceedings of the Inter-national Working Conference on Mining Soft-ware Repositories, Jun 2012.

[34] A. E. Hassan, “The road ahead for MiningSoftware Repositories,” in Frontiers of Soft-ware Maintenance, Oct 2008, pp. 48–57.

[35] P. Bhattacharya, L. Ulanova, I. Neamtiu, andS. C. Koduru, “An empirical analysis of bugreports and bug fixing in open source an-droid apps,” in Proceedings of the EuropeanConference on Software Maintenance andReengineering, Mar 2013, pp. 133–143.