the power of the crowd: mobile testing for scale and global coverage

John Carpenter PEng, CTO www.mob4hire.com

Sept 30, 2010

The Power Of The Crowd

Mobile Testing for Scale and Global Coverage

“5 out of 6 app developers will not break even”2

“[Only] one in 8 - will break even its programming costs in six

months of downloads” 2

“Trism has made $250,000 for its developer, Steve Demeter, in

just two month” 1

1. Wired Magazine.  http://www.wired.com/gadgetlab/2008/09/app-store-is-a/#ixzz0upLJqgzq

3. Telegraph UK. http://www.telegraph.co.uk/technology/apple/5163678/Apples-iPhone-is-a-developers-goldmine.html

2. Tomi Ahonen http://communities-dominate.blogs.com/brands/2010/03/paid-smartphone-apps-half-get-under-1000-downloads-what-does-this-mean.html

“Joel Comm, whose virtual flatulence app, iFart, was at one point earning him £5,000 every day” 3

Would you buy it?

Don’t Crash –

Do Your Unit Testing. White Box Testing.

Don’t Suck–

How do Users Respond to Your App?

Get Sticky–

User Feedback / Customer Loyalty Starts Here?

What Features Will Go In The Next Version

7 years ago developing a mobile application,

...to deploy on 3 carriers for around 3% of phones

took a team of 15, 6 months and cost ~$500,000

1. Engadget . http://www.engadget.com/2004/10/02/gps-cellphone-swordfish-game/

 [Blister] says that they are the first location based fishing game, which we're sure is true, since we're not even sure there is a market for one of these

things.1

Today,

...expectations are higher. Budgets are lower.

$30,000 and take less than 6 weeks..

1. Android Lib Stats. http://www.androlib.com/appstats.aspx

Today,User Feedback is Even More Important

..69% of users discovery apps through rankings, reviews and recommendation

The Crowd are Your Customers

The Crowd Are Amateurs

Focus

Language

Incentives

Rules

Tools

FLIRT model for planning open collaboration

Sami Viitamakiwww.samiviitamaki.com

Focus

Language

Incentives

Rules

Tools

Sami Viitamakiwww.samiviitamaki.com

What do you need done?

What results are you looking for? In what

format?

What is the objective of engaging the community?

Who is managing it?

Sami Viitamakiwww.samiviitamaki.com

Does the Tester Understand the Tasks?

Is There a Mutual Understanding and

Respect?

Focus

Language

Incentives

Rules

Tools

Sami Viitamakiwww.samiviitamaki.com

Is There a Reasonable Incentive to Complete The

Task?

Are There Reasonable Safe Guards Against Unknowns?

What Non-Monetary Value Can You Offer?

Focus

Language

Incentives

Rules

Tools

Sami Viitamakiwww.samiviitamaki.com

Are You Protecting Your IP? Copyright Issues?

What Is The Criteria For Completion?

What Constraints on Testing Are Needed?

Focus

Language

Incentives

Rules

Tools

Sami Viitamakiwww.samiviitamaki.com

Do You Have The Ability To Monitor The Work?

How Do You Correct Deficiencies?

How Do You Handle App Distribution?

Focus

Language

Incentives

Rules

Tools

Rule 1 Don’t Crowd Source Your Development

Rule 2 Don’t Internal Source Your Usability Testing

“Thankfully, the theory behind such poor scoring isn’t an inherent bug or an outright failure to work, but users’ annoyance at not being able to access

the app without stumping up the necessary £9.99-per-month premium

subscription.”. T3 UK

LIVEPROJECTS

http://www.mob4hire.com/the-mob/projects

1,700 developers registered in the Mob4Hire community

Thanks!

john@mob4hire.com

www.mob4hire.comwww.mob4hire.blogspot.comwww.twitter.com/mob4hirem.mob4hire.com