Enhancing Mobile Apps QualityManual Testing Revisited

Presenter : Saurabh Agarwal, Mindfire Solutions

About Me:

• Saurabh Agarwal, Software QA Engineer, Mindfire Solutions

• Skills: Mobile and Web Application Testing• Contact:

Skype: mfsi_saurabhEmail: saurabha@mindfiresolutions.com

Focus areas for today

• The versatile nature and ecosystem of mobile apps

• A peek inside the technology behind the curtains

• Getting started with mobile app testing - Pick up that device!

• What are those "bugs" actually?• Avoiding the extremes while Reporting Issues

Why Failing Hurts?“App store ratings are extremely important to us. You can launch a beautifully designed native application, but if it crashes, then it will receive a poor rating and users will go elsewhere. Our goal is to launch nothing short of a 4.5 star app every time - no exceptions. Anyone can read your app store rating. There’s no way to hide poor quality in the world of mobile.”- Michael Croghan, Mobile Solutions Architect, USA Today

The versatile nature and ecosystem of mobile apps

More Combinations Screen size

Input methods Access to personal data

Competition for resources Use in all types of situations

Time Crunch Require constant correct updates

Guidelines and App Store Submissions Emotions

A peek inside the technology behind the curtains

• When testing Web / Desktop apps we are not usually concerned about the inside of the physical device.

• In case of device apps, it’s important to understand what is inside the device so that we can test apps thoroughly and understand how they might fail in ways that a PC or web app will not.

On the outside• Hard case (for structural support and holding the device together)• High-resolution touch screen (for viewing and interacting with apps)• Speakers and microphone (to listen to sounds and input sound)• Volume control and mute buttons (to control volume and turn off the

ringer)• Buttons:• – Navigation (to set your OS to a known state, go back, etc.)• – Sleep/wake/lock (for when the device is not in use)• – Some devices may also have a full keyboard• Camera (to capture images and video)• Headphone/speaker jack (for speakers, microphones and other

accessories)• Connector (for charging and accessories)

On the inside

• Logic Board• Memory• Battery• SIM Card• Memory Card

More Stuff!

• WiFi Antenna• GPS Antenna• Bluetooth• NFC (near-field communication)• Cellular netword antennas for data and voice

(GSM, CDMA, GPRS, 3G)• Cameras (Front, Back)• Vibration motor

And more stuff! • Ambient light

censor• Proximity Sensor• Gesture sensor • Accelerometer• Magnetometer• Gyroscope• Barometer• Temperature sensor• Humidity sensor• Fingerprint scanner• Heart Rate sensor• Hall Sensor

Done with the Dissection ;)

Getting started with mobile app testing - Pick up that device!

• Use the software in a way you expect your users to

• Gather user information (including yourself!)• The first launch test – First impressions• Something that makes you feel uncomfortable

using the app, enough to make you feel bored, frustrating, clueless enough to delete the app?

• Never blame yourself for feeling confused because of technology.

What are those "bugs" anyway?

• James Bach: “A bug is something that bugs someone who matters.”

Clear Program Malfunction Something that annoys User

• It may be functionally correct—that is, it meets a specification—but how the user feels about the app is important.

Watch out for these device specific areas• Usability and Ergonomics• User Interface• Functionality – Including Crashes and Hangs• Orientation and Sensors• Network Connectivity• Resource Starving• Installation testing• Data security• Performance• Cross platform tests• Interrupts• How the user “feels” about it

Case Study – By Jonathan KohlOne of the most difficult intermittent bugs that I had to track down in a mission-critical application had to do with a freeze up, or gimbal lock. The devices would freeze up, (requiring a hard reboot) during sales presentations, product demonstrations and beta tests. This was unacceptable, because how do you sell an app when it behaves like this?

Case Study – A Restaurant SearchConsider that trip to a local restaurant and how mobile technology has become enmeshed in that experience. How many of the following activities can you relate to? •Searching for a restaurant nearby (using location-based services or GPS) •Selecting a restaurant based on cuisine, location or price. •Reading user reviews and ratings to help make a decision. •Plotting out and following directions to the restaurant on a map. •After arriving, checking in to a social networking application, alerting people in your network of your current location. •Searching the web to answer questions about the items on the menu. •Translating a meal that is in a foreign language. •Checking the nutritional information for a meal. •Once the food arrives, taking a picture of it and uploading it to your social networking profile. •Friends commenting about the photo and your meal. •Throughout the meal, posting and responding to comments about the meal on your social networks. •At the conclusion of a meal, posting positive, neutral or negative comments about the experience to restaurant review applications. •If the experience was poor, ranting on public social media.

Usability Testing – Again!!

• Paramount for device apps• Functionality• Layout and Design• Interaction• Bottom-line - To get to the top of the App

Store, you’ll either need to launch an amazing app, or be among the best in your field in terms of usability.

Fine Tuning the approach

• Need to consider a testing strategy that combines different testing options that together provide you with the best overall testing result that balances the tradeoff between cost, quality, and time-to-market.

Questions?

References

• http://go.utest.com/iOS7tips-ebook.html• Tap Into Mobile Application Testing by

Jonathan Kohl - https://leanpub.com/testmobileapps

• http://www.ministryoftesting.com/2013/06/going-mobile-testing-beyond-the-device/

Thank You!

PS – Your Feedback is important!