Smiler Yang

2012/12/13

Quality Assurance in Software Industry

Outline

• About me

• Roles in a project

• Why a software needs QA

• QA tasks

• Testing Methodology

• Examples

• Some stories in real world

• Happy and Unhappy

• Q & A

Roles in a project

• PM (Product Manager)

• JM (Project Manager)

• DM (Developer Manager)

• QM (QA Manager)

• Architecture

• RD

• QA

• UI Designer

• Technical Writer

Why a software needs QA?

• Did you test your program?

– I think it’s “Yes”

– But you might not consider everything…

So, we need---

• Different people

• Different knowledge

• Different environment

QA tasks

• Requirement Clarification

• Test Plan

• Test Spec

• Test Case

• Environment Preparation

• Functional Testing

• System Testing

• Exploratory Testing

• Bug Reporting

• Bug Verification

• Document Testing

• Test Automation

• Beta training

• Customer on-site

Testing Methodology

Testing Methodology (1/5)

• Black Box Testing – Perform the tests on the basis of functional properties and

requirements of a software

– Focus on the selected test data and the interpretation of test results

– Needs experience…

– Includes • Functional Test

• Error-handling Test

• Boundary Test

• Performance Test

• Stress Test

• Usability Test

• I18N Test (internationalization test)

• …

Testing Methodology (2/5)

• White Box Testing – Need to understand overall system architecture

– Inspect code of function implementation

– Focus on testing the paths of control flow and data flow • Go through all independent paths

• Exercise logical decisions

• Executed all loops with boundary condition checked

Testing Methodology (3/5)

• Test Plan

– Purpose of Test Plan

• Sync Testing Direction with JM/DM/RD

• To Give all testing directions/information to QA Engineer

– Information required in a test plan

• MRD/PRD/Functional Spec/High Level Design

– Schedule of each development process, Feature List, System Requirement, Product Team Information

• Configuration Compatibility Testing

– hardware compatibility / software (OS/Application) compatibility

• What is the user expecting to the product?

– How does Customer use the product?

– on what kind of the environment?

Testing Methodology (4/5)

• Define the Scope of testing of Features and Environments

– Doing and not doing

– Decide the priority of Features and Environment

• Define of Bug Severity

– Severity: Critical/Serious/Non-Critical/Enhancement/Suggestion

• Define Testing methods

• Resource Planning

– People

– Environment

• Preparation of the testing

– Knowledge: QA knowledge, product knowledge, …

– Environment: Software, Hardware,…

• Process for Testing/Management

– Bug Tracking System

– Checking method of the progress

– Bug Review Meeting

– …

Testing Methodology (5/5)

• Test Case

– What is Test Case?

• The set of input data, operation steps and expected result

• Operation steps includes conditions, configurations, environment information, …

– What is good Test Case?

• The purpose of the testing is to improve the quality by finding the errors.

• Test case should cover all requirements.

• Test case should take the wrong input/output and wrong operation into consideration.

• Test case should have clear description.

Examples

Example of Test Case Creation

• Use Case:

– Account Login in a web portal

• Test cases??

– Hint: • Functional Test

• Error-handling Test

• Boundary Test

• Performance Test

• Stress Test

• Usability Test

• I18N Test

Example of Bug Reporting

• Issue: User failed to login the web portal

• How to describe a problem?

• The information of a bug should include:

– Title

– Environment

– Version/build number

– OS/Platform

– Priority

– Severity

– Defect Type

– Frequency

– Description

– Steps

– Expected results

– …

Some stories in real world

QA and JM

<Before the milestone day>

• JM: Any issues of the candidate build?

• QA: So far we found #358 is the problem for…, and #359 is the issue when user…., and…..

• JM: Have RD found the root cause for these issues?

• ….

• …

• Then “3M” may have a meeting to review the issues.

QA and RD

< After QA submit a bug to RD>

• RD: This problem didn’t appear in my machine.

• QA: It did happen in my testing environment!

• …

• ..

• .

QA and Customer

<One day…>

• Customer: The service is not available after I used the system for only 2 days.

• QA: What did you do on the machine today? Do you think it’s normal yesterday?

• Customer: I don’t know. I just moved the machine from my office to server room.

• QA: Why did you move the machine?

• …

• ..

A day of a QA

• Read emails

• Clean environment and install new build.

• Testing, bug verification, …

• Find new bug and submit it to RD.

• Discussion for the current issues.

• Discussion for the new features.

• Write test plan / test spec / test case

• Clarify, reproduce issues and collect more debug log…

• Review document, write automation, …..

• Reading, study group, …

Happy and Unhappy

• Happy

– Improve the quality of the software

– Provide the solution that customers really need

– Learn new things

– …

• Unhappy

– Too many tasks

– “Why didn’t you find the bug before the release?”

– Schedule VS Quality

– …

Q & A

Thank you~