ipc2008 slide qa in depth best practises

Quality Assurance in PHP Projects

In-Depth Best-Practises

Max Horváth & Wojciech Duda, studiVZ Ltd. 2

Who are we?

•  Max Horváth

•  Actively involved in web development for the past 10 years.

•  Quality Manager at studiVZ Ltd.

Who are we?

•  Wojciech Duda

•  Involved in software testing for the past 6 years.

•  QA Engineer at studiVZ Ltd.

Who are YOU?

•  What is your experience with: •  QA in PHP projects?

•  Metrics of software?

•  Testing of software?

Schedule

Schedule / Lots of coffee

•  10:45am – 11:15am Coffee Break

•  12:15am – 01:30pm Lunch

•  03:15pm – 03:45pm Coffee Break

Schedule (Part I)

•  09:30am – 10:00am Intro / Tutorial

•  10:45am – 11:15am Coffee Break

•  11:15am – 12:15am Tutorial

•  12:15am – 01:30pm Lunch

Schedule (Part II)

•  01:30pm – 02:30pm Tutorial

•  03:15pm – 03:45pm Coffee Break

•  03:45pm – 05:00pm Q&A

Objectives of this workshop

•  To introduce you to the quality assurance process and key quality assurance activities

•  To explain the role of standards in quality assurance

•  To explain the concept of a software metric, predictor metrics and control metrics

•  To explain how measurement may be used in assessing software quality

Topics covered

•  Quality assurance and standards

Topics covered

•  Quality planning

Topics covered

•  Quality control

Topics covered

•  Software measurement and metrics

What is Quality Assurance?

•  QA is part of all good production processes.

•  Quality Assurance (QA) is part of the process that ensures:

•  Quality Assurance (QA) is part of the process that ensures: •  quality in work

•  activities are being performed effectively

•  the product meets requirements

•  Involves defining appropriate quality standards and procedures and ensuring that these are followed

•  Should aim to develop a ‘quality culture’ where quality is seen as everyone’s responsibility

What is quality?

•  Quality, simplistically, means that a product should meet its specification

What is unique about Software QA?

It is not equal

•  Software QA has some differences:

It is not equal

•  Software QA has some differences: •  Functions, benefits and production costs are not as

easily measured as a physical product.

It is not equal

•  When a software product is delivered, it is not necessarily complete ... it lives, it evolves, it gets updated.

It is not equal

•  Finished is a term applied for the purpose of releasing it for use.

It is not equal

•  Finished is a term applied for the purpose of releasing it for use.

•  Software specifications are usually incomplete and often inconsistent

The challenge

•  Processes and methods to manage, monitor, and measure quality in web applications can be as fluid and elusive as the defects they are meant to keep in check.

The challenge

•  Processes and methods to manage, monitor, and measure quality in web applications can be as fluid and elusive as the defects they are meant to keep in check.

•  For processes and methods to be most effective, you need to implement both standards and company guidelines.

Quality Assurance is not limited to Quality Control

•  Testing is an essential part of QA

•  But QA tests the entire process:

•  But QA tests the entire process: •  Your development process / efficiency

•  Functional accuracy of your software releases

•  Ideally preventing future software defects

The benefits of QA

•  Improved client satisfaction:

The benefits of QA

•  Improved client satisfaction: •  profitable relationships

The benefits of QA

•  good testimonials

The benefits of QA

•  good testimonials

•  waves of referrals

The benefits of QA

•  Reduced cost of development:

The benefits of QA

•  Reduced cost of development: •  fewer defects

The benefits of QA

•  streamlined and simplified stages of development

The benefits of QA

•  retesting runs smoothly

The benefits of QA

•  deployment on time

The benefits of QA

•  deployment on time

•  on budget

The benefits of QA

•  Reduced cost of maintenance:

The benefits of QA

•  Reduced cost of maintenance: •  good development is less troublesome to support

The benefits of QA

•  Reduced cost of maintenance: •  good development is less troublesome to support

•  support is costly

Quality Assurance activities

•  Quality management

•  Quality management •  Establish organizational procedures and standards for

quality

•  Quality planning •  Select applicable procedures and standards for a

particular project and modify these as required

quality

•  Quality control •  Ensure that procedures and standards are followed by

the software development team

•  Important:

•  Important: •  Quality management should be separate from project

management to ensure independence

Quality assurance and standards

•  Standards are the key to effective quality management

•  They may be international, national, organizational or project standards

•  Product standards define characteristics that all components should exhibit e.g. a common programming style

•  Process standards define how the software process should be enacted

Importance of standards

•  Encapsulation of best practice - avoids repetition of past mistakes

•  Framework for quality assurance process - it involves checking standard compliance

•  Provide continuity - new staff can understand the organization by understand the standards applied

Problems with standards

•  Not seen as relevant and up-to-date by software engineers

•  Involve too much bureaucratic form filling

•  Unsupported by software tools so tedious manual work is involved to maintain standards

Testing in PHP Projects

•  Testing is one of the most important tasks of QA.

•  We are going to talk about these testing techniques and tools:

•  We are going to talk about these testing techniques and tools: •  Unit Tests (PHPUnit)

•  Continuous Integration (phpUnderControl)

•  Software Metrics (PHPUnit / PHP_CodeSniffer)

•  Acceptance Tests (PHPUnit / Selenium)

Testing Types

•  Unit Tests

Testing Types

•  Unit Tests •  Executable code fragments test the correctness of

parts – units – of the software (system under test).

Testing Types

•  System / Integration Tests

Testing Types

•  System / Integration Tests •  Conducted on a complete, integrated system to

evaluate the system's compliance with ist specified requirements.

Testing Types

•  Non-Functional Tests

Testing Types

•  Non-Functional Tests •  Test of performance, stability or security of

application.

Testing Types

application.

•  Acceptance Tests

Testing Types

application.

•  Acceptance Tests •  Ensure that the code does what the customer wants.

Testing Tools

•  To make testing viable, good tool support is needed.

Testing Tools

•  To make testing viable, good tool support is needed.

•  This is where a testing framework such as PHPUnit comes into play.

Testing Tools

•  Unit Tests •  PHPUnit

•  PHPT

•  SimpleTest

Testing Tools

•  Acceptance Tests •  Selenium Suite

•  Selenium Remote Control •  PHPUnit to talk Selenese to Selenium RC

Testing Tools

•  Non-Functional Tests •  Performance Tests

•  ab •  Apache JMeter

Testing Tools

•  Non-Functional Tests •  Performance Tests

•  ab •  Apache Jmeter

•  Security Tests •  Chorizo •  Selenium and Chorizo

PHPUnit

•  PHPUnit is a member of the xUnit family of testing frameworks (SUnit, jUnit, pyUnit...).

PHPUnit

•  PHPUnit is a member of the xUnit family of testing frameworks (SUnit, jUnit, pyUnit...).

•  Written for PHP 5.

PHPUnit

•  The xUnit frameworks provide a way to automate unit tests.

PHPUnit

•  Make it easy to test, and you'll do it more often.

PHPUnit

•  Make it easy to test, and you'll do it more often.

•  If you use version control, test before every check-in.

PHPUnit

•  The PHPUnit Pocket Guide

PHPUnit

•  Tells you everything you need to know when writing PHPUnit test cases.

PHPUnit

•  Tells you everything you need to know when writing PHPUnit test cases.

•  Buy the book, or read the HTML version at: •  http://www.phpunit.de/pocket_guide/index.en.php

PHPUnit

Let's look at a test case!

PHPUnit <?php require_once 'PHPUnit/Framework.php';

class ArrayTest extends PHPUnit_Framework_TestCase { protected $fixture;

protected function setUp() { // Create the Array fixture. $this->fixture = array(); }

public function testNewArrayIsEmpty() { // Assert that the size of the Array fixture is 0. $this->assertEquals(0, sizeof($this->fixture)); }

public function testArrayContainsAnElement() { // Add an element to the Array fixture. $this->fixture[] = 'Element';

// Assert that the size of the Array fixture is 1. $this->assertEquals(1, sizeof($this->fixture)); } } ?>

PHPUnit

•  Run a PHPUnit test case with the phpunit command.

PHPUnit

phpunit UnitTest UnitTest.php

PHPUnit

phpunit UnitTest UnitTest.php

•  Tells PHPUnit to look for a class UnitTest in the UnitTest.php file (filename is optional).

PHPUnit

•  PHPUnit can output test logs to a variety of formats.

PHPUnit

•  PHPUnit can output test logs to a variety of formats.

•  This allows results to be used for Continuous Integration.

Continuous Integration

•  Continuous integration is a way to automatically run your test suite regularly.

•  Once a day …

•  Once an hour …

•  Every time someone checks in code …

What is integration?

•  Integrating a change (code, database structure …) into the source code repository without breaking the project.

When is integration becoming important?

•  For any change that might break functionality.

•  More than one developer and/or multiple components.

•  More developers, more components = more complexity.

•  It is a software engineering practice.

•  It describes how to integrate changes frequently.

•  It stabilizes the product through self-testing code.

•  It uses rapid feedback to solve integration issues fast and putting the pain upfront the development cycle. (Remember: software defects are more expensive the later they are fixed.)

•  Goal: produce as many working builds as possible.

•  Without unit tests Continuous Integration does not help a lot.

•  Start using PHPUnit or extend your test suite:

•  Start using PHPUnit or extend your test suite: •  Each identified bug deserves a unit test to prevent

regression.

•  Develop unit tests for each new feature.

regression.

•  Develop unit tests for each new feature.

•  You don’t need full code coverage. Find the right level!

How to get started with Continuous Integration

•  Keep everything under version control.

•  Keep everything under version control. •  Automation does not work if pieces are missing.

•  Clone your production environment:

•  Clone your production environment: •  At least versions of main components should match.

•  Automate the deployment of that environment.

•  Continuous Integration is more than setting up a CI server.

•  You have to live Continuous Integration.

•  Keep in mind: CI is a customized , tailored process:

•  Keep in mind: CI is a customized , tailored process: •  CI in company A is not necessarily the same as in

company B.

Benefits of Continuous Integration

•  Enforce your build policies.

•  Inform developers of build statuses.

•  Make builds and reports available to everyone.

•  Automate deployments to various testing environments.

Your team and Continuous Integration

•  The team should always see the monitor.

•  Transparency is important – Developers need to see status and bugs.

•  Define some punishment for breaking the build.

CruiseControl / phpUnderControl

•  The mother of all CI servers.

•  phpUnderControl features:

•  phpUnderControl features: •  Support for different code metric views.

•  It includes, but is not limited to, plugins for email notification, Ant, Phing, and various source control tools.

•  A web interface is provided to view the details of the current and previous builds.

•  phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects:

•  phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: •  PHPUnit

•  PHPDocumentor

•  PHP_CodeSniffer

•  PHP_Depend

•  PHP_Depend •  PHP_CompatInfo

•  … your own tools

Code metrics

•  A software metric is a measured value for a quality of software.

Code metrics

•  A software metric is a measure of some property of a piece of software or is specifications.

•  You cannot control what you cannot measure.

Measuring code quality

•  Lines of code.

•  Text-based metric for code size

•  Several definitions:

•  Several definitions: •  Lines of Code (LOC)

•  Comment Lines of Code (CLOC)

•  Non-Comment Lines of Code (NCLOC)

•  Executable Lines of Code (ELOC)

•  Ratios are of interest:

•  Ratios are of interest: •  I.e. ratio of CLOC / (E)LOC

•  Code coverage

•  Tells how many lines of code are covered by your unit tests.

•  100% Code coverage is a required criteria, but it doesn’t tell if your tests do test your code sufficient enough.

•  Which statements, branches, and paths are executed when the tests run?

•  Which statements, branches, and paths are executed when the tests run? •  C0-Coverage: Statement Coverage

•  C1-Coverage: Branch Coverage

•  C2-Coverage: Path Coverage

•  Cyclomatic Complexity

•  Metric for the complexity of a code unit

•  Counting the branching points:

•  Counting the branching points: •  if, for, foreach, while, case, catch, &&, ||, ternary

operator (?:)

•  Interpretation:

•  Interpretation: •  Higher complexity leads to more errors

•  Higher complexity makes testing harder

Analyze your code quality

•  Use the following tools:

•  Use the following tools: •  PHPUnit

•  PhpDocumentor •  Generate API documentation

•  PHP_CodeSniffer •  Coding Style •  Correctness of API Documentation

•  Php_Depend •  Analyze code dependencies

•  Zend Code Analyzer •  Unreachable Code •  Unused parameters and variables

Collect code analysis

•  Either collect all analysis results in your CI server or build a tool to let your developers see a wrap-up of all analyzer results.

•  Categorize code analysis results in different levels.

•  Higher rated issues need to be fixed first.

Continual metrics

•  Metric values are not definite values.

Continual metrics

•  Wire metrics and analysis into continuous integration.

Continual metrics

•  Discuss on iteration boundaries.

Continual metrics

•  Discuss on iteration boundaries.

•  It is important to note that variations of metrics need more attention than its values.

Issue tracking

•  Issues and tasks need to be:

Issue tracking

•  Issues and tasks need to be: •  organized

Issue tracking

•  allocated

Issue tracking

•  allocated

•  scheduled

Issue tracking

•  allocated

•  scheduled

•  tracked

Issue tracking

•  allocated

•  scheduled

•  tracked

•  That’s where an issue tracker comes into play.

Issue tracking

•  An Issue tracker is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work.

Issue tracking

•  An Issue tracker is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work.

•  Issues may include the time a bug was reported, its severity, the erroneous program behavior, and details on how to reproduce the bug; as well as any programmers who may be working on fixing it.

Issue tracking

•  Typical issue tracking systems support the concept of the life cycle for a bug/feature which is tracked through the status assigned to the issue.

Issue tracking

•  Typical issue tracking systems support the concept of the life cycle for a bug/feature which is tracked through the status assigned to the issue.

•  You can also assign the expected time needed to fix a bug so you can plan how many issues may be solved in one iteration.

Issue tracking

•  If supported by your issue tracker and version control system use a hook script to let developers mark commits to an active issue.

Issue tracking

•  If supported by your issue tracker and version control system use a hook script to let developers mark commits to an active issue.

•  If you require an issue ID in the commit message developers are forced to use the issue tracking software.

Pair Programming

•  Helps write better code by dividing responsibility between two people.

Pair Programming

•  Observer – strategy

Pair Programming

•  Observer – strategy •  Improvements

Pair Programming

•  Future problems

Pair Programming

•  Driver – tactics

Pair Programming

•  Driver – tactics •  Completing the task

Pair Programming

•  Observer functions as safety net and guide.

Pair Programming

•  Observer functions as safety net and guide.

•  Observer and Driver can exchange roles.

Pair Programming

•  Benefits

Pair Programming

•  Benefits •  Design quality

Pair Programming

•  Reduced cost due to fewer defects

Pair Programming

•  Learning & Training

Pair Programming

•  Shared knowledge of product

Pair Programming

•  Shared knowledge of product

•  Improved discipline and time management

Pair Review/Buddy System

•  Review your code/tests regularly with different partners in short meetings.

•  Solve difficult problems.

•  Exchange best practices, ideas, elegant solutions.

•  Just spread knowledge.

Code Reviews: Too time consuming?

•  Code Reviews may seem to be to be too time consuming when time pressure increases.

•  Without specific approaches to manage the reviews they might be in fact too time consuming.

•  These issues are simply problems with some review processes.

Code Reviews: Your return on investment

•  Regular peer review is a proven process with demonstrable ROI.

•  The benefits vary from team to team but commonly include:

•  The benefits vary from team to team but commonly include: •  Identifying bugs and defects early.

•  Sharing expertise and encouraging knowledge transfer.

•  Encouraging adherence to internal standards and style conventions.

•  Identifying individual strengths and weaknesses.

Code Reviews: Processes are your friend!

•  Defining Code Review Iterations is as important as defining how to do them!

Code Reviews: Processes are your friend!

•  Defining Code Review Iterations is as important as defining how to do them!

•  Don‘t skip any Code Reviews as changes to your source code are more expensive the later you do them!

Code Reviews: What are the best tools?

•  A human being!

•  There is pair programming – why not make it pair reviewing?

•  An initial Code Review is faster and more reliable to be done, if both Reviewer and Developer can communicate directly with each other.

Code Reviews: Web-based tool: Crucible

•  Crucible is a web based Code Review tool by Atlassian Software Systems Pty Ltd.

•  Commercial tool – but available free of charge for Open Source projects.

•  Crucible's mission is to streamline the process aspects so development teams can access the benefits.

•  This is how it looks like:

•  You create a new review from your version control system:

•  Your review is open:

•  Both reviewer and developer can communicate about the code:

•  Commits can either be done after committing to a version control system or using pre-commit diff files.

Testing your web application

•  Now that we covered the code analysis what about the product you are creating?

•  We want to have automated tests for our web application that do regression testing, compare results and check whether our application runs flawlessly in all of our supported browsers.

•  Challenges:

•  Challenges: •  Modern GUI app testing (AJAX & Co.)

•  OS / browser / platform

•  JavaScript engines / versions / levels

•  There is the Selenium product family:

•  Selenium Core

•  Selenium IDE

•  Selenium RC

•  Selenium Core:

•  Selenium Core: •  JavaScript engine

•  runs in browser

•  controls web application in other frame

•  exposes its API

•  Selenium IDE:

•  Selenium IDE: •  Firefox plugin

•  directly communicates with Selenium Core in the browser

•  Records / replays scenarios

•  great to get started (interactively)

•  Selenium RC:

•  Selenium RC: •  remote control

•  server component and proxy

•  allows test automation in various ways

•  has a capability called Selenium Grid, to drive multiple instances of the Selenium RC server

Advantages of Selenium

•  Tests run directly in browser

•  Implemented entirely using browser technologies:

•  Implemented entirely using browser technologies: •  JavaScript

•  DHTML

•  Frames

•  Selenium interacts with the DOM served to the browser

•  Selenium is agnostic of server side technology

Concepts of Selenium

•  Element Locators:

•  Element Locators: •  specify HTML elements

•  Patterns:

•  Patterns: •  for pattern matching values

•  Actions:

•  Actions: •  manipulate application state

•  Accessors:

•  Accessors: •  store results in variables

•  Assertions:

•  Assertions: •  verify that the application is in a certain state

Useful Selenium Tools

•  For Firefox!

•  For Firefox! •  Web Developer Toolbar

•  Firebug

•  XPather

•  Firebug

•  XPather

•  Xpath Checker

Maintaining Selenium Tests

•  Test code is just like production code.

•  Needs refactoring, abstraction.

•  Try to maintain meaning – in test names, variable names, comments, and test suite organization.

Test Fragility

•  Tests are fragile.

Test Fragility

•  They break when your code changes.

Test Fragility

•  They break when your code changes.

•  Stabilize them!

Test Fragility

•  Minimize influences:

Test Fragility

•  Minimize influences: •  Use Mock Objects

Test Fragility

•  Recreate dependencies – SetUp

Test Fragility

•  Clean up your test environment – TearDown

Test Fragility

•  Clean up your test environment – TearDown

•  Use non-fragile locators

Mock Objects

•  Let Unit Tests test the code not the dependencies:

Mock Objects

•  Let Unit Tests test the code not the dependencies: •  Simulate complex objects

Mock Objects

•  Simulate database connections

Mock Objects

•  Simulate uncontrollable variables – time, temperature etc.

Mock Objects

•  Simulate uncontrollable variables – time, temperature etc.

•  Simulate objects that do not exist yet

Test SetUp

•  Use the SetUp method to recreate testable states for functional tests:

Test SetUp

•  Use the SetUp method to recreate testable states for functional tests: •  Use methods that are unit-tested

Test SetUp

•  Fill database with necessary data

Test SetUp

•  Fill database with necessary data

•  Tests can be run on additional fresh test environments.

Test TearDown

•  Use the TearDown method to clean up your test environment:

Test TearDown

•  Use the TearDown method to clean up your test environment: •  Remove no longer necessary database entries

Test TearDown

•  Clean caches, no longer needed log files

Test TearDown

•  Reduces uncontrollable influences on tests.

Test TearDown

•  Reduces uncontrollable influences on tests.

•  Keeps test execution quick.

Non-Fragile Locators

•  When using Selenium, use non-fragile locators:

•  When using Selenium, use non-fragile locators: •  Keep locators independent from DOM

•  Use ids or names

•  Use wildcards where applicable

•  Dom-Locators (JavaScript expressions)

•  Page elements can be moved around, but your tests will still work.

Release process

•  Make sure you have a standard procedure for releasing / launching new versions of your product.

Release process

•  Assign responsibility or have a dedicated release management.

Release process

•  Assign responsibility or have a dedicated release management.

•  Use the tools to the fullest.

Release process: Versioning

•  Use the versioning system – choose a stable revision and branch.

•  Development can continue without breaking the stable branch.

•  If you find defects before next major version you can release stable versions by fixing them in the stable branch.

•  Reintegrate fixes for bugs into the trunk so they are do not resurface.

Release process: Delta Builder

•  Tool to collect data about stable branch.

•  List changes to the codebase since last release.

•  Have developers review their changes.

•  Highlight important pieces of your product – configuration files, most used libraries etc.

•  Match versioning data with issue tracker – have all completed tasks been really checked into the versioning system?

Release process: Tests

•  Dedicate time to test the stable branch before release.

•  Run all your tests as first step – last remaining defects found can be fixed before release.

•  Make sure all your tests pass.

•  A manual test can help you assure stability – find defects not identifiable by tests.

Release process: Checklists

•  Minimize human error

•  Collect all tasks for the release

•  Collect all tasks for the release •  Running tests

•  Building dependencies

•  Deploying all components of your product

•  Communicating – notify all those that need to know!

Checklists in general

•  Once again - Minimize human error!

•  All repetitive but complicated tasks:

•  All repetitive but complicated tasks: •  Creating the skeleton of a new module

•  Manually check the functionality of a module

•  Set up a new system ready to be shipped

•  …

Questions?

Thank You!

ipc2008 slide qa in depth best practises

Education

construction practises

hr practises wipro

global hr practises

xcelsius best practises[1]

data centre best practises workshop

best practises sauvegardes vmware

datastage best practises 1

symposium ketenintegratie - presentatie best practises

best practises in mobile fundraising

best practises - germanwatch

digital practises portfolio

android best practises

081007- requirement practises - agile

safety practises

best practises in dfpcl

top 10 wms practises

practises for product owners

environmental practises in helsinki,

legible practises (3.65mb)

802.1x best practises