behavioural driven development (bdd) automated testing

STeP-IN SUMMIT 2014

11th

International Conference on Software Testing

June 2014 at Bangalore, Hyderabad, Pune - INDIA

Behavioural driven development (BDD) automated testing with Specflow & Specrun

by

Mradul Kapoor, Consultant,

Deloitte and

Jigesh Shah, Consultant Deloitte

Copyright: STeP-IN Forum

Published with permission for restricted use in ‘STeP-IN SUMMIT 2014’ in agreement with full copyrights from owner(s) / author(s) of material. All rights reserved. No part of this

publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without

the prior consent of the owner(s) / author(s). This edition is manufactured in India and is authorized for distribution only during ‘STeP-IN SUMMIT 2014’ as per the applicable

conditions.

Practices Experience Knowledge Automation

Produced By Hosted By

www.stepinforum.org www.qsitglobal.com

http://www.stepinforum.org/

http://www.qsitglobal.com/

1 of 16

Produced by Hosted by

STeP-IN SUMMIT 2014 Pre-Conference Tutorials Behavioural driven development (BDD)

automated testing with Specflow & Specrun

A practical view on Behavior Driven Development (BDD) automation using Speclog + Specflow

Spec-ing your automation using Specflow

1

• Are we thinking as per business??

• Constructive or Destructive or Both??

• Collaboration Breeds Innovation

Testing Challenges

2 of 16




2

Having conversations

is more important than capturing

conversations

is more important than automating

conversations

3

Behavior Driven Development (BDD)

• BDD tests are executable specifications of desired behavior and functionality of

the system

– Expressed in language of the problem domain

– Automated scripts

– Regression suite

• Development is driven from the outside-in (user story to script)

• Driven by Business Value

• Realized through the Interface (GUI)

• Tests are defined in a natural language (Gherkins)

3 of 16




4

Workflow of BDD

5

• Establish the goals of different stakeholders

• Drawing out features which will achieve those goals using feature injection

• Involve stakeholders through outside–in software development

• Use examples to describe the behavior of the application

• Automate those examples to provide feedback and regression testing

• Use 'should' when describing the behavior of software

• Use mocks to stand-in for modules of code which have not yet been written

BDD Practices

4 of 16




6

• Example – "Refunded or exchanged items should be returned to stock.“

• Customer, Product Manager, Developer or QA engineer might clarify the requirements by

breaking this down into specific scenarios

• Scenario 1: Refunded items should be returned to stock

BDD Example

7

• Scenario 1: Refunded items should be returned to stock

– Given a customer previously bought a black sweater from me

– and I currently have three black sweaters left in stock

– When he returns the sweater for a refund

– Then I should have four black sweaters in stock

BDD Example

5 of 16




8

High Level Tool Comparison

Features\Tools Cucumber Jbehave SpecFlow

Platform Support Java\Ruby Java .Net

Excel Integration No No Yes

Documentation Yes Yes Yes

Regular Expressions Yes Yes Yes

Auto Complete No No Yes

Traceability No No Speclog

9

Specflow

• BDD testing framework

– Integrates nicely with Visual Studio

• Acceptance tests manifest themselves as features and scenarios

• Specflow generates automated test from features

– Specflow tests run as normal xUnit tests

– Visual Studio test runner, build server

6 of 16




10

How Specflow Scenarios are executed

11

Specflow – Feature File

• Describe some piece of functionality of the system

• Maintained in a .feature file

• Every feature has a title and is written after the keyword Feature

• Plain readable text

– Understandable by all parties, including business

– Gherkin format, popularized by Cucumber

• Features contains one or more scenarios

7 of 16




12

Feature File(Example)

Feature: Addition Example

In order to avoid silly mistakes

As a math novice

I want to be told the sum of two numbers

Scenario: Add two numbers

Given I have entered 50 into the calculator

And I have entered 70 into the calculator

When I press add

Then the result should be 120 on the screen

13

Scenarios

• A scenario describes a single acceptance test for a feature

• Most features are composed of multiple scenarios

• Specflow generates a test for each scenario

– The name of the test is generated from the scenario title

8 of 16




14

Scenario Example




When I press add


15

Scenario Step

• Used to compose a scenario

• Custom code to automate your application

• Describe preconditions, triggering action, and verification of outputs for the

acceptance test

• Given: preconditions

• When: triggering action

• Then: behavior and state verifications

9 of 16




16

Scenario Step Example

[Feature File]

Given I Log into an Application

is matched to the following step definition binding…

[Fixture(Code)]

[Given(@"I Log into an Application")]

public void login() {

. . .

}

17

Background

• Common preconditions for all scenarios in a feature

• Contains one or more scenario steps

• Executed before any other steps in the scenario

• Specflow generates a method from the background element in the feature file in the test

class

– Invoked from each scenario test case in the test class

10 of 16




18

Background:

Given I have initialized the Sum- variable to 0




When I press add


Background Example

19

Tags

• Markers that can be applied to features and scenarios

• Useful for selectively mixing in behavior

• Used to categorize scenarios

– Some unit test frameworks support this categorization tagging

• Custom tags you define

11 of 16




20

Tags Example

@AddTwoNumbers




When I press add


21

Scenario Outlines

• Data-driven scenarios or scenario templates

• Consists of

– Scenario template specification with data placeholders

– Set of examples providing values for placeholders

• Specflow generates parameterized test logic for the scenario outline and individual test

method for each example set

12 of 16




22

Scenario Outline


Given I have entered <Number1> into the calculator

And I have entered <Number2> into the calculator

When I press add

Then the result should be <Total> on the screen

Examples:

| Number1 | Number2 | Total |

|10 | 10 | 20 |

| 50 | 70 |120 |

23

• Allows additional code execution at various points during the test execution lifecycle.

• The hooks are global but can be restricted to run only for features or scenarios with a

specific tag

Hooks

Hooks Description

[BeforeTestRun] [AfterTestRun] automation logic that has to run before/after the entire test run

[BeforeFeature] [AfterFeature] automation logic that has to run before/after executing each feature

[BeforeScenario] or [Before]

[AfterScenario] or [After]

automation logic that has to run before/after executing each scenario or

scenario outline example

[BeforeScenarioBlock]

[AfterScenarioBlock]

automation logic that has to run before/after executing each scenario block

(e.g. between the "givens" and the "whens")

[BeforeStep] [AfterStep] automation logic that has to run before/after executing each scenario step

13 of 16




24

[BeforeScenario(“Initialize")]

public static void BeforeWebScenario()

{

AddTwoNumbers();

}

Hooks Example

25

Regular Expression

• Provides parameterization for the steps with common Step Definition

[Feature]


[Fixture]

[Given (@" I have entered (.*) into the calculator ")]

Public void EnterNumbers(int Number1)

{

}

(.*) => string

(\d+) => integer

14 of 16




26

Specflow Integration with Other Tools

Specflow

Feature/ Scenarios

Scenario: x

Given abc

Feature

Scenario 1 Scenario 2

Step 1 Step 2 Step 1

Step Definition

Given abc

Navigate to URL

Login to the System

Step 1

Step 2

Step 3 Step 4

Step Implementation

@Step(“Login”)

Public void login() WebDriver /

Watin/Coded UI Project Libraries

Web Browser

Application

NUNIT/

Xunit/

MBUnit/

SpecRun

27

• Automate Application on Web

– Specflow + Selenium

– Specflow + Watin

• Automate Application on Desktop

– Specflow + White

• Automate Application on Mobile

– Specflow + Ranorex

– Specflow + Selenium

• Image Automation

– Specflow + Sikuli

Specflow on various Platforms

15 of 16




28

Why Specflow can be used

• Never have to rewrite your specs/tests, since the behaviour does not change, it's only

the implementation that does.

• Collaboration between Business stakeholders, Business Analysts, QA team and

Developers

• Ubiquitous language

• Traceability is relatively easy

• Enables easy adaptation to requirement changes

• Localized to 35+ languages

29

• SpecLog introduces the concept of workspaces to the product backlog, where you can map

requirements in story maps.

• Requirements can be arranged in multiple workspaces and you can visualize

dependencies and refinements of requirements on the map

• Supports collecting acceptance criteria and examples for existing user stories.

• Can be used to insert UI scribbles and screen shots as well as data and business rule

examples

• Synchronizes requirements across multiple clients is available to support collaboration.

• Can be used to tag requirements to support your example collection flow

SpecLog

16 of 16




30

Leading practices

• Write high-level specifications

• Specifications should remain stable over time

• Build a scripting interface for manipulating your system under test (SUT)

• Focus specifications on isolated behaviors

• Think of specifications in Given-When-Then format

• Use the Page Object pattern

behavioural driven development (bdd) automated testing

Documents