requirement management & domain modelling marko ”narsuman” rintamäki 2012

Requirement ManagementRequirement Management

Requirement Management & Domain Modelling

Marko ”NarsuMan” Rintamäki

2012

Requirement ManagementRequirement Management

What is Requirement Manangement?

What means a feature, requirement, use cases, and user story

Requirement Management continues…Requirement Management continues…

Between 40% and 60% of software failures and defects are the result of poor software management and requirements definition. In plain English, this means that about half of the problems encountered could have been avoided by making it clear, from the very beginning, what the customer expected from the respective project. This is to say that the programming was fine and the developers did their job well – only they did a different job from what they were supposed to.

http://www.softwareprojects.org

Read also http://www.projectshrink.com/why-requirements-change-270.html

http://www.softwareprojects.org/

ChangeManagement

Old School Process model?

http://en.wikipedia.org/wiki/Waterfall_model

Requirement Management

Product Life Cyle

Agile Process?


Sprint 1 Sprint 2 Sprint 3 Sprint 4Sprint 0

http://en.wikipedia.org/wiki/Agile_software_development

Product Life Cyle

Requirement Specification

http://en.wikipedia.org/wiki/Software_Requirements_Specification

Example SystemExample System

http://www.sysml.org/http://www.sysml.org/

UML

OMT

OMT

System Engineering

Google: SysML, UML, Systems engineering, Software Design,

CodeCode

Software Engineering - ArchitectureSysML

UML

OMT

.NET, JAVA, C++

Software Engineering - Design


http://www.sysml.org/

Different points of view on example systemDifferent points of view on example system

Seller

User

Developer

Why Requirements?Why Requirements?

IDEALIDEAL

Requirements help

Design

Requirements help

Design

REQ-XREQ-X

REQ-YREQ-Y

REQ-ZREQ-Z

REQ-OREQ-O

REQREQ

ScalabilityScalability

StabilityStability

PerformancePerformance

SecuritySecurity


StressStress

UsabiltyUsabilty

Different points of view on systemRequirement Category‘s

Requirements from product development point of viewRequirements from product development point of view

„scalability“„scalability“

„Stability“

„Functionality“

Problem Domain vs. Solution DomainProblem Domain vs. Solution Domain

What we want?How to do it?

Business/Stake/Customer Requirements

Architecture/Design/Technology Requirements

Requirements and Architecture DesignRequirements and Architecture Design

Business and customer

Development Team

BusinessRequirements

BusinessRequirements

TechnicalRequirements

TechnicalRequirementsNEEDNEED SOLUTIONSOLUTION

Design & ArchitectureDesign & Architecture

AgreementAgreement

“One of sides should not dominate in design process”“One of sides should not dominate in design process”

DefinesDefines

Feature X * nFeature X * n

FunctionalRequirements


Non-FunctionalRequirements


Use CasesUse Cases

Vision of product

Simple Requirement Management ProcessSimple Requirement Management Process

Problem Domain Solution Domain

Solution Proposal

Customer/Marketing/business

User StorysUser Storys

FEATUREVISION/NEED/

PROPOSAL

FEATUREVISION/NEED/

PROPOSAL

YOU!YOU!

Design documents & implementation

Test Case

Requirement tracebilityRequirement tracebility





FeatureFeature

DesignDesign

ImplementationImplementation

TestTest

BugBug

Stake holderRequirementStake holderRequirement

Code…..…..………..Fix

..

Implement/Fix TASK

Implement/Fix TASK Test TASKTest TASKDesign

TASKDesignTASK

ResearchTASK

ResearchTASK

How to find requirements?How to find requirements?

● You can find requirements from several sources: Users, Stakeholders, Business and Development team and many others

● Requirements are trying to define nature of feature/system/solution more specific than common written document does. This information is helping development team to design a solution for a need

● There is several common methods to define and gather requirements.

– Traditional Requirement modeling (http://en.wikipedia.org/wiki/Requirements_management)

– RUP/UML based Use Case modeling (http://en.wikipedia.org/wiki/IBM_Rational_Unified_Process)

– Agile XP oriented User Story’s (http://en.wikipedia.org/wiki/Agile_software_development)

● Your task is to figure out a small difference between them

Read more http://en.wikipedia.org/wiki/Requirements_management

Traditional Requirement ModelingTraditional Requirement Modeling

• A requirement shall be a complete sentence.• Sentence has to be understandable, measurable and

testable

• ReqId1 - Tractor has four wheels• ReqId2 - Tractor has one exhaust pipe• ReqId3 - Engine of tractor is capable of use flexi fuel• ReqId4 – The tractor has a hook for trailer• ReqId5- The tractor shall have a enhanced driving system

• ReqId1 - Tractor has four wheels• ReqId2 - Tractor has one exhaust pipe• ReqId3 - Engine of tractor is capable of use flexi fuel• ReqId4 – The tractor has a hook for trailer• ReqId5- The tractor shall have a enhanced driving system

Google: requirement specification template and SRS Software Requirement Specification

Google: requirement specification template and SRS Software Requirement Specification

Functional and non-functional requirementsFunctional and non-functional requirements





Functional Requirement

"User can select application from ui

by using wheel button”

”Tractor can be driven both directions”

Functional Requirement


by using wheel button”

”Tractor can be driven both directions”

Non-Functional Requirement

"Performance Requirement"

”Tractor Startup should take minimum 10 seconds”

”Usability Requirement”

”User interface should be able to control using simple wheel quide”

”The hook can last max 20Kkg trailer load”

Non-Functional Requirement


”Tractor Startup should take minimum 10 seconds”

”Usability Requirement”

”User interface should be able to control using simple wheel quide”

”The hook can last max 20Kkg trailer load”

How it works? How fast it is?How stable it is?

Do some googling!!

Create a wiki page!!

AboutUserStory

Do some googling!!


AboutUserStory

Functional Requirement Category?

Functionality

Implementation should work like this way

Functionality

Implementation should work like this way

http://en.wikipedia.org/wiki/Functional_requirement

Non-Functional Requirement examplesNon-Functional Requirement examples

Scalability

How our implementation is scaling in situation X?

Scalability


Stability

Is our implementation stable in a situation

like zzzZZZ?

Stability


like zzzZZZ?

Security

Is our implementation secure enough against

attack type xxx?

Security


attack type xxx?

PerformanceHow good performance our implementation provides against competitor?

PerformanceHow good performance our implementation provides against competitor?

StressHow much we can stress our implentation without

a problems?

StressHow much we can stress our implentation without

a problems?

UsabiltyIs implementation usable for target

customer?

UsabiltyIs implementation usable for target

customer?

MaintenanceIs implementation easy to maintain?

MaintenanceIs implementation easy to maintain?

http://en.wikipedia.org/wiki/Non-functional_requirement

User Interface????

Usability of functionality?





User Inteface Layouts

Scalability


Scalability


Stability


like zzzZZZ?

Stability


like zzzZZZ?

Functionality

How our implementation should work?

Functionality

How our implementation should work?

Security


attack type xxx?

Security


attack type xxx?

Performance

How good performance our implementation provides against competitor?

Performance

How good performance our implementation provides against competitor?

Stress

How much we can stress our implentation without

a problems?

Stress

How much we can stress our implentation without

a problems?

Usabilty

Is implementation usable for target

customer?

Usabilty

Is implementation usable for target

customer?

Maintenance

What about maintenance?

Maintenance

What about maintenance?

Training Task: Reverse Engineering requirements

http://store.steampowered.com/





? ?

Functional RequirementsFunctional Requirements Non-Functional RequirementsNon-Functional Requirements

Requirement Analysis

http://en.wikipedia.org/wiki/Requirements_analysis

UML Modelling & Requirements

Use CaseUse Case

USE CASE

Written scenario for action. Also execeptions included

Use Case: Open IFDK Main Application

Actor: IFDK User

Step1: Gadget User touches home button

Step2: UI wakeup initiated (if standby)

Step3: Home screen is activated

Setp4: User browses applications specific icons using wheel button

Step5. Icons are moving on screen left and right

Step6: User selects application by pushing wheel button

Step7: Application starts up <4 seconds

Execptions:

1. If application cannot start there will be note on screen about problem

USE CASE


Use Case: Open IFDK Main Application

Actor: IFDK User








Execptions:


Use Case

A use case in software engineering and systems engineering, is a description of steps or actions between a user (or "actor") and a software system which leads the user towards something useful.[1]

Wikipedia

http://en.wikipedia.org/wiki/Software_engineering

http://en.wikipedia.org/wiki/Systems_engineering

http://en.wikipedia.org/wiki/Actor_(UML)

http://en.wikipedia.org/wiki/Use_case

Practice: Create Use CasesPractice: Create Use Cases

ACTOR

Use Case

SYSTEM Actor:

System: IFDK kit

Use Cacenario: Standby mode after boot

1. User turn’s on IFDK by pressing red button on front panel

2. Screen wil flash and show welcome text ”Hello my friend!”

3. User interface opens after seconds

4. Screen will show three selection buttons

5. After 30 seconds user inteface goes to standby mode

Exeption:

1. If user activates screenby tapping it standby counter will be reseted

Actor:

System: IFDK kit

Use Cacenario: Standby mode after boot

1. User turn’s on IFDK by pressing red button on front panel

2. Screen wil flash and show welcome text ”Hello my friend!”

3. User interface opens after seconds

4. Screen will show three selection buttons

5. After 30 seconds user inteface goes to standby mode

Exeption:

1. If user activates screenby tapping it standby counter will be reseted

Define Stakeholders?Define Customer?

USE CASE:_____________________________________________

Actor:__________________________________

Scenario:

Execeptions:

USE CASE:_____________________________________________

Actor:__________________________________

Scenario:

Execeptions:

Why Use Cases are used?Why Use Cases are used?

Use cases are used for define functional requirements!

(FDD) Feature Driven Development(FDD) Feature Driven Development

Core SoftwareCore Software

Is product a combination of features?Is product a combination of features?

Calory CounterCalory Counter

Drum MetronomeDrum Metronome

Table Drum ModeTable Drum Mode Standby ModeStandby Mode

MIDI SupportMIDI Support

Touch Screen with single tapTouch Screen with single tap

Is product a combination of features?Is product a combination of features?


Drum MetronomeDrum Metronome

Table Drum ModeTable Drum Mode

Standby ModeStandby Mode



FeatureFeature

FeatureFeature

Feature is functionality of product/software which can be seen as one module of whole product

Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface

Feature is functionality of product/software which can be seen as one module of whole product

Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface

Do some googling!!


AboutUserStory

Do some googling!!


AboutUserStory

Idea#1Idea#1

Who is defining a Feature?Who is defining a Feature?

Customer

I would like to haveInternal Flame Drum Kit

Could you deliver it to us?

I would like to haveInternal Flame Drum Kit

Could you deliver it to us?

Actually We have severalfeatures for it here

Actually We have severalfeatures for it here

Ok!What's a plan

Ok!What's a plan

Nice looking feature propoals.We have to do some evaluationNice looking feature propoals.

We have to do some evaluation

Idea#2Idea#2

Idea#3Idea#3 Idea#4Idea#4

Idea#1Idea#1 -Technology?-Knowledge-Resource-Solution?-Priority?

Practice: Defining FeaturesPractice: Defining Features

Play problem domain card game with team to

search for features?



Non-Functional

Requirements

Non-Functional

Requirements

FunctionalityFunctionality

StabilityStability

SecuritySecurity


UsabilityUsability

User Interface Design?User Interface Design?

Feature XFeature X

Traditional Requirement Modelling and FeaturesTraditional Requirement Modelling and Features

..............

System TestingSystem Testing

Integration TestingIntegration Testing

Unit TestingUnit Testing

Customer RequirementsCustomer Requirements

System RequirementsSystem Requirements

Design RequirementsDesign Requirements

Component RequirementsComponent

Requirements

Acceptance TestingAcceptance Testing

Why we need requirements from testing point of view?Why we need requirements from testing point of view?


„Developer's Area“

„Test Engineers Area“

”Traditional Testing Levels”


Feature example 1 (Invented on course 2009-2010)Feature example 1 (Invented on course 2009-2010)

Calory Counter:

Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application

Energy usage

Feature example 2 (Invented on course 2009-2010)Feature example 2 (Invented on course 2009-2010)

Table Drummer:

Player drums table board instead of drum can. IFDK kit is able to use DSP algorithm to detect correct drum sound from environment.

In training mode IFDK is trained to detect drum sounds for environment.


DSPAlgorithm

DSPAlgorithm

Customer/Business Requirements?Customer/Business Requirements?

Calory CounterCalory CounterDrum MetronomeDrum Metronome


Simple Training ModeSimple Training Mode



Customer Type 1 Customer Type 2 Customer Type 3 Customer Type 4

Who are our target customers?

Customer StrategyCustomer Strategy

Drum MetronomeDrum Metronome Table Drum ModeTable Drum Mode

Simple Training ModeSimple Training Mode



Customer Type 1 Customer Type 2 Customer Type 3 Customer Type 4

What is our key customer?

Primary Target


Secondary Target

Requirement

Requirement

Requirement USE CASE #2

USE CASE #1

USE CASE #3

Requirement

Requirement

Requirement

USE CASE #1 User Story #1

User Story #2

User Story #3

RequirementRequirement

RequirementRequirement USE CASE #2USE CASE #2

USE CASE #1

USE CASE #3USE CASE #3

RequirementRequirement USE CASE #1USE CASE #1 User Story #1User Story #1

Requirement

Requirement


USE CASE #1

USE CASE #3

Requirement

Requirement

Requirement

USE CASE #1 User Story #1

User Story #2

User Story #3




USE CASE #1





USE CASE #1USE CASE #1 User Story #1User Story #1

User Story #2User Story #2

User Story #3User Story #3

Requirement

Requirement


USE CASE #1

USE CASE #3

Requirement

Requirement

Requirement

USE CASE #1RequirementRequirement


USE CASE #1RequirementRequirement



Features and release planningFeatures and release planning

Release 0.1

Release 1.1 Release 1.2

Feature: Simple Training Mode

Feature: Table Drum mode

Feature Touch Screen with single tap

Release 1.0

TIME TO MARKET!! For Target Group 3TIME TO MARKET!! For Target Group 3

CORE/Platform Software Development



DefinesDefines






Use CasesUse Cases

Vision of product



Solution Proposal

Customer User StorysUser Storys

FEATUREVISION/NEED/

PROPOSAL

FEATUREVISION/NEED/

PROPOSAL

YOU!YOU!


Test Case

Brief History of Use CasesBrief History of Use Cases

● Defined by Ivar Jacobson http://en.wikipedia.org/wiki/Use_case

● Used with UML (Unified Modeling Language) http://en.wikipedia.org/wiki/Unified_Modeling_Language

● RUP (Rational Unified Process)● http://fi.wikipedia.org/wiki/RUP

http://en.wikipedia.org/wiki/Unified_Modeling_Language

User StoryUser Story

● http://en.wikipedia.org/wiki/User_story

http://en.wikipedia.org/wiki/User_story

Brief History of User StorysBrief History of User Storys

● Used in agile development process for requirement definition and gathering

● Backlog Item

● Simple way to describe needed functionality

DefinesDefines






Use CasesUse Cases

Vision of product



Solution Proposal

Customer User StorysUser Storys

FEATUREVISION/NEED/

PROPOSAL

FEATUREVISION/NEED/

PROPOSAL

YOU!YOU!


Test Case


USER STORY Examples

Simple phrase describes a need. This can lead to several other storys!

"As user I would like to open application easily”

As a user I would like to select application from icon list on screen

As a user I would like to configure amount of application icons on screen

"As a user I would like to use wheel for speeding up selection process"

"As a user I would like to initate application fast enough"

"As a tractor driver I would like to have enhanced driving system”

USER STORY Examples


"As user I would like to open application easily”

As a user I would like to select application from icon list on screen

As a user I would like to configure amount of application icons on screen

"As a user I would like to use wheel for speeding up selection process"

"As a user I would like to initate application fast enough"

"As a tractor driver I would like to have enhanced driving system”


Practice: Create User StorysPractice: Create User Storys

USER STORY:

As a bad behavin person I cannot access IFDK using wlan without encryption

How to test?Acceptance Criteria?

USER STORY:

As a bad behavin person I cannot access IFDK using wlan without encryption


USER STORY:

As a member of audience I can hear effect sound that player is having electrical shocks


USER STORY:

As a member of audience I can hear effect sound that player is having electrical shocks


Agile Requirement ManagementAgile Requirement Management

Epic Story

User StoryUserStory0001

RequirementId0002

RequirementId0003

EpicStory0001

As a userI would like to use

productWhich is fast to power

on

As a userI would like to use

productWhich is fast to power

on

As a Customer I wouldlike to have topquality product

As a Customer I wouldlike to have topquality product

NOTE:Gadget should

have >30fpsUI performace

NOTE:Gadget should

have >30fpsUI performace

NOTE:Gadget should

Startup <5seconds

NOTE:Gadget should

Startup <5seconds

AcceptanceCriterias?

EPIC STORY: As a user I would like to use product which is fast to startup

Story Points:

Notes:

• Gadget should startup <5seconds

How to show it is tested?(eg. Acceptance Criteria)

Use timer to verify startup time. Measure time from power on to main screen activated.

• Time

EPIC STORY: As a user I would like to use product which is fast to startup

Story Points:

Notes:




• Time

USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up

Story Points:

Notes:




• Time

USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up

Story Points:

Notes:




• Time

USER STORY:

Story Points:

Notes:


USER STORY:

Story Points:

Notes:


USER STORY:

Story Points:

Notes:


USER STORY:

Story Points:

Notes:


RequirementMeasurable

Testable

RequirementMeasurable

Testable

Use Case, User Story, Requirement

USE CASE


Use Case: Open Application

Actor: Gadget User








Execptions:


USE CASE


Use Case: Open Application

Actor: Gadget User








Execptions:


USER STORY


"As user I would like to open application easily"

"As a user I would like to use wheel for simplify ui

interaction"

"As a user I would like to initate application fast

enough"

USER STORY


"As user I would like to open application easily"

"As a user I would like to use wheel for simplify ui

interaction"

"As a user I would like to initate application fast

enough"

Non Functional Requirement


"Application Startup should take minimum 4 seconds"

Non Functional Requirement


"Application Startup should take minimum 4 seconds"

Functional Requirements


by using wheel button"

Functional Requirements


by using wheel button"


http://www.opfro.org/

http://www.google.fi/url?sa=t&rct=j&q=requirement%20specification%20example&source=web&cd=1&ved=0CBgQFjAA&url=http%3A%2F%2Fwww.opfro.org%2FComponents%2FWorkProducts%2FRequirementsSet%2FSystemRequirementsSpecification%2FSystemRequirementsSpecificationExample.doc&ei=aiWETqjnG-P04QTMzNivDw&usg=AFQjCNFd4r5LLJgQj3foixq_jxZHjk78pQ&sig2=ILMCdBta4j8JVTCbQfgZgQ&cad=rja

Interesting Links ?Interesting Links ?


You have vision of productWhat means testing in brief?

You have vision of productWhat means testing in brief?

FeaturesFeatures Test CaseTest Case

Use CasesUse Cases


Product Design & Implementation

RequirementsRequirements

Test CaseTest Case

Test CaseTest Case

Test CaseTest Case

Ready to testReady to test

Testing & Quality Assurance

Can we deliverProduct

Ready to Deliver

Ready to Deliver

? CustomerNot ready

todeliver

SW Development Process (Waterfall)SW Development Process (Waterfall)

RequirementGathering/Evaluation

RequirementGathering/Evaluation

DesignDesign


Verification &Validation

Verification &Validation

MaintenanceMaintenanceError ManagmentProcessError ManagmentProcess

Task1Task1Task1Task1 Task1Task1 Task1Task1

Mile Stone 1Mile Stone 1 Mile Stone 2Mile Stone 2 Mile Stone 3Mile Stone 3

Task1Task1 Task1Task1

SW Development Process (Agile)SW Development Process (Agile)

User Story YUser Story Y

SprintSprint

Task1Task1

Task2Task2

Task3Task3

SprintSprint SprintSprint SprintSprint

Task4Task4

Task5Task5

Task6Task6

User Story XUser Story X

DesignImplementation

Verification


Verification

Product Backlog


Verification


Verification


Verification


Verification


Verification


Verification

User Story ZUser Story Z

Task7Task7

Task8Task8

Task9Task9

User Story ZUser Story Z

Customer/BusinessRequirements


Sub SystemRequirementsSub System

Requirements

ComponentRequirementsComponent

RequirementsComponent / Unit

TestingComponent / Unit

Testing

IntegrationTesting

IntegrationTesting

SystemTesting

SystemTesting

AcceptanceTesting

AcceptanceTesting

SystemRequirements

SystemRequirements

IFDK SystemVerification and Validation

IFDK SystemVerification and Validation

Use CasesUse Cases


FeaturesFeatures

RequirementsRequirements

Validation = Are we building the right product?Verification = Are we building the product right?Validation = Are we building the right product?

Verification = Are we building the product right?

Architecture&Design&

Implementation


Implementation

ProductProductVALIDATIONVALIDATION

VERIFICATIONVERIFICATION

IFDK Product IdeasIFDK Product Ideas

V-Model in testingV-Model in testing

Customer Requirements

Sub System Requirements

Component Requirements Component/Unit Testing

Integration Testing

System Testing

Acceptance Testing

System Requirements

Verification and Validation

Verification = Are we building the product right?Validation = Are we building the right product?

Component Design

ClassClass

AttributesAttributes

ClassClass

MethodsMethods

ClassClass


ClassClass

MethodsMethods

●Testing according ISEB standard

Functional Testing

Testing based on an analysis of the specification of the functionality of a component or system. See also black box testing.)

Non-Functional Testing

Testing the attributes of a component or system that do not relate to functionality, e.g. scalabilty, stability, reliability, efficiency, usability, maintainability and portability.

http://www.bcs.org/upload/pdf/glossary-current.pdf

Black Box vs White Box Testing

Unit Testing is White Box testingUnit Testing is White Box testingSystem Testing is Black BoxTestingSystem Testing is Black BoxTesting

????

We need to capsule all projects as sub projects, which have more independence

• Project Teams has to have own test engineer –> Integration Test Engineer• this lead’s to better defect prevention• Integration Test Engineer is part of project team, part of team at start of project• Test Engineer provides valuable information for design and enables better testability for project product

Define new roles for validation engineers

• Check next page

Lower level for test automation

• Enables regression test (method to block new implementation defects)• Support for executing more complex test scenarios• Continuous test execution (stability, performance gain etc)

●Notes

Test Level1 – Component/Unit testing

• Component/Class level unit testing (eg. xUnit framework)

Test Level2 - Integration Testing (Feature, Application)

• Integration test for feature/application with “stub” interface components• Functionality SW Testing• Non-Functional SW Testing according needs (eg. feature/application specific performance, stability)

Test Level3 – Regression Testing (Target HW+Android platform, tool, terminal etc.)

• Functional SW testing for all possible features• Non-Functional SW testing according needs• Non-Functional HW Testing according needs

Test Level4 – System Integration Testing (Target Platform, Tool, Terminal etc)

• Functional SW testing for all new features• Non-Functional SW Performance, Stability, Scalability

Test Level5 – Acceptance Testing

• Functional/Non-Functional test according customer requirements

HWT1 – Hardware Testing• HW Integration test with limited environment.• Conducted performance verified

●Testing Level descriptions

●Testing Levels (with TAF = test automation framework)IFDK Release v1.0

Feature Project 2

Regression Testing (ISEB)System Integration

TAF

Component/Unit Testing (ISEB)

IntegrationTesting (ISEB)

SystemTesting (ISEB)

AcceptanceTesting (ISEB)

Feature Project 1

TAF

TAF Project

TAF enables

TAF Supports

Verification and Validation

Refrerences & LinksRefrerences & Links

http://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdfhttp://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdf

VERIFICATION & VALIDATIONVERIFICATION & VALIDATION

Validation = Are we building the right product?

Verification = Are we building the product right?

Create a Test Case!

Functional?

Your SourcesFor Test Case

Functional Test Case:

●Verify functionality of XXXX

Non-Functional Test Cases

●Verify Stability of XXXX●Verify Performance of XXX●Verify Security of XXXX●Verify Usability of XXXX●Verify Scalability of XXXX●etc...

Non-Functional?

Which Type?

●Requirement●Use Case●Feature●User Story

●Customer's Idea●Brainstorm●Intitution●Exploratory

Check also.....

●Correct functionality path●Miss-usage of functionality●Boundary Check

Check also.....●Check Possiblity to automated testing?

How to create Test Case???

Regression Test Case??

Write a Case

Write a Case

Verify what?Using configuration?

With tools?

What is verdict?

●Verify drum track player pause mode functionality.●Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury”●Test should be done using android emulator environment and using your hands, ears and eyes”

Add Information about case

●Pre State:●Android emulator is running●Release X is installed on emulator

●Test Case Steps:●1. Open drum kit player application●2. Select song ”Show must go on”●3. Start to play●4. Press Pause and check song is paused●5. Check memory usage from system application●6. Press Play●7. jump to 4 several time (<10)●8. Listen song to the end●9. Exit player using ”exit button”

●End State:●IFDK Kit in main screen mode

●Test Case Id●Test Case owner/writer●Date●comments

●If Pause is working result is PASS. If Pause mode failed result is FAIL

Define pre-stateDefine Steps

Define end-state

What Information Test Case should contain?

Component /Unit TestingComponent /Unit Testing

ClassClass


ClassClass

MethodsMethods

ClassClass


ClassClass

MethodsMethods

ClassClass


ClassClass

MethodsMethods

ClassClass


TestClassTestClass

MethodsMethods

Unit Test Frame WorkUnit Test Frame WorkResultResult

Integration testing using emulatorIntegration testing using emulator

Test Case

SUT/DUTIFDK

What is generated as results from test case execution

LOG FILE

EVENTS

NOTIFICATIONS

SUT = System Under TestDUT = Device Under Test

ENVIRONMENTANDROID EMULATOR

TOOLSTOOLSTOOLSscripts/grep

TEST CASE

●TEST CASE ID XXXXX

●Step1●Step2●Step3.●Step4.

INCIDENT(Huomio)

Bug/Defect(Vika)

Defect Database

IFDKSystem

EXECUTE TEST !

Test Engineer

Notes

Reports

What means error reporting?

Component TestingComponent Testing



Sub SystemRequirementsSub System

Requirements

ComponentRequirementsComponent

RequirementsComponent / Unit

TestingComponent / Unit

Testing

IntegrationTestingIntegrationTesting

SystemTestingSystemTesting

AcceptanceTestingAcceptanceTesting

SystemRequirements

SystemRequirements


Implementation


Implementation

ProductProduct

VALIDATIONVALIDATION

VERIFICATIONVERIFICATION

How to verify component implementation

How to verify component implementation

-Unit Testing-Code Coverage-Branch Coverage-Complexity Analyse

-Unit Testing-Code Coverage-Branch Coverage-Complexity Analyse

Code ComplexityCode Complexity

Example tool CCCC

Branch coverageBranch coverage

The percentage of branches that have beenexercised by a test suite. 100% branch coverageimplies both 100% decision coverage and 100%

statement coverage.

Code CoverageCode Coverage

An analysis method that determines which parts ofthe software have been executed (covered) by thetest suite and which parts have not been executed,

e.g. statement coverage, decision coverage orcondition coverage.

http://en.wikipedia.org/wiki/Code_coverage

http://www.atlassian.com/software/clover/

http://en.wikipedia.org/wiki/Code_coverage

Release/Configuration Management &Integration Testing

Day 6Day 6

Putting all tools together!Putting all tools together!

Continous Integration

http://hudson-ci.org/

Design verification -Unit Testing

Design verification -Unit Testing

Unit Test example Re-run with MeegoUnit Test example Re-run with Meego

Example of xUnit in QT environment

Integration TestingIntegration Testing


Sub SystemRequirements

ComponentRequirements

Component / UnitTesting

IntegrationTesting

SystemTesting

AcceptanceTesting

SystemRequirements


Implementation

Product

VALIDATION

VERIFICATION

Why Integrate first? Avoid Big Bang!Why Integrate first? Avoid Big Bang!

HW ComponentHW Component

Data BaseData BaseComponent/Application 10% testedComponent/Application 10% tested

Web ServiceWeb Service

Tested Component/ApplicationTested Component/Application

Integration Test with stubsIntegration Test with stubs

Tested Component/ApplicationTested Component/Application

LogLog

STUB/MOCK ComponentSTUB/MOCK Component

Scripted STUB InterfaceScripted STUB Interface

ControlConfigureControlConfigure

SimulatedInterfaceSimulatedInterface

Messages/EventsMessages/Events


Control InterfaceControl Interface

in practice #1 IFDK android setupin practice #1 IFDK android setup

Tested ComponentApplication


Activate/ControlActivate/Control



ControlConfigureControlConfigure



WEB SERVER simulatingFacebook interfaceWEB SERVER simulatingFacebook interface


Trace/LogTrace/Log

in practice #2 server component testingin practice #2 server component testing



Trace/LogTrace/Log

Activate/ControlActivate/Control

Mock Server/DaemonMock Server/Daemon


AutomatedTest Interface

AutomatedTest Interface



WEB SERVERWEB SERVER


Operating SystemOperating System

NeededFake Application

NeededFake Application

Junit Scripted InterfaceJunit Scripted Interface

Release Management and Integration Testing

Release Management and Integration Testing

IFDK ApplicationIFDK Application

Facebook Web ServiceFacebook Web Service

HW ComponentCalore Meter

Enabled Drum Stick

HW ComponentCalore Meter

Enabled Drum Stick

Calore Meter SW Component 10% testedCalore Meter SW Component 10% tested

Data Base Schema DesignData Base Schema Design

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

Stubsneeded

First System TestWith all componentsFirst System TestWith all components

Trace/Log as feedbackTrace/Log as feedback

Log contains important information. Log

Simple tool for log analysing in Linux ”grep” command ,

TAIL -F /var/log/messages | grep error

-Specific Inhouse log capturing and analyse tools

AgendaAgenda

Brief guidance for NEST Project Platform 1.3

Background story for reference project IFDK

Some Definitions

SW Development Process

Release + Configuration Management

Testing Levels and Error Management

Hands On: Test Link + Bugzilla

Error Reporting, Metrics and daily usage

Closed Software Project vs Open Source Project

Discussion

Error/Bug/Defect ReportError/Bug/Defect Report

●Defect/Burg/Error ID●Reporter●Time●Founded where●Which way?●Test Case●Test Setup/Configuration●Describe scenario?●Attachements? Picture/Log/etc..

●Defect/Burg/Error ID●Reporter●Time●Founded where●Which way?●Test Case●Test Setup/Configuration●Describe scenario?●Attachements? Picture/Log/etc..

About Error ManagementAbout Error Management

RequirementManagementRequirementManagement

ImplementationProcess

ImplementationProcess

Verification&

Validation

Verification&

Validation

FailureReportFailureReport

DefinitionsDefinitions

Failure -

Fault, Defect, Bug -

Incident, Failure, Error

Example forum thread: http://www.allinterview.com/showanswers/36257.html

ISTQB syllabus

http://www.allinterview.com/showanswers/36257.html

Release & Configuration Management

Release & Configuration Management

Release ManagementRelease Management

Version 0.1

Version 0.1 Version 0.2Version 0.2 Version 0.3Version 0.3

Version 0.2.1Version 0.2.1

Version 0.2.2.1Version 0.2.2.1


Version 0.4Version 0.4TrunkTrunk

Customer 1Customer 1

Customer 1Customer 1Version 0.2.3Version 0.2.3



Release & Configuration Managmement

Version 0.1

Version 0.1 Version 0.2Version 0.2 Version 0.3Version 0.3




Version 0.4Version 0.4TrunkTrunk






FeaturesFeatures

Release 1.0Release 1.0



FeaturesFeatures

FeaturesFeatures

Release PlanningRelease Planning

Test ManagementTest Management

Validaton& Verificaton (Testing) Management

Version 0.4Version 0.4


Version 0.2.2

Version 0.2.2

Test PlanTest CasesFor

Features

Test PlanTest CasesFor

Features

Tested Release/configurationTested Release/configuration

Error/DefectReport

Error/DefectReport

Error/DefectReport

Error/DefectReport

Error/DefectReport

Error/DefectReport

Feature PackFeature Pack

Project ManagerProject Manager

Designer/CoderDesigner/Coder

IntegrationTest EngineerIntegrationTest Engineer

Test ManagerTest Manager

System TestingSystem Testing

IFDK System TestingIFDK System Testing

Feature Unit/Integration TestingFeature Unit/Integration Testing

IFDK System Acceptance TestingIFDK System Acceptance Testing

SystemTest EngineerSystemTest Engineer

Test AutomationEngineerTest AutomationEngineer

AcceptanceTest EngineerAcceptanceTest Engineer

ValidationValidation

VerificationVerification

IFDK Verification/Validation (Testing Organization)IFDK Verification/Validation (Testing Organization)

Regression Testing

Regression Testing

Integration Testing

Integration Testing

System Testing

System Testing

Acceptance Testing

Acceptance Testing

Test AutomationTest Automation

Unit Testing

Unit Testing

Product ReleaseProduct Release

Error and change managementError and change management

Example Sources for error reportExample Sources for error report

CRMCRM

Field TestingField Testing

Testing ProcessTesting Process

CustomerFeedback /Customer

Feedback /

Error ReportError Report ChangeRequest?Change

Request?

N x IncidentsN x Incidents

Change ManagementChange Management

Sometimes founded defect can lead to change

Bug?Bug?

Change Request?Change Request?Not ClearRequirementsNot Clear

Requirements

Feature ?Feature ?

Error Reporting and Process

Hands On: Bugzilla Error DatabaseHands On: Bugzilla Error Database

http://www.bugzilla.org/http://www.bugzilla.org/installation-list/

http://www.bugzilla.org/http://www.bugzilla.org/installation-list/

What is Bugzilla?

Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.

What is Bugzilla?

Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.

http://www.bugzilla.org/

http://www.bugzilla.org/

Other Defect Database Solutions

JIRA – Commercial

Requisite Pro – Commercial

Rational Synergy - Commercial

Mantis – Open Source

BugzillaBugzilla

Reporting, Metrics and daily usageReporting, Metrics and daily usage

CMMI Process framework

CMMI – covers ”error management” in several process areas

SP3.2 Analyze Verification results

Typical Work Products: Trouble reports

- Analyze the verificationd ata on defects

- Record all results of the analysis in a report

- Provide infromation on how defects can be resolved (including verfiocation methods, criteria, and verification environment) and initiate corrective action

Project Monitoring and Control

SG2 Manage Corrective Action to Closure

SP2.1 Analyze Issues

SP2.2 Take corrective Action

SP2.3 Manage corrective Action

SG2 Validate Product or Product Components

SP2.2 Analyze Validation Results. Change request management & Configuration Management process

SG2 Track and Control Changes

SP2.1 Track Change Requests

SP2.2 Control Configuration Items

Traditional SW Project vs Open Source Project

Traditional SW Project vs Open Source Project

Open Source – Crowd Sourcing

SW Relase tested without coordination by group of volunteers

Release tested by customer

Field Testing

Test Group

requirement management & domain modelling marko ”narsuman” rintamäki 2012

Documents

design requirements

ideal requirements

requirements definition

design process slide

software design

specification slide

user story slide

requirement manangement