practitioner flashcards fronts

8/14/2019 Practitioner Flashcards Fronts

1/15

A Technical Review (also known asa peer review), is considered to be aformal review type, even though noManagers are expected to attend. It

involves a structured encounter, inwhich a peer/s analyse the workwith a view to improve the quality ofthe original work.

Ideally led by the Moderator

Attended by peers /technical experts

Documentation is required

No Management presence

Decision making

Solving technical problems

A walkthrough is a set of proceduresand techniques designed for a peergroup, lead by the author to reviewsoftware code. It is considered to be

a fairly informal type of review. Thewalkthrough takes the form ameeting, normally between one andtwo hours in length.

Led by the Author

Attended by a peer group

Varying level of formality

Knowledge gathering

Defect finding

An inspection is a formal type ofreview. It requires preparation onthe part the review team membersbefore the inspection meeting takes

place. A follow-up stage is also arequirement of the inspection. Thisensures that any re-working iscarried out correctly.

Led by a Moderator

Attended by specified roles

Metrics are included

Formal process

Entry and Exit Criteria

Defect finding

An informal review is an extremelypopular choice early on in thedevelopment lifecycle of bothsoftware and documentation. The

review is commonly performed bypeer or someone with relevantexperience, and should be informaland brief.

Low cost

No formal process

No documentation required

Widely used review

Software Validation and Verification

can involve analysis, reviewing,

demonstrating or testing of all

software developments. This will

include the development process

and the development product itself.

Verification and Validation is

normally carried out at the end of

the development lifecycle (after all

software developing is complete).

But it can also be performed much

earlier on in the development

lifecycle by simply using reviews.

Validation involves the actual

testing. This should take place after

verificat ion phase has been

completed.

Validation: confirmation by

examination and provision of

objective evidence that the

particular requirements for a specific

intended use have been fulfilled.

Validation: Are we building the right

product?

Verification would normally involve

meetings and reviews and to

evaluate the documents, plans,

requirements and specifications.

This can be achieved by using

reviews and meetings etc.

Verification:confirmation byexamination and provision of

objective evidence that specified

requirements have been fulfilled.

Verification: Are we building the

product right?

The Waterfall model is also known

as the Sequential model. Each

stage follows on from the previous

one. The testing is performed in

block as the last stage.

Planning or Test creation is notconsidered until the actual softwarecode has been written. This canresult in problems being found muchlater in the project lifecycle than isdesirable.

Walkthrough ReviewTechnical Review Inspection Review Informal Review

V & V Validation Verification Waterfall Model


2/15

The V-Model is an industry standard

framework that shows clearly the

software development lifecycle in

relation to testing. It also highlightsthe fact that the testing is just as

important as the software

development itself. The relationships

between development and testing

are clearly defined.

The V-Model improves the presenceof the testing activities to display amore balanced approach.

The Spiral model is an incrementaltesting approach to bothDevelopment and Testing. This isused most effectively when the

users do not know all of therequirements. From what is known,initial requirements can be defined.Then from these the code and testcases are created. As time goes on,more details of the requirements areknown and implemented in furtheriterations of design, coding andtesting phases. The system isconsidered to be complete, whenenough of the iterations have takenplace.

RAD represents Rapid ApplicationDevelopment, and is a softwaredevelopment process that wasdeveloped in the mid 1980s. It was

developed to overcome the rigidityof such processes as The WaterfallModel.

Elements:

Prototyping

Iterative Development

Time-boxing

Team Members

Management Approach

RAD Tools

DSDM (Dynamic Systems

Development Methodology) is

basically a high level framework of

already proven RAD techniques, andalso management controls that are

used to increase the chances of

successful RAD projects.

The high level framework allows for

a process that can be easi ly

modified for individual projects

specific needs. But, this quite simple

framework also results in poor

implementation due to lack of detail.

As a Tester, your focus will be fixed

on the test process. But we must

consider other processes that exist,

and also their interaction with the

test process.

Project Management

Change Management

Configuration Management

Software Development

Technical Writing

Technical Support

Component testing is also known as

Unit, Module, or Program Testing. In

simple terms, this type of testing

focuses simply on testing of the

individual components themselves.

It is common for component testing

to be carried out by the Developer of

the software. This however has a

very low rating of testing

independence.

This type of Integration testing isconcerned with ensuring theinteractions between the softwarecomponents at the module levelbehave as expected. ComponentIntegration Testing is also often

referred to as Integration Testing inthe Small.

It is commonly performed after anyComponent Testing has completed,and the behaviour tested may coverboth functional and non-functionalaspects of the integrated system.

Requirements-based Testing issimply testing the functionality of thesoftware/system based on therequirements. The tests themselvesshould be derived from thedocumented requirements and not

based on the software code itself.This method of functional testingensures that the users will be gettingwhat they want, as the requirementsdocument basically specifies whatthe user has asked for.

Spiral ModelV - Model RAD DDSM

Process Interfaces Component TestingComponent Integration

TestingRequirements BasedFunctional Testing


3/15

Different types of users may use thedeveloped software in different

ways. These ways are analysed andbusiness scenarios are thencreated. User profiles are often usedin Business Process FunctionalTesting. Remember that all of thefunctionality should be tested for,not just the most commonly usedareas.

Testing the ability of the system tobe able to bear loads. An examplewould be testing that a system could

process a specified amount oftransactions within a specified timeperiod. So you are effectivelyloading the system up to a highlevel, then ensuring it can stillfunction correctly whilst under thisheavy load.

A program/system may haverequirements to meet certain levelsof performance. For a program, this

could be the speed of which it canprocess a given task. For anetworking device, it could mean thethroughput of network traffic rate.Often, Performance Testing isdesigned to be negative, i.e. provethat the system does not meet itsrequired level of performance.

Stress Testing simply means puttingthe system under stress. The testingis not normally carried out over a

long period, as this would effectivelybe a form of duration testing.Imagine a system was designed toprocess a maximum of 1000transactions in an hour. A stress testwould be seeing if the systems couldactually cope with that manytransactions in a given time period. Auseful test in this case would be tosee how the system copes whenasked to process more than 1000.

A major requirement in todayssoftware/systems is security,particularly with the internetrevolution. Security testing isfocused at finding loopholes in theprograms security checks. Acommon approach is to create test

cases based on known problemsfrom a similar program, and testthese against the program undertest.

This is where consideration is takeninto account of how the user will usethe product. It is common forconsiderable resources to be spenton defining exactly what thecustomer requires and simple it is touse the program to achieve there

aims. For example; test cases couldbe created based on the GraphicalUser Interface, to see how easy itwould be to use in relation to atypical customer scenario.

This type of testing may focus onthe actual memory used by aprogram or system under certainconditions. Also disk space used bythe program/system could also be afactor. These factors may actuallycome from a requirement, and

should be approached from anegative testing point of view.

Volume Testing is a form of SystemsTesting. It primary focus is toconcentrate on testing the systemswhile subjected to heavy volumes ofdata. Testing should be approachedfrom a negative point of view toshow that the program/system

cannot operate correctly when usingthe volume of data specified in therequirements.

Load TestingBusiness ProcessFunctional Testing

Performance Testing Stress Testing

Security Testing Useability Testing Storage Testing Volume Testing


4/15

A complicated program may alsohave a complicated installationprocess. Consideration should bemade as to whether the program willbe installed by a customer or aninstallation engineer. Customerinstallations commonly use somekind of automated installationprogram. This would obviously haveto under go significant testing initself, as an incorrect installationprocedure could render the targetmachine/system useless.

Documentation in todaysenvironment can take several forms,as the documentation could be aprinted document, an integral help

file or even a web page. Dependingof the documentation media type,some example areas to focus oncould be, spelling, usability,technical accuracy etc.

Recovery Testing is normally carriedout by using test cases based onspecific requirements. A system maybe designed to fail under a given

scenario, for example if attacked bya malicious user; theprogram/system may have beendesigned to shut down. Recoverytesting should focus on how thesystem handles the failure and howit handles the recovery process.

This type of Integration Testing isconcerned with ensuring theinteractions between systemsbehave as expected. It is commonly

performed after any Systems Testinghas completed. Typically not allsystems referenced in the testing arecontrolled by the developingorganization. Some systems maybecontrolled by other organizations, butinterface directly with the systemunder test.

User Acceptance Testing or UAT iscommonly the last testing performedon the software product before itsactual release. It is common for thecustomer to perform this type oftesting, or at least be partiallyinvolved. Often, the test ingenvironment used to perform UserAcceptance Testing is based on a

model of the customer senvironment. This is done to try andsimulate as closely as possible theway in which the software productwil l actually be used by thecustomer.

This type of Acceptance Testing isaimed at ensuring the acceptancecriteria within the original contracthave indeed been met by thedeveloped software. Normally anyacceptance criteria is defined whenthe contract is agreed. RegulationAcceptance Testing is performed

when there exists specificregulations that must be adhered to,for example, there may be safetyregulations, or legal regulations.

This form of acceptance testing iscommonly performed by a SystemAdministrator and would normally beconcerned with ensuring thatfunctionality such as;backup/restore, maintenance, andsecurity functionality is present andbehaves as expected.

Alpha Testing should be performedat the developers si te, andpredominantly performed by internaltesters only. Often, other companydepartment personnel can act astesters. The marketing or salesdepartments are often chosen forthis purpose.

Documentation TestingInstallability Testing Recovery TestingSystem Integration

Testing

UAT Contract & RegulationAcceptance Testing

Operational AcceptanceTesting

Alpha Testing


5/15

Beta Testing is commonly performedat the customers site, and normallycarried out by the customersthemselves. Potential customers areoften eager to trial a new product ornew software version. This allowsthe customer to see anyimprovements at first hand andascertain whether or not it satisfiestheir requirements. On the flip side,it gives invaluable feedback to thedeveloper, often at little or no cost.

It is imperative that when a fault isfixed it is re-tested to ensure thefault has indeed been correctlyfixed.

Re-test:Whenever a fault is detected andfixed then the software should bere-tested to ensure that the originalfault has been successful lyremoved.

When checking a fixed fault, youcan also consider checking thatother existing functionality has notbeen adversely affected by the fix.

This is called Regression Testing.

Regression Test:Regression testing attempts toverify that modifications have notcaused unintended adverse sideeffects in the unchanged software(regression faults) and that themodified system still meets itsrequirements.

A Test Plan should be a singledocument that basically containswhat is going to be tested, why it isgoing to be tested, and how it is

going to be tested. I t is alsoimportant to clarify what is not goingto be tested in the software producttoo. With regards to using a standardTest Plan layout, then we can look tothe advice given by theIEEE(Institute of Electrical andElectronic Engineers) located in theInternational Standard IEEE Std929-1998.

A standard test process that iscommonly used exists within theBS7925-2 Standard for SoftwareComponent Testing:

Test Planning

Test Specification

Test Execution

Test Checking & Recording

Checking for TestCompletion

This should apply to both new

projects and maintenance work.

Normally fairly short in length, the

test policy should be a high-level

document, and should contain the

following items:

Definition of testing

The testing process

Evaluation of testing

Quality levels

Improvement approach

Based on the test policy, the test

strategy is designed to give an

overview of the test requirements for

a programme or even organization.

Information relating to risks should

be documented here, specifically

the risks that will be addressed by

the testing, and the specific tests

that will be used against each risk.

Exactly how the test strategy for a

particular project will be

implemented is displayed in the

project plan. The project test plan will

normally be referenced from the

overall project plan. In relation to the

test strategy, the project plan shoulddetail items from the test strategy

that it is complying with, and also

items it is not complying with.

Re-TestBeta Testing Regression TestingTest PlanDocument

Generic TestProcess

Test Policy Test Strategy Project Plan


6/15

The specific details of the approach

taken by the testing for a specific

test phase is shown in this

document. It can be thought of asbeing based on the project plan, but

with greater amounts of detail

included, such as testing activities

based on day to day plan, or

expected amounts of man hours to

complete individual tasks.

Risk management comprises of thefollowing three components:

Risk Identification

Risk Analysis

Risk Mitigation

Risk management should be aconsideration for everyone involvedin the project.

The following techniques can all beused to identify risks associated withproducts and projects. The list is byno means rigid, as many

organisations will have there owntechniques.

Expert Interviews

Independent Assessment

Risk Templates

Lessons Learned

Risk Workshops

Brainstorming andChecklists

So what does the term Risk

Analysis actually mean? It is simply

studying the identified risks. A

simple formula can be used tocalculate risk:

Frequency (likelihood) X Severity

(impact)

By using the above formula we can

produce a figure, otherwise known

as the exposure.

Risk mitigation is simply the

response to the analysed risks. A

choice must be made as what action

should be carried out once a risk

has been identified. Some possible

choices could be:

Do nothing

Take preventative action

(test it)

Contingency plan (what we

should do if the predicted

fault actually occurs)

What this method allows you to dois effectively partition the possibleprogram inputs. For each of theabove input fields, it should notmatter which values are entered aslong as they are within the correctrange and of the correct type.

So the point of equivalenceportioning is to reduce the amountof testing by choosing a smallselection of the possible values tobe tested, as the program willhandle them in the same way.

By the use of equivalencepartitioning, a tester can performeffective testing without testingevery possible value. This methodcan be enhanced further by anothermethod called Boundary ValueAnalysis. After time, an experiencedTester will be often realise that

problems can occur at theboundaries of the input and outputspaces. When testing only a smallamount of possible values, theminimum and maximum possiblevalues should be amongst the firstitems to be tested.

The classification tree method is alsoknown as a decision tree methodand the terms can be usedinterchangeably as they mean thesame thing. A decision tree can belearned by splitting the source setinto subsets based on an attributevalue test. This process is repeated

on each subset in a recursively. Therecursion is completed when splittingis either not possible, or a singleclassification can be applied to eachelement of the subset.

Risk ManagementPhase Test Plan Risk Identification Risk Analysis

Risk Mitigation EquivalencePartitioning

Boundary ValueAnalysis

Classification TreeMethod


7/15

This type of Black-box testing isbased on the concept of states andfinite-states, and is based on thetester being able to view thesoftwares states, transition betweenstates, and what will trigger a statechange. Test cases can then bedesigned to execute the statechanges.

This testing method involves usinga model of the source code whichidentifies statements. Thesestatements are the categorized as

being either executable or non-executable. In order to use thismethod, the input to eachcomponent must be identified. Also,each test case must be able toidentify each individual statement.Lastly, the expected outcome ofeach test case must be clearlydefined

This test method uses a model ofthe source code which identifiesindividual decisions, and theiroutcomes. A decision is defined as

being an executable statementcontaining its own logic.

This logic may also have thecapability to transfer control toanother statement. Each test case isdesigned to exercise the decisionoutcomes. In order to use thismethod, the input to eachcomponent must be identified.

Branch Condition Testing uses a

model of the source code, and

identifies decisions based on

individual Boolean operands withineach decision condition. A decision

is defined as being an executable

statement containing its own logic.

An example of a decision would be a

loop in a program.

Branch Condition Combination

Testing uses a model of the source

code, and identifies decisions based

on combinations of Boolean

operands within decision conditions.

This logic may also have the

capability to transfer control to

another statement. The decision

condition is a Boolean expressionwhich is evaluated to determine the

outcome of the decision.

Requirements-based Testing issimply testing the functionality of thesoftware/system based on therequirements. The tests themselvesshould be derived from thedocumented requirements and notbased on the software code itself.This method of functional testingensures that the users will begetting what they want, as therequirements document basicallyspecifies what the user has askedfor.

This is where consideration is takeninto account of how the user will usethe product. It is common forconsiderable resources to be spenton defining exactly what thecustomer requires and how simple itis to use the program to achievethere aims. For example; test casescould be created based on the

Graphical User Interface, to seehow easy it would be to use inrelation to a typical customerscenario.

Volume Testing is a form of SystemsTesting. It primary focus is toconcentrate on testing the systemswhile subjected to heavy volumes ofdata. Testing should be approachedfrom a negative point of view toshow that the program/systemcannot operate correctly when usingthe volume of data specified in therequirements.

Statement TestingState Transition

TestingBranch Decision

TestingBranch Condition

Testing

Branch ConditionCombination Testing Requirements BasedFunctional TestingUseability Testing Volume Testing


8/15

A program/system may haverequirements to meet certain levelsof performance. For a program, thiscould be the speed of which it canprocess a given task. For anetworking device, it could mean thethroughput of network traffic rate.Often, Performance Testing isdesigned to be negative, i.e. provethat the system does not meet itsrequired level of performance.

Stress Testing simply means puttingthe system under stress. The testingis not normally carried out over along period, as this would effectively

be a form of duration testing.Imagine a system was designed toprocess a maximum of 1000transactions in an hour. A stress testwould be seeing if the systemscould actually cope with that manytransactions in a given time period.A useful test in this case would be tosee how the system copes whenasked to process more than 1000.

Dynamic analysis is a testingmethod that can provide informationon the state of software. It canachieve this dynamically i.e. it

provides information when thesoftware is actually running. It iscommonly used to exercise parts ofthe program that use memoryresources e.g.:

Memory allocation

Memory usage

Memory de-allocation

Memory leaks

Unassigned pointers

Static Analysis is a set of methodsdesigned to analyse software codein an effort to establish it is correct,prior to actually running the software.

As we already know, the earlier wefind a fault the cheaper it is to fix. Soby using Static Analysis, we caneffectively test the program evenbefore it has been written. Thiswould obviously only find a limitednumber of problems, but at least it issomething that can be done veryearly on in the development lifecycle.

Control flow graphs display the logicstructure of software. The flow oflogic through the program ischarted. It is normally used only byDevelopers as it is a very low levelform testing, often used inComponent Testing.

It can be used to determine thenumber of test cases required totest the programs logic. It can alsoprovide confidence that the detail ofthe logic in the code has beenchecked.

Cyclomatic Complexity is a softwaremetric that is used to measure thecomplexity of a software program.Once we know now how complexthe program is, we then know howeasy it will be to test.

C = E N + P

C = Cyclomatic Complexity

E = number of edges

N = number of nodes

P = number of components

The most basic form of a complexitymetric is the Lines of Code metric,or LOC metric. Its purpose likeother complexity metrics is toestimate the amount of effort thatwill be required not only to developsuch a program, but also assist inestimating how much effort will be

required to test it.

In its simplest form we could use theLOC metric by literally counting thenumber of lines of code in theprogram.

The idea behind Data-flow Analysisis to work-out the dependenciesbetween items of data that are usedby a program. When a program isran, it rarely runs in a sequentialorder i.e. starting at line 1 andfinishing at line 100. What usuallyhappens is that the dependencies ofthe data within the program will

determine the order. Data-flowAnalysis can be used to f inddefinitions that have no interveninguse. Data-flow analysis is also usedto detect variables that are usedafter it has effectively been killed.

Stress TestingPerformance Testing Dynamic Analysis Static Analysis

Control FlowGraphing

Cyclomatic Complexity Lines of Code Data Flow Analysis


9/15

Why can one Tester find more errorsthan another Tester in the samepiece of software? More often thannot this is down to a techniquecalled Error Guessing. To besuccessful at Error Guessing, acertain level of knowledge andexperience is required. A Tester canthen make an educated guess atwhere potential problems may arise.This could be based on the Testersexperience with a previous iterationof the software, or just a level ofknowledge in that area of technology.

This type of testing is normallygoverned by time. It consists ofusing tests based on a test chapterthat contains test objectives. It is

most effective when there are littleor no specifications available. Itshould only really be used to assistwith, or compliment a more formalapproach. It can basically ensurethat major functionality is working asexpected without fully testing it.

This type of testing is considered tobe the most informal, and by many itis considered to be the leasteffective. Ad-hoc testing is simply

making up the tests as you goalong. Often, it is used when there isonly a very small amount of time totest something. A common mistaketo make with Ad-hoc testing is notdocumenting the tests performedand the test results. Even if thisinformation is included, more oftenthan not additional information is notlogged such as, software versions,dates, test environment details etc.

A Tester normally selects test inputdata from what is termed an inputdomain. Random Testing is simplywhen the Tester selects data from

the input domain randomly. As youcan tell, there is little structureinvolved in Random Testing. Inorder to avoid dealing with the abovequestions, a more structured Black-box Test Design could beimplemented instead. However,using a random approach could savevaluable time and resources if usedin the right circumstances.

A Quality Assurance (QA) standardsimply specifies that testing shouldbe performed.

Example: ISO 9000

An industry specific standard willdetail exactly what level of testing isto be performed.

Examples:

Railway Signalling standard

DO-178B

Nuclear Industry standard

MISRA guidelines for motorvehicle software

Pharmaceutical standards

Testing standards will detail how toperform the testing. Ideally, atest ing standard should bereferenced from a QA or Industryspecific standard.

Example: BS7925-1, BS7925-2

Review: A process or meeting duringwhich a work product, or set of workproducts, is presented to projectpersonnel, managers, users or otherinterested parties for comment orapproval. [IEEE]

A review should be performed whenall of the supporting documentationis available. This can include designdocuments, requi rementsdocuments, standards documents,basically any documentation that haseither been inf luential or isapplicable to the document to bereviewed.

Exploratory TestingError Guessing Ad-hoc Testing Random Testing

Quality AssuranceStandards

Industry SpecificStandards

Testing Standards Review Definition


10/15

Organisations will commonly havedifferent named roles than thoselisted below, but this will give you anidea of a commonly used set of

roles used throughout the world.

Manager

Moderator

Author

Reviewer

Scribe

An example of a typical reviewprocess is below. This is probablythe most documented reviewprocess you will find in the software

development world, and is open tointerpretation:

Planning

Kick-off

Preparation

Meeting

Rework

Follow-up

Exit Criteria

We term an incident; any significant,unplanned event that occurs duringtesting that requires subsequentinvestigation and/or correction. The

incident should be raised when theactual result differs from theexpected result. After the inevitableinvestigation of the incident, theremay be a reason other than asoftware fault, for example:

Test environment incorrectlyset up

Incorrect Test Data used

Incorrect Test Specification

This standard aims to provide astandard approach to classificationof anomalies found in software. Itincludes descript ions of the

processes involved in a software lifecycle, including details on howanomalies should be recorded andsubsequently processed. It consistsof four sequential steps;Recognition, Investigation, Action,Disposition. Each of those steps hasthree administrative activities whichare; Recording, Classifying,Identifying Impact.

A maturity model is basically a

collection of elements that are

structured in such a way that they

can describe characteristics of

processes and their effectiveness. A

maturity model can provide:

A starting point A shared vision

A structure for organising

actions

Use of previous experience

value of improvements

The Capability Maturity Model,

simply put, is a baseline of practices

that should be implemented in order

to develop or maintain a product.

The product can be completely

software, or just partially software.

The SW-CMM focuses on thesoftware practices whereas with the

CMMI, you may find both software

and systems practices.

The CMM defines five maturitylevels which form the top-levelstructure of the CMM itself. Eachlevel is basically a foundation thatcan be built upon to improve theprocess in sequence. Starting withbasic management practices andprogressing through successiveproven levels.

Initial

Managed

Defined

Quantitatively Managed

Optimising

The software process capabilitydefines what can be achieved byundertaking a specific softwareprocess. It achieves this bydescribing the range of expectedresults. There are six capabilitylevels.

Incomplete

Performed

Managed

Defined

Quantitatively Managed

Optimising

Review ProcessStructure

Review Roles Incident Management IEEE Std. 1044-1993

Maturity ModelDefinition

SEI Capability MaturityModel (CMMI)

CMM Maturity Levels CMM Capability Levels


11/15

The ISO/IEC 15504 is also knownas SPICE. Spice stands forSoftware Process Improvement andCapability dEtermination. It isessentially a framework forassessing software processes.Rather than concerning itself withspecific standards, ISO/ISEC 15504concerns itself with is thecapabili ties provided by anorganisations structure. Thesestructures include its managementstructure and its process definitionstructure.

The Illinois Institute of Technology(IIT) developed the Testing MaturityModel (TMM) in 1996. The mainreason for developing the TMM was

that existing maturity models didntproperly address real testing issues.It was designed to complement theexisting CMM. The main purpose ofthe TMM is to support assessmentand improvement drives within anorganisation. The model comprisesof a Maturity Model and anAssessment Model.

The maturity levels are basicallydefined levels of maturity that canbe achieved by showing thatspecific practices have been carried

out. The TMM has five differentachievable levels:

Initial

Phase Definition

Integration

Management/Measurement

Optimisation/defectPrevention and QualityControl

Developed by Koomen and Pol in1997, the Test Process ImprovementModel or TPI was created with thegoal of simplifying the sometimesover-complicated testing process.

The TPI model itself identifies thegood and bad parts of a testingprocess. The maturity of the processcan also be assessed by using theTPI. The TPI consists of thefollowing four components:

A Maturity Model

A Test Maturity Matrix

A Checklist

Improvement Suggestions

The TPI model consists of threematurity levels and fourteen scales.The individual levels contain severaldifferent scales. The scalesthemselves provide indication ofwhich key areas requireimprovement.

Scales 1 to 5 focus on bringthe testing process undercontrol

Scales 6 to 10 focus onestablishing test processefficiency

Scales 11 to 14 focus ontest process optimisation

The TPI takes into account the

different aspects of a test process,

including design techniques, test

tool usage and reporting. Structured

evaluation of various key areas,

highlights the test processes

strengths and weaknesses. The

state of a key area is determined byassigning a level to it, commonly A

to B to C etc. The levels are

increased based on time, cost and

quality.

This type of tool is designed toassist with verification and validationof requirements, for example;consistency checking.

By examining the code instead ofrunning test cases through the code,this type of tool can provideinformation on the actual quality ofthe software. Cyclomatic complexityis one such characteristic that canbe obtained by using this type oftool.

TMM DefinitionISO/IEC 15504 (SPICE)

DefinitionTMM Maturity Levels TPI Definition

TPI Model TPITest Maturity Matrix

RequirementsTesting Tools

Static AnalysisTools


12/15

This type of tool can generate testcases from specifications, which arenormally stored in a CASE toolrepository. Some variations of this

type of tool can also generate testcases from analysing the code itself.

Data can be selected from existingtest specific databases by using thistype of tool. Advanced types of thistool can utilise a range of database

and file formats.

These are an extremely populartype of tool. They provide captureand replay facilities for WIMPinterface based applications. The

tools can simulate mousemovement, mouse clicks andkeyboard inputs. The tools can evenrecognize windows and buttons,thus making them extremelyversatile. The test procedures arenormally written in a specificscripting language. This tool isanother popular choice forregression testing.

If the software under test does not

have a user interface, then test

harnesses and drivers can be used

to execute the software. These typesof tools can be bought off the shelf,

but more commonly they are built for

a specific purpose.

Creates actual test scripts based oninformation held within a testspecification. Simulators arecommonly used where it isimpracticable to use them, forexample software to control a spaceprobes trajectory.

This type of tool comprises of two

components; Load Generation and

Test Transaction Measurement.,

Load Generation is commonly

performed by running the

application using its interface or by

using drivers. The number of

transactions performed this way arethen logged. Performance test tools

will commonly be able to display

reports and graphs of load against

response time.

Run-time information on the state of

the executing software is achieved

by using Dynamic Analysis Tools.

These tools are ideally suited for

monitoring the use and allocation of

memory. Faults such as memory

leaks, unassigned pointers can be

found, which would otherwise be

difficult to find manually.

Debugging tools are often used by

programmers to try and reproduce

code related errors in order to

investigate a problem. The debugger

allows the program to be run line by

line. This enables halted the program

on demand to examine and set

program variables.

Test Input DataPreparation Tool

Test Design Tools Test Running Tools Test Harnesses

Test Script GeneratorsPerformance Test

Tools

Dynamic AnalysisTools

Debugging Tools


13/15

This type of tool is used to highlightdifferences between actual resultsand expected results. Off the shelfComparison Tools can normally dealwith a range of file and database

formats. This type of tool often hasfilter capabilities to allow ignoring ofrows or columns of data or evenareas on a screen

Test Management Tools commonly

have multiple features. Test

Management is mainly concerned

with the management, creation and

control of test documentation. Moreadvanced tools have additional

capabilities such as test

management features, for example;

result logging and test scheduling.

This type of tool provides objective

measures of structural test coverage

when the actual tests are executed.

Before the programs are compiled,

they are first instrumented. Oncethis has been completed they can

then be tested. The instrumentation

process allows the coverage data to

be logged whilst the program is

running. Once testing is complete,

the logs can provide statistics on the

details of the tests covered.

These tools are simply used tocheck that no broken hyperlinks existon a web site.

These tools are typically used fortesting e-commerce and e-businessapplications. The main purpose ofthis tool is to check web sites toensure that they are available tocustomers and also to producewarnings if problems are detected.

These tools are commonly used fortesting e-commerce and e-businessapplications, and sometimes websites. A security testing tool willcheck for any parts of a web basedsystem that could cause potentialsecurity risks if attacked.

A Test Oracle is used toautomatically generate expectedresults. They are commonly used insituations where an old system isupgraded with a new system withthe same functionality, so the oldsystem can be used as an Oracle.

A suggested tool selection andevaluation process is:

Determine the actual problem orrequirement

Ensure that there are no obviousalternative solutions

Prepare a business case

Identify any constraints

Identify any specific required toolfeatures or characteristics

Prepare a short-list of possiblesuitable tools

Perform a detailed evaluation

Perform a competitive trial, ifneeded

Test ManagementTools

Comparison ToolsCoverage Measurement

ToolsHyperlink Testing

Tools

Monitoring Tools Security TestingTools Test Oracles Tool SelectionProcess


14/15

The last thing we want is tointroduce a tool into theorganisation, only to find a fewweeks down the line it fails resultingin potentially disastrous scenarios.

In order to avoid this situation, wecan implement a pilot project. Thebenefits of using a pilot project are;

Gaining experience usingthe tool

Identify any test processchanges

Identify any shortcomingssuitability of the tool

An implementation team can beformed to evaluate a new toolconsisting of:

A Champion A Change Agent

A Tool Custodian

A User is someone who has hadexperience actually using thesoftware under test, or similar typesof software. This knowledge can be

useful to determine the type of faultsthat a typical user may comeacross, which to most developmentswould also have the most impact.The User would probably not havesufficient knowledge to test thesoftware to extreme depths though.

A developers background may haveprovided them with experience withcode, design or requirementsanalysis. This knowledge can be

extremely useful, as the developerwould probably have some ideawhen looking at the software of howit was developed, and so wouldprobably know where to look forweaknesses.

Previous knowledge of testing hasits obvious advantages. They shouldbe able to analyse a specification,design test cases, execute testcases, and produce results andreports. An individual with previoustesting experience would also havethe right mindset for testing, as theywould already know the reasoning

behind why testing is performed.

Shapers: Challenging, dynamic,thrives on pressure. The drive andcourage to overcome obstacles.Prone to provocation. Offendspeople's feelingsImplementer: Disciplined, reliable,conservative and eff icient.Somewhat inflexible. Slow torespond to new possibilities

Completer Finisher: Painstaking,conscientious, anxious. Searchesout errors and omissions. Deliverson time. Inclined to worry unduly.Reluctant to delegate.

Coordinator: Mature, confident, agood chairperson. Clarifies goals,promotes decision-making,delegates well. Can often be seenas manipulative.Team Worker: Co-operative, mild,perceptive and diplomatic. Listens,builds, averts friction. Indecisive incrunch situations.Resource Investigator: Extrovert,enthusiastic, communicative.Explores opportunities. Developscontacts. Over - optimistic. Losesinterest after short period.

Plant: Creative, imaginative,unorthodox. Solves diff icultproblems. Ignores incidentals. Toopre-occupied to communicateeffectively.Monitor Evaluator: Sober,strategic and discerning. Sees alloptions. Judges accurately. Lacksdrive and ability to inspire others.Specialist: Single-minded, self-

starting, dedicated. Providesknowledge and skills in rare supply.Contributes only on a narrow front.Dwells on technicalities.

Test ToolImplementation Team

Pilot Projects User Skills Developer Skills

Tester Skills Belbins ActionOriented Roles

Belbins PeopleOriented Roles

Belbins ThoughtOriented Roles


15/15

The Test Leader will commonlycome from a testing backgroundand have a full understanding ofhow testing is performed. They willalso possess good managerial

expertise. They are also responsiblefor ensuring that test coverage issufficient and will be required toproduce reports.

The Tester obviously provides theskills necessary to perform theTesting itself. This role can includetest design and test execution.

Automated testing skills are also apossible requirement of this role.

Preparation of test data

Execute tests

Review other peoples tests

Review Test Plan

Involvement in automationof tests

Create Test Specifications

The client is effectively the projectsponsor, and will provide the budgetfor the project. The Client can alsobe the business owner.

Management skills are provided bythe Project Manager. The ProjectManager will be actively involvedthroughout the project and will

provide feedback to the client.

A Developer will provide the skills towrite the actual software code andperform Unit Testing. They may alsobe called upon at a later stage toprovide bug fixes and technicaladvice.

The Business Analyst will provideknowledge of the business andanalysis skills. The Business Analystwill also be responsible for creatingUser Requirements based on talkswith the Users.

Systems design will be provided bythe Systems Analyst. The SystemsAnalyst will also be responsible fordeveloping the FunctionalSpecif icat ion from the UserRequirements.

Technical detail and support to thesystem design is the responsibility ofthe Technical Designer. This rolemay include databaseadministration.

The TesterThe Test Leader The Client The Project Manager

The Developer The Business Analyst The Systems Analyst The Technical Designer

practitioner flashcards fronts

Documents