7/27/2019 3testing

1/43

Usability Testing

In Usability Testing, a small-set of target end-users, of a software system, use it to expose usability defects. This

testing mainly focuses on the users-ease to use the application, flexibility in handling controls and ability of the

system to meet its objectives.This testing is recommended during the initial design phase of SDLC, which gives

more visibility on the expectations of the users

Need for Usability Testing

Aesthetics and design are important. How well a product looks usually determines how well it works.There are

many software applications / websites, which miserably fail, once launched, due to following reasons

7/27/2019 3testing

2/43

Where do I click next?

Which page needs to be navigated?

Which Icon or Jargon represents what?

Error messages are not consistent or effectively displayed

Session time not sufficient.

Usability Testing identifies usability errors in the system early in development cycle and can save a

product from failure.

Goals of Usability Testing: Goal of this testing is to satisfy users and it mainly concentrates on the following

parameters of a system:

Effectiveness of the system

Is the system is easy to learn?

Is the system useful and adds value to the target audience?

Is Content, Color, Icons, Images used are aesthetically pleasing ?

Efficiency

Navigation required to reach desired screen/webpage should be very less. Scroll bars shouldnt be used

frequently.

Uniformity in the format of screen/pages in your application/website.

Provision to search within your software application or website

Accuracy

No outdated or incorrect data like contact information/address should be present.

No broken links should be present.

User Friendliness

Controls used should be self-explanatory and must not require training to operate

Help should be provided for the users to understand the application / website

Alignment with above goals helps in effective usability testing

Usability Testing Process

Usability testing process consists of the following phases

Planning:- During this phase the goals of usability test are determined. Having volunteers sit in front of

your application and recording their actions is not a goal. You need to determine critical functionalities and

objectives of system. You need to assign tasks to your testers, which exercise these critical functionalities. During

this phase , usability testing method, number & demographics of usability testers , test report formats are also

determined

Recruiting: During this phase, you recruit the desired number of testers as per your usability test plan.

Finding testers who match your demographic (age , sex etc ) and professional ( education , job etc .) profile can

take time.

UsabilityTesting: During this phase, usability tests are actually executed.

7/27/2019 3testing

3/43

Data Analysis: Data from usability tests is thoroughly analyzed to derive meaningful inferences and give

actionable recommendations to improve overall usability of your product.

Reporting: Findings of the usability test is shared with all concerned stakeholders which can include

designer, developer, client, and CEO

Methods of Usability Testing

There are two methods available to do usability testing

1. Laboratory Usability Testing

2. Remote Usability Testing

Laboratory Usability Testing:. This testing is conducted in a separate lab room in presence of the

observers. The testers are assigned tasks to execute. The role of the observer is to monitor behavior of the testers

and report outcome of testing. The observer remains silent during the course of testing. In this testing both

observers and testers are present in same physical location.

Remote Usability Testing : Under this testing observers and testers are remotely located. Testers

access the System Under Test, remotely and perform assigned tasks. Testers voice , screen activity , testers

facial expressions are recorded by an automated software. Observers analyze this data and report findings of the

test. Example of such a software - http://silverbackapp.com/

How many users do you need ?

Research ( Virzi, 1992 and Neilsen & Landauer , 1993) indicates that 5 users are enough to uncover

80% of usability problems. Some researchers suggest other numbers.

The truth is , the actual number of user required depends on the complexity of the given application and your

usability goals. Increase in usability participants results into increased cost , planning , participant managementand data analysis.

But as a general guideline , if you on a small budget and interested in DIY usability testing 5 is a good number to

start with. If budget is not a constraint its best consult experienced professionals to determine number of users.

http://www.google.com/url?q=http%3A%2F%2Fsilverbackapp.com%2F&sa=D&sntz=1&usg=AFQjCNFoWbhl-XjU1mwU5q0zXlc-kX4Ywghttp://www.google.com/url?q=http%3A%2F%2Fsilverbackapp.com%2F&sa=D&sntz=1&usg=AFQjCNFoWbhl-XjU1mwU5q0zXlc-kX4Ywg

7/27/2019 3testing

4/43

Bestpractices- Usability testing

The primary goal of this testing is to find crucial usability problems before the product is launched . Following

things have to be considered to make testing success:

Start the testing during the early stage of design and development

Its a good practice to conduct usability testing on your competitors product before you begin

development. This will help you determine usability standards for your target audience

Select the appropriate users to test the system(Can be experts/non-experts users/50-50 of Experts and

Non-Experts users)

Use a bandwidth shaper . For instance , your target audience has poor network connectivity , limit network

bandwidth to say 56 Kbps for your usability testers.

Testers need to concentrate on critical & frequently used functionalities of the system.

Assign a single observer to each tester. This helps observer to accurately note testers behavior. If anobserver is assigned to multiple testers, results may be compromised

Educate Designers and Developers that this testing outcomes is not a sign of failure but its a sign of

Improvement

Pros and Cons of Usability testing:

As with anything in life, usability testing has its merits and de-merits. Lets look at them

Pros:

It helps uncover usability issues before the product is marketed.

It helps improve end user satisfaction

It makes your system highly effective and efficient

It helps gather true feedback from your target audience who actually use your system during usability test.

You do not need to rely on opinions from random people.

Cons:

Cost is a major consideration in usability testing. It takes lots of resources to set up a Usability Test Lab.

Recruiting and management of usability testers can also be expensive .However, these costs pay

themselves up in form of higher customer satisfaction, retention and repeat business. Usability testing is

therefore highly recommended.

Functional Testing - Stuff You Must Know!

unctional testing verifies that each function of the software application operates in conformance with the

requirement specification. This testing mainly involves black box testing and it is not concerned about the source

code of the application.Each and every functionality of the system is tested by providing appropriate input, verifying

the output and comparing the actual results with the expected results. This testing involves checking of User

Interface, APIs, Database, security, client/ server applications and functionality of the Application Under Test. The

testing can be done either manually or using automation

7/27/2019 3testing

5/43

What do you test in Functional Testing?

The prime objective of Functional testing is checking the functionalities of the software system. It mainly

concentrates on -

Mainline functions: Testing the main functions of an application

Basic Usability: It involves basic usability testing of the system. It checks whether an user can freely

navigate through the screens without any difficulties.

Accessibility: Checks the accessibility of the system for the user

Error Conditions: Usage of testing techniques to check for error conditions. It checks whether suitable

error messages are displayed.

Functional Testing Process:

In order to functionally test an application, following steps must be observed.

Understand the Requirements

Identify test input (test data)

Compute the expected outcomes with the selected

test input values

Execute test cases

Comparison of actual and computed expected result

7/27/2019 3testing

6/43

Functional Vs Non-Functional Testing:

Functional Testing

Non-Functional TestinG

Functional testing is performed using the

functional specification provided by the client and

verifies the system against the functional

requirements.

Non-Functional testing checks the Performance,

reliability, scalability and other non-functional aspects of

the software system.

Functional testing is executed first Non functional testing should be performed after

functional testing

Manual testing or automation tools can be used for

functional testing

Using tools will be effective for this testing

Business requirements are the inputs to functional

testing

Performance parameters like speed , scalability are

inputs to non-functional testing.

Functional testing describes what the product does Nonfunctional testing describes how good the product

works

Easy to do manual testing Tough to do manual testing

Types of Functional testing are

Unit Testing

Smoke Testing Sanity Testing

Integration Testing

White box testing

Black Box testing

User Acceptance testing

Regression Testing

Types of Non functional testing are

Performance Testing

Load Testing Volume Testing

Stress Testing

Security Testing

Installation Testing

Penetration Testing

Compatibility Testing

Migration Testing

Functional testing tools:

There are several tools available in the marker to perform functional testing. They are explained as follows:

7/27/2019 3testing

7/43

JUnit Used mainly for Java applications and this can be used in Unit and system testing

soapUI This is an open source functional testing tool, mainly used for Web service testing. It supports

multiple protocols such HTTP, SOAP and JDBC.

Watir This is functional testing tool for web applications. It supports tests executed at the web browser

and uses ruby scripting language

Conclusion:

Functional testing is process of testing functionalities of the system and ensures that the system is

working as per the functionalities specified in the business document. The goal of this testing is to check whether

the system is functionally perfect!!!

Little know Details on "End to End Testing"

Unlike System Testing, End-to-End Testing not only validates the software system under test but also checks its

integration with external interfaces. Hence, the name End-to-End. The purpose of End-to-End Testing is to

exercise a complete production-like scenario. Along with the software system, it also validates batch/data

processing from other upstream/downstream systems.

End to End Testing is usually executed after functional and system testing. It uses actual production like data and

test environment to simulate real-time settings. End-to-End testing is also called Chain Testing

Why End to End Testing ?

Modern software systems are complex and are interconnected with multiple sub-systems

A sub-system may be different from the current system or may be owned by another organization. If any one of

the sub-system fails, the whole software system could collapse. This is major risk and can be avoided by

End-to-End testing. End-to-End testing verifies the complete system flow. It increase test coverage of various

7/27/2019 3testing

8/43

sub-systems. It helps detect issues with sub-systems and increases confidence in the overall software product.

End to End testing Process:

The following diagram gives an overview of the End to End testing process.

The chief activities involved in End to End Testing are -

Study of end to end testing requirements

Test Environment setup and hardware/software requirements

Describe all the systems and its subsystems processes.

Description of roles and responsibilities for all the systems

Testing methodology and standards

End to end requirements tracking and designing of test cases

Input and output data for each system

How to create End-to-End Test Cases?

End to End Testing Design framework consists of three parts

1. Build user functions

2. Build Conditions

3. Build Test Cases

Lets look at them in detail: -

7/27/2019 3testing

9/43

Build User Functions

Following activities should be done as a part of build user functions:

List down the features of the system and their interconnected components

List the input data, action and the output data for each feature or function

Identify the relationships between the functions Determine whether the function can be reusable or independent

For example Consider a scenario where you login into your bank account and transfer some money to

another account from some other bank (3rdparty sub-system)

1. Login into the banking system

2. Check for the balance amount in the account

3. Transfer some amount from your account to some other bank account (3rdparty sub-system)

4. Check the your latest account balance

5. Logout of the application

Build Conditions based on User Function

Following activities are performed as a part of build conditions:

Building a set of conditions for each user function defined

Conditions include sequence, timing and data conditions

For example Checking of more conditions like

Login Page

Invalid User Name and Password

Checking with valid user name and password

Password strength checking

Checking of error messages

Balance Amount

Check the current balance after 24 hours.(If the transfer is sent to different bank) Check for the error message if the transfer amount is greater than the current balance amount

Build Test Scenario

Building the test scenario for the user function defined

In this case,

Login into the system

Check of bank balance amount

Transfer the bank balance amount

Build Multiple Test cases

Build one or more test cases for each scenario defined .Test cases may include each condition as single

test case.

Metrics for end to end testing:

7/27/2019 3testing

10/43

Following are few of many metrics used for End to End Testing.

Test Case preparation status: It givesTest Case preparation progress against planned

Weekly Test Progress- Provides week-wise details of percentage test completion- Failed, not executed &

executed against planned for execution tests.

Defects Status & Details- It gives Percentage of open & closed defects by week. Also, week-wise defects

distribution based on severity and priority

Environment Availability -Total number of hours up / Total number of hours scheduled per day for

testing

Difference between End to End Testing and System Testing:

End to End Testing System Testing

Validates the software system as well as

interconnected sub-systems

Validates just the software system as per the

requirements specifications.

It checks the complete end-to-end process flow. It checks system functionalities and features.

All interfaces, backend systems will be considered

for testing

Functional and Non-Functional Testing will be

considered for testing

Its executed once system testing is completed. Its executed after integration testing.

End to End testing involves checking external

interfaces which can be complex to automate. Hence

Manual Testing is preferred.

Both Manual and Automation can be performed for

system testing

Conclusion

End to end testing is the process verifying a software system along with its sub-systems. The biggest

challenge in this testing is to have enough knowledge of the whole system as well as interconnected sub-system.

Positive Vs Negative testing

Software testing is process of verifying and validating the software or application and checks whether it is working

as expected. The intent is to find defects and improve the product quality. There are two ways to test the software ,

viz, Positive Testing and Negative Testing.

Positive testing can be performed on the system by providing the valid dataas input. It checks whether

an application behaves as expected with the positive input. . This is to test to check the application that does what it

is supposed to do so

For example

7/27/2019 3testing

11/43

There is a text box in an application which can accept only numbers. Entering values upto 99999 will be

acceptable by the system and any other values apart from this should not be acceptable. To do positive testing, set

the valid input values from 0 to 99999 and check whether the system is accepting the values.

Negative Testing can be performed on the system by providing invalid data as input. It checks whether an

application behaves as expected with the negative input. This is to test the application that does not do anything

that it is not supposed to do so. For example

For the example above Negative testing can be performed by testing by entering alphabets characters

from A to Z or from a to z. Either system text box should not accept the values or else it should throw an error

message for these invalid data inputs.In both the testing, following needs to be considered:

Input data

Action which needs to be performed

Output Result

Testing Technique used for Positive and Negative Testing:

Following techniques are used for Positive and negative validation of testing is:

Boundary Value Analysis

Equivalence Partitioning

Boundary Value Analysis:

This is one of the software testing technique in which the test cases are designed to include values at the

boundary. If the input data is used within the boundary value limits, then it is said to be Positive Testing. If the inputdata is picked outside the boundary value limits, then it is said to be Negative Testing.

7/27/2019 3testing

12/43

For example

A system can accept the numbers from 0 to 10 numeric values. All other numbers are invalid values.

Under this technique , boundary values 0 , 10 and -10 will be tested.

Equivalence Partitioning:

This is a software testing technique which divides the input date into many partitions .Values from each

partition must be tested atleast once. Partitions with valid values are used for Positive Testing. While ,partitions

with invalid values are used for negative testing.

7/27/2019 3testing

13/43

For example-

Numeric values Zero to ten can be divided to two( or three )partition. In our case we have two partitions -10

to -1 and 0 to 10. Sample values (5 and -5) can be taken from each part to test the scenarios.

Conclusion:

Testing helps deliver quality software application and ensures the software is bug free before the

software is launched. For effective testing, use both - Positive and Negative testing which give enough confidence

in the quality of the software. Real time users are can input any values and those needs to be tested before

release.

Static Vs Dynamic Testing

Under Static Testing code is not executed. Rather it manually checks the code, requirement documents, anddesign documents to find errors. Hence, the name static.

Main objective of this testing is to improve the quality of software products by finding errors in early stages of the

development cycle. This testing is also called as Non-execution technique or verification testing.

Static testing involves manual or automated reviews of the documents. This review, is done during initial phase of

testing to catch defect early in STLC. It examines work documents and provides review comments

Work document can be of following:

Requirement specifications

Design document

Source Code Test Plans

Test Cases

Test Scripts

Help or User document

Web Page content

UnderDynamic Testing code is executed. It checks for functional behavior of software system , memory/cpu usage

and overall performance of the system. Hence the name Dynamic

7/27/2019 3testing

14/43

Main objective of this testing is to confirm that the software product works in conformance with the business

requirements. This testing is also called as Execution technique or validation testing.

Dynamic testing executes the software and validates the output with the expected outcome. Dynamic testing is

performed at all levels of testing and it can be either black or white box testing.

Testing Techniques used for Static Testing:

Informal Reviews: This is one of the type of review which doesnt follow any process to find errors in the

document. Under this technique , you just review the document and give informal comments on it.

Technical Reviews:A team consisting of your peers, review the technical specification of the software

product and checks whether it is suitable for the project. They try to find any discrepancies in the

specifications and standards followed. This review concentrates mainly on the technical document

related to the software such as Test Strategy, Test Plan and requirement specification documents.

Walkthrough: The author of the work product explains the product to his team. Participants can ask

questions if any. Meeting is led by the author. Scribe makes note of review comments

Inspection: The main purpose is to find defects and meeting is led by trained moderator. This review is a

formal type of review where it follows strict process to find the defects. Reviewers have checklist to review

the work products .They record the defect and inform the participants to rectify those errors.

Static code Review: This is systematic review of the software source code without executing the code. It

checks the syntax of the code, coding standards, code optimization, etc. This is also termed as white box

testing .This review can be done at any point during development.

7/27/2019 3testing

15/43

Testing Techniques used for Dynamic Testing:

Unit Testing:Under unit testing , individual units or modules is tested by the developers. It involves

testing of source code by developers.

Integration Testing: Individual modules are grouped together and tested by the developers. The purpose

is to determine that modules are working as expected once they are integrated.

System Testing: System testing is performed on the whole system by checking whether the system or

application meets the requirement specification document.

Also , Non-functional testing like performance, security testing fall under the category of dynamic testing.

Difference between Static and Dynamic Testing:

Static Testing Dynamic Testing

Testing done without executing the program Testing done by executing the program

This testing does verification process Dynamic testing does validation process

Static testing is about prevention of defects Dynamic testing is about finding and fixing the defects

Static testing gives assessment of code and

documentation

Dynamic testing gives bugs/bottlenecks in the software

system.

Static testing involves checklist and process to be

followed

Dynamic testing involves test cases for execution

This testing can be performed before compilation Dynamic testing is performed after compilation

Static testing covers the structural and statement Dynamic testing covers the executable file of the code

7/27/2019 3testing

16/43

coverage testing

Cost of finding defects and fixing is less Cost of finding and fixing defects is high

Return on investment will be high as this process

involved at early stage

Return on investment will be low as this process involves

after the development phase

More reviews comments are highly recommended

for good quality

More defects are highly recommended for good quality.

Requires loads of meetings Comparatively requires lesser meetings

Conclusion:

Verification and Validation are two measures used to check that the software product meets the requirements

specifications. Static testing involves verification whereas dynamic testing involves validation. Together they help

improve software quality.

Adhoc Testing Stuff You Must Know !

Adhoc testing is an informal testing type with an aim to break the system. This testing is usually an unplanned

activity . It does not follow any test design techniques to create test cases. In fact is does not create test cases

altogether! This testing is primarily performed if the knowledge of testers in the system under test is very high.

Testers randomly test the application without any test cases or any business requirement document.

Adhoc Testing does not follow any structured way of testing and it is randomly done on any part of application. Main

aim of this testing is to find defects by random checking. Adhoc testing can be achieved with the testing technique

called Error Guessing. Error guessing can be done by the people having enough experience on the system to

geuss the most likely source of errors.

This testing requires no documentation/ planning /process to be followed. Since this testing aims at finding

defects through random approach, without any documentation, defects will not be mapped to test cases. Hence,

sometimes, it is very difficult to reproduce the defects as there are no test-steps or requirements mapped to it.

When execute Adhoc Testing?

Adhoc testing can be performed when there is limited time to do elaborative testing. Usually adhoc testing

is performed after the formal test execution. And if time permits, adhoc testing can be done on the system).Adhoc

7/27/2019 3testing

17/43

testing will be effective only if the tester is knowledgeable of the System Under Test.

Types of adhoc testing

There are different types of Adhoc testing and they are listed as below:

Buddy Testing Two buddies mutually work on identifying defects in the

same module. Mostly one buddy will be from development

team and another person will be from testing team. Buddy

testing helps the testers develop better test cases and

development team can also make design changes early.

This testing usually happens after unit testing completion.

Pair testing Two testers are assigned modules, share ideas and work

on the same machines to find defects. One person can

execute the tests and another person can take notes on

the findings. Roles of the persons can be a tester and

scriber during testing.

Comparison Buddy and Pair Testing

Buddy testing is combination of unit and system testing

together with developers and testers but Pair testing is

done only with the testers with different knowledge

levels.(Experienced and non-experienced to share their

ideas and views)

Monkey Testing Randomly test the product or application without testcases with a goal to break the system.

Best practices of Adhoc testing

Following bestpratices can ensure effective Adhoc Testing.

Good business knowledge

Testers should have good knowledge of the business and clear understanding of the requirements-

7/27/2019 3testing

18/43

Detailed knowledge of the end to end business process will help find defects easily. Experienced testers find

more defects as they are better at error guessing.

Test Key Modules

Key business modules should be identified and targeted for adhoc testing.. Business critical modules

should be tested first to gain confidence on the quality of the system.

Record Defects

All defects need to be recorded or written in a notepad . Defects must be assigned to developers for fixing.

For each valid defect ,corresponding test cases must be written & must be added to planned test cases.

These defect findings should be made as lesson learnt and these should be reflected in our next system

while we are planning for test cases.

Conclusion:

The advantage of Adhoc testing is to check for the completeness of testing and find more defects then

planned testing. The defect catching test cases are added as additional test cases to the planned test cases.Adhoc Testing saves lot of time as it doesnt require elaborate test planning , documentation and test

case design.

Lets explore - Exploratory Testing!

Under scripted testing, you design test cases first and later proceed with test execution. On the contrary,

exploratory testing is simultaneous process of test design and test execution all done at the same time.

Scripted Test Execution is usually a non-thinking activity where testers executes the test steps and compare the

actual results with expected results. Such test execution activity can be automated does not require much cognitive

skills.

Exploratory testing, on the other hand, is all about discovery, investigation and learning. This emphasizes is on

personal freedom and responsibility of the individual tester. Test cases are not created in advance but testers

check system on the fly. They may note down ideas about what to test before test execution. The focus of

exploratory testing is more on testing as a thinking activity.

7/27/2019 3testing

19/43

Though the current trend in testing is to push forautomation, exploratory testing is a new way of thinking.

Automation has its limits

Here are some major differences between Scripted and exploratory testing:

Scripted Testing Exploratory Testing

Directed from requirements Directed from requirements and exploring during

testing

Determination of test cases well in advance Determination of test cases during testing

Confirmation of testing with the requirements Investigation of system or application

Emphasizes on prediction and decision making Emphasizes on adaptability and learning

Involves confirmed testing Involves Investigation

Is about Controlling tests Is about Improvement of test design

Like making a speech you read from a draft Like making a conversion its spontaneous

The script is in control The testers mind is in control

7/27/2019 3testing

20/43

Exploratory testing -

Is not random testing but it is adhoc testing with purpose of find bugs

Is structured and rigorous

Is cognitively (thinking) structured as compared to procedural structure of scripted testing. This structure

comes from Charter, time boxing etc.

Is highly teachable and manageable

Is not a technique but it is an approach. What actions you perform next is governed by what you are doing

currently

Exploratory Test Preparation:

Exploratory test preparation goes through following 5 stages detailed below and it is also called session

based test management (SBTM Cycle):

1. Create a Bug Taxonomy (classification )

Categorize common types of faults found in the past projects

Analyze the root cause analysis of the problems or faults

Find the risks and develop ideas to test the application.

2. Test Charter

Test Charter should suggest

i. what to test

ii. how it can be tested

iii. What needs to be looked

Test ideas are the starting point of exploration testing

Test charter helps determine how the end user could use the system

3. Time Box

This method includes pair of testers working together not less than 90 minutes

There should not be any interrupted time in those 90 minutes session

Time box can be extended or reduced by 45 minutes

This session encourages testers to react on the response from the system and prepare for the

correct outcome

4. Review Results:

Evaluation of the defects

Learning from the testing

Analysis of coverage areas

5. Debriefing:

Compilation of the output results

Compare the results with the charter

Check whether any additional testing is needed

During exploratory execution, following needs to be done:

Mission of testing should be very clear

Keep notes on what needs to be tested, why it needs to be tested and the assessment of the product

quality

Tracking of questions and issues raised during exploratory testing

Better to pair up the testers for effective testing

The more we test, more likely to execute right test cases for the required scenarios

It is very important to take document and monitor the following

Test Coverage Whether we have taken notes on the coverage of test cases and improve the quality of

the software

7/27/2019 3testing

21/43

Risks Which risks needs to be covered and which are all important ones?

Test Execution Log Recordings on the test execution

Issues / Queries Take notes on the question and issues on the system

Smarter exploratory testing finds more errors in less time.

Pros and Cons of Exploratory Testing:

Pro

Con

This testing is useful when requirement

documents are not available or partially

available

It involves Investigation process which helps

find more bugs than normal testing-

Uncover bugs which are normally ignored by

other testing techniques

Helps to expand the imagination of testers by

executing more and more test cases which

finally improves productivity as well

This testing drill down to smallest part of

application and covers all the requirements

This testing covers all the types of testing

and it covers various scenarios and cases

Encourages creativity and intuition

Generation of new ideas during test

execution

this testing purely depends on the tester skills

limited by domain knowledge of the tester

Not suitable for Long execution time

Challenges of Exploratory Testing:

There are many challenges of exploratory testing and those are explained below:

7/27/2019 3testing

22/43

Learning to use the application or software system is a challenge

Replication of failure is difficult

Determining whether tools need to be used can be challenging

Determine the best test cases to execute can be difficult

Reporting of the test results is challenge as report doesnt have planned scripts or cases to compare with

the actual result or outcome

Documentation of all events during execution is difficult to record

Dont know when to stop the testing as exploratory testing has definite test cases to execute.

When use exploratory testing?

Exploratory testing can be used extensively when

The testing team has experienced testers

Early iteration is required

There is a critical application

New testers entered into the team

Conclusion:

Exploratory testing is performed to overcome the limitations of scripted testing. It helps in improving test

case suite. It empathizes on learning and adaptability.

All About Compatibility Testing

What is compatibility?

Compatibility is nothing but capability of existing or living together. In normal life, Oil is not compatible with water,

but milk can be easily combined with water.

What is compatibility testing?

7/27/2019 3testing

23/43

In computer world, compatibility is to check whether your software is capable of running on different hardware,

operating systems, applications , network environments or mobile devices.

Compatibility Testing is a type of the Non-functional testing

Types of compatibility tests

Lets look into compatibility testing types

Hardware : It checks software to be compatible with different hardware configurations .

Operating Systems: It checks your software to be compatible with different Operating Systems like

Windows , Unix , Mac OS etc.

Software: It checks your developed software to be compatible with other softwares.For example: MS

Word application should be compatible with other softwares like MS Outlook,MS Excel , VBA etc.

Network: Evaluation of performance of system In network with varying parameters such as Bandwidth,

Operating speed, Capacity. It also checks application in different networks with all parameters mentioned

earlier.

Browser: It checks compatibility of your website with different browsers like Firefox , Google Chrome ,

Internet Explorer etc.

Devices : It checks compatibility of your software with different devices like USB port Devices, Printers and

Scanners, Other media devices and Blue tooth.

Mobile: Checking you software is compatible with mobile platforms like Android , iOS etc.

7/27/2019 3testing

24/43

Versions of the software: It is verifying you software application to be compatible with different versions of

software.For instance checking your Microsoft Word to be compatible with Windows 7, Windows 7 SP1 ,

Windows 7 SP 2 , Windows 7 SP 3.

There are two types of version checking

Types of Version Checking

Backward compatibility Testing is to verify the behavior of the developed hardware/software with the older

versions of the hardware/software.

Forward compatibility Testing is to verify the behavior of the developed hardware/software with the newer

versions of the hardware/software.

Tools for compatibility testing

Adobe Browser Lab Browser Compatibility Testing:This tool helps check your application in different

browsers.

Secure Platform Hardware Compatibility tool:This tools includes necessary drivers for a specific hardware

platform and it provides information on tool to check for CD burning process with CD burning tools.

Virtual Desktops Operating System Compatibility: This is used to run the applications in multiple operating

7/27/2019 3testing

25/43

systems as virtual machines. N Number of systems can be connected and compare the results.

Compatibility testing process:

1. Initial phase of compatibility testing is to define the set of environments or platforms the application is

expected to work on.

2. Tester should have enough knowledge on the platforms / software / hardware to understand the expected

application behavior under different configurations.

3. Environment needs to be set-up for testing with different platforms, devices, networks to check whether

your application runs well under different configurations.

4. Report the bugs .Fix the defects. Re-test to confirm defect fixing.

Conclusion:

The most important use of compatibility testing is to ensure whether developed software works under different

configurations (as stated in requirements documentation). This testing is necessary to check whether the

application is compatible with clients environment.

GUI Testing

What is GUI ?

There are two types of interfaces in a computer application.Command Line Interface is where you type text and

computer responds to that command.GUI stands for Graphical User Interface where you interact with the computer

using images rather than text.Following are the GUI elements which can be used for interaction between the user

and application:

What is GUI Testing?

Graphical User Interface (GUI) testing is the process of testing the systems GUI of the System Under Test. GUI

testing involves checking the screens with the controls like menus, buttons, icons, and all types of bars tool bar,

menu bar, dialog boxes and windows etc.

What do you Check in GUI Testing ?

7/27/2019 3testing

26/43

The following checklist will ensure detailed GUI Testing.

Check all the GUI elements for size, position, width, length and acceptance of characters or numbers. For

instance, you must be able to provide inputs to the input fields.

Check you can execute the intended functionality of the application using the GUI

Check Error Messages are displayed correctly Check for Clear demarcation of different sections on screen

Check Font used in application is readable

Check the alignment of the text is proper

Check the Color of the font and warning messages is aesthetically pleasing

Check that the images have good clarity

Check that the images are properly aligned

Check the positioning of GUI elements for different screen resolution.

Approach of GUI Testing:

GUI testing can be done through three ways:

Manual Based Testing : Under this approach, graphical screens are checked manually by testers in

conformance with the requirements stated in business requirements document.

Record and Replay : GUI testing can be done using automation tools. This is done in 2 parts. During

Record , test steps are captured into the automation tool. During playback, the recorded test steps are executed on

the Application Under Test. Example of such tools - QTP .

7/27/2019 3testing

27/43

Model Based Testing

A model is a graphical description of systems behavior. It helps us to understand and predict the system

behavior. Models help in a generation of efficient test cases using the system requirements. Following needs to be

considered for this model based testing:

Build the model

Determine Inputs for the model

Calculate expected output for the model

Run the tests

Compare the actual output with the expected output

Decision on further action on the model

Some of the modeling techniques from which test cases can be derived:

Charts Depicts the state of a system and checks the state after some input.

Decision Tables Tables used to determine results for each input applied

Model based testing is an evolving technique for the generating the test cases from the requirements .Its main

advantage, compared to above two methods, is that it can determine undesirable states that your GUI can

attain.

Conclusion:

7/27/2019 3testing

28/43

Success of a software product immensely depends on how the GUI interacts with the user and ease in using its

various features. Hence GUI testing is very important. Manual GUI testing can sometimes be repetitive and boring

and hence error prone. Automation is highly recommended for GUI testing.

Learn API testing in 10 minutes!!!

Before going to API testing, first lets understand -

What is an API?

API is an acronym forApplication Programming Interface.It enables communication between two separate

software system. A software system implementing an API contains functions/sub-routines which can be executed

by another software system.

What is API Testing?

API testing is checking Application Programming Interface of a Software System. In order to check API , it

needs to be called from the calling software . Necessary test environment, database, server, application setup

needs to be prepared before doing API Testing.API testing is focused on the functionality of the softwares

business logic and it is entirely different from GUI testing. It mainly concentrates on the business logic layer of the

software architecture. This testing wont concentrate on the look and feel of an application.

Set-up of API environment and different types of test cases:

Normally, setting up the API environment is very complex as it doesnt involve user interface. Database

and server should be configured as per the application requirements. API Function should be called to checkwhether that API is working.Output of API could be

Any type of data

Status (say Pass or Fail)

Call to another API function.

The output is compared with the expected outcome. API test cases are written based on the input data that users

are expected to key-in.

7/27/2019 3testing

29/43

For example There is an API function which should add two integer numbers .

1.Long add(int a, int b)

The numbers have to be given as input parameters. Output should be summation of two integer numbers . This

output needs to be verified with expected outcome.

Calling needs to be done such as

1.add (1234, 5656)

Exceptions have to be handled if number is exceeding the integer limit.

Consider the below API function -

1. Lock()

2. Unlock()

3. Delete()

They return any value such as True (in case of success) or false (In case of error) as an output .A more accurate

test case would be , can call the functions in any of the script and later check for changes either in the database or

the Application GUI.

Calling of another API / Event

In this case , we call one of the API function which in turn will call another function.For example First API function

can be used for deleting a specified record in the table and this function in turn call another function to REFRESH

the database.

7/27/2019 3testing

30/43

Approach of API Testing:

Following points helps the user to do API Testing approach:

1. Understanding the functionality of the API program and clearly define the scope of the program

2. Apply testing techniques such as equivalence classes, boundary value analysis and error guessing and

write test cases for the API

3. Input Parameters for the API need to be planned and defined appropriately4. Execute the test cases and compare expected and actual results.

Difference between Unit and API Testing:

Following are the differences between unit testing and API testing

Unit Testing API Testing

Performed by developers Performed by testers

Separate functionality is tested End to End functionality has been tested

Developer can access the source code Testers usually cannot access the code

UI testing is also involved Only API functions are tested

Only Basic functionalities are tested All functional issues are tested

API Test Automation:

Since API and unit testing both target source code, similar tools can be used for testing both.

If an API method is using .NET code, then the tool which is supporting should have .NET

Example for Automation tools are

NUnit for .Net

JUnit for Java

7/27/2019 3testing

31/43

Best Practices of API Testing:

There are some best practices for effective API testing:

Planning the test cases What and How to test the API functions

Important to organize the test properly so that those API can be organized properly

Parameters selection should be explicitly mentioned in the test case itself Call sequencing should be performed and well planned

Prioritize API function calls so that it will be easy for testers to test

Proper testing and test cases should be done while handling one time call functions like Delete,

CloseWindow, etc.

Challenges of API testing:

There are various challenges when we are doing API testing:

No GUI available to test the application which is difficult to give input values

Verifying and validating the output in different system is little difficult for testers

Parameters selection and categorization needs to be known to the testers Exception handling function needs to utilized and used properly

Testers should be knowledgeable in coding

Conclusion:

API consists of set of classes / functions / procedures which represent the business logic layer. If API is not tested

properly, it may cause problems not only the API application but also in the calling application.

What is Security Testing ?

What is Security?

Security is set of measures to protect an application against unforeseen actions that cause it to stop functioning or

being exploited. Unforeseen actions can be either intentional or unintentional.

What is Security testing?

7/27/2019 3testing

32/43

Security Testing ensures, that system and applications in an organization, are free from any loopholes that may

cause a big loss. Security testing of any system is about finding all possible loopholes and weaknesses of the

system which might result into loss of information at the hands of the employees or outsiders of the Organization.

The goal of security testing is to identify the threats in the system and measure its potential vulnerabilities. It also

helps in detecting all possible security risks in the system and help developers in fixing these problems through

coding.

Types of Security Testing:

There are seven main types of security testing as per Open Source Security Testing methodology manual.

They are explained as follows:

Vulnerability Scanning : This is done through automated software to scan a system against known

vulnerability signatures.

Security Scanning: It involves identifying network and system weaknesses, and later provides solutions

for reducing these risks. This scanning can be performed for both Manual and Automated scanning.

Penetration testing: This kind of testing simulates an attack from malicious hacker. This testing involves

7/27/2019 3testing

33/43

analysis of a particular system to check for potential vulnerabilities to an external hacking attempt.

Risk Assessment :This testing involves analysis of security risks observed in the organization. Risks are

classified as Low, Medium and High. This testing recommends controls and measures to reduce the

risk.

Security Auditing : This is internal inspection of Applications and Operating systems for security flaws.

Audit can also be done via line by line inspection of code

Ethical hacking: Its hacking an Organization Software systems. Unlike malicious hackers ,who steal for

their own gains , the intent is to expose security flaws in the system.

Posture Assessment: This combines Security scanning, Ethical Hacking and Risk Assessments to show

an overall security posture of an organization.

Integration of security processes with the SDLC:

It is always agreed, that cost will be more ,if we postpone security testing after software implementation

phase or after deployment. So, it is necessary to involve security testing in SDLC life cycle in the earlier phases.

Lets look into the corresponding Security processes to be adopted for every phase in SDLC

SDLC Phases Security Processes

Requirements

Security analysis for requirements and check

abuse/misuse cases

Design

Security risk analysis for designing. Development of testplan including security tests

Coding and Unit Testing

Static and Dynamic Testing and Security white box testing

7/27/2019 3testing

34/43

Integration Testing

Black Box Testing

System Testing

Black Box Testing and Vulnerability scanning

Implementation

Penetration Testing, Vulnerability Scanning

Support

Impact analysis of Patches

Test plan should include

Security related test cases or scenarios

Test Data related to security testing

Test Tools required for security testing

Analysis on various tests outputs from different security tools

Sample Test Scenarios for Security Testing:

Sample Test scenarios to give you a glimpse of security test cases -

Password should be in encrypted format

Application or System should not allow invalid users

Check cookies and session time for application

For financial sites, Browser back button should not work.

Methodologies

In security testing, different methodologies are followed, and they are as follows:

Tiger Box:This hacking is usually done on a laptop which has a collection of OSs and hacking tools. This

testing helps penetration testers and security testers to conduct vulnerabilities assessment and attacks.

Black Box:Tester is authorized to do testing on everything about the network topology and the technology.

Grey Box: Partial information is given to the tester about the system, and it is hybrid of white and black box

models.

Hackers - Access computer system or network without authorization

Crackers Break into the systems to steal or destroy data

7/27/2019 3testing

35/43

Ethical Hacker Performs most of the breaking activities but with permission from owner

Script Kiddies or packet monkeys Inexperienced Hackers with programming language skill

Myths and Facts of Security testing:

Lets talk on an interesting topic on Myths and facts of security testing:

Myth #1 We dont need a security policy as we have a small business

Fact : Everyone and every company need a security policy

Myth #2 There is no return on investment in security testing

Fact : Security Testing can point out areas for improvement that can improve efficiency and reduce

downtime, enabling maximum throughput.

Myth #3: Only way to secure is to unplug it.

Fact: The only and the best way to secure organization is to find Perfect Security. Perfect security can be

achieved by performing posture assessment and compare with business, legal and industry justifications.

Myth #4:Internet isnt safe. I will purchase software or hardware to safeguard the system and savebusiness.

Fact: One of the biggest problems is to purchase software and hardware for security. Instead,

organization should understand security first and then apply it.

Conclusion:

Security testing is most important testing for an application and check whether confidential data stays

confidential. In this type of testing, tester plays a role of the attacker and play around the system to find security

related bugs. This security testing is very important in IT industry to protect data by all means.

Model Based Testing Stuff You Must Know!

Before moving forward to Model Based testing, let first understand what a model is -

What is model?

A model is a description of a systems behavior. Behavior can be described in terms of input sequences, actions,

conditions, output and flow of data from input to output. It should be practically understandable and can be

reusable shareable must have precise description of the system under test.

There are numerous models available and it describes different aspects of the system behavior. Examples of the

model are:

Data Flow Control Flow

Dependency Graphs

Decision Tables

State transition machines

7/27/2019 3testing

36/43

Model Based Testing

Model based testing is a testing technique where run time behavior of a software under test is checked

against predictions made by a formal specification or model. In other means, it describes how system behaves in

response to an action ( determined by a model). Supply action ,and see, if the system responds as per the

expectation.It is a light-weight formal method to validate a system. This testing can be applied for both hardwareand software testing.

The above model explains about simplified approach of writing poem in notepad and possible actions related to

each step. For each and every action(like starting, Entering poem, Saving), Test Case can be generated and output

can be verified.Model based testing is a new approach to software testing. The evolution of Software Testing is

shown below -

7/27/2019 3testing

37/43

Types of MBT:

There are two types of Model based testing. First is

Offline / a priori Generation of Test Suites before executing it . Test suite is nothing but collection of test cases.

Online / on-the-fly

Generation of Test Suites during test execution

Different Models in Testing:

In order to understand the MBT, it is necessary to understand some of the models explained below. Lets go

through one by one:

Finite State Machines

This model helps testers to assess the result depending on the input selected. There can be various

combinations of the inputs which results in a corresponding state of the system.The system will have specific

state and current state which is governed by set of inputs given from the testers.

Consider the example-

There is a system which allows employees to log- in to the application. Now, current state of the employee is Out

and it became In once he signs- in to the system. Under the in state , employee can view, print, scan

documents in the system .

7/27/2019 3testing

38/43

State Charts

It is an extension of Finite state machine and can be used for complex and real time systems. State charts are

used to describe various behaviors of the system. It has definite number of states. The behavior of the system is

analyzed and represented in the form of events for each state.

For example

Defects are raised in Defect management tool with the status as New. Once it is fixed by developers, it has to be

changed to status Fixed. If defect is not fixed change status to Re-open. State chart should be designed in such a

way that it should call for an event for each state.

7/27/2019 3testing

39/43

Unified Modeling Language (UML)

Unified Modeling Language (UML) is a standardized general-purpose modeling language.UML includes a set of

graphic notation techniques to create visual models of that can describe very complicated behavior of the system.

UML has notations such as:

Activities

Actors

Business Process

Components

Programming language

Challenges of Model Based Testing:

Deployment of MBT in every organization obviously requires high amount of investments and effort.

Following are drawbacks of MBT:

Necessary Skills required in testers

Learning curve time will be more

Difficult to understand the model itself

Benefits of Model Based Testing:

Following are benefits of MBT:

Easy test case/suite maintenance

Reduction in Cost

Improved Test Coverage

Can run different tests on n number of machines

Early defect detection

7/27/2019 3testing

40/43

Increase in defect count

Time savings

Improved tester job satisfaction

Conclusion

Testers construct mental models anyway during their testing. Those mental models can be transformed to modelson paper. This helps testers to achieve readability and re-usability.

Learn Mutation Testing in 10 minutes

In Mutation Testing, we mutate (change) certain statements in the source code and check if the test cases are

able to find the errors. It is a type of white box testing which is mainly used for unit testing. The changes in

mutant program are kept extremely small, so it does not affect the overall objective of the program.

The goal of Mutation Testing is to assess the quality of the test cases which should be robust enough to fail

mutant code. This method is also called as Fault based testing strategy as it involves creating fault in the program

Mutation was originally proposed in 1971 but lost fervor due to high costs involved. Now, again it has picked steam

and is widely used for languages such as Java and XML.

7/27/2019 3testing

41/43

How to execute mutation testing?

Following are the steps to execute mutation testing:

Step 1: Faults are introduced into the source code of the program by creating many versions called mutants. Each

mutant should contain a single fault, and the goal is to cause the mutant version to fail which demonstrates theeffectiveness of the test cases.

Step 2: Test cases are applied to the original program and also to the mutant program. A test case should be

adequate, and it is tweaked to detect faults in a program.

Step 3: Compare the results of original and mutant program.

Step 4: If the original program and mutant programs generate the same output, then that the mutant is killed by the

test case. Hence the test case is good enough to detect the change between the original and the mutant program.

Step 5: If the original program and mutant program generate different output, Mutant is kept alive. In such cases ,

more effective test cases need to be created that kill all mutants.

How to Create Mutant Programs?

A mutation is nothing but a single syntactic change that is made to the program statement. Each mutant program

should differ from the original program by one mutation.

7/27/2019 3testing

42/43

Original Program Mutant Program

If (x>y)

Print Hello

Else

Print Hi

If(x)

Print Hello

Else

Print Hi

What to change in a Mutant Program?

There are several techniques that could be used to generate mutant programs. Lets look at them

Operand replacementoperators

Expression ModificationOperators

Statement modification Operators

Replace the operand with

another operand(x with y

or y with x) or with the

constant value.

Replace an operator or

insertion of new operators

in a program statement.

Programmatic statements are

modified to create mutant programs.

Example-

If(x>y) replace x and y

values

If(5>y) replace x by

constant 5

Example-

If(x==y)

We can replace == into >=

and have mutant program

as

If(x>=y) and inserting ++

in the statement

If(x==++y)

Example-

Delete the else part in an if-else

statement

Delete the entire if-else statement to

check how program behaves

Some of sample mutation operators:

GOTO label replacement

Return statement replacement

Statement deletion

Unary operator insertion(Like

and ++)

Logical connector replacement

Comparable array name

replacement

Removing of else part in the

if-else statement

Adding or replacement ofoperators

Statement replacement by

changing the data

Data Modification for the

variables

Modification of data types in the

program

7/27/2019 3testing

43/43

Automation of Mutation Testing:

Mutation testing is extremely time consuming and complicated to execute manually. To speed up the process, it is

advisable to go for automation tools. Automation tools reduce cost of testing as well.

List of tools available -

Ninja Turtles- .net mutation testing tool

Mutagenesis PHP mutation testing frameworkHeckle Ruby Mutation Testing Tool

Jester Mutation Testing Tool for Java

Mutation Score:

The mutation score is defined as the percentage of killed mutants with the total number of mutants.

Mutation Score = (Killed Mutants / Total number of Mutants) * 100

Test cases are mutation adequate if the score is 100%. Experimental results have shown that mutation testing is

an effective approach for the measuring the adequacy of the test cases. But, the main drawback is that the high

cost of generating the mutants and executing each test case against that mutant program.

Advantages and Disadvantages of Mutation Testing:

Following are the advantages of Mutation Testing:

It is a powerful approach to attain high coverage of the source program.

This testing is capable comprehensively testing the mutant program.

Mutation testing brings a good level of error detection to the software developer.

This method uncovers ambiguities in the source code, and has the capacity to detect all the faults in the

program.

Customers are benefited from this testing by getting most reliable and stable system.

On the other side, following are the disadvantages of Mutant testing:

Mutation testing is extremely costly and time consuming since there are many mutant programs that need to

be generated.

Since its time consuming, its fair to say that this testing cannot be done without an automation tool.

Each mutation will have the same number of test cases than that of the original program. So, a large number

of mutant programs may need to be tested against the original test suite. As this method involves source code changes, it is not at all applicable for black box testing.

Conclusion:

Do you want an exhaustive testing of your application? Answer is Mutation testing. It is the most comprehensive

technique to test a program. This is the method which checks for the effectiveness and accuracy of a testing

http://www.google.com/url?q=http%3A%2F%2Fjester.sourceforge.net%2F&sa=D&sntz=1&usg=AFQjCNEkIkiSDNwAX2ctaKXwMR9W586wRQhttp://www.google.com/url?q=http%3A%2F%2Fglu.ttono.us%2Farticles%2F2006%2F12%2F19%2Ftormenting-your-tests-with-heckle&sa=D&sntz=1&usg=AFQjCNGZ3F5n4rLuWMvUFPgZBdwk3vxZ8Ahttps://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fpadraic%2Fmutagenesis&sa=D&sntz=1&usg=AFQjCNGxlcHTCdHjO2lATT0bHBpzQsNiSwhttp://www.google.com/url?q=http%3A%2F%2Fwww.mutation-testing.net%2F&sa=D&sntz=1&usg=AFQjCNG8Bt7BdXprLbxxl723Ar8fKMGGcA