unit 1 basic concepts of testing & quality
Post on 16-Aug-2015
79 Views
Preview:
TRANSCRIPT
1
BASIC CONCEPT OF TESTING AND QUALITYUNIT - 1
Prepared By: Ravi J Khimani, CSE Department, SLTIET, Rajkot
2
QUALITY REVOLUTION
Quality – critical issue in product development In the past, near about 1940, Quality
Revolution began. Due to – Global Competition, OutSourcing, Off-
Shoring and increasing customer expectation. Developing quality product in tighter schedule
is big issue for companies.
3
QUALITY REVOLUTION
In old approach, Efforts to improve quality is centered around the end of product development.
In new approach, improve quality encompasses all phases: from Requirement Analysis to Final Delivery of Product.
4
QUALITY REVOLUTION
An effective quality process must focus on: Paying attention to customer expectation Making efforts to continuously improve quality. Integrate measurement with design and
development. Develop system level perspectives with
concentrating on methodology and process. Eliminate waste through continuous improvement
5
QUALITY REVOLUTION
Quality movement started around 1940-1950 William Edward Deming, gives literature on
Statistical Quality Control (STQ) STQ is discipline based on measurements and
statistics. In which, decisions are made and plans
developed on the collection and evaluation of facts and data.
6
QUALITY REVOLUTION Deming gave Shewhart Method for STQ. In which, he gave Plan – Do – Check – Act
(PDCA) cycle, also known as Shewhart Cycle.
7
QUALITY REVOLUTION Between 1950 – 1970, companies came up
with innovative principle, known as “Lean Principle”.
Which is, “A systematic approach to identify and eliminate waste through continuous improvement, flowing the product at the pull of the customer expectations.”
For Example, to compress the time period in banking payments.
8
QUALITY REVOLUTION In the era of 1950, Joshep M Juran of US
proposed raising level of Quality Management from Manufacture Dept. to overall organization.
That Quality Management is known as Total Quality Control (TQC).
It includes companywide activities, audits, quality circle and promotions of quality management principles.
9
QUALITY REVOLUTION That TQC emphases on,
Quality comes first not short-term profits. Customer comes first not the producers. Decisions are based on Data and Facts. Management is participatory and respectful of
all employees Management is driven by cross-functional
committees covering product planning, design, sales, marketing, manufacturing, purchasing, distribution.
10
QUALITY REVOLUTION
One of the TQC method developed by Japan, is known as Ishikawa OR cause-and-effect method.
It says, product quality comes from four causes, materials, machines, methods, measurements.
11
QUALITY REVOLUTION Traditionally, the TQC and Lean Concept are
applied to Manufacturing Department. The software development process uses
these concepts through “Software Capability Maturity Model (CMM)” model, to guide the production of qualitative software.
Which provides framework to discuss software production issues.
12
SOFTWARE QUALITY What is Software Quality?
Quality is a complex concept – means different things to different people.
Kitchenham and Pfleeger discuss software quality in five different ways: Transcendental View:
It envisages quality as something that can be recognized but is difficult to define.
It can be also applied to complex area of regular life. Not specific to only Software Quality.
13
SOFTWARE QUALITY User View:
It perceives quality as fitness for purpose. Key question: “Does product satisfy user needs?”
Manufacturing View: Here quality level is determined by considering
product’s specification. Product View:
Quality is viewed as inherent characteristics. Like Internal Characteristics, external characteristics.
14
SOFTWARE QUALITY Value Based View:
Depends on amount a customer is willing to pay for it.
In mid-1970, McCall, Richards and Walters were gave the software quality in two terms, Quality Factors Quality Criteria
15
SOFTWARE QUALITY A quality Factor
represents behavioral characteristics of system. For example, Correctness, Reliability, Efficiency,
Testability, Maintainability, Modularity and Reusability.
A Quality Criteria Is an attribute of Quality Factor that can be
related to Software Development. For example, Modularity, Correctness
16
SOFTWARE QUALITY There are various software quality models have
been proposed for Software Quality and its related attributes
Two most popular models are CMM [Software Capability Maturity Model] ISO [International Standard Organization]
In the field of Testing two popular models are available, TPI [Test Process Improvement] TMM [Test Maturity Model]
17
FAILURE, FAULT, ERROR AND DEFECT In software testing, we can find references to
these four terms. In fault – tolerant computing community,
Failure: A failure is said to occur whenever the external behaviour of a system does not confirm to that prescribed in the system specification.
Error: An error is a state of the system. In the absence of any corrective action by the system, an error state could lead to a failure which would not be attributed to any event subsequent to the error.
18
FAILURE, FAULT, ERROR AND DEFECT Fault: A fault is the adjudged (true) cause of an
error. A fault may remain undetected for a long time,
until some event activates it. When an event activates a fault,
it brings the program into an intermediate error state.
If computation is allowed to proceed from an error state without any corrective action, the program eventually causes a failure.
19
FAILURE, FAULT, ERROR AND DEFECT in fault-tolerant computing, corrective
actions can be taken to take a program out of an error state into a desirable state such that subsequent computation does not eventually lead to a failure.
The process of failure manifestation can therefore be succinctly represented as a behaviour chain as follows:
fault →error →failure.
20
FAILURE, FAULT, ERROR AND DEFECT In software, a system may be defective
due to design issues; certain system states will expose a defect, resulting in the development of faults are defined as incorrect signal values or decisions within the system.
In practical purpose, both term, Defect and Fault are synonymous terms.
21
FAILURE, FAULT, ERROR AND DEFECT For Example,
Consider a small organization. Defects in the organization’s staff promotion policies can cause
improper promotions, viewed as faults. The resulting ineptitudes & dissatisfactions are errors in the
organization’s state. The organization’s personnel or departments probably begin to malfunction as result of the errors, in turn causing an overall degradation
of performance. The end result can be the organization’s failure to achieve its goal.
22
ROLE OF TESTING Testing plays an important role in
achieving and assessing the quality of a software product .
we can improve the quality of the products as we repeat a test - find defects – fix cycle during development.
But how can we perform system level tests before releasing product?
23
ROLE OF TESTING Friedman and Voas say, “software
testing is a verification process for software quality assessment and improvement.”
Software quality assessment is divided into two categories, Static Analysis Dynamic Analysis
24
1. STATIC ANALYSIS [ROLE OF TESTING] It is based on the examination of a number of documents,
requirements documents, software models, design documents source code.
It also includes code review, inspection, walk-through, algorithm analysis, and proof of correctness.
It does not involve actual execution of the code under development.
Instead, it examines code and reasons over all possible behaviours that might arise during run time.
For Ex., Compiler optimizations are standard static analysis.
25
2. DYNAMIC ANALYSIS [ROLE OF TESTING] It involves actual program execution with input
values. During execution, it observes the behavioural and
performance properties of the program. For practical considerations, a finite subset of the
input set can be selected. Therefore, in testing, we observe program behaviours
and reach a conclusion about the quality of the system.
Careful selection of a finite test set is crucial to reaching a reliable conclusion.
26
ROLE OF TESTING [CONT…] Both Analysis techniques are complementary
in nature, and for better effectiveness, both must be performed repeatedly and alternated.
Practitioners and researchers need to remove the boundaries between static and dynamic analysis and create a hybrid analysis that combines the strengths of both approaches
27
OBJECTIVES OF TESTING The stakeholders in a test process are the
programmers, the test engineers, the project managers, and the customers.
A stakeholder is a person or an organization who influences a system’s behaviours or who is impacted by that system.
Different stakeholders view a test process from different perspectives, as follows,
28
OBJECTIVES OF TESTING It does work:
While implementing a program unit, the programmer may want to test whether or not the unit works in normal circumstances.
The programmer gets much confidence if the unit works to his or her satisfaction.
The same idea applies to an entire system as well—once a system has been integrated, the developers may want to test whether or not the system performs the basic functions.
Here, for the psychological reason, the objective of testing is to show that the system works, rather than it does not work.
29
OBJECTIVES OF TESTING It does not work:
Once the programmer (or the development team) is satisfied that a unit (or the system) works to a certain degree, more tests are conducted with the objective of finding faults in the unit (or the system).
Here, the idea is to try to make the unit (or the system) fail. Reduce the risk of failure:
Software systems contain faults, which cause the system to fail.
“failing from time to time” gives rise of failure rate. As faults are discovered and fixed during more tests, Thus, a higher level objective of performing tests is to bring
down the risk of failing to an acceptable level.
30
OBJECTIVES OF TESTING Reduce the cost of testing:
The different kinds of costs associated with a test process include
the cost of designing, maintaining, and executing test cases, the cost of analyzing the result of executing each test case, the cost of documenting the test cases, and the cost of actually executing the system and documenting it.
Therefore, the less the number of test cases designed, the less will be the associated cost of testing.
However, producing a small number of arbitrary test cases is not a good way of saving cost.
Objective – more test cases – more reliable software. Its define effectiveness of test cases.
31
TEST CASE Basic Form, it’s a pair of,
<input, expected outcome> In stateless system, outcome depends only on
the current input. For example, calculate square root of number, A compiler for C language.
32
TEST CASE In state-oriented system, outcome depends on
current input as well as current state of system. For this test case consists of Sequence of <input,
expected outcome> pairs, according to state. It also consists of Decision and Timing factor. For example, ATM Machine,
33
EXPECTED OUTCOME Outcome of program execution is complex entity,
which includes, Value produced State change (program or database) Set of values interpreted together for valid outcome.
ORACLE: is important concept of Test Design. Oracle is entity – Program, Process, Human
Expert or body of data – that tells us the expected outcome of test.
34
EXPECTED OUTCOME Expected Outcome should be computed
while designing test cases. Before program execution and with selected
inputs. Will eliminate implementation biasing. The process is of – execute program –
observe actual output – verify actual output – use that verified output in subsequent test case.
35
CONCEPT OF COMPLETE TESTING It means “there are no undiscoverable faults at the end
of test phase”. It’s near impossible, due to
Input domain is to large Program may have large number of state based on valid and
invalid inputs. Timing constraint Design (for program) issues too complex Impossible to create all possible execution environments like
weather, temperature, altitude, pressure, etc… “Input value which is valid but is not property
timed is called inopportune Input”.
36
CENTRAL ISSUE IN TESTING Impossible to test complete input
domain to discover faults. Select subset of Domain is best thing to
test program.
38
TEST ACTIVITIES Identify objective to be tested
Clear purpose must be associated with every test case.
Select Inputs Based on specification, source code and
expectation Compute the expected outcome
Can be done from high-level understanding of the test objectives and specification of program under test.
39
TEST ACTIVITIES Set up execution environment of program
Prepare right execution environment For ex, Initialize local system, external to program like
making network connection, right database system available.
For ex, Initialize any remote, external system like distributed system to run client code.
Execute the Program With selected input and observe actual output. Test Coordination concept is used in synchronizing
different components of Test Cases, like different physical location.
40
TEST ACTIVITIES Analyze Test Result
Comparison should be precise. After analyzing, Test Verdicts should be assigned,
like Pass, Fail, inconclusive. Test Report must be written, contains
reasons of failure of test casesanalysis of failureA indication to actual outcome, test
case, input values tested, environment of execution.
41
VERIFICATION AND VALIDATION Two similar concept Verification
Determine that product of given development phase satisfies the requirements established before starting of that phase.
Like requirement specification, design specification, code, user manual.
“Activities that check the correctness of a development phase are called Verification Activities”.
42
VERIFICATION AND VALIDATION Validation
Activities that determine and confirm that product meets its desired use.
Aim at confirming that product meets user’s expectations.
Focus on final product. Should be executed at early stage of
development cycle, not at the end of development cycle.
43
VERIFICATION VS. VALIDATIONVerification Validation
Confirming that Building product correctly
Confirming that building correct product
Review interim works like specification, design specification, code, user manual.
Is performed at the end of development phase of each part of system.
Verification considers quality attributes like consistency, correctness, completeness.
Validation considers only correctness and satisfaction.
Can be applied through Statistic Analysis Techniques like inspection, walkthrough, reviews, checklists, etc…
Can be applied by running system in its actual environment using verity of Tests.
44
CORRECTNESS Correctness from software engineering
perspective can be defined as the stickiness to the specifications that determine how users can interact with the software and how the software should be have when it is used correctly.
If the software behaves incorrectly, it might take considerable amount of time to achieve. Or sometimes cannot achieve.
45
CORRECTNESS Below are some of the important rules for effective
programming which are consequences of the program correctness theory. Defining the problem completely. Develop the algorithm and then the program logic. Reuse the proved models as much as possible. Prove the correctness of algorithms during the design
phase. Developers should pay attention to the clarity and
simplicity of your program. Verifying each part of a program as soon as it is developed.
46
SOURCES OF INFORMATION FOR TEST CASES A software development process generates a large body
of information, such as requirements specification, design document, and source code.
In order to generate effective tests at a lower cost, test designers analyze the following sources of information: Requirements and functional specifications. Source code Input and output domains Operational Profiles Fault model [Error Guessing, Fault Seeding, Mutation
Analysis]
47
TEST LEVELS Performed at different levels involving
complete system. Software goes through four stages: Unit,
Integration, System and Acceptance level testing.
First three inside developing organization Last one is outside developing organization These four stages of testing is known as V-
Model.
50
CONTROL FLOW GRAPH A rectangle represents sequential computation. A diamond box represents decision point.
Two branches one decision box has, True or False. A small circle represents a merge point.
We are not labeling merge point.
51
TESTING TYPES Black Box Testing
Black-box testing is a method of software testing that examines the functionality of an application based on the specifications. It is also known as Specifications based testing.
Independent Testing Team usually performs this type of testing during the software testing life cycle.
This method of test can be applied to each and every level of software testing.
There are different techniques involved in Black Box testing,
like Equivalence Class, Boundary Value Analysis, Domain Tests, Orthogonal Arrays, Decision Tables, State Models, All-pairs testing
52
TESTING TYPES White Box Testing
White box testing is a testing technique, that examines the program structure and derives test data from the program logic/code. The other names of white box testing are clear box testing, open box testing, logic driven testing or path driven testing or structural testing.
White Box Testing Techniques: Statement Coverage - This technique is aimed at exercising
all programming statements with minimal tests. Branch Coverage - This technique is running a series of tests
to ensure that all branches are tested at least once. Path Coverage - This technique corresponds to testing all
possible paths which means that each statement and branch is covered.
53
TESTING TYPES Model Based Testing
Model-based testing is a software testing technique in which the test cases are derived from a model that describes the functional aspects of the system under test.
It makes use of a model to generate tests that includes both offline and online testing.
Importance Unit testing wont be sufficient to check the functionalities To ensure that the system is behaving in the same
sequence of actions. Model-based testing technique has been adopted as an
integrated part of the testing process. Commercial tools are developed to support model-based
testing.
54
TESTING TYPES Interface Testing
Interface Testing is performed to evaluate whether systems or components pass data and control correctly to one another. It is to verify if all the interactions between these modules are working properly and errors are handled properly.
Verify that communication between the systems are done correctly
Verify if all supported hardware/software has been tested
Verify if all linked documents be supported/opened on all platforms
Verify the security requirements or encryption while communication happens between systems
Check if a Solution can handle network failures between Web site and application serve
55
TESTING TYPES Unit Testing
Tests individual program units like function, class After surety of units working properly, modules
are assembled to larger subsystem. By Software Developer.
Integration Testing Performed on assembled units of system in unit
testing. By Software Developer and Integration test
engineers.
56
TESTING TYPES System Testing
Object is to construct a reasonably stable system that can withstand to System-Level Testing.
System level testing includes functionality testing, security testing, reliability testing, stability, stress, performance, load testing.
Aim to discover most of the faults and verify fixes are working.
System testing composed of create test plan, design test suit, prepare test environments, executing tests.
57
TESTING TYPES Regression Testing
Perform throughout life cycle of system development.
Performed when component of system is modified.
Idea to verify that modification does not affect existing work and does not introduce new faults.
It is distinct level of testing Sub-phase of unit, integration and system-level
testing.
58
TESTING TYPES Acceptance Testing
After completion of all above testing, product is delivered to customer.
Customer performs their own series of tests, called acceptance tests.
Objective of this testing to measure Quality of Product.
Key notion is customer’s expectations from the system.
top related