testing web software applications

Testing Web Software Applications

Jeff Offutt, PhDJeff Offutt, PhDInformation & Software EngineeringInformation & Software Engineering

George Mason UniversityGeorge Mason UniversityFairfax, VA USAFairfax, VA USA

www.ise.gmu.edu/~ofut/www.ise.gmu.edu/~ofut/[email protected]@ise.gmu.edu

Roger Alexander, Colorado State UniversityAnneliese Andrews, Washington State UniversityYe Wu George Mason University

Joint work with:

Supported by NSF and NIST.

© Offutt 1999-2003. All Rights Reserved. 2June 2003

Outline of Talk

1. Introduction to Web Software Applications2. Testing Object-oriented Software3. Difficulties with Analyzing and Testing Web Software4. Integration Testing of Loosely Coupled Software5. Testing the Dynamic Flow of Control6. System Testing Web Software Applications7. Future Methods and Technologies


Introduction to Web Software Applications

WebEngng


Web Software Engineering

• Modern web sites are now too complicated for individuals tomanage.

• They need to be engineered by teams of people with diverse talents:– Programming skills– Graphics design– Usability– Information layout and engineering– Data communications– Data base

We need We need web site engineeringweb site engineering


Web Sites and Software

• Web Page : Data that fits in one browser screen• Web Site : A number of connected web pages• Web Site Software : Software that makes web sites

dynamic and interactive• Dynamic Web Page : A web page that is generated on

demand by a program (such as a servlet or JSP)

networkClient Server WebSoftware


Important Web SoftwareQuality Attributes

1. Reliability2. Usability3. Security Customers have little “site loyalty”

and will switch quickly, thus time to market is much less importantthan in other application areas.

(but still important!)

4. Availability5. Scalability6. Maintainability7. Performance & Time-to-market

Based on an informal survey of about a dozen software development managers, 2000.


Complexities of Web Site Software

• Heterogeneous– Diverse hardware platforms– Diverse software platforms– Diverse software languages– Diverse software models (imperative, declarative, functional, …)– Diverse organizations building the software

• Concurrent and distributed• High quality requirements• High degree of reuse and third party components• New essential problems


Multi-tiered Web Software Systems

middlewaremiddlewarenetwork

Client WebServer

ApplicationServer

DBServer

JavaHTMLCGIJSP, etc

BrowserJavascripts

Client-server … 3-tier … N-tier …


Challenges of N-Tier Architectures• Communication and distribution is usually handled by bought

middleware (CORBA, EJB, DCOM, etc)• Software becomes heterogeneous and parallel• Advantages:

– More powerful applications– Many services to many clients– Enhanced security, scalability and availability

• Disadvantages:– Designing truly reusable objects is difficult– More complicated to design and model– Performance risks– Reliability is more difficult to achieve– More difficult to maintain software– A lot to learn about the new technologies


Web Software

• Client-side– HTML– Scripting languages (Javascript)

• Server-side– CGI– Compiled modules (Java servlets, ASP)– Scripted-page modules (JSP, ASP, PHP)– Data storage (beans, DB)– Web servers


Compiled Modules - Servlets

• Servlets are small Java classes that perform a service• Servlet container or engine

– connects to network– catches requests– produces responses– requests are handled by objects

• Servlets receive requests and data• Servlets have full access to the server:

– Database– Other software componentsShared memory with other servlets


Scripted-page Modules - Java Server Pages

• JSPs turn servlets "inside-out":– Instead of HTML in Java …– Java in HTML

• JSPs are to translated to servlets, compiled, then executed

• This encourages separation of tasks:

Page Layout

Graphics designer

WritingHTML

?

Integration w/JSP

Webby Javaprogrammer

ApplicationDevelopment

Java, JavaBeans

Javaprogrammer


Session Management & Tracking

• HTTP client server communication is connnectionless– Connection is terminated as soon as the request is made and

fulfilled (simple and resistant to network problems)

• But how can a server keep track of state of different clients?

Session: A series of related interactions between a client and a web server (similar to a use case)

Request with a TokenClientC

ServerSResponse with a Token


Web Software Technologies

• The new technologies were created partly because of the decoupled, networked aspect of the web

• But the major motivation has been to support the very high quality requirements of web software


Testing of Web Software Applications

• Web software applications are composed of– HTML– Client-side scripts (Javascript)– Client-side Web software components

• Compiled modules (servlets)• Scripted page modules (JSPs)• Data abstraction modules (Javabeans)

– Traditional object-oriented classes

• First we will explore how to test object-oriented classes …


Testing Object-oriented Software

OOTest


Inheritance

Declared type: The type given when an object reference is declaredClock w1; // declared type Clock

Actual type: The type of the current objectw1 = new Watch(); // actual type Watch

A

C

BIn Java, the method that is executed is the In Java, the method that is executed is the lowestlowest version of the method defined version of the method defined between the actual and root types in the between the actual and root types in the inheritance hierarchyinheritance hierarchy


Polymorphism

• The same variable can have different types depending on the program execution

• If B inherits from A, then an object of type B can be used when an object of type A is expected

• If both A and B define the same method M(B overrides A), then the same statement might call either A’s version of M or B’s version


Example DU Pairs and Anomalies

Method Uses DefsA::h () {A::u, A::w}

A::i () {A::u}

A::j () {A::v} {A::w}

A::l() {A::v}

B::h() {B::x}

B::i() {B::x}

C::i()

C::j() {C::y} {C::y}C::l() {A::v}

Consider what happens when an overriding method has a differentdef-set than the overridden method

def-use

def-use DUanomaly

DUanomaly

B

+h ()+i ()

-x

A-u-v-w

+h()+I()+j()+l()

C

+i ()+j ()+l ()

-y


Polymorphism Headaches (Yo-Yo)A

+d ()+g ()+h ()+i ()+j ()+l ()

A d() j()g() h() i() l()implicitimplicitimplicitimplicitInstantiated

type

B k()h() i()

C i() j() l()

Object is of type AA::d ()


Polymorphism Headaches (Yo-Yo)A

+d ()+g ()+h ()+i ()+j ()+l ()

A d() j()g() h() i() l()implicitimplicitimplicitimplicit

B+h ()+i ()+k ()

B h() i() k()implicit

j()i()

Instantiated

type

Instantiated

type

B k()h() i()

C i() j() l()

A d() g() h() l()

C i() j() l()

Object is of type BB::d ()


Polymorphism Headaches (Yo-Yo)

C+i ()+j ()+l ()

B+h ()+i ()+k ()

A+d ()+g ()+h ()+i ()+j ()+l ()

C j()i() l()

j()i()

i() k()

B h() i() k()implicit

j()i()

A d() j()g() h() i() l()implicitimplicitimplicitimplicitInstantiated

type

B k()h() i()

C i() j() l()

A d() g() h() l()

Instantiated

typeC i() j() l()

A d() g() h() l()

B h()

Instantiated

type

Object is of type C, C::d ()


Potential for Faults in OO Programs

• Complexity is relocated to the connections among components

• Less static determinism – many faults can now only be detected at runtime

• Inheritance and Polymorphism yield vertical and dynamicintegration

• Aggregation and use relationships are more complex• Designers do not carefully consider visibility of data and

methods


Testing OO Software

1) Intra-method testing: Testing individual methods within classes

2) Inter-method testing: Pairs of methods within a class are tested in concert

3) Intra-class testing: Testing a single class, usually using sequences of method calls

4)4) InterInter--class testingclass testing: More than one class is tested at : More than one class is tested at the same time (integration)the same time (integration)


Coupling-Based Testing

• Test data and control connections

• Derived from previous work for non-procedural programs

• Based on insight that integration occurs through couplings among software artifacts

CallerF x = 14

y = G (x)

print (y)

print (a)

b = 42

return (b)

G (a)Callee

last-def-before-return

last-def-before-call

first-use-in-callee

call site

first-use-after-call


Polymorphic Call Set

Set of methods that can potentially execute as result of a method call through a particular instance contextpcs(o.m) = {W::m, Y::m, X::m}

public void f ( W o )

{

…

j o.m();

…

l o.l();

…

k o.n();

}


Coupling Sequences

• Pairs of method calls within body of method under test:– Made through a common instance context– With respect to a set of state variables that are commonly

referenced by both methods– Consists of at least one coupling path between the two method

calls with respect to a particular state variable• Represent potential state space interactions between

the called methods with respect to calling method

• Used to identify points of integration and testing requirements


Example Coupling Sequence

o bound to instance of W

h def (o)

Client f

i o.m()

j o.l()

k o.n()n ()

use (W::v)use (W::u)

m ()

def (W::v)

l ()

def (W::u)

W-v :-u :

+m()+n()+l()

Z

+m()+n()

X-x :+n()

Y-w :+m()+l()

Couplingsequence with respect to W::v

Couplingsequence with respect to W::u


Example Coupling Sequence (2)

o bound to instance of Z

h def (o)

Client f

i o.m()

j o.l()

k o.n()

m ()

def (Z::x)

n ()

use (Z::x)use (Z::x)

l ()

def (W::u)

Couplingsequence with respect to Z::x

W-v :-u :

+m()+n()+l()

X-x :+n()

Y-w :+m()+l()

Z

+m()+n()

-x :


Testing Requirements

• Want to test the ways in which f can interact with instance bound to object o:– Interactions occur through the coupling sequences

• Need to consider the set of interactions that can occur:– What types can be bound to o?– Which methods can actually execute? (polymorphic call sets)

• Test all couplings with all possible type bindings


All-Poly-Coupling-Defs-and-Uses

Every coupling path must be executed for every member of the type family defined by the context of a coupling sequence and forevery coupling variable in the sequence

• Handles inheritance and polymorphism• Takes definitions and uses of variables into account

But Web software has much more than inheritance But Web software has much more than inheritance and polymorphism and polymorphism ……


Difficulties with Analyzing and Testing Web Software

WebSoftware


New Essential Problems of Web Site Software

1. Web site software is extremely loosely coupled– Coupled through the Internet – separated by space– Coupled to diverse hardware and software applications– Web services will dynamically couple with other services

after deployment – without human intervention!

2. Web software services offer dynamically changing flow of control– Web pages are created by software on user request– The interaction points (forms, buttons, etc.) vary depending on state: the

user, previous choices, server-side data, even time of day– Examples: amazon.com, netflix.com, washingtonpost.com


Problem 1: Loosely Coupled

Web-based systemsConnected with network protocols

Loose and extremely loose coupling

Traditional systemsConnected by calls and message passing

High and moderate coupling

How can we ensure the reliability of this type of system?How can we ensure the reliability of this type of system?


WebPics

Welcome Yong-Rae Kwon!

Search

Recommended Movies

X XXXXX

Examine queue

View account

(Warning: Queue empty)

WebPics

whan young hap ni da Byoung-Ju Choi!

Search

Recommended Movies

A C DB

Examine queue

View account

Frequent customer bonus

Problem 2: Dynamic Flow of Control

How can we ensure the reliability of this type of system?How can we ensure the reliability of this type of system?


Problems for PractitionersFun for Researchers

• How to write requirements and specifications for web software services?

• How to design and model web software?• How to test loosely coupled software whose control

flow is determined dynamically?• How to safely and reliably perform maintenance?• How can existing software development processes be

adapted to this new type of software?


Integration Testing ofLoosely Coupled Software

IntegXMLTest


Testing Loosely Coupled Software

• Unit and module may not be affected

• Software integration, however, is completely different

• Previous integration testing strategies focused on couplings among the software components– Couplings were analyzed by looking at implementation– But source is often not available for web components!

• The essential part of integration is how the software components communicate


Communicating in Loosely Coupled Systems

• In traditional software, components and component authors negotiate details about structure of data that is exchanged– Types– Formats– Order

• This is difficult in extremely loosely coupled software– Authors cannot communicate– Components may not know who they interact with until execution time

• XML is a standard that allows data items to be exchanged:– Independent of type– Without regard to format– In arbitrary order

• XML tags allow components to infer information about type, format and order


The Problem Context

Web Component A Request XML message from A Web Component B

HTTP

• Heterogeneous web software interactions• Components communicate by using an agreed upon standard

for data exchange – XML

TestingTesting: A technique to check interaction:: A technique to check interaction:

Interaction Data Diversity (IDD)Interaction Data Diversity (IDD)

Response XML message from B

Example XML Document and its DTD

<?XML version = "1.0">

<AUTHORIZED_USERS>

<AUTHORIZED_USER>

<USER_ID>steffi</USER_ID>

<PASSWORD>sso</PASSWORD>

</AUTHORIZED_USER>

</AUTHORIZED_USERS>

<!ELEMENT AUTHORIZED_USERS (AUTHORIZED_USER) * >

<!ELEMENT AUTHORIZED_USER (USER_ID, PASSWORD)>

<!ELEMENT USER_ID (#PCDATA)>

<!ELEMENT PASSWORD (#PCDATA)>


Interaction Data Diversity (IDD) Analysis

• Traditional mutation analysis:– modifies the program source– is primarily used for unit and module testing– uses test cases that are input values to program units

• Interaction data diversity:– modifies the of web service data interactions (the messages)– is used for integration testing of web software components– test cases are XML messages between web software

components

• IDD creates test cases as XML messages from the XML grammar description (DTDs)

Current Constraints

• notMemberOf : Data value not in the valid set of values

• relationOf : Data values related with each other through a relational operator (>, <, =, , , )

• mapOf : Data value meet a specific syntax mapping


IDD Illustration

Execute T

Interaction I

response

request WebComponent

C2

WebComponent

C1

InteractionData Diversity

System(IDDS)

Interaction DataDiversity Operators

request

response

WebComponent

C1

WebComponent

C2

Execute TDD

InteractionsI1, I2, …DD

InteractionsI1, I2, …DD

InteractionsI1, I2, …

Generates

ISM Example<-- DTD for Computer Accounts --><!ELEMENT USER_ID (#PCDATA)><!ELEMENT PASSWORD (#PCDATA)>

<!ELEMENT AUTHORIZED_USERS (AUTHORIZED_USER) * ><!ELEMENT AUTHORIZED_USER (USER_ID, PASSWORD)>

<!ELEMENT SIGNON_REQUEST (USER_ID, PASSWORD)>

<!ELEMENT SIGNON_RESPONSE (USER_ID)><!ATTLIST SIGNON_RESPONSE AUTHENTICATION (ALLOW | DENY)

#REQUIRED>


Example: Authentication

Web Component A Request XML message from A

Response XML message from B

HTTP

Web Component B

• User login authentication for A is provided by B• A sends XML message requesting B to authenticate a user• B responds to A with an XML message

Request Message<SIGNON REQUEST>

<USER ID>steffi</USER ID><PASSWORD>sso</PASSWORD>

</SIGNON REQUEST>

Response Message<SIGNON RESPONSE

AUTHENTICATION="ALLOW"><USER ID>steffi</USER ID>

</SIGNON RESPONSE>

Use notMemberOf to Modify Request Message

(Joyce, sso) is notMemberOf authorized user database

Modified Request Message

<SIGNON REQUEST><USER ID>Joyce</USER ID><PASSWORD>sso</PASSWORD>

</SIGNON REQUEST>

Response Message

<SIGNON RESPONSE AUTHENTICATION="DENY"><USER ID>Joyce</USER ID>

</SIGNON RESPONSE>

Response is different from that of original message,Response is different from that of original message,so software succeedsso software succeeds


Use relationOf to Modify Request Message

“sso” != “ysma”


<SIGNON REQUEST><USER ID>Steffi</USER ID><PASSWORD>ysma</PASSWORD>

</SIGNON REQUEST>

Response Message

<SIGNON RESPONSE AUTHENTICATION="ALLOW"><USER ID>Steffi</USER ID>

</SIGNON RESPONSE>

Response is same as that of original message,Response is same as that of original message,so test has found a failure!!so test has found a failure!!

Use mapOf to Modify Request Message

USER ID is alphanumeric


<SIGNON REQUEST><USER ID>;Steffi&</USER ID><PASSWORD>sso</PASSWORD>

</SIGNON REQUEST>

Response Message

<SIGNON RESPONSE AUTHENTICATION=“DENY"><USER ID>Steffi</USER ID>

</SIGNON RESPONSE>

Response is different from that of original message,Response is different from that of original message,so software succeedsso software succeeds


Testing the Dynamic Flow of Control

AtomicSections


Testing Software with Dynamic Flow of Control

• Dynamic web pages are created when users make requests• Technologies include servlets:

– A request is sent with a list of parameters– The servlet performs some computation, perhaps using other components

such as databases and beans– The servlet returns results to the client as a web page– The web page includes Javascript, buttons, and form fields

• The program the user interacts with changes dynamically– Javascript, buttons, and fields on the client can change anytime– The software components the servlet interact with can change any time

Control flow graphs cannot be created Control flow graphs cannot be created staticallystatically


Variant and Invariant Portions

• The web pages can vary

• From the users' perspective, each user can interact with a different program

• Unlike traditional software, we cannot determine potentialflows of control before execution

• But all the pieces of the web pages and the programs are all contained in the software

• The pieces are invariant, but the way they are combined varies

Atomic SectionsA section of HTML including scripts) such that if part of the seA section of HTML including scripts) such that if part of the section is ction is sent to a client, the entire section is (sent to a client, the entire section is (all or nothing propertyall or nothing property))

• A static web page file (HTML and Javascript)or

• A piece of a web page file generated by a program– Can be an HTML section

or– HTML with a static structure and content variables– A content variable provides data to the web page, but not

structure

• Can be extracted from code


Atomic Section ExamplePrintWriter out = response.getWriter();

Atomicsections

out.println ("<HTML>")

out.println ("<HEAD><TITLE>" + title +“ </TITLE></HEAD>)"

out.println ("<BODY>")

P1 =

for (int i=0; I < myVector.size(); i++)

if (myVector.elementAt(i).size > 10)

out.println("" + myVector.elementAt(i) +

"");

P2 =

else

out.println ("" + myVector.elementAt(i) + "");P3 =

out.println ("</BODY></HTML>");P4 =

out.close();Content

variables


Composite Sections

A composite section is composed of atomic sections combined with the following rules:

• Basis: p is an atomic section

• Sequence (p p1 p2) : p is composed of p1 followed by p2

• Selection (p p1 p2) : p is either p1 or p2, but not both

• Iteration (p p 1*) : server selects repeated copies of p1

• Aggregation (p p1 {p2} ) : p2 is contained inside p1


Modeling Web Pages

• Web pages are modeled as composite sections combined with the three rules and regular expressions

• The previous example produces:p p1 (p2 | p3)* p4

• The composite section models for web software components can be produced automatically

• Web software components also communicate with each other, which is modeled with transitions


Transitions Among Web Components

There are three types of transitions:

1. Link Transition (p q1 | q2 | … ) : A transition from one composite section to another

2. Composite Transition (s = p1 | p2 | … ) : A transition from a servlet to one of the composite sections it produces

3. Operational Transition (p q) : A transition that the user imposes on the software, using the back button, refresh, or URL rewriting


Web Service Application Model

• A web application is modeled as a quintuple { S, C, T, AS, CS }:– S: Start page– C: Set of composition rules for each component– T: Set of transition rules among the components– AS: Set of atomic sections– CS: Set of composite sections

• Tests are created by deriving sequences of transitions among the web software components and composite sections


Example

• GradeServlet login reports grades to students– S = { index.html }– C = {GradeServlet = p1 ((p2 p3*) | p4) p5,

SendEmail = …}– T = { S GradeServlet,

GradeServlet.p4 (SendEmail | GradeServlet) }• Test derivations

– S GradeServlet p1 p2 p3 p5– S GradeServlet p1 p4 p5– S GradeServlet p1 p4 p5 SendEmail …– S GradeServlet p1 p4 p5 Previous S

GradeServlet p1 p4 p5


System Testing Web Software Applications

SystemTest


Application Testing

• Three aspects need testing:1. Single functions local to web pages2. Navigation among web pages3. State dependent behavior on web pages

• Web applications are modeled with finite state machines– FSMs can lead to a state space explosion


Modeling Web Applications

• Partitioning Web Applications – Subsystems – FSM for each subsystem– Major functions

• FSMs for each subsystem

• Test sequences for combined subsystems


Logical Web Pages

• Logical web page (LWP): A portion of a web page that offers a specific service to the users– Login section– Single form– Search box

• Individual web pages may contain several LWPs

• LWPs are the fundamental element of our model


Partitioning Web Applications

• Components: Sets of web pages that implement a single logical function

• Subsystems: Groups of components and individual web pages that together implement a major function

• Components and subsystems are derived from:– User level functionalities– Navigation layout– Couplings among components

• Clusters: Collections of one or more components that together form a cohesive functional unit


Cluster Finite State Machines

• Cluster Finite State Machines (CFSM): A finite state model of the behavior of the cluster

• Nodes: derived from subsystems and components

• Edges: represent navigation among pages

• Result: A collection of independent finite state machines that are:– Small enough to generate test sequences from– Clearly define the information that propagates among CFSMs


Aggregate Finite State Machines

• Aggregate Finite State Machines (AFSM): A finite state model of the high-level aggregate behavior of the web application

• Nodes: Each cluster is one node

• Edges: navigation points among clusters

• Result: A high level abstract FSM that models the overall behavior of the web application


Testing CFSMs & AFSMs

• Traditional graph testing criteria are applied:– Transition coverage: Find values that cause each transition in

each FSM to be covered– Transition-pair coverage: Find values that cause each pair of

transitions to be covered

• Applied at both the cluster and aggregate FSM level


Future Methods and Technologies

Future???


Technology Predictions for 2010

• 1990: Internet, Command-based email, FTP• 2000: A Web-based world

It would be foolish to make 10 year predictions about the web!


Needs for Modeling & TestingWeb Applications

• High level design patterns for Web applications– Model-View Controller (MVC) is helpful but more are needed

• Modeling languages for describing Web applications– UML is insufficient

• Analysis techniques that handle:– Session data– Dynamic integration with web services

• Techniques for performing maintenance and regression testing

• Best practices for detailed design and implementation• Better education among developers!


Open Issues in Web Testing

• Precise criteria for generating transition sequences• Translating sequences to inputs is hard:

– User inputs– State on the server

• Checking the results requires checking the output web pages and changes to state on the server

• Testing session management issues• Interactions among multiple users• Guidelines for developing safe inheritance

hierarchies• Guidelines or standards for safe use of polymorphism


Relevant Papers on My Web Site

http://www.ise.gmu.edu/~ofut/

• OO:www.ise.gmu.edu/~ofut/rsrch/abstracts/integ.html

• Web:www.ise.gmu.edu/~ofut/rsrch/abstracts/web.html

testing web software applications

Documents