Composing a Framework to Automate Testing of

Operational Web-Based Software

Reham Alhejiali, Chris Cederstrom, Ranjitha Kashyap

Motivation

Web based application:Example: E-commerceCharacteristic:1.Short time to market.2.Extensively used.3.High reliability expected.4.Demand for continuous availability.5.Large user community.

Goal

-To address the need for an automated cost- effective testing strategy for web application.

- To investigate the scalability and evolving the tests suites to automatically adapt as the application operation profile changes.

Outline

1.General framework for testing web-based application.

2. State of the art.

3.The testing framework .

4. Demonstrates the use of the framework with case study.

5. conclusion.

``````````````````````````````

Figure 1. General Framework for Testing Web-based Applications

State of the Art

• User-Session-based testing

-Test cases: - Created by capturing real user interactions and utilizing the user sessions as

representative test cases of user behavior. -Testers could use the collected user sessions during maintenance to enhance the

original test suite. -The user sessions provide the test data that represents usage not anticipated during

earlier testing stages and that evolves as operational profiles change.

-Webking and Rational Robot

Generated test cases may not be adequate.

-Elbaum et al(2003)

Reduction technique employed is not scalable.

continue:

• Functional testing:

-link, form testers

-Test the functionality of application.

• Program-based testing

-Test cases: based on the data flow between objects in the model.

Liu et al (2000)

-WATM

-Test based on the data flow between objects in the model.

-ICFG

Di Lucca et al(2002)

-Developed a web application model and set of tools for the evaluation and automation of testing web application.

-Test case generation not completely automated.

Testing Framework and Initial Prototype

Test Case Generation

• Using a Web server to log user sessions for test case generation

• Modified the AccessLog class of the Resin web server to log user sessions in a specified format:o IP addresso Timestampo Requested URL o Cookieso Name-value of GET/POST requestso Referrer URL

• Each user session is a collection of user requests in the form of URL and name-value pairs.

Replay Tool & Coverage Analysis

• Replaying user sessions are viewed using wget for each entry in the user session log

• These requests will emulate user interactions and input

• Cookies are used to recognize the user and sessions

• Coverage analysis is to evaluate the effectiveness of the generated test suite

• Coverage is calculated by Clover

• Clover returns the analysis of statements and methods used

Coverage Analysis with Clover

Clover: http://www.atlassian.com/software/clover/overview

Oracle and Fault Detection

• Oracle generate expected results using a properly working application

• Run the user sessions on the working application and new version and store results in database

• Results of working application is compared to output from test runs to confirm validity and detect faults

Case Study

• A medium sized web application like bookstore was considered for test

• Front end had 11 jsps and classes

• LOC:9,748, Methods: 385

• Collected 123 User sessions

WEBAccessLog

User

User Session Log

0.197.37.159 [03/Feb/2004:16:14:05 −0500] GET /apps/bookstore/Default.jsp ] −−cookies=off] −

10.197.37.159 [03/Feb/2004:16:16:27 −0500] GET /apps/bookstore/Registration.jsp ] −−cookies=off −−header "Cookie:JSESSIONID=a7mpavbwGTf6"] http://dwalin.cis.udel.edu:8080/apps/bookstore/Default.jsp

10.197.37.159 [03/Feb/2004:16:17:22 −0500] GET /apps/bookstore/Registration.jsp

?member_login=bobmason& member_password=14921492&member_password2=14921492& first_name=bob&last_name=mason& email=bobmason%40udel.edu&address=&phone=& card_type_id=&card_number=& FormName=Reg&FormAction=insert& member_id=&PK_member_id= ]

−−cookies=off −−header "Cookie:JSESSIONID=a7mpavbwGTf6"] http://dwalin.cis.udel.edu:8080/apps/bookstore/Registration.jsp

Test case generation and execution

AccessLog

Access Log Parser

10.197.37.159: GET.apps.bookstore.Default.jsp GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Default.jsp GET.apps.bookstore.Login.jsp POST.apps.bookstore.Login.jsp

;

10.82.161.133: GET.apps.bookstore.Default.jsp GET.apps.bookstore.images.icon_reg.gif GET.apps.bookstore.images.icon_home.gif GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Default.jsp

;

Concept Analysis Tool

objects[001]: 0.82.161.133

attributes[001]: GET.apps.bookstore.images.icon_home.gif

GET.apps.bookstore.images.icon_reg.gif

Test Suite Reducer

Replay Tool

Example Of Using Clover

Coverage Report for Bookstore

Seeded Fault- Technique to measure Test Coverage

- Known bugs are randomly added to the source code

- During testing, the percentage of known bugs not found indicates the real bugs that remain.

Fault Description Example

Logic error in control or data flow //String bookDir = bookProp.getProperty(title)bookDir = null;

Link error in hyperlink location String link1 = "registration.jsp"String link2 = "shoppingCart.jsp"<a href="<%=link2>">Registration</a>

Datastore error in code that interacts with data store

/* Select book_id, name, quantity from store where book_id=123 */Select title, name, quantity from store where book_id=123

Cost and Scalability

Component Execution Time Component Output Output Space

Test Case Generation 19s Reduced Test Suite 1MB

Replay Tool Execution

16m56s Pages Retrieved 152 MB

Coverage Analysis 1m52s Coverage Reports/Clover Database

3.5MB

Test Oracle 14s Diff Output 0.5M

Conclusion

• Discussed the framework for automating the testing process for web-based software

• Focuses on usability, costs, and scalability to evolving the test suite

• Used as one of the measures of fault detection for future evaluation

• Code Coverage