SIMULATING ERRORS IN WEB SERVICES

International Journal of Simulation: Systems, Sciences and Technology 2004

Nik Looker, Malcolm Munro and Jie Xu

Outline

Introduction Quality of Service Fault Injection Fault Model Test Case Conclusion and Future Work

Introduction

Web Service technology is a key factor in the success of any e-Science or e-Commerce project.

High Quality of Service (QoS) to allow the production of highly reliable systems.

Fault injection is a well-proven method of assessing the dependability of a system

Introduction

This fault injection method is a modified version of Network Level Fault Injection.

This method differs from standard Network Level Fault Injection techniques

Fault injector decodes the SOAP messages and can inject meaningful faults into a message

This enables API-level parameter value modification to be performed as well as standard Network Level Fault Injection.

Quality of Service

Availability the quality aspect of whether a Web Service is present and

ready for use. Accessibility

the quality aspect that represents the degree the Web Service is capable of servicing a request.

Integrity the quality of the Web Service maintaining the correctness

of any interaction. Performance

the quality aspect that is defined in terms of the throughput of a Web Service and the latency.

Quality of Service

Reliability the quality aspect that represents the capability of

maintaining the service and service quality. Regulatory

the quality aspect that the service corresponds to rules, laws, standards and specifications.

Security the quality aspect that defines confidentiality for parties

using a service

Fault Injection Techniques

Network Level Fault Injection Corruption, loss or reordering of network messages at the n

etwork interface. Runtime injection technique that injects faults into captured

network packets. Software Implemented Fault Injection (SWIFI)

instrumenting the operating system protocol stack Perturbation

Modify program states without mutating existing code statements.

testing fault tolerance mechanisms

Fault Injection Techniques

Corruption of Memory Space Platform Specific

Code Mutation Certification Problems because code changed

Syscall Interposition Platform Specific

Fault Injection Tools

Ferrari (Fault and Error Automatic Real-Time Injection) uses a system based around software traps

to inject errors into a system. traps are activated by either a call to a

specific memory location or a timeout. FTAPE (Fault Tolerance and

Performance Evaluator) inject faults into memory, registers and disk

accesses

Fault Injection Tools

Xception processor’s exception handling capabilities are us

ed to trigger fault injection. DOCTOR (IntegrateD sOftware Fault InjeCTi

On EnviRonment) allows injection of memory and register faults, and

network communication faults. combination of timeout, trap and code modificatio

n.

Fault Injection Tools

Orchestra a script driven fault injector which is based around

Network Level Fault Injection.

WS-FIT

Web Service – Fault Injection Technology Method and tools implement mechanism of fault inje

ction and also provide a framework for the creation and execution of test cases

Fault injector that allows network level fault injection Dependability Assessment of SOAP based SOA

SOAP Based WSDL Defined Interfaces Combined with conventional specification

Fault Model

Fault Types that can effect a SOAP based system: Physical Faults

Effecting memory or processor registers Software Faults

Programming errors and design errors Resource Management Faults

Memory leakage and exhaustion of resource Communication Faults

Message deletion, duplication, delay, reordering or corruption Life-Cycle Faults

Premature object destruction through starvation of keep-alive messages and delayed asynchronous responses

Enhanced Fault Model

Failure Modes

Crash of a service Crash of a hosting web server (or the host itself) Hang of a service Corruption of input data Corruption of output data Duplication of messages Omission of messages Delay of messages

Why WS-FIT testing?

Particularly interested in Assessing Quality of Service Assessing Fault Tolerance of Systems Developing Methods and Tools

So that we can Detect defects in platform code and design Gather dependability metrics on platforms to allow

comparisons Use WS-FIT to test individual Web Services

WS-FIT

Consists of 3 parts: Hook code in SOAP

Library Fault Injector Framework Script

Trigger Injection Results

Instrumented SOAP API

Trigger

Script

Results

LogFIT

Original SOAPProtocol Stack

Service

Send Hook

Original Sending Layer

Intercept Retransmit

SOAP API Hooks

SOAP API Hooks One for sending messages. One for receiving messages.

Hooks consisted of simple socket code To pass messages to fault injector. Receive (possibly) modified message from

injector.

Instrumenting a Service

Instrumented SOAP API

Trigger

Script

Results

LogFIT

Original SOAPProtocol Stack

Service

Send Hook

Original Sending Layer

Intercept Retransmit

What the Framework Provides Logging Function (XML format for easy analysis)

Detects faults and logs faults Logs injected fault packets

Trigger and Injection mechanism: Trigger on Message Type and parameter. Inject Fault through User Script.

Implemented Framework implemented in Java for portability Scripts implemented in Python

Executed in the JVM by Jython

User Script

Result class One instance of this for duration of test. Used to store static results. Currently only tag count but will be enhanced in later

experiments. Trigger Class

One instance per packet. Returns Boolean value to indicate if injector class should be run. Use data from result class to determine if the correct point in

message stream has been reached. Injector Class

One instance per packet Inject fault into packet

WS-FIT

GUI

The GUI provides An easy to use environment to create and

execute tests Ability to create a skeleton test script from WSDL

definitions Populate tests through user defined Fault Model Execution of test scripts from tool Real-time visualization of RPC message

parameters

Quality of Service

Experiment to demonstrate WS-FIT can be used to assess QoS criteria

WS-FIT will be used to modify the performance of a system by injecting a latency (without code modification being required)

Test Scenario

The test system implements a simple, self regulating, heater unit.

The hardware used is simple and relies on the software drivers to prevent failure states being reached.

Under normal operation the ‘Unit’ will give a logarithmic temperature rise so the Controller uses timers to modify to a linear temperature rise.

Will be used in a chemical experiment A substance must be heated to 60°C Must reach 60ºC in a specified period of time Must have a linear temperature rise

The test will introduce a latency to decrease the throughput and hence the performance of the system.

Test System

InstrumentedSOAP API

Thermocouple

SOAP API

Heater Coil

SOAP API

Controller

Client

SOAP API

Trigger

Script

Results

Log

Send Hook

Fault Injector

Natural Operation of Heater

Measured Temperature Without Injected Faults

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Time (seconds)

Tem

pera

ture

(degre

es C

)

Non Linear Client

Linear Client

Model

Model comparied to Linear Client

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Time (seconds)

Tem

pera

ture

(degre

es C

)

Model

Linear Client

Applying a Fault Model to aParameter

Injecting Latency

Injecting a Latency Into the System

-10

0

10

20

30

40

50

60

0 100 200 300 400 500 600 700 800

Time (seconds)

Tem

pera

ture

(d

eg

rees

C)

Linear Client with Latency

Linear Client without Latency

Summary

WS-FIT can be effectively used to: Introduce SOAP protocol faults

Introduce both random and non-random protocol faults Assess Quality of Service

Such as throughput modification

WS-FIT Relatively non-invasive Can be used to assess both middleware and individual services Gives real-time feedback through visualization of RPC parameter

s

http://www.wsfit.org/