generating interface grammars from wsdl for automated verification of web services (talk @ icsoc...

Hallé, Hughes, Bultan, Alkhalaf

Sylvain Hallé, Graham Hughes,Tevfik Bultan, Muath Alkhalaf

Generating Interface Grammarsfrom WSDL for AutomatedVerification of Web Services

University of CaliforniaSanta Barbara, USA


A motivating scenario



$



?



express checkoutexpress checkout

?




!




!

?


More concretely


More concretely

PPPP


More concretely

<SetExpressCheckoutRequest>

</SetExpressCheckoutRequest>

<PaymentDetails

</PaymentDetailsItem><PaymentAction> </PaymentAction>

</PaymentDetails>

> 1234 50 456 1 Sale

<Token> </Token><OrderTotal> </OrderTotal><PaymentDetailsItem><Number> </Number><Quantity> </Quantity>

PPPP


More concretely



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale


<SetExpressCheckoutResponse>

</SetExpressCheckoutResponse> 1234<Token> </Token>

PPPP


More concretely



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale




<GetExpressCheckoutDetails>

</GetExpressCheckoutDetails>

1234 abcd<Token> </Token><PayerID> </PayerId>

PPPP


More concretely



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale







<GetExpressCheckoutDetailsResponse>

</GetExpressCheckoutDetailsResponse>

1234 abcd ...

<Token> </Token><PayerID> </PayerId><PaymentDetails> </PaymentDetails>

PPPP


More concretely



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale









1234 abcd ...


<DoExpressCheckoutPaymentRequest>

</DoExpressCheckoutPaymentRequest>

1234 abcd Sale

<Token> </Token><PayerID> </PayerId><PaymentAction> </PaymentAction>

PPPP


More concretely



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale









1234 abcd ...




1234 abcd Sale


<DoExpressCheckoutPaymentResponse>

</DoExpressCheckoutPaymentResponse>

1234 7890

<Token> </Token><PaymentInfo>

</PaymentInfo><TransactionID> </TransactionId>

PPPP


More concretely



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale









1234 abcd ...




1234 abcd Sale


<DoExpressCheckoutPaymentResponse>

</DoExpressCheckoutPaymentResponse>

1234 7890

<Token> </Token><PaymentInfo>

</PaymentInfo><TransactionID> </TransactionId>

PPPPMany ways to make the exchange fail


More concretely



More concretely


<Name> </Name>


<PaymentDetails>

<FooBar> </FooBar> </PaymentDetails>

1234 50 John Doe ...

bazbaz

<Token> </Token><OrderTotal> </OrderTotal><CustomerAddress>

</CustomerAddress>



More concretely


<Name> </Name>


<PaymentDetails>


1234 50 John Doe ...

bazbaz


</CustomerAddress>

PPPP

unexpectedelement

Many ways to make the exchange fail


More concretely


<Name> </Name>


<PaymentDetails>


1234 50 John Doe ...

bazbaz


</CustomerAddress>

PPPP

PaymentActionmissing



More concretely


<Name> </Name>


<PaymentDetails>


1234 50 John Doe ...

bazbaz


</CustomerAddress>

PPPPMany ways to make the exchange

1. Messagewith wrong

fail

structure

PaymentActionmissing


More concretely



More concretely


<Number> </Number><Quantity> </Quantity>


<PaymentDetails>

</PaymentDetailsItem> <PaymentAction> </PaymentAction> </PaymentDetails>

<Token> </Token> <OrderTotal> </OrderTotal> <PaymentDetailsItem> 5

123450

4 61

Sale



More concretely




<PaymentDetails>



123450

4 61

Sale





More concretely




<PaymentDetails>



123450

4 61

Sale



PPPP

should beequal



More concretely




<PaymentDetails>



123450

4 61

Sale



PPPP

should beequal

Many ways to make the exchange

2. Messagewith goodstructure

fail


More concretely




<PaymentDetails>



123450

4 61

Sale



PPPP

should beequal


2. Messagewith goodstructure but

values

fail

wrong


More concretely



More concretely



1234 abcd Sale




More concretely



1234 abcd Sale


PPPP

the first occurrenceof a token must be ina SetExpressCheckoutrequest



More concretely



1234 abcd Sale


PPPP



3. Messagewith goodstructure andgood values

fail


More concretely



1234 abcd Sale


PPPP



3. Messagewith goodstructure andgood valuesin the wrong

fail

sequence


More concretely



1234 abcd Sale


PPPP



3. Messagewith goodstructure andgood valuesin the wrong

fail

sequence

???


More concretely

???


We want to do, ...as automatically as possible

Proposed solution



�

Proposed solution



...impersonate the client, send test sequences to the service

�

Proposed solution




DRIVER

�

Proposed solution




! Check if service does what we expect/understand

DRIVER

�

Proposed solution





DRIVER

� �

Proposed solution





...impersonate the service, generate responses to the client

DRIVER

� �

Proposed solution






DRIVER STUB

� �

Proposed solution






! Environment closedmodel checking possibleÞ

DRIVER STUB

� �

Proposed solution


Related work


Related work

Stub/driver


Related work

Structure

Stub/driver

Constraints

Values

Sequence


Related work

Structure

Stub/driver

Constraints

Values

Sequence

soapUI with "mock web services"


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Java API for XML Web Services; IBM Web Service Validation Tool (validation only)


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Cacciagrano et al., WS-FM 2006(validation only)


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Bai et al., SOSE 2005Bartolini et al., ICSOC 2008


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Hallé & Villemaire, CAV 2009(validation only)


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Hughes & Bultan, ISSTA 2007


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Hughes & Bultan, AFM 2007


Related work

Structure

Stub/driver

Constraints

Values

Sequence

Hughes & Bultan, TAV-WEB 2008


Related work

Structure

Stub/driver

Constraints

Values

Sequence

The present paper


Tool architecture


Tool architecture

Control-flowconstraints


Tool architecture

Interfacegrammar

Control-flowconstraintsG


Tool architecture

WSDL

Interfacegrammar



Tool architecture

WSDL

Interfacegrammar

Control-flowconstraintsTranslator G


Tool architecture

WSDL

Interfacegrammar

Control-flowconstraintsTranslator

Interfacecompiler

G


Tool architecture

WSDL

Interfacegrammar


Servicedriver

Translator

Interfacecompiler

G


Tool architecture

WSDL

Interfacegrammar


Servicedriver

Serverstub

Translator

Interfacecompiler

G


Tool architecture

WSDL

Interfacegrammar


Servicedriver

Serverstub

Translator

Interfacecompiler

automated

G


Recall: grammar

Interface grammar


Recall: grammar

Interface grammar

A login S | S login®B logout S | S logout®

S A B | e®


Recall: grammar

Interface grammar

production rule


S A B | e®


Recall: grammar

Interface grammar

production rule

start symbol


S A B | e®


Recall: grammar

Interface grammar

production rule

start symbolnonterminal


S A B | e®


Recall: grammar

Interface grammar

production rule


terminal


S A B | e®


Recall: grammar

Interface grammar

production rule


terminal

empty symbol

S A B | e®A login S | S login®B logout S | S logout®


Recall:

Sufficient to express : "every login can be matched

to exactly one logout in the future"

grammar

control-flowconstraints

Interface grammar

production rule


terminal

empty symbol



Two uses for the grammar:

1. that a given sequence belongs to that grammarCheck

Interface grammar

What about login login logout logout ?

SA B

login S Blogin A B B

...login login logout logout



Two uses for the grammar:

2. sequences belonging to that grammarGenerate

Interface grammar


SA B

login S Blogin B

...login logout


Interface grammar

DRIVER


Interface grammar

G

DRIVER


Interface grammar

G

DRIVER

?


Interface grammar

G

DRIVER

1


Interface grammar

G

DRIVER

2


Interface grammar

G

DRIVER

2

?2 ÎG


Interface grammar

G

DRIVER

2

üû/


Interface grammar

G

STUB


Interface grammar

G

STUB

1


Interface grammar

G

STUB

1

?1 ÎG


Interface grammar

G

STUB

1

üû/


Interface grammar

G

STUB

?

1


Interface grammar

G

STUB

1

2


Interface grammar

G

STUB

2

2


Interesting consequence:

We use the same grammar for:

! Checking messages! Generating messages

Both to produce:

! A stub! A driver

G

Interface grammar

G


Sufficient to express : "every login can be matched

to exactly one logout in the future"

control-flowconstraints

Interface grammar



Extensions to classical grammar

Interface grammar




Interface grammar


argument





Interface grammar


argument

the same x





Interface grammar


argument

the same x

®

checks that x is an intgenerates an int x





Interface grammar


argumentsemantic predicate

semantic action

the same x

®






Sufficient to express control-flow : "every login with argument x

can be matched to exactly one logout withargument x in the future"

and valueconstraints

Interface grammar


argumentsemantic predicate

semantic action

the same x

®





Example: generate a sequence

Interface grammar


® int(x) A(x) B(x) | e

Sint(x) A(x) B(x)

A(1) B(1)!login(1) S B(1)

!login(1) S !logout(1)!login(1) int(x) A(x) B(x) !logout(1)

S



Example: generate a sequence

Sint(x) A(x) B(x)

A(1) B(1)!login(1) S B(1)

!login(1) S !logout(1)!login(1) int(x) A(x) B(x) !logout(1)

Interface grammar


® int(x) A(x) B(x) | eS



Interfacegrammar


Servicedriver

Serverstub

Interfacecompiler

Interface grammar

G


Interfacegrammar


Servicedriver

Serverstub

Interfacecompiler

Interface grammar

still missingmessage structure

G


We have production rules to check/generate data types...

primitive

Interface grammar






How can we get similar rules for data types?

primitive

composite

Interface grammar




!login(x) áá???ññ®



How can we get similar rules for data types?

primitive

composite

Interface grammar




!login(x) áá???ññ®WSDL


From WSDL to interface grammar rules

WSDL provides a (machine-readable) definition of a message's structure




Short-hand notation: (MSL; Brown et al., WWW 2001)

Model Schema Language

SetExpressCheckoutRequest[ Token[ ] , PaymentDetails[ OrderTotal[ ], PaymentDetailsItem[ Number[ ], Quantity[ ] ] ]]

string

int

intint

{0,1}

{1,8}{0,8}




Short-hand notation: (MSL; Brown et al., WWW 2001)

Model Schema Language



<PaymentDetails


</PaymentDetails>

> 1234 50 456 1 Sale


SetExpressCheckoutRequest[ Token[ ] , PaymentDetails[ OrderTotal[ ], PaymentDetailsItem[ Number[ ], Quantity[ ] ] ]]

string

int

intint

{0,1}

{1,8}{0,8}


We build a (recursive) mapping p from MSL constructs to sets of production rules

Example:

Mapping for other MSL constructs: see the paper (straightforward)


p[[Element[e] ]] ={0,1}

® ááx=nullññ |ááx=new Element();ññ

Element(x)È p[[e(x)]]{ }


Interfacegrammar







Interfacegrammar


WSDL






Interfacegrammar


WSDL(MSL)






Interfacegrammar


WSDL(MSL)

Translator






Interfacegrammar





WSDL(MSL)

Translator G

® ááx=new Login();ññ logininside(x,y)!login(x)

® ááy=new UserId(); x.userId=y;ññ...

logininside(x,y)



® ááx=new Login();ññ logininside(x,y)!login(x)


Java code, classesprovided by Apache Axis


We extracted interface grammars for two real-world web services:

Experimental results

E-Commerce ServiceExpress Checkout



Java/Axis driver generated test sequences to the actual service



�



Java/Axis driver generated test sequences to the actual service

Pair Java client/Java stub model checked with Java Pathfinder



�

�


! Driver message generation time: 1 sec./message! Typical test sequence: ! Found between implementation and

documentation

! Multiple contract violations found in sample client using Java Pathfinder

! Maximum checking time: / memory! Found between impl. and doc. with driver

5 seconds2 mismatches

20 sec. 43 MB2 mismatches



1. Long-running WS interactions involve constraints overmessage , and

2. We developed interface grammars to model these constraintsin a uniform notation

3. Using the same IG, we can generate a service and a client to perform model checking / test

generation

4. We tested the approach with two real-worldWS and found betweendocumentation and implementation

structure values sequence

automaticallystub driver

mismatches

Take-home points

generating interface grammars from wsdl for automated verification of web services (talk @ icsoc...

Documents