opodis'04 a protocol for recording provenance in service-oriented grids paul groth, michael...

OPODIS'04

A protocol for recording provenance in service-oriented Grids

Paul Groth, Michael Luck, Luc Moreau

University of Southampton

OPODIS'04

Or…How to show your work.

In a Grid

OPODIS'04

Contents

1. What is Provenance and why you should care

2. The Grid and Provenance

3. An Architectural Vision

4. PReP

6. What’s next

7. Conclusion

OPODIS'04

A Definition

Main Entry: prov·e·nance Pronunciation: 'präv-n&n(t)s, 'prä-v&-"nän(t)sFunction: nounEtymology: French, from provenir to come forth, originate, from Latin provenire, from pro- forth + venire to come Date: 17851 : ORIGIN, SOURCE2 : the history of ownership of a valued object or work of art or literature

The provenance of something is the

documentation of process that led to that something

i.e. showing your work

OPODIS'04

The importance of provenance

Process is IMPORTANT

Art Wine Drug Discovery Financial Auditing Aerospace …

OPODIS'04

The Grid

The Grid problem is defined as coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organisations [Foster, Kesselman, Tuecke].

OPODIS'04

… and the Provenance Problem

Given a set of services in an open grid environment that decide to form a virtual organisation with the aim to produce a given result;

How can we determine the process that generated the result, especially after the virtual organisation has been disbanded?

OPODIS'04

Provenance Problem cont.

Provenance recording should be part of the infrastructure

Currently, the Web Services protocol stack and the Open Grid Services Architecture do not provide any support for recording provenance.

Methods are generally adhoc and do not interoperate.

OPODIS'04

More Definitions

Provenance The provenance of some data is the documentation of

process that led to the data.

Interaction Provenance For some data, interaction provenance is the

documentation of interactions between actors that led to the data.

Actor Provenance For some data, actor provenance is documentation

that can only be provided by a particular actor pertaining to the process that led to the data.

OPODIS'04

An Architecture

OPODIS'04

Motivating Requirements

Verifiability Accountability Reproducibility

Repeat Reproduce

Preservation Scalability Generality Customisability

OPODIS'04

An Architecture with Provenance Support

OPODIS'04

PReP- Provenance Recording Protocol

client serviceinvocation

result

recordinvocationand result

recordinvocationand result

negotiate

Why record 2 views?

ProvenanceStore

ProvenanceStore

ProvenanceStore

OPODIS'04

Multiple Provenance Stores

client serviceinvocationresult

ProvenanceStore

invocationand result record

invocationand result record

client serviceinvocationresult

ProvenanceStoreinvocation

and result recordinvocation

and result record

client serviceinvocationresult

ProvenanceStore

invocationand result record

invocationand result record

Provenance stores may be shared or different

OPODIS'04

Linking Stores

client serviceinvocationresult

ProvenanceStore

invocationand result record

invocationand result record

client serviceinvocationresult

ProvenanceStoreinvocation

and result recordinvocation

and result record

client serviceinvocationresult

ProvenanceStore

invocationand result record

invocationand result record

Link

OPODIS'04

PReP in detail

Model PReP using asynchronous message passing. Maps well to any implementation Helpful for scalability

Four Phase Protocol Negotiation Invocation Provenance Recording Termination

OPODIS'04

PReP’s messages

ProposeReplyInvoke ResultRecord NegotiationRecord InvocationRecord ResultSubmission FinishedAdditional Provenance

Record Negotiation AckRecord Invocation AckRecord Result AckSubmission Finished AckAdditional Provenance Ack

OPODIS'04

PReP’s messages

ProposeReplyInvoke ResultRecord NegotiationRecord InvocationRecord ResultSubmission FinishedAdditional Provenance

Record Negotiation AckRecord Invocation AckRecord Result AckSubmission Finished AckAdditional Provenance Ack

Used for connecting provenance stores and for recording provenance about actors.

OPODIS'04

Provenance Store – An abstract state machine

Formalise the protocol by formalising the individual entities in the protocol

Know exactly how the Provenance Service responds to receipt of messages

Use to show a liveness property Something good will eventually

happen

OPODIS'04

Client and Service

State transition diagram Cannot formalise internals, only the

response to PReP Show Termination Property

OPODIS'04

OPODIS'04

What’s next

PReP Version 2Greater Flexibility

• In configuration and recorded infoSecurity

• non-repudiation• mutual authentication

Fault Tolerant Specification What if you have a lot of data?

Look at scalability

OPODIS'04

Conclusion

Provenance is important Provenance recording must be part of

the infrastructure. Standards. PReP provides is a first step. An implementation (PReServ) is

available.

OPODIS'04

http://www.pasoa.org

Where you can download PReServpg03r@ecs.soton.ac.uk

The End