Storage management and caching in PAST, a large-scale, persistent peer-

to-peer storage utility

Antony Rowstron, Peter Druschel

Introduction

PAST is an Internet-based, peer-to-peer global storage utility.

It aims to provide strong persistence, high availability, scalability and security.

PAST employs Pastry as the routing scheme.

Pastry is an efficient routing scheme used to reliably route client requests to the appropriate nodes.

PAST Operations

fileId=Insert(name, owner-credentials, k, file)fileId is 160-bit, while nodeId is 128-bit;

fileId

nodeId

file=Lookup(fileId) Reliably retrieve a copy of the file

Reclaim(fileId, owner-credentials)Notice: Reclaim doesn’t guarantee that the file is no longer

available after it was reclaimed.

128 Bits 32 Bits

128 Bits

M L

Storage Management

Design Goals: Allow high global storage utilization Gracefully degrade as the system approaches t

he maximal utilization

Load balancing: Replica diversion File diversion

Replica Diversion

Replica diversion is to accommodate differences in the storage capacity and utilization of nodes within a leaf set.

B

A

C

An entry points to B

File is stored in B due to the less capacity in A

A is the right node to store the file, but has insufficient space

C is the k+1th closest node, which stores the pointer in case of the failure of node A

File Diversion

The k closest nodes to the fileId could not accommodate the file nor divert the replicas; then, a file diversion is invoked.

The client generates a new fileId, and retries to insert the file.

Repeat the second step up to 3 times.If it still fails, an error is reported to the app

lication.

Caching

PAST nodes use the “unused” portion of their advertised disk space to cache files.

Caching in PAST is based on GreedyDual-Size (GD-S) policy, which was originally developed for caching Web proxies.

Experimental Results

*This graph is from the paper.

Question?