Serverless Network File Systems

Network File Systems Allow sharing among independent file

systems in a transparent manner Mounting a remote directory in NFS

Use remote procedure calls Traditional Network File Systems, like

NFS, use a central server to provide the file system services.

This work present an alternative, serverless network file system called xFS.

NFS Requirements / Metrics Performance / Speed Availability Scalability Fault Tolerance / Recovery

Limitations of Central Server Systems All read misses and disk writes go to the

server – Performance Bottleneck Not scalable – Too many clients can hurt

performance. Expensive to increase server hardware

or add servers. Require server replication for high

availability – increases cost and complexity. Also, latency to duplicate the data.

Serverless Network File Systems Increased Performance - distributes

control processing and data storage among cooperating workstations.

Scales easily to simplify system management.

Fault tolerance through distributed RAID and Log structured file system.

Migrates responsibility of failed components to other workstations.

Background RAID – Write portion of data to each disk.

High performance (parallel accesses) Availability xFS uses RAID striping for files across a stripe

group. Small writes hurt performance – must do parity

update. Log-structured File System (LFS)

Append-only file system. Leaves holes – need cleanup. Quick writes can be delayed to help RAID Helps recovery (checkpoints on disk)

Background (con’t) Multiprocessor Cache Consistency

Statically divide physical memory evenly among processors.

Each processor manages the cache consistency state for its own physical memory.

xFS does this for files. The node storing the files keeps up with consistency.

In xFS it is dynamic – files can be managed by different nodes.

Goals of xFS Provide a scalable way to subset

storage servers into groups to provide efficient storage.

Scalable, distributed metadata and cache consistency management.

Flexibility to dynamically reconfigure responsibilities after failures.

System entities Clients – want to access data in the system Storage Servers – store the system’s files Metadata Managers – hold cache

consistency state and disk location metadata.

Cleaners – clean up the LFS after writes

Entities may lie on the same system or on different systems.

Serverless File Service “Anything, Anywhere” – all data and

metadata can be located on and move to any node in the system.

File access is faster because they are distributed across multiple workstations!

How does the system locate the data? Key maps: manager map, imap, file

directories, stripe group maps.

Manager Map Table indicating which machines

manage which file indices File indices listed in the parent directory file.

Globally Replicated Updated dynamically

On machine failure or reconfiguration of file managers

Can work as a load balancing mechanism. Not yet implemented, but a possibility

Imap Imaps are held by a file’s manager Maps a file’s index number to the

disk address of the index node (inode).

The index node gives the file offset and pointers to each data block.

Similar to standard OS implementation

RAID Stripe Groups Better to stripe files over a group of

servers instead of all servers in the system.

Improves availability – Each group stores its own parity. Allows recovery from multiple failures.

Stripe Group Map – tells which nodes are a member of the group.

Must reference this map before reading or writing data to the file system.

Cache Consistency Token-based scheme Client must request and acquire

write ownership from the file’s manager.

Manager invalidates other cached copies

Cache Consistency (con’t) Client keeps write ownership until

another client requests it. It then must flush the changes to the disk.

xFS guarantees that the up-to-date copy is given to the node requesting the data.

Traditional network file systems do not always guarantee this.

Management Distribution Policies xFS tries to assign files used by a client

to a manager co-located on that machine.

When a client creates a file, xFS assigns the manager on that machine to the file. Improves locality Reduces network hops to satisfy requests -

40%

Reconfiguration Not yet implemented in this version… When system detects configuration

change, a global consensus algorithm is envoked.

Leader is chosen to run the algorithm given a list of active nodes.

Generates a new manager map and distributes it across the nodes.

Security in xFS Only appropriate in a restricted

environment Machines cooperating over a fast

network Must trust one another’s kernels to

enforce security