Download - Spinnaker Networks, Inc

of 30 NFS Industry ConferenceOctober 22-23, 2002

NFS

INDUSTRY

CONFERENCE

Spinnaker Networks, Inc.

www.spinnakernet.com

301 Alpha DrivePittsburgh, PA 15238(412) 968-SPIN


NFS

INDUSTRY

CONFERENCE

• ”Everything you know is wrong”– … at least eventually

– space requirements change

– “class of service” changes

– desired location changes

Storage Admin’s Problem


NFS

INDUSTRY

CONFERENCE

• System scaling– add resources easily

– without client-visible changes

• Online reconfiguration– no file name or mount changes

– no disruption to concurrent accesses

• System performance

Solution


NFS

INDUSTRY

CONFERENCE

Spinnaker Design

• Cluster servers for scaling– using IP (Gigabit Ethernet) for cluster links

– separate physical from virtual resources– directory trees from their disk allocation

– IP addresses from their network cards

– we can add resources without changing client’s view of system


NFS

INDUSTRY

CONFERENCE

Spinnaker Design

• Within each server– storage pools

– aggregate all storage with single service class– e.g. all RAID 1, RAID 5, extra fast storage

– think “virtual partition” or “logical volume”


NFS

INDUSTRY

CONFERENCE

Spi

nnaker

Archit

ect

ure

• Create virtual file systems (VFSes)– a VFS is a tree with a root dir and subdirs– many VFSes can share a storage pool

– VFS allocation changes dynamically with usage– without administrative intervention– can manage limits via quotas

– Similar in concept to– AFS volume / DFS fileset


NFS

INDUSTRY

CONFERENCE

Spi

nnaker

Archit

ect

ure

A

adam ant

B

BachBobs

Eng

net disk

Spin

DeptsUsers

A B Eng

Storage Pool

Storage Pool


NFS

INDUSTRY

CONFERENCE

Spi

nnaker

Archit

ect

ure

• Create global “export” name space– choose a root VFS

– mount other VFS, forming a tree– by creating mount point files within VFSes

– export tree spans multiple servers in cluster– VFSes can be located anywhere in the cluster

– export tree can be accessed from any server

– different parts of tree can have different CoS


NFS

INDUSTRY

CONFERENCE

Gl

obal

Na

mi

ng

and

VFSes

A

adam ant

Eng

net disk

Spin

Depts Users

A B Eng

Users

A B Eng

Spin

Depts Users Users

A

adam ant

Eng

net disk

Storage Pool

Storage Pool

B

BachBobs

B

BachBobs


NFS

INDUSTRY

CONFERENCE

Clustered Operation

• Each client connects to any server– requests are “switched” over cluster net

– from incoming server

– to server with desired data

– based on– desired data

– proximity to data (for mirrored data)


NFS

INDUSTRY

CONFERENCE

Cluster Organization

SpinServer cluster

Windows desktop

Windows desktop

UNIX workstation

Windows desktop

SpinServer 1

SpinServer 2

SpinServer 3

Q1 eng P2

\acct

Q1 Q2

\eng

P1 P2

P1 Q2

\acct

Q1 Q2

acct

\eng

P1 P2


NFS

INDUSTRY

CONFERENCE

Client Access

Gigabit Ethernet

Fibre Channel

Client Access

Gigabit Ethernet

Fibre Channel

Disk Process• Caching• Locking

X

Gigabit EthernetSwitch

Network Process• TCP termination• VLDB lookup• NFS server over SpinFS

SpinFS Protocol

Network Process• TCP termination• VLDB lookup• NFS server over SpinFS

Disk Process• Caching• Locking

Server/Network Implementation


NFS

INDUSTRY

CONFERENCE

• At enterprise scale, security is critical– don’t have departmental implicit “trust”

• Kerberos V5 support– For NFS clients

– Groups from NIS

– For CIFS using Active Directory

Security


NFS

INDUSTRY

CONFERENCE

• A virtual server consists of– a global export name space (VFSes)– a set of IP addresses that can access it

• Benefits– additional security fire wall

– a user guessing file IDs limited to that VS

– rebalance users among NIC cards– move virtual IP addresses around dynamically

Virtual Servers


NFS

INDUSTRY

CONFERENCE

• 94 MB/sec read– single stream read, 9K MTU

• 99 MB/sec write– single stream write, 9K MTU

• All files much larger than cache– real I/O scheduling was occurring

Performance – single stream


NFS

INDUSTRY

CONFERENCE

Benefits

• Scale single export tree to high capacity– both in terms of gigabytes

– and ops/second

• Keep server utilization high– create VFSes wherever space exists

– independent of where data located in name space

• Use expensive class of storage– only when needed

– anywhere in the global name space


NFS

INDUSTRY

CONFERENCE

Benefits

• Use third-party or SAN storage– Spinnaker sells storage

– but will support LSI storage, others

• Kerberos and virtual servers– independent security mechanisms

– cryptographic authentication and

– IP address-based security as well


NFS

INDUSTRY

CONFERENCE

• Free data from its physical constraints– data can move anywhere desired within a cluster

• VFS move– move data between servers online

• VFS mirroring– Mirror snapshots between servers

• High availability configuration– multiple heads supporting shared disks

Near Term Roadmap


NFS

INDUSTRY

CONFERENCE

VFS Movement

• VFSes move between servers– balance server cycle or disk space usage

– allows servers to be easily decommissioned

• Move performed online– NFS and CIFS lock/open state preserved

– Clients see no changes at all


NFS

INDUSTRY

CONFERENCE

VFS Move

SpinServer cluster

Windowsdesktop

Windowsdesktop

SpinServer1

SpinServer2

Q1 eng

\acct

Q1 Q2

\eng

P1 P2

Q2 acct

acct


NFS

INDUSTRY

CONFERENCE

VFS

M

irror

• Multiple identical copies of VFS– version number based

– provides efficient update after mirror broken

– thousands of snapshots possible

– similar to AFS replication or NetApp’s SnapMirror


NFS

INDUSTRY

CONFERENCE

Failover Pools

• Failover based upon storage pools– upon server failure, peer takes over pool

– each pool can failover to different server

– don’t need 100% extra capacity for failover


NFS

INDUSTRY

CONFERENCE

Failover Configuration

SpinServer 1

SpinServer 2

P1

SpinServer 3

P4

P2

P3


NFS

INDUSTRY

CONFERENCE

Additional Benefits

• Higher system utilization– by moving data to under-utilized servers

• Decommission old systems– by moving storage and IP addresses away

– without impacting users

• Change storage classes dynamically– move data to cheaper storage pools when possible

• Inexpensive redundant systems– don’t need 100% spare capacity


NFS

INDUSTRY

CONFERENCE

• Caching– helps in MAN / WAN environments

– provide high read bandwidth to single file

• Fibrechannel as access protocol– simple, well-understood client protocol stack

• NFS V4

Extended Roadmap


NFS

INDUSTRY

CONFERENCE

Summary

• Spinnaker’s view of NAS storage– network of storage servers

– accessible from any point

– with data flowing throughout system– with mirrors and caches as desired

– optimizing various changing constraints

– transparently to users


NFS

INDUSTRY

CONFERENCE

ThankYou

Mike KazarCTO


NFS

INDUSTRY

CONFERENCE

Design Rationale

• Why integrate move with server?– VFS move must move open/lock state

– Move must integrate with snapshot

– Final transition requires careful locking at source and destination servers


NFS

INDUSTRY

CONFERENCE

Design Rationale

• Why not stripe VFSes across servers?– Distributed locking is very complex

– and very hard to make fast

– enterprise loads have poor server locality

– as opposed to supercomputer large file patterns

– Failure isolation

– limit impact of serious crashes

– partial restores difficult on stripe loss


NFS

INDUSTRY

CONFERENCE

Design Rationale

• VFSes vs. many small partitions– can overbook disk utilization

– if 5% of users need 2X storage in 24 hours– can double everyone’s storage, or

– can pool 100 users in an SP with 5% free

Download - Spinnaker Networks, Inc

Top Related