Lecturer Notes

Storage Area Network( Unit One )

Ms. Madhu G. Mulge( M-Tech CSE )

M.B.E.'S College of Engineering, Ambejogai.

1ME CSE [sem-II]

Introduction

2ME CSE [sem-II]

Designing storage area networks requires an understanding of the upper-layerapplications that depend on storage resources as well as the storagearchitectures available to satisfy them.

A framework has been developed to help storage architects determine whichstorage solutions may be most appropriate.Example : SAN , NAS, etc

Sharing storage resources among multiple servers or workstations requires a peer-to-peer network that joins targets to initiators.

SAN’s functions : creation of management, security, and other auxiliary services has trailed somewhat behind the deployment of shared storage networks.

The composition of that network and the type of storage data traversing it vary from one architecture to another.

ME CSE [sem-II] 3

Shared Storage Architectures Storage Area Networks (SAN)

Fibre Channel Gigabit Ethernet

Network-Attached Storage (NAS)Ethernet (Fast Ethernet or Gigabit Ethernet)

ApplicationsOnline Transaction Mining DatabaseWeb services

Shared Storage Model establishes the general relationship between userapplications that run on servers and hosts and the underlying StorageDomain.

Fig 1: The SNIA(Storage Networking Industry Association) Shared Storage Model overview

4ME CSE [sem-II]

The storage domain subdivides into three main categories:

File/record subsystem :The file/record subsystem is the interface between upper-layer applications

and the storage resources.SQL Server and Oracle

Block aggregation layer : Useful information appears to the upper-layer application as records or

files, both formats are ultimately stored on disk or tape as contiguous bytesof data known as blocks.

Block subsystem : Data Identified with specific records or files, the blocks themselves are

written to or read from physical storage.

Storage Domain is positioned as an auxiliary subsystem, i.e. Services Subsystem contains a number of storage-specific functions, including management, security, backup, availability, capacity, and planning.

5ME CSE [sem-II]

Fig 2: Direct-attached storage in the SNIA Shared Storage Model

6ME CSE [sem-II]

The Shared Storage Model is possible to insert server and storage components toclearly differentiate between direct-attached, SAN, and NAS configurations.

Replacing direct-attached connections with a more flexible network interconnectionenables new storage solutions.

In Direct attached storage a server with logical volume management (LVM) andsoftware RAID (redundant array of independent disks) running on the host system.

As the server receives information from the application that must be written to disk, the software RAID stripes blocks of data across multiple disks in the block layer.

Software RAID executes the mechanics to striping blocks of data to multiple disks.

The logical volume manager (LVM) presents a coherent image of the database / data to the upper-layer application in the form of volume (for example, E: drive), directories, and subdirectories.

7ME CSE [sem-II]

Fig 3: Changing the relationship between servers and storage via a SAN8ME CSE [sem-II]

9ME CSE [sem-II]

SANs alter the relationship between servers and storage targets, Instead ofbeing bound by direct parallel SCSI cabling, servers and storage are nowjoined through a peer-to-peer network.

As in direct-attached storage, logical volume management, software RAID,and hardware RAID still play their roles, but the connectivity between serversand storage devices allows you to attach any server to any storage target.

The SAN infrastructure can be Fibre Channel, Gigabit Ethernet, or, with theappropriate IP storage switches, both.

To accurately represent actual customer deployments, Fig 3 model could alsodepict multiple SANs servicing various upper layer applications andproviding attachment to common or distinct storage resources.

For heterogeneous environments, it might be useful to call out which SANsuse Fibre Channel fabrics exclusively and which involve a mix of FibreChannel and IP components.

ME CSE [sem-II] 10

Fig 4: Multiple SANs within the Shared Storage Model

ME CSE [sem-II] 11

Fig 5: The role of NAS in the Shared Storage Model

ME CSE [sem-II] 12

The Shared Storage Model, positions of NAS (Array X and Y) devicespartly in the file/record subsystem layer, extending down to the blockSubsystem.

NAS devices serve up files and so naturally include the block aggregationfunctions required to put data to disk.

NAS device is essentially a file server with its own storage resources.

NAS devices typically use NFS (Network File System) or CIFS(Common Internet File System) protocols to transport files over the localarea network (LAN) to clients.

ME CSE [sem-II] 13

Example: Carlson Companies

ME CSE [sem-II] 14

Storage and Networking ConceptsStorage and networking have evolved as two distinct technologies.

Storage: as embodied by the concepts of initiators.Targets: assumes a master/slave relationship between

connected devices.

SAN technology must provide:

High-performance transport of data. Efficient placement of data after it has safely arrived. Resolve the contradiction created by thrusting master servers and storage

slaves into a democratic peer infrastructure.

ME CSE [sem-II] 15

Networking in front of the Server

The front of the server faces outward toward the user community via anetwork interface.

Network dispersed computer resources across an enterprise by using Ethernetas the physical transport and TCP/IP as the networking protocol.

The OSI model divides networking into seven functional layers.

As user data is passed down by an application for transmission across thenetwork, it is wrapped in successive envelopes of information, roughlycorresponding to the layers represented in some specific manner.

ME CSE [sem-II] 16

Layer Number

Layer Name

Description

7 Application Interface for email, file transfer, etc.

6 Presentation Application specific

5 Session End-to-end session control: NetBIOS, Application specific.

4 Transport Transmission: TCP, UDP

3 Network Routing protocol : IP

2 Data link Access method: Ethernet, Token ring, etc.

1 Physical Physical transport: Twisted pair etc.

ME CSE [sem-II] 17

The most prevalent LAN transport is Ethernet over twisted-pair copper wiring.Ethernet transmission rates are :

10Mbps (megabits per second)100Mbps (also known as Fast Ethernet)1,000Mbps or 1Gbps (Gigabit Ethernet)

10Mbps and 100Mbps speeds are most commonly used for attachment ofindividual PCs or workstations, with effective data rates of slightly morethan 1MBps (megabytes per second) and 12MBps, respectively.

If you restrict the network to a single speed, bottlenecks will occur as moreusers attempt to access shared resources.

Ethernet offers deterministic bandwidth in powers of ten, it is possible to design scalable non-blocking infrastructures.

The core itself can be built with one or more 10Gb interswitch links toprovide a high-performance LAN backbone.

Ethernet LANs are normally limited to a single building or campus. To share computer resources with remote sites, wide area networks (WANs) use telecommunications lines, microwaves, or satellites to transmit data.

ME CSE [sem-II] 18

Fig 5: Scaling the network using departmental and core switches

ME CSE [sem-II] 19

Several fundamental principles of networking are common to both LANand WAN technologies.

Networking principles include the following: Serial transportAccess methodAddressing Packetizing of data Routing of packets Upper-layer protocol support

ME CSE [sem-II] 20

SCSI bus Architecture

ME CSE [sem-II] 21

The SCSI protocol was developed to provide efficient transport of databetween computers and peripheral devices such as disks, printers, scanners,and other resources.

SCSI sits below the file/record layer in the SNIA Shared Storage Model and receives requests to send or retrieve blocks of data from a peripheral device.

The responsibility of the SCSI protocol is simply to ensure that the write task iscompleted and to report success to the operating system, no matter how thephysical storage is configured.

SCSI targets are identified by the operating system in a three-part :

The Bus designator is one of several SCSI interfaces installed on a hostsystem.

The Target represents a single storage resource on a bus daisy chain. The LUN (logical unit number) designation identifies the SCSI

client within the target.

ME CSE [sem-II] 22

Fig 6 : SCSI Architectural Model (SAM-2)

ME CSE [sem-II] 23

The SCSI architecture defines the relationship between initiators (hosts) andtargets (for example, disks) as a client/server exchange.

The SCSI-3 application client resides in the host and represents the upper-layer application, file system, and operating system I/O requests.

The SCSI-3 device server sits in the target device, responding to requests.

The client/server, requests and responses are exchanged across some form of underlying transport, which is governed by the appropriate SCSI-3 service delivery protocol for that transport.FCP protocol or iSCSI for gigabit serial links.

The SCSI-3 commands that serve I/O requests from the host application are differentiated from the SCSI-3 transport protocols that actually move data via the service delivery subsystem.

Juggling multiple transactions to one or more targets requires contextswitching: the ability to quickly switch from one job to another as SCSIresponses are processed.

ME CSE [sem-II] 24

The bus/target/LUN identifier may be further mapped to the addressingrequirements of a specific transport.

Reads and writes of data between SCSI initiators and targets are performedwith a series of SCSI commands, delivery requests, delivery actions, andresponses.

SCSI commands and parameters are specified in the command descriptorblock (CDB).

The CDB is encapsulated within the Fibre Channel Protocol informationunits (IUs), whereas in iSCSI it is carried in the iSCSI protocol data unit(PDU).

A traditional parallel SCSI adapter card may represent a single bus, with that bus supporting multiple daisy chained disks.

Alternatively, a Fibre Channel host bus adapter (HBA) or iSCSI networkinterface may be viewed by the operating system as a SCSI bus.

ME CSE [sem-II] 25

The Parallel SCSI BusThe most common method for connecting disk and tape devices to file and

application servers has been via traditional SCSI bus cabling.One of the difficulties presented by parallel bus architecture is a

phenomenon known as skew.Skew refers to the difference in arrival time for each bit comprising a data

word.

Fig 7: Timing skew on parallel bit transmission

ME CSE [sem-II] 26

Parallel SCSI requires termination of any unused ports.

As multiple SCSI devices are daisy-chained together, end devices must beterminated to avoid erratic signal interference.

Proper cabling and termination are critical for stable parallel SCSI operation,and improper termination can cause data corruption or transaction failures.

Parallel SCSI cabling to multiple SCSI host bus adapters installed in theserver (as shown in Fig 8).

Parallel SCSI has restrictions in terms of distance and number of devices supported on a SCSI bus make it difficult to support larger storage capacities.

The primary weakness of parallel SCSI for enterprise applications it binds storage resources to a single server.

Storage resources cannot be shared easily by multiple servers, and unusedstorage capacity behind one server cannot be utilized by another.

ME CSE [sem-II] 27

Fig 8: Server with multiple SCSI adapter cards installed

Network Solution : direct-attached storage configurations, the limitations ofparallel SCSI are being addressed by Serial Attached SCSI (SAS).

Efficiencies in local storage connectivity. increase immune to the skew and distance limitations of parallel cabling.

ME CSE [sem-II] 28

Network-Attached Storage (NAS)NAS accomplishes the central goal of storage networking: the sharing of

storage resources through the separation of servers and storage over acommon network.

In redundant configurations, NAS can also provide highly available, non-disruptive storage access.

NAS performance is determined in part by the bandwidth available on themessaging network.

NAS device can rely on parallel SCSI, IDE, or Fibre Channel for storageconnectivity.

NAS vendors is to make their products more appliance-like and to reduce theoverhead of NFS/CIFS protocols over TCP/IP.

The concept of “Appliance"—a device that is simply plugged in to the network andrequires little administration.

NAS vendors support the Distributed File Access System (DAFS) over VirtualInterface (VI)

ME CSE [sem-II] 29

Fig 9: NAS appliance can use any type of physical storage

ME CSE [sem-II] 30

Another trend in NAS technology is the separation of storage from the NASprocessor or head, so that storage capacity increased.

NAS technology is the separation of storage from the NAS processor or head,so that storage capacity can be more easily expanded using SAN techniquesor shared with SAN-attached servers (as shown in fig 10).

Advantage: higher performance and scalable storage capacity.

Limitations: presents vulnerabilities in the event of loop failure.

Solution : Embedded-loop switch technology is used on the backplane of some NAS storage arrays to bypass failures and maintain stable operation

ME CSE [sem-II] 31Fig 10 : NAS head with shared SAN storage

ME CSE [sem-II] 32

Networking behind the Storage

Storage Area Networks introduce network relationships behind the server.

SANs enable more efficient use of storage resources and lower administrative costs through storage consolidation.

Replacing the dedicated parallel SCSI cabling bond between servers andstorage with peer-to-peer networking does two things: Only provides more flexibility in the deployment of storage resources. Frees storage from the exclusive ownership by a single server.

SAN may be isolated from the mainstream messaging network through protocol segregation (Fibre Channel) or network segregation (virtual LAN, or VLAN).

SAN infrastructure has provided a foundation for new storage applications such as server clustering, LAN-free and server-free tape backup, and storage consolidation.

ME CSE [sem-II] 33

Fig 8: A simple storage area network

ME CSE [sem-II] 34

Fibre Channel provides both 1Gbps and 2Gbps speeds and so offers anadvantage in raw bandwidth for device Attachment.

SANs are a new type of network optimized for storage.

SANs require new management tools to supervise new storage relationships.

ME CSE [sem-II] 35

Fibre channel internalsFibre Channel standards define a multilayered architecture for moving data

across the network.

These layers are numbered from FC-0 to FC-4.

Fibre Channel is a gigabit transport, with current implementations at 1Gbpsand 2Gbps.

Fibre Channel layers fall within the lower levels of the OSI model and can beconsidered a link layer network

The standards provide definitions :Physical-layer attributes Transport controls Upper-level support of TCP/IP SCSI-3 High Performance Parallel Interface (HiPPI) Other protocols.

ME CSE [sem-II] 36

Layer Number

Functions Contents

FC-4 Upper layer protocolinterface

FC-3 Common services

FC-2 Data Delivery

FC-1 Ordered sets/bytesencoding

FC-0 Physical interface

Layer Number

Functions Contents

FC-4 Upper layer protocol interface

Ex: SCSI-3, IP

FC-3 Common services Under construction (Encryption and compression)

FC-2 Data Delivery Framing , flow control, class services

FC-1 Ordered sets/bytes encoding

8b/10b encoding , link controls

FC-0 Physical Interface Optical/ electrical, cable plant

Fibre Channel Layers

Fibre Channel standards define a multilayered architecture’s Functions:

FC-4:- Establishes the interface between Fibre Channel and upper-level applications.

Example: For host bus adapters, this function is typically fulfilled by the device driver provided by the vendor.

FC-3 layer :- Under development Example: It may include facilities for data encryption and compression.

FC-2 layer:- Blocks of data handed by the upper-level application Example: various classes of services and flow control mechanisms.

FC-1 and FC-0—Focus on the actual transport of data across the network.

FC-1 :- Facilities for encoding & decoding data for shipment at gigabit speeds and defines the command structure for accessing the media.

FC-0:- Establishes standards for various media types, allowable lengths, and signaling.

ME CSE [sem-II] 38

Fibre Channel's layered architecture is implemented on three transporttopologies:

Point-to-point: -A dedicated connection between two devices only, typically a server

and a disk.

Arbitrated loop: -A shared medium, similar to Token Ring or Fiber Distributed Data

Interface (FDDI), and employs a special superset of commands to controlaccess to the medium by multiple devices.

Switched fabric: -one or more switches providing higher-level services and switched

bandwidth of 100MBps or 200MBps (200MBps and 400MBps full duplex)per port.

ME CSE [sem-II] 39

Data EncodingProper transmission and reception of a digital bit stream would not be

possible if raw bytes of data were simply herded into a shift register andshoved one bit at a time onto the transport.

Without a byte encoding scheme : it would be impossible to detect when one bit ended and another began—in other words, it would be impossible to recover the clocking used to issue the stream.

converts each 8-bit byte into two possible 10-bit characters.Example:- Raw stream of hex FF bytes

1111 1111 = 101011 0001 or 010100 1110 (The 8b/10b mechanism)

Each of the two 10-bit products of this conversion should have no more than six total 1's or 0's, and about half will have an equal number of 1's and 0's.The 10-bit characters with more 1's than 0's have positive disparity.With more 0's than 1's have negative disparity.Balance of 1's and 0's results in neutral disparity.

ME CSE [sem-II] 40

Fibre Channel uses one of these nondata characters as a special command character. Known as the K28.5 command character.

Unique comma sequence is used by data recovery circuitry as a trigger for defining the boundaries of 10-bit characters.

ME CSE [sem-II] 41

Ordered SetsTo move data across the network, Fibre Channel uses a concise command

syntax referred to as ordered sets.

Ordered sets fall into three general categories: start of frame, end of frame,and primitive signals.

One subset is used to indicate the start and end of data frames and the class of service used to ship the frame.

A group of ordered sets known as primitive signals is used to indicate actionsor events on the transport.

The start of frame (SOF) delimiter K28.5 D21.5 D22.2 D22.2, for example,is used in a frame header to initiate a connectionless level of service.

An end of frame (EOF) ordered set—for example, K28.5 D21.4 D21.6D21.6—would indicate that the transmission of this frame was complete.

ME CSE [sem-II] 42

Fibre Channel defines 11 different types of SOF ordered sets, and 8 EOFwords.

The specific SOF or EOF used informs the recipient as to what further action(such as an acknowledgment).

Primitive sequences require that at least three consecutive words berecognized before any action is taken.

supersets of primitive signals and primitive sequences have been definedspecifically for arbitrated loop and Fibre Channel fabric operation.

ME CSE [sem-II] 43

Framing Protocol

To move data from one Fibre Channel-attached device to another, the datablocks handed down by the upper-layer protocol of the sender must beorganized into discrete packets for shipment across the transport.

Frames are always prefaced with an ordered set start of frame (SOF)delimiter.single 4-byte word that defines the class of service used and specifieswhether the frame is the first in a series or simply one in a series of relatedframes.24-byte frame header contains destination and source addressing, as

well as control fields that indicate the frame's content (controlinformation or data type) and position within a series of sequentialframes.

Data integrity within the frame is verified with an 32-bit cyclic redundancycheck (CRC).

ME CSE [sem-II] 44

Classes of ServiceTo ensure proper data transport, applications may require different levels ofdelivery guarantees, bandwidth, and connectivity between communicators.Fibre Channel standards provide five classes of delivery service.Class 1 service defines a dedicated connection between two devices.

a file server and a disk array with acknowledgment of frame deliveryClass 2 service does not require a dedicated connection between talkingpairs but it does provide acknowledgment of frame delivery.Class 3 service is connectionless, but unlike Class 2, there is no notificationof delivery.

Similar to datagram services in LAN topologies (for example, UserDatagram Protocol/Internet Protocol, or UDP/IP).

Class 4 service introduces the concept of virtual circuits to Fibre Channelarchitecture.

The switch must maintain and monitor potentially hundreds of virtualcircuits. Ex:- SNMP.

Class 6 service has been defined to provide multicast service withacknowledged delivery.

ME CSE [sem-II] 45

Fibre Channel provides several flow control mechanisms based on a systemof credits.

Credit scheme is based on the number of buffers that an end node maintainsfor storing incoming data.

Transmission credit is initially established when two communicating nodeslog in and exchange communication parameters.

End-to-end flow control (EE_Credit).

Buffer-to-buffer credit (BB_Credit) is used by Class 2 and Class 3 serviceand relies on the receiver-ready (R_RDY) ordered set to replenish credits.

Flow Control

ME CSE [sem-II] 46

Naming and Addressing ConventionsThe unique identity of participants in a Fibre Channel environment ismaintained through a hierarchy of fixed names and assigned addressidentifiers.

In Fibre Channel terminology, a communicating device is a node.

Each node has a fixed 8-byte node name assigned by the manufacturer.Institute of Electrical and Electronics Engineers (IEEE) for a range ofaddresses.

World-Wide Name (WWN).

The 64-bit name provides a unique identity throughout the SAN.

The switch maintains a table of the device's WWN and the assigned 24-bitaddress in the Simple Name Server (SNS).

ME CSE [sem-II] 47

Thank U…~!~