oses: 11. fs impl. 1 operating systems v objectives –discuss file storage and access on secondary...

OSes: 11. FS Impl. 1

Operating SystemsOperating Systems

ObjectivesObjectives– discuss file storage and access on discuss file storage and access on

secondary storage (a hard disk)secondary storage (a hard disk)

Certificate Program in Software DevelopmentCSE-TC and CSIM, AITSeptember--November, 2003

11. File System Implementation(S&G, Ch. 11)


ContentsContents

1.1. A Hard DiskA Hard Disk

2.2. File System OrganizationFile System Organization

3.3. Allocation MethodsAllocation Methods

4.4. File System ManagementFile System Management

5.5. Directory ImplementationDirectory Implementation

6.6. Improving PerformanceImproving Performance


1. A Hard Disk1. A Hard Disk Fig. 2.5, p.33

:

rotation

armplatter

cylinder c

track t

read-writehead

spindleactuator

sector s


NotesNotes

To read/write disk sectorTo read/write disk sector– move head to cylinder (seek)move head to cylinder (seek)– wait for sector to rotate to head (latency)wait for sector to rotate to head (latency)– read or write (transfer)read or write (transfer)

ControllerController– one controller for several disksone controller for several disks– intelligent controllersintelligent controllers– cachingcaching

continued


I/O transfer is carried out in blocksI/O transfer is carried out in blocks– one or more sectorsone or more sectors

Sector size: 32 - 4096 bytesSector size: 32 - 4096 bytes– usually 512 bytesusually 512 bytes


2. File System Organization2. File System OrganizationFig. 11.1, p.370

application program

logical file system

file-organization module

basic file system

I/O control

hardware devices


FeaturesFeatures

Basic File SystemBasic File System– identifies a hardware block using identifies a hardware block using

drive/cylinder/track/sector numbersdrive/cylinder/track/sector numbers

I/O ControlI/O Control– consists of device drivers and interrupt handlersconsists of device drivers and interrupt handlers– translates hardware block commands (e.g. translates hardware block commands (e.g.

“retrieve block 123”) into hardware specific “retrieve block 123”) into hardware specific instructionsinstructions

continued


File Organization ModuleFile Organization Module– translates logical block addresses into physical translates logical block addresses into physical

blocks and then to drive / cylinder / track / blocks and then to drive / cylinder / track / sector numberssector numbers


2.1. Open File Table2.1. Open File Table

An An open()open() searches for a file and then searches for a file and then records its details in an records its details in an open file tableopen file table– returns an index to the table entryreturns an index to the table entry

called acalled a file descriptor file descriptor in UNIX in UNIX

– the index is used by subsequent file operationsthe index is used by subsequent file operations


Open File Table DiagramOpen File Table DiagramFig. 11.2, p.372

test.cmail.txt : :

rw- rw- rw-rw- --- ---

. . .

. . .01

: :

n

file name permissions access dates

pointer todisk blockindex


3. Allocation Methods3. Allocation Methods How is space allocated to files on a disk?How is space allocated to files on a disk? AimsAims::

– utilize disk space effectivelyutilize disk space effectively– allow fast file accessallow fast file access

Three main methods:Three main methods:– contiguous allocationcontiguous allocation– linked allocationlinked allocation– indexed allocationindexed allocation


3.1. Contiguous Allocation3.1. Contiguous Allocation

Each file occupies a set of contiguous Each file occupies a set of contiguous (continuous) blocks on the disk.(continuous) blocks on the disk.

Supports fast sequential and direct (random) Supports fast sequential and direct (random) accessaccess– e.g. the 5th block of a file starting at block 19 is e.g. the 5th block of a file starting at block 19 is

at block 24 (19+5)at block 24 (19+5)


DiagramDiagram Fig. 11.3, p.374

0 1 2 3

4 5 6 7

8 9 10 11

15141312

19181716

23222120

27262524

31302928

file start lengthcount 0 2tr 14 3mail 19 6list 28 4f 6 2

directory

'physical block' view


ProblemsProblems

Finding space for a new file or a resized Finding space for a new file or a resized file.file.

Determining size requirements.Determining size requirements.

External fragmentation of the hard disk.External fragmentation of the hard disk.


3.2. Linked Allocation3.2. Linked Allocation

Each file is stored as a linked list of disk Each file is stored as a linked list of disk blocksblocks– the disk blocks may be anywhere on the diskthe disk blocks may be anywhere on the disk



0 101 2 3

4 5 6 7

8 9 10 11

15141312

19181716

23222120

27262524

31302928

file start endjeep

directory

16 25

1

-1

9 25


AdvantagesAdvantages

Solves the problems with contiguous Solves the problems with contiguous allocationallocation– the space allocation problemthe space allocation problem– no external fragmentationno external fragmentation

Fast for sequential access.Fast for sequential access.


DisadvantagesDisadvantages

Direct access is not supported (efficiently).Direct access is not supported (efficiently).

Space overhead of pointersSpace overhead of pointers– solutionsolution: cluster disk blocks: cluster disk blocks

Reliability if a pointer ‘fails’Reliability if a pointer ‘fails’– solutionsolution: use richer (bigger) links: use richer (bigger) links


3.2.1. File-allocation Table (FAT)3.2.1. File-allocation Table (FAT)Fig. 11.5, p.378

618

eof

339

217. . .test

directory entry

name startblock

0

217

339

618

no. of disk blocks FAT


NotesNotes

The table has one entry for each disk block The table has one entry for each disk block on the disk, and is indexed by block on the disk, and is indexed by block number.number.

Records the sequence of blocks used by Records the sequence of blocks used by each file.each file.

No need for pointers in the actual disk No need for pointers in the actual disk blocks.blocks.

continued


Used by MS-DOS and OS/2Used by MS-DOS and OS/2

DrawbackDrawback: the disk head needs to do many : the disk head needs to do many seeks:seeks:– move to the start of the FATmove to the start of the FAT– then move to the required FAT entrythen move to the required FAT entry– then move to the actual disk blockthen move to the actual disk block


3.3. Indexed Allocation3.3. Indexed Allocation

A modification of linked allocation where A modification of linked allocation where the disk block pointers for a file are all the disk block pointers for a file are all placed in an placed in an index blockindex block..



0 1 2 3

4 5 6 7

8 9 10 11

15141312

19181716

23222120

27262524

31302928

file index blockjeep

directory

19

91611025-1-1-1

19


NotesNotes

Supports direct access.Supports direct access.

Wasteful if only a few pointers are Wasteful if only a few pointers are stored in the index block.stored in the index block.


3.3.1. Variations3.3.1. Variations

Linked SchemeLinked Scheme– link together the (smaller) index blocks used for link together the (smaller) index blocks used for

a filea file

Multilevel IndexMultilevel Index– use a first-level index block to point to use a first-level index block to point to

second-level index blocks which point to the second-level index blocks which point to the actual disk blocksactual disk blocks

– supports much bigger filessupports much bigger files

continued


Combined SchemesCombined Schemes– e.g. the UNIX file index block (e.g. the UNIX file index block (inodeinode))

modeowners (2)

timestamps (3)size

block count

single indirectdouble indirecttriple indirect

direct blocks

data

data

.

.

.

data

data

.

.

.

.

.

.

data

data

.

.

.

data

data

data

data12

:::

:

:

:

Fig. 11.7, p.381


3.4. Performance3.4. Performance

Allocation methods vary in their:Allocation methods vary in their:– storage efficiencystorage efficiency– disk block access timesdisk block access times

Measurements depend on the file mix in Measurements depend on the file mix in the OSthe OS– e.g. no. of sequential access files vs. no. of e.g. no. of sequential access files vs. no. of

direct/random access filesdirect/random access files

continued


Contigous allocationContigous allocation– constant time access for sequental or direct constant time access for sequental or direct

access filesaccess files

Linked allocationLinked allocation– should not be used for direct access filesshould not be used for direct access files

continued


Indexed allocationIndexed allocation– faster if the index block for the file is already faster if the index block for the file is already

in memoryin memory

– large files require more index block large files require more index block searching, so are slower to accesssearching, so are slower to access


Performance OptimisationsPerformance Optimisations

Since CPU speeds are often 10,000 times Since CPU speeds are often 10,000 times faster than hard disk speeds, then very faster than hard disk speeds, then very complex disk accessing optimisations, complex disk accessing optimisations, coded in software/hardware are coded in software/hardware are worthwhile.worthwhile.


4. File Space Management4. File Space Management

Use a free-space list to record which blocks are Use a free-space list to record which blocks are free (i.e. free (i.e. notnot allocated to a file or directory). allocated to a file or directory).

Implementation approaches:Implementation approaches:– bit vectorbit vector– linked listlinked list– groupinggrouping– countingcounting


4.1. Bit Vector4.1. Bit Vector

Represent each free block by a ‘1’ bit, each Represent each free block by a ‘1’ bit, each allocated block as a ‘0’ bit in a bit vector.allocated block as a ‘0’ bit in a bit vector.

Example: free blocks are Example: free blocks are 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, 27,…2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, 27,…

Bit vector:Bit vector:001111001111110001100000011100000...001111001111110001100000011100000...

continued


OSes contain fast bit manipulation OSes contain fast bit manipulation instructions to find a free block or blocks instructions to find a free block or blocks quickly.quickly.

ProblemProblem: size of the bit vector: size of the bit vector– 1 bit required for each disk block1 bit required for each disk block


4.2. Linked List4.2. Linked List

Link together all Link together all the free disk the free disk blocks.blocks.

Problems:Problems:– traversal speedtraversal speed– pointer failurepointer failure

Fig. 11.8, p.384

0 1 2 3

4 5 6 7

8 9 10 11

15141312

19181716

23222120

27262524

31302928

free-spacelist head


4.3. Grouping4.3. Grouping

Store the addresses of the Store the addresses of the nn free blocks free blocks in the first free disk block.in the first free disk block.

Use pointers to link together multiple Use pointers to link together multiple disk blocks of this information.disk blocks of this information.


4.4. Counting4.4. Counting

In contiguous allocation schemes, free In contiguous allocation schemes, free blocks are usually contiguous.blocks are usually contiguous.

Instead of making a list of Instead of making a list of nn free disk free disk blocks, store the address of the first free blocks, store the address of the first free block and a counter (containing block and a counter (containing nn).).


5. Directory Implementations5. Directory Implementations Linear list of filesLinear list of files

– simple to codesimple to code– time-consuming to searchtime-consuming to search

solutionsolution: cache recently used directory info.: cache recently used directory info.

Hash tableHash table– compute the index into a list of files based on a compute the index into a list of files based on a

hash function (applied to the file name)hash function (applied to the file name) collisionscollisions cost of directory reorganizationcost of directory reorganization


6. Improving Performance6. Improving Performance

Most disk controllers include local memory Most disk controllers include local memory to store previously accessed tracksto store previously accessed tracks– repeated accesses are fasterrepeated accesses are faster– called a called a track cachetrack cache (or (or track buffertrack buffer))

Some systems store recently accessed disk Some systems store recently accessed disk blocks in memoryblocks in memory– called a called a disk cachedisk cache (or (or block bufferblock buffer) )

continued


Some systems support Some systems support RAM disksRAM disks ((virtual disksvirtual disks) in memory) in memory– users can carry out standard disk users can carry out standard disk

operations, but they are much faster since operations, but they are much faster since the data never leaves memorythe data never leaves memory



harddisk

trackcache

ram disk

disk cache

CPU

main memory

disk controller

oses: 11. fs impl. 1 operating systems v objectives –discuss file storage and access on secondary...

Documents