file systems cns 3060 operating systems. topicstopics files directories file system implementation...
Post on 19-Dec-2015
280 views
TRANSCRIPT
File SystemsFile SystemsFile SystemsFile SystemsCNS 3060CNS 3060
OperatingSystems
TopicsTopicsTopicsTopics
Files Directories File system implementation Example file systems
OperatingSystems
Long-term Long-term Information StorageInformation Storage
Long-term Long-term Information StorageInformation Storage
Must store large amounts of data
Information stored must survive the termination of the process using it
Multiple processes must be able to access the information concurrently
Must store large amounts of data
Information stored must survive the termination of the process using it
Multiple processes must be able to access the information concurrently
OperatingSystems
File StructureFile StructureFile StructureFile Structure
Three kinds of files byte sequence (O/S doesn’t care what’s inside) record sequence (O/S understands record structure) tree (Database systems)
Three kinds of files byte sequence (O/S doesn’t care what’s inside) record sequence (O/S understands record structure) tree (Database systems)
OperatingSystems
o read all bytes/records from the beginningo cannot jump around, could rewind or back up. o convenient when medium was mag tape
Sequential AccessSequential AccessSequential AccessSequential Access
OperatingSystems
Random AccessRandom AccessRandom AccessRandom Access
o bytes/records read in any order – disk driveso essential for data base systemso read can be … - move file marker (seek), then read or … - read and then move file marker
o bytes/records read in any order – disk driveso essential for data base systemso read can be … - move file marker (seek), then read or … - read and then move file marker
OperatingSystems
Typical File AttributesTypical File AttributesTypical File AttributesTypical File AttributesOperatingSystems
File OperationsFile OperationsFile OperationsFile Operations1. Create2. Delete3. Open4. Close5. Read6. Write
1. Create2. Delete3. Open4. Close5. Read6. Write
7. Append8. Seek9. Get attributes10. Set Attributes11. Rename
7. Append8. Seek9. Get attributes10. Set Attributes11. Rename
OperatingSystems
Memory Mapped FilesMemory Mapped FilesMemory Mapped FilesMemory Mapped Files
In some cases, it is convenient to map a file into the address space of a running process. File access is then done by normal reads and writes of memory. The result is much faster and to some, much easier than actual writing to the file.
Memory mapping is done by changing the system’s internal tables so that the file becomes backing store (ala paging) for the memory region into which the file is mapped.
OperatingSystems
Hierarchical Directory Hierarchical Directory SystemsSystems
Hierarchical Directory Hierarchical Directory SystemsSystems
Root
A B C
A
C C
B
C
B
B B
C C
C
OperatingSystems
Path NamesPath NamesPath NamesPath Names
A UNIX directory tree
A UNIX directory tree
OperatingSystems
Directory OperationsDirectory OperationsDirectory OperationsDirectory Operations
1. Create2. Delete3. Opendir4. Closedir
1. Create2. Delete3. Opendir4. Closedir
5. Readdir6. Rename7. Link8. Unlink
5. Readdir6. Rename7. Link8. Unlink
OperatingSystems
A File System A File System ImplementationImplementationA File System A File System
ImplementationImplementation
A possible file system layoutWhen the system is started, the BIOS reads in and executes the Master Boot Record (MBR). The MBR locates the active partition and finds it’s boot block. The boot block loads the O/S.
A possible file system layoutWhen the system is started, the BIOS reads in and executes the Master Boot Record (MBR). The MBR locates the active partition and finds it’s boot block. The boot block loads the O/S.
sector 01 partition is marked as active
OS Loader
OperatingSystems
superblock i-node table data
The superblock contains informationabout the file system itself, for example,how big is the file system
superblock i-node table data
This is an array of i-node structures. An i-nodeis identified by it’s position in the array.
I-NodesI-NodesI-NodesI-Nodes
When a file is opened, it’s i-node is loaded from disk into memory. Thus only a small amount of memory is required, and only while the file is opened.
I-nodes are fixed in size.
OperatingSystems
superblock i-node table data
All directories and files are stored in the data areaof the file system. Everything in the file system is stored in “blocks ”. Blocks are fixed in size and representthe smallest unit of storage in the file system.
Creating a new file involves the following operations:
superblock i-node table data
1. The O/S finds an unused i-node
47
Creating a new file involves the following operations:
superblock i-node table data
1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node
attributes
47
Creating a new file involves the following operations:
superblock i-node table data
1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node3. The kernel find free blocks and stores the file data
attributes
20
0
63
5
82
1
||||
||||
|
||||
||||
|
||||
||||
|
47
Creating a new file involves the following operations:
superblock i-node table data
1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node3. The kernel find free blocks and stores the file data4. The kernel stores the block number in the i-node
attributes
20
0
63
5
82
1
||||
||||
|
||||
||||
|
||||
||||
|
200
635
821
47
Creating a new file involves the following operations:
superblock i-node table data
1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node3. The kernel find free blocks and stores the file data4. The kernel stores the block number in the i-node5. The kernel adds an entry to the directory
attributes
20
0
63
5
82
1
||||
||||
|
||||
||||
|
||||
||||
|
200
635
821
47
47 myFile.txt
TerminologyTerminologyTerminologyTerminologyTrack
Read/writehead
SectorBlock or cluster
A block is the minimum unit of storage for data. Block sizes are defined by the O/S and the file system.
Internal Fragmentation: Occurs when all of a block is not used by a file.
External Fragmentation:Occurs when blocks usedto store a file are notcontiguous.
OperatingSystems
ProblemProblemProblemProblemTrack
Read/writehead
SectorBlock or cluster
A block is the minimum unit of storage for data. Block sizes are defined by the O/S and the file system.
OperatingSystems
Given that you need n blocks on the diskto hold the contents of a file, how do youallocate those blocks to the application?
1
23
4
Contiguous File Contiguous File AllocationAllocation
Contiguous File Contiguous File AllocationAllocation
The simplest file allocation scheme is to take blocks sequentially from the disk, as they are needed for each file. This has two major advantages:
o It is simple to implement. You only need to keep track of the starting block and the number of blocks in the file.
o It is very efficient. Only one seek is required to read in the entire file. (a seek is the operation that moves the read/write head over the correct track.)
OperatingSystems
ProblemProblemProblemProblemTrack
Read/writehead
SectorBlock or clusterOperatingSystems
1
2
4
Create a file of 3 blocks
Contiguous File Contiguous File AllocationAllocation
OperatingSystems
Create a file of 3 blocks
Create a file of 5 blocks
Contiguous File Contiguous File AllocationAllocation
OperatingSystems
Create a file of 3 blocks
Create a file of 5 blocks
Create a file of 4 blocks
Contiguous File Contiguous File AllocationAllocation
OperatingSystems
Create a file of 3 blocks
Create a file of 5 blocks
Create a file of 4 blocks
Create a file of 6 blocks
Contiguous File Contiguous File AllocationAllocation
OperatingSystems
Create a file of 3 blocks
Create a file of 5 blocks
Create a file of 4 blocks
Create a file of 6 blocks
Now . . . Delete the 2nd file
What’s the problem with this design?
External Disk Fragmentation.
Contiguous File Contiguous File AllocationAllocation
OperatingSystems
you have to have a filethat is 5 blocks or less
to fit here!
Linked List AllocationLinked List AllocationLinked List AllocationLinked List Allocation
+ Every block on disk can be used. Disk blocks can be anywhere.+ The directory only need store the address of the first block of the file.- Each block sacrifices the space required to store the pointer- Random access of blocks in the file is slow.
OperatingSystems
OperatingSystems
1
2 3
4
what if you want to randomly access this block?
Random Access
OperatingSystems
1
2 3
4
So ... you have to access this block,because it contains the pointer
to the next block.The directory only contains theaddress of the first block.
OperatingSystems
1
2 3
4
Now you need to access this blockto get the location of the 3rd block
. . . but this will more than likely involvea disk seek, i.e. move the disk head
A File Allocation Table (fat)A File Allocation Table (fat)A File Allocation Table (fat)A File Allocation Table (fat)
The fat table usually resides in a fixed location at the beginning of on the disk.
No space is taken up in the file for pointers.
OperatingSystems
OperatingSystems
1
2 3
4
Why cache the fat?
fat
move the disk head toread the first entry in
the fat
OperatingSystems
1
2 3
4
Why cache the fat?
fat
now move the disk head toread in the first block of
the file.
OperatingSystems
1
2 3
4
Why cache the fat?
fat
move the disk head backto read the next entry in
the fat
OperatingSystems
1
2 3
4
Why cache the fat?
fat
Move the disk head toread in the next block
in the file
When the fat is in cacheWhen the fat is in cacheWhen the fat is in cacheWhen the fat is in cache
+ Random access is easier – the chain is entirely in memory - The biggest disadvantage is that the FAT resides in memory assume a 20GB disk with 1024KB block-size. The FAT needs 20 million entries (60-80MB)
OperatingSystems
Unix uses i-nodes!
OperatingSystems
Directory ImplementationsDirectory ImplementationsDirectory ImplementationsDirectory Implementations
A simple Directory* File attributes stored in the directory* Disk address stored in the directory (first block)* Fixed size entries (so fixed length file names)
(MS/DOS & Windows3.x)
mail attributes
attributes
attributes
attributes
attributes
disk address
disk address
disk address
disk address
disk address
games
homework
music
photos
OperatingSystems
Directory ImplementationsDirectory ImplementationsDirectory ImplementationsDirectory Implementations
Each directory entry pointsto an i-node. File attributes are stored in the i-node.
(Unix)
mail address of i-node
games
homework
music
photos
address of i-node
address of i-node
address of i-node
address of i-node
Handling Long File Handling Long File NamesNames
Handling Long File Handling Long File NamesNames
Fragmentation Issues(take a directory entry out)Page Faults may occur(directory spans multiple pages)
OperatingSystems
Handling Long File Handling Long File NamesNames
Handling Long File Handling Long File NamesNamesOperating
Systems
File SharingFile Sharing
Unix allows different processes to share files …
The Process Table: every process has an entry in the process table includes all open file descriptors owned by the process - file descriptor flags - a pointer into the file tableprocess table entry
fd flags ptr
fd 0:fd 1:fd 2:
...
The File Table (per process) table of all open files - status flags for the file (read, write, append, etc) - the current file offset - a pointer to the i-node for this file
process table entry
fd flags ptr
fd 0:fd 1:fd 2:
...
file table entry
file status flags
current file offset
i-node pointer
The i-node Table one for each open file read from disk when the file is opened includes a pointer to the file’s i-node - file permissions - file owner - file size - device file is physically located on - pointers to the actual file blocks on disk - etc
process table entry
fd flags ptr
fd 0:fd 1:fd 2:
...
file table entry
file status flags
current file offset
i-node pointer
i-node
permissions
user & group ids
File size
Time stamps
. . .
Pointer to first disk block
process table entry
fd flags ptr
fd 0:fd 1:fd 2:
...
file table
file status flags
current file offset
i-node pointer
file status flags
current file offset
i-node pointer
i-node
i-node
A Single Process With Two Open FilesA Single Process With Two Open Files
permissions
user & group ids
File size
Time stamps
. . .
Pointer to first disk block
permissions
user & group ids
File size
Time stamps
. . .
Pointer to first disk block
process table entry
fd flags ptr
fd 0:fd 1:fd 2:
...
file table
file status flags
current file offset
v-node pointer
file table
file status flags
current file offset
v-node pointer
i-node
process table entry
fd flags ptr
fd 0:fd 1:fd 2:
...
Two Processes Sharing the Same FileTwo Processes Sharing the Same File
Note that as each process writes to the file, the file offset isupdated in the file table for that process, to reflect the number of bytes written. If this causes the file offset to exceed the filesize, the file size is updated in the files i-node.
permissions
user & group ids
File size
Time stamps
. . .
Pointer to first disk block
Sharing FilesSharing FilesSharing FilesSharing Files
This file appears in User B’s directory aswell as in User C’s directory.
This file appears in User B’s directory aswell as in User C’s directory.
In the simple directory case, both directoriesmust contain the block addresses for the file.
If user B adds to the file, the appended diskblocks will not show up in C’s directory.
This problem is solved using i-nodes, sinceeach directory entry only need point to thei-node.
OperatingSystems
B’s directory
i-node
file
C ’s directory
data
OperatingSystems
Deleting a Shared FileDeleting a Shared FileDeleting a Shared FileDeleting a Shared File
User C creates a file
i-node
User C’sdirectory
OperatingSystems
i-node owner = Ccount = 1
Deleting a Shared FileDeleting a Shared FileDeleting a Shared FileDeleting a Shared File
User B links to the shared file.
i-node
User C’sdirectory
OperatingSystems
i-node owner = Ccount = 2
User B’sdirectory
What do you do whenUser C deletes the file?
You can’t delete the file,Because the B’s Directory will point to an invalid i-node.
Deleting a Shared FileDeleting a Shared FileDeleting a Shared FileDeleting a Shared File
User B links to the shared file.
i-node
User C’sdirectory
OperatingSystems
i-node owner = Ccount = 1
User B’sdirectory
What do you do whenuser C deletes the file?
You can’t delete the file,because the B’s Directory will point to an invalid i-node.
All you can do is delete C’s directoryEntry and leave the file. This works, but may cause accounting problems.
Using what is called a symbolic link,B links to one of C’s files by creating anew file type called a Link. The Link filejust contains the path name of the fileit links to.
Using Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksOperating
Systems
User C’sdirectory
i-node type = file
User B’sdirectory
i-nodetype = link
datafile path file
Now, when C deletes the file,the i-node pointed to by B’sdirectory is valid. However,attempts to access the file will fail.
Using Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksOperating
Systems
User C’sdirectory
User B’sdirectory
i-nodetype = link
path file???
Disk Space Disk Space ManagementManagement
Block SizeBlock Size
Disk Space Disk Space ManagementManagement
Block SizeBlock Size
As we have seen, disk space is usually managed in fixed size blocks. The question arises, how big should the blocks be?
If the block is too large, then a lot of space is wasted.
If the block size is too small, then performance suffersbecause reading each block requires a rotational delayand a seek.
OperatingSystems
Determining Optimal Block Size
Determining Optimal Block Size
• Dark line (left hand scale) gives the data rate of a disk• Dotted line (right hand scale) gives disk space efficiency• Note that disk space utilization and data rate are in direct conflict!
• Dark line (left hand scale) gives the data rate of a disk• Dotted line (right hand scale) gives disk space efficiency• Note that disk space utilization and data rate are in direct conflict!
for 2KB files
wastedspace
data rate is almost completelydominated by seek time androtational delay of the disk.
Block Size
Managing Free Disk Managing Free Disk BlocksBlocks
Managing Free Disk Managing Free Disk BlocksBlocks
Linked Listbit map
n bytes to hold a disk block number 1 bit per blockFixed size
OperatingSystems
MS-DOS File SystemMS-DOS File SystemMS-DOS File SystemMS-DOS File SystemThe MS-DOS File System was patterned after
the CP/M File System
Directory entry – fixed 32 byte length
File Name
8
Ext
3
Attribute• read-only• archive• hidden• system file
bytes 1
Reserved
10
TimeDate
First block number in FAT
Size
2 2 2 4
Accurate to within 2 seconds
Pinned to 1980
OperatingSystems
FAT-12FAT-12FAT-12FAT-12
MS-DOS defines a block (called a cluster by Microsoft) as some multiple of 512 bytes.
FAT-12 blocks were 512 bytes. Each cluster is represented in the File Allocation table by a 12 bit number. Thus, the maximum partition size was about 2MB and the File location Table contained 4096 2-byte entries.
OperatingSystems
Fat-16Fat-16Fat-16Fat-16
As drives got bigger, the File Allocation Table structure had to change to accommodate the bigger sizes. In a FAT-16 table, each cluster was now represented by a full 16 bit entry in the file allocation table. Additional block sizes of 8 KB, 16KB, and 32 KB supported 2 GB partitions.
But what’s the problem with bigger block sizes?
OperatingSystems
Fat-32Fat-32
FAT-32 was introduced in the second release of Windows 98.FAT-32 represents each cluster in the file table with a 28-bitentry. The maximum partition size for a FAT-32 system is 2 Terabytes. A big advantage of FAT-32 is that for anequivalent partition size, FAT-32 blocks can be much smallerthan FAT-16 blocks. Thus, the file system is more efficient.
The downside is that the File Allocation Table is much bigger.For a 4KB block and a 2GB partition, the FAT takes up2 MB of RAM.
The Windows 98 File System
The Windows 98 File System
File Name
8
Ext
3
Attribute• read-only• archive• hidden• system file
bytes 1 10
TimeDate
Lower 16 bits offirst block number
Size
2 2 2 4
NT bit (for compatibility)Adds time accuracy to creation date/time - within 10ms
Last write
Creation date/time
Last access time
Upper 16 bits offirst block number
Long File NamesLong File NamesLong File NamesLong File Names
Windows 98 provides 2 file names for each file. * The standard DOS 8 + 3 file name * A long file name
The algorithm for creating an 8 + 3 file nameis to truncate the file name to 6 characters andadd ~1 to the name ( or ~2 if the name already exists).
OperatingSystems
Each long file name is stored in the directory, in front of thenormal directory entry for the file. Up to 13 Unicode charactersare stored in the following format. Multiple entries are used toProvide storage for the entire long file name.
Sequence
5 characters 0
Attribute byte0x0F
checksum
6 characters 0 2 chars
OperatingSystems
The Big Long Name
T h e B I g ~ 1
1 T h e B
A
A
A
NT
0
0
S
C
C
CreationTime
i g L o n
. . . low
0
0
g N
65 a m e
OperatingSystems
Windows NT File System
NTFS does not use sectors. Insteadit uses a cluster, which is some number(power of 2 ) sectors. This makes thefile system independent of sector size.
bootsector datamaster file table
systemfiles
free space bitmap
MFT Records
RecordHeader
attribute-value pairs
Attributeheader
data
Attributeheader
data
one of: standard info file name attribute list (location of additional mft records, if needed) volume name data . . .