cse 513 introduction to operating systems class 8...
TRANSCRIPT
CSE 513Introduction to Operating Systems
Class 8 - Input/OutputFile Systems
Jonathan WalpoleDept. of Comp. Sci. and Eng.
Oregon Health and Science University
I/O devices
q Device (mechanical hardware)q Device controller (electrical hardware)q Device driver (software)
Devices and their controllers
q Components of a simple personal computer
Monitor
Bus
How to communicate with a device?
q Hardware supports I/O ports or memorymapped I/O for accessing device controllerregisters and buffers
Wide performance range for I/O
Performance challenges: I/O hardware
q How to prevent slow devices from slowing downmemory
q How to identify I/O addresses withoutinterfering with memory performance
Single vs. Dual Memory Bus Architecture
(a) A single-bus architecture(b) A dual-bus memory architecture
Hardware view of Pentium
Structure of a large Pentium system
Performance challenges: I/O software
q How to prevent CPU throughput from beinglimited by I/O device speed (for slow devices)
q How to prevent I/O throughput from beinglimited by CPU speed (for fast devices)
q How to achieve good utilization of CPU and I/Odevices
q How to meet the real-time requirements ofdevices
Programmed I/O
Steps in printing a string
Programmed I/O
q Polling/busy-waiting approach
copy_from_user(buffer,p,count);for(i=0;i<count;i++){ while (*p_stat_reg != READY); *p_data_reg = p[i]; }
return();
Interrupt-driven I/O
q Asynchronous approachv give device data, do something else!v resume when device interrupts
copy_from_user(buffer,p,count);enable_interrupts();while (*p_stat_reg != READY);*p_data_reg=p[0];scheduler();
if (count==0){ unblock_user();} else { *p_data_reg = p[i]; count--; i++;}ack_interrupt();return_from_interrupt();
Interrupt driven I/O
(b)
Hardware Support for Interrupts
How interrupts happens. Connections between devices and interruptcontroller actually use interrupt lines on the bus rather thandedicated wires
DMA
q Offload all work to a DMA controllerv avoids using the CPU to do the transferv reduces number of interruptsv DMA controller is like a co-processor doing
programmed I/O
copy_from_user(buffer,p,count);set_up_DMA_controller();scheduler();
ack_interrupt();unblock_user();return_from_interrupt();
DMA
Software engineering-related challenges
q How to remove the complexities of I/O handlingfrom application programsv standard I/O APIs (libraries and system calls)v generic categories across different device types
q How to support a wide range of device types ona wide range of operating systemsv standard interfaces for device driversv standard/published interfaces for access to kernel
facilities
I/O Software Layers
Layers of the I/O Software System
Interrupt Handlers
q Interrupt handlers are best hiddenv have driver starting an I/O operation block until
interrupt notifies of completion
q Interrupt procedure does its taskv then unblocks driver that started it
q Steps must be performed in software afterinterrupt completed
− Save regs not already saved by interrupt hardware− Set up context for interrupt service procedure
Interrupt Handlers
l Set up stack for interrupt service procedurel Ack interrupt controller, reenable interruptsl Copy registers from where savedl Run service procedurel Set up MMU context for process to run nextl Load new process' registersl Start running the new process
Device Drivers
q Communications between drivers and device controllers goes overthe bus
I/O Software: Device Drivers
q Device drivers “connect” devices with theoperating systemv Typically a nasty assembly-level job
• Must deal with hardware changes• Must deal with O.S. changes
v Hide as many device-specific details as possible
q Device drivers are typically given kernelprivileges for efficiencyv Can bring down O.S.!v How to provide efficiency and safety???
Device-Independent I/O Software Functions
Functions of the device-independent I/O software
Providing a deice-independent block size
Allocating and releasing dedicate devices
Error reporting
Buffering
Uniform interfacing for device drivers
Device-Independent I/O Software Interface
(a) Without a standard driver interface(b) With a standard driver interface
Device-Independent I/O Software Buffering
(a) Unbuffered input(b) Buffering in user space(c) Buffering in the kernel followed by copying to user space(d) Double buffering in the kernel
Copying Overhead in Network I/O
Networking may involve many copies
User-Space I/O Software
Layers of the I/O system and the main functionsof each layer
Devices as files
q Before mounting,v files on floppy are inaccessible
q After mounting floppy on b,v files on floppy are part of file hierarchy
Disk Geometry
q Disk head, platters, surfaces
cylinder
Track
Sector
Physical vs. logical disk geometry
q Constant Angular Velocity vs. Constant LinearVelocity
Disks
Disk parameters for the original IBM PC floppy disk and aWestern Digital WD 18300 hard disk
CD-ROMs
CD-ROM data layout
Logical data layout on a CD-ROM
CD-R Structure
q Cross section of a CD-R disk and laserv not to scale
q Silver CD-ROM has similar structurev without dye layerv with pitted aluminum layer instead of gold
Double sided dual layer DVD
Plastic technology
q CDsv Approximately 650 Mbytes of datav Approximately 74 minutes of audio
q DVDsv Many types of formats
• DVD-R, DVD-ROM, DVD-Videov Single layer vs. multi-layerv Single sided vs. double sidedv Authoring vs. non-authoring
Disk Formatting
A disk sector
Disk formatting with cylinder skew
Disk formatting with interleaving
q No interleavingq Single interleavingq Double interleaving
Disk scheduling algorithms
q Time required to read or write a disk blockdetermined by 3 factorsv Seek timev Rotational delayv Actual transfer time
q Seek time dominates
q Error checking is done by controllers
Disk scheduling algorithms
q First-come first serve
q Shortest seek time first
q Scan à back and forth to ends of disk
q C-Scan à only one direction
q Look à back and forth to last request
q C-Look à only one direction
Disk scheduling algorithms
Error handling complicates scheduling
q A disk track with a bad sectorq Substituting a spare for the bad sectorq Shifting all the sectors to bypass the bad one
RAID levels 0 to 2
RAID levels 3 to 5
Part B
File Systems
Long-term Information Storage
q Must store large amounts of data
q Information stored must survive the terminationof the process using it
q Multiple processes must be able to access theinformation concurrently
File naming and file extensions
Typical file extensions.
File Structure
q Three kinds of file structurev byte sequencev record sequencev tree
File Types
(a) An executable file (b) An archive
File access
q Sequential accessv read all bytes/records from the beginningv cannot jump around, could rewind or back upv convenient when medium was mag tape
q Random accessv bytes/records read in any orderv essential for data base systemsv read can be …
• move file marker (seek), then read or …• read and then move file marker
File attributes
File operations
CreateDeleteOpenCloseReadWrite
AppendSeekGet attributesSet AttributesRename
Exmple Program Using File System Calls (1/2)
Example Program Using File System Calls (2/2)
Memory-mapped files
(a) Segmented process before mapping filesinto its address space
(b) Process after mapping existing file abc into one segment creating new segment for xyz
Directories - single level
q A single level directory systemv contains 4 filesv owned by 3 different people, A, B, and C
Directories - two-level
Letters indicate owners of the directories and files
Hierarchical directory systems
Path names in Unix
Directory operations
CreateDeleteOpendirClosedir
ReaddirRenameLinkUnlink
File system implementation on disk
A possible file system disk layout
Implementing files - contiguous allocation
(a) Contiguous allocation of disk space for 7 files(b) State of the disk after files D and E have been removed
Implementing files - linked allocation
Storing a file as a linked list of disk blocks
Implementing Files - FAT
Linked list allocation using a file allocation table (FAT) in RAM
Implementing files - index nodes
An example i-node
Implementing directories
(a) A simple directoryfixed size entriesdisk addresses and attributes in directory entry
(b) Directory in which each entry just refers to an i-node
Implementing directories
q Two ways of handling long file names in directoryv (a) In-line, (b) In a heap
Shared files - hard links
File system containing a shared file
Problems with shared files
(a) Situation prior to linking(b) After the link is created(c)After the original owner removes the file
Disk space management and performance
q Dark line (left hand scale) gives data rate of a diskq Dotted line (right hand scale) gives disk space efficiencyq All files 2KB
Block size
Disk space management
(a) Storing the free list on a linked list(b) A bit map
Disk Space Management (3)
(a) Almost-full block of pointers to free disk blocks in RAM- three blocks of pointers on disk
(b) Result of freeing a 3-block file(c) Alternative strategy for handling 3 free blocks
- shaded entries are pointers to free disk blocks
Disk space management - quotas
Quotas for keeping track of each user’s disk use
Maintaining File System Consistency
q Crashes can cause file system to havecorrupted data
q First: bitmap vs. linked storage maps
q fsck / scandiskv Block-level à ensure that all blocks on disk are
accounted for• traverse inodes counting allocated blocks• compare result to free list
v File-level à ensure that all files are accounted for• traverse directory system counting files
File system consistency
q File system states(a) consistent(b) missing block(c) duplicate block in free list(d) duplicate data block
Maintaining File System Consistency
q File level consistencyv Have directories with i-node #’s of filesv Have link status in the i-nodes themselvesv Compare!
Performance issues
q Buffer cachev A buffer in between the application and the FS
• Hold buffer of accessed data• Allows data to be reused by other apps• Can implement prefetch strategies to minimize
latency• Serves a buffer for writing out application data
– Delayed writes allow the writes to happen whenconvenient
– Trade-off between consistency andperformance
v Hash table indexed to minimize the searching
File system performance - buffer cache
The buffer cache data structures
File system performance - data placement
q I-nodes placed at the start of the diskq Disk divided into cylinder groups
v each with its own blocks and i-nodes
Log-Structured File Systems
q With CPUs faster, memory largerv disk caches can also be largerv increasing number of read requests can come from cachev thus, most disk accesses will be writes
q LFS Strategy structures entire disk as a logv have all writes initially buffered in memoryv periodically write these to the end of the disk log
• long contiguous writes to disk• requires large contiguous free space!• data not updated in place
v when file opened, locate i-node, then find blocks
Example: The Unix file system
q File systems structure
q The file system in UNIX is an integral part of theoperating system. Files are used for:v Terminal handling /dev/tty***v Pipes “ls | more”v Directoriesv Sockets (for networking)
q The design of the UNIX file system allows the user towrite code that has a uniform interface.
Boot block superblock i-nodes data block data block …
Unix file system structures
q Superblock - tells the UNIX O.S. what theformat of the disk is, the # of blocks, thenumber of i-nodes, etc...
q I-nodes - are the metadata data structuresfor files and directoriesv Files - holds information such as owner, permissions,
date of modification, where the data isv Directories are just a special case of files that have
the pairs <file-name, i-node #> stored in the file
The Unix name space
q Filesv Represented with an I-node and disk blocks that
hold the datav File names are not stored with the i-node (they’re in
the directories that refer to them)• This is why there is a link count in the inode
q Directoriesv Directories are also filesv Entries are disblocks with <filename, i-node #> pairsv Special directory marking in i-node
Unix directory entries
A UNIX V7 directory entry
The Unix name space
/ inode #22
. 22
.. 22usr 24var 35home 26txt 23
/ inode #24
. 24
.. 22src 38conf 53
/ inode #53
## config..## Frame rate 33Frame size ..
Pathname translation in Unix
The steps in looking up /usr/ast/mbox
Unix I-node structure
q UNIX files consist of i-node and data blocks
q UNIX uses an indexed allocation scheme withv 10 direct pointers to blocksv 1 indirect pointer to blocksv 1 double indirect
pointer to blocksv 1 triple indirect
pointer to blocks
i-node
...
...
...
...
...
triple indirect pointer
Unix I-node structrure
A UNIX i-node
Maximum file size in Unix
q Several parameters determine how many (and of whatsize) files can be representedv No. of bits in a disk addressv No. of bits in a virtual memory referencev Disk block size
q Example:v 10 direct, 1 indirect, 1 double indirect, 1 triple indirectv No. bits in disk address --> 16-bitsv No. of bits in virtual memory reference --> 32-bitsv Disk block size --> 1024 kbytes
v What is the maximum file size in this system?
The “mount” command in Unix
q Commandsv “mount” allows file systems to be added to the
names spacev “umount” takes a file systems out of the name space
q Mount pointsv Mount points can be any directory in the name spacev Once a file system is mounted, the entire subtree
at the mount point is no longer accessible (until thefile system is unmounted)
Some UNIX file system features
q mounting - allows other file systems (disks),whether local or remote to be “mounted” into auniform file system name space
/
mntusr var
home X11
bin
Associating files with processes
q To provide uniform access to data, UNIX has a level ofindirection that is used for opening files
process 1
process n
...
Open File Table ......
.........
......
.........
...
Files
Each open file entry holds, permissions (R/W), file offset
File system vs. virtual memory
Process
Paging
FSDisk
Swap
OpenFile
Table
A file-centric view of IPC
q All input and output in UNIX are handledthrough the open file table structurev This is how UNIX can provide a single uniform
interface for local or remote data access
q Pipes in UNIX are nothing more thanv A writing process that has its “stdout” linked to a
“pipe” file (instead of a /dev/tty*** file)v A reading process that has its “stdin” linked to a
“pipe” file (instead of a /dev/tty*** file)