cse 513 introduction to operating systems class 8...

CSE 513Introduction to Operating Systems

Class 8 - Input/OutputFile Systems

Jonathan WalpoleDept. of Comp. Sci. and Eng.

Oregon Health and Science University

I/O devices

q Device (mechanical hardware)q Device controller (electrical hardware)q Device driver (software)

Devices and their controllers

q Components of a simple personal computer

Monitor

Bus

How to communicate with a device?

q Hardware supports I/O ports or memorymapped I/O for accessing device controllerregisters and buffers

Wide performance range for I/O

Performance challenges: I/O hardware

q How to prevent slow devices from slowing downmemory

q How to identify I/O addresses withoutinterfering with memory performance

Single vs. Dual Memory Bus Architecture

(a) A single-bus architecture(b) A dual-bus memory architecture

Hardware view of Pentium

Structure of a large Pentium system

Performance challenges: I/O software

q How to prevent CPU throughput from beinglimited by I/O device speed (for slow devices)

q How to prevent I/O throughput from beinglimited by CPU speed (for fast devices)

q How to achieve good utilization of CPU and I/Odevices

q How to meet the real-time requirements ofdevices

Programmed I/O

Steps in printing a string

Programmed I/O

q Polling/busy-waiting approach

copy_from_user(buffer,p,count);for(i=0;i<count;i++){ while (*p_stat_reg != READY); *p_data_reg = p[i]; }

return();

Interrupt-driven I/O

q Asynchronous approachv give device data, do something else!v resume when device interrupts

copy_from_user(buffer,p,count);enable_interrupts();while (*p_stat_reg != READY);*p_data_reg=p[0];scheduler();

if (count==0){ unblock_user();} else { *p_data_reg = p[i]; count--; i++;}ack_interrupt();return_from_interrupt();

Interrupt driven I/O

(b)

Hardware Support for Interrupts

How interrupts happens. Connections between devices and interruptcontroller actually use interrupt lines on the bus rather thandedicated wires

DMA

q Offload all work to a DMA controllerv avoids using the CPU to do the transferv reduces number of interruptsv DMA controller is like a co-processor doing

programmed I/O

copy_from_user(buffer,p,count);set_up_DMA_controller();scheduler();

ack_interrupt();unblock_user();return_from_interrupt();

DMA

Software engineering-related challenges

q How to remove the complexities of I/O handlingfrom application programsv standard I/O APIs (libraries and system calls)v generic categories across different device types

q How to support a wide range of device types ona wide range of operating systemsv standard interfaces for device driversv standard/published interfaces for access to kernel

facilities

I/O Software Layers

Layers of the I/O Software System

Interrupt Handlers

q Interrupt handlers are best hiddenv have driver starting an I/O operation block until

interrupt notifies of completion

q Interrupt procedure does its taskv then unblocks driver that started it

q Steps must be performed in software afterinterrupt completed

− Save regs not already saved by interrupt hardware− Set up context for interrupt service procedure

Interrupt Handlers

l Set up stack for interrupt service procedurel Ack interrupt controller, reenable interruptsl Copy registers from where savedl Run service procedurel Set up MMU context for process to run nextl Load new process' registersl Start running the new process

Device Drivers

q Communications between drivers and device controllers goes overthe bus

I/O Software: Device Drivers

q Device drivers “connect” devices with theoperating systemv Typically a nasty assembly-level job

• Must deal with hardware changes• Must deal with O.S. changes

v Hide as many device-specific details as possible

q Device drivers are typically given kernelprivileges for efficiencyv Can bring down O.S.!v How to provide efficiency and safety???

Device-Independent I/O Software Functions

Functions of the device-independent I/O software

Providing a deice-independent block size

Allocating and releasing dedicate devices

Error reporting

Buffering

Uniform interfacing for device drivers

Device-Independent I/O Software Interface

(a) Without a standard driver interface(b) With a standard driver interface

Device-Independent I/O Software Buffering

(a) Unbuffered input(b) Buffering in user space(c) Buffering in the kernel followed by copying to user space(d) Double buffering in the kernel

Copying Overhead in Network I/O

Networking may involve many copies

User-Space I/O Software

Layers of the I/O system and the main functionsof each layer

Devices as files

q Before mounting,v files on floppy are inaccessible

q After mounting floppy on b,v files on floppy are part of file hierarchy

Disk Geometry

q Disk head, platters, surfaces

cylinder

Track

Sector

Physical vs. logical disk geometry

q Constant Angular Velocity vs. Constant LinearVelocity

Disks

Disk parameters for the original IBM PC floppy disk and aWestern Digital WD 18300 hard disk

CD-ROMs

CD-ROM data layout

Logical data layout on a CD-ROM

CD-R Structure

q Cross section of a CD-R disk and laserv not to scale

q Silver CD-ROM has similar structurev without dye layerv with pitted aluminum layer instead of gold

Double sided dual layer DVD

Plastic technology

q CDsv Approximately 650 Mbytes of datav Approximately 74 minutes of audio

q DVDsv Many types of formats

• DVD-R, DVD-ROM, DVD-Videov Single layer vs. multi-layerv Single sided vs. double sidedv Authoring vs. non-authoring

Disk Formatting

A disk sector

Disk formatting with cylinder skew

Disk formatting with interleaving

q No interleavingq Single interleavingq Double interleaving

Disk scheduling algorithms

q Time required to read or write a disk blockdetermined by 3 factorsv Seek timev Rotational delayv Actual transfer time

q Seek time dominates

q Error checking is done by controllers

Disk scheduling algorithms

q First-come first serve

q Shortest seek time first

q Scan à back and forth to ends of disk

q C-Scan à only one direction

q Look à back and forth to last request

q C-Look à only one direction

Disk scheduling algorithms

Error handling complicates scheduling

q A disk track with a bad sectorq Substituting a spare for the bad sectorq Shifting all the sectors to bypass the bad one

RAID levels 0 to 2

RAID levels 3 to 5

Part B

File Systems

Long-term Information Storage

q Must store large amounts of data

q Information stored must survive the terminationof the process using it

q Multiple processes must be able to access theinformation concurrently

File naming and file extensions

Typical file extensions.

File Structure

q Three kinds of file structurev byte sequencev record sequencev tree

File Types

(a) An executable file (b) An archive

File access

q Sequential accessv read all bytes/records from the beginningv cannot jump around, could rewind or back upv convenient when medium was mag tape

q Random accessv bytes/records read in any orderv essential for data base systemsv read can be …

• move file marker (seek), then read or …• read and then move file marker

File attributes

File operations

CreateDeleteOpenCloseReadWrite

AppendSeekGet attributesSet AttributesRename

Exmple Program Using File System Calls (1/2)

Example Program Using File System Calls (2/2)

Memory-mapped files

(a) Segmented process before mapping filesinto its address space

(b) Process after mapping existing file abc into one segment creating new segment for xyz

Directories - single level

q A single level directory systemv contains 4 filesv owned by 3 different people, A, B, and C

Directories - two-level

Letters indicate owners of the directories and files

Hierarchical directory systems

Path names in Unix

Directory operations

CreateDeleteOpendirClosedir

ReaddirRenameLinkUnlink

File system implementation on disk

A possible file system disk layout

Implementing files - contiguous allocation

(a) Contiguous allocation of disk space for 7 files(b) State of the disk after files D and E have been removed

Implementing files - linked allocation

Storing a file as a linked list of disk blocks

Implementing Files - FAT

Linked list allocation using a file allocation table (FAT) in RAM

Implementing files - index nodes

An example i-node

Implementing directories

(a) A simple directoryfixed size entriesdisk addresses and attributes in directory entry

(b) Directory in which each entry just refers to an i-node

Implementing directories

q Two ways of handling long file names in directoryv (a) In-line, (b) In a heap

Shared files - hard links

File system containing a shared file

Problems with shared files

(a) Situation prior to linking(b) After the link is created(c)After the original owner removes the file

Disk space management and performance

q Dark line (left hand scale) gives data rate of a diskq Dotted line (right hand scale) gives disk space efficiencyq All files 2KB

Block size

Disk space management

(a) Storing the free list on a linked list(b) A bit map

Disk Space Management (3)

(a) Almost-full block of pointers to free disk blocks in RAM- three blocks of pointers on disk

(b) Result of freeing a 3-block file(c) Alternative strategy for handling 3 free blocks

- shaded entries are pointers to free disk blocks

Disk space management - quotas

Quotas for keeping track of each user’s disk use

Maintaining File System Consistency

q Crashes can cause file system to havecorrupted data

q First: bitmap vs. linked storage maps

q fsck / scandiskv Block-level à ensure that all blocks on disk are

accounted for• traverse inodes counting allocated blocks• compare result to free list

v File-level à ensure that all files are accounted for• traverse directory system counting files

File system consistency

q File system states(a) consistent(b) missing block(c) duplicate block in free list(d) duplicate data block

Maintaining File System Consistency

q File level consistencyv Have directories with i-node #’s of filesv Have link status in the i-nodes themselvesv Compare!

Performance issues

q Buffer cachev A buffer in between the application and the FS

• Hold buffer of accessed data• Allows data to be reused by other apps• Can implement prefetch strategies to minimize

latency• Serves a buffer for writing out application data

– Delayed writes allow the writes to happen whenconvenient

– Trade-off between consistency andperformance

v Hash table indexed to minimize the searching

File system performance - buffer cache

The buffer cache data structures

File system performance - data placement

q I-nodes placed at the start of the diskq Disk divided into cylinder groups

v each with its own blocks and i-nodes

Log-Structured File Systems

q With CPUs faster, memory largerv disk caches can also be largerv increasing number of read requests can come from cachev thus, most disk accesses will be writes

q LFS Strategy structures entire disk as a logv have all writes initially buffered in memoryv periodically write these to the end of the disk log

• long contiguous writes to disk• requires large contiguous free space!• data not updated in place

v when file opened, locate i-node, then find blocks

Example: The Unix file system

q File systems structure

q The file system in UNIX is an integral part of theoperating system. Files are used for:v Terminal handling /dev/tty***v Pipes “ls | more”v Directoriesv Sockets (for networking)

q The design of the UNIX file system allows the user towrite code that has a uniform interface.

Boot block superblock i-nodes data block data block …

Unix file system structures

q Superblock - tells the UNIX O.S. what theformat of the disk is, the # of blocks, thenumber of i-nodes, etc...

q I-nodes - are the metadata data structuresfor files and directoriesv Files - holds information such as owner, permissions,

date of modification, where the data isv Directories are just a special case of files that have

the pairs <file-name, i-node #> stored in the file

The Unix name space

q Filesv Represented with an I-node and disk blocks that

hold the datav File names are not stored with the i-node (they’re in

the directories that refer to them)• This is why there is a link count in the inode

q Directoriesv Directories are also filesv Entries are disblocks with <filename, i-node #> pairsv Special directory marking in i-node

Unix directory entries

A UNIX V7 directory entry

The Unix name space

/ inode #22

. 22

.. 22usr 24var 35home 26txt 23

/ inode #24

. 24

.. 22src 38conf 53

/ inode #53

## config..## Frame rate 33Frame size ..

Pathname translation in Unix

The steps in looking up /usr/ast/mbox

Unix I-node structure

q UNIX files consist of i-node and data blocks

q UNIX uses an indexed allocation scheme withv 10 direct pointers to blocksv 1 indirect pointer to blocksv 1 double indirect

pointer to blocksv 1 triple indirect

pointer to blocks

i-node

...

...

...

...

...

triple indirect pointer

Unix I-node structrure

A UNIX i-node

Maximum file size in Unix

q Several parameters determine how many (and of whatsize) files can be representedv No. of bits in a disk addressv No. of bits in a virtual memory referencev Disk block size

q Example:v 10 direct, 1 indirect, 1 double indirect, 1 triple indirectv No. bits in disk address --> 16-bitsv No. of bits in virtual memory reference --> 32-bitsv Disk block size --> 1024 kbytes

v What is the maximum file size in this system?

The “mount” command in Unix

q Commandsv “mount” allows file systems to be added to the

names spacev “umount” takes a file systems out of the name space

q Mount pointsv Mount points can be any directory in the name spacev Once a file system is mounted, the entire subtree

at the mount point is no longer accessible (until thefile system is unmounted)

Some UNIX file system features

q mounting - allows other file systems (disks),whether local or remote to be “mounted” into auniform file system name space

/

mntusr var

home X11

bin

Associating files with processes

q To provide uniform access to data, UNIX has a level ofindirection that is used for opening files

process 1

process n

...

Open File Table ......

.........

......

.........

...

Files

Each open file entry holds, permissions (R/W), file offset

File system vs. virtual memory

Process

Paging

FSDisk

Swap

OpenFile

Table

A file-centric view of IPC

q All input and output in UNIX are handledthrough the open file table structurev This is how UNIX can provide a single uniform

interface for local or remote data access

q Pipes in UNIX are nothing more thanv A writing process that has its “stdout” linked to a

“pipe” file (instead of a /dev/tty*** file)v A reading process that has its “stdin” linked to a

“pipe” file (instead of a /dev/tty*** file)