chapter11
TRANSCRIPT
CS2028 - Unix Internals
Chapter 1
General Overview of the Unix
History
System structure
User Perspective
Operating System Services
Assumptions about hardware
AgendaAgenda
3
What is Unix?
What is an operating system?
The lowlevel (first layer loaded) software which handles the interface to system hardware (input/output devices, memory, file system, etc), schedules tasks, and provides common core services such as a basic user interface.all of the software that comes with a system before applications are installed.
4
What Is Unix?
Examples of Operating Systems:Unix and Unixlike: A/UX, AIX, *BSD (Free, Micro, Net, Open, etc), Darwin (Mac OS X), HPUX, Hurd, IRIX, Linux, LynxOS, Minix, QNX, SELinux, Solaris, System V, Triance, TRUSIX, Tru64, UnixWare, VSTa, z/OS, etc.Embedded: BeOS, Chorus OS, eCos, FreeRTOS, IOS, JUNOS, LynxOS, QNX, VRTX, VxWorks, Windows CE, RTLinux, RTAI, Symbian, etc.Others: AOS, JavaOS, MorphOS, Primos, Windows 3.1/95/98/NT/XP/2000/2003, etc.
5
What is Unix? A portable, multi-tasking and multi-user operating system
Portable: runs on many different hardware architectures (Intel x86 and IA-64, Alpha, MIPS, HP PA-RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.).
Preemptive multi-tasking: several programs can run at the same time (time slices, interrupts, and task switching).
Multi-user: many users can share the computer system at the same time.
6
What is Unix? Other Features
Uses a simple, uniform file model which includes devices and access to other services in a flexible, hierarchical file system.
Written in a high-level language (“ C” ) making it easy to read, understand, change and port.
The command prompt is a simple user process, the Unix shell, which is also a convenient job programming language.
Includes support for regular expressions which are convenient for complex searching.
7
Why Do I Need to Know This?
Ubiquitous: Most big computers and much of the Internet infrastructure runs on some variant of Unix (SUN, SGI, HP, etc.).
Linux is the fastest growing operating system in the market. Source code availability provides the ability to “ get under the
hood” of operating system design and function (and TCP/IP). By understanding the Unix/Linux community and its culture, it is
easy to collect critical information.
HISTORY Multics: 1965 Bell with General Electronic Company and
Massachusetts Institute of Technology.GE645. Multiuser, Large data storage & power, data sharing.
UNIX: 1969 Thompson and Ritchie at AT&T Bell Labs – Unix File system – Thompson (Space travel program) – PDP 7
UNIX moved to PDP – 11 in 1971 UNIX in C (Ritchie) 1977 – UNIX ported into non PDP machine , Interdata 8/32 BSD: 1978 Berkeley Software Distribution- UNIX system III –
V(1983) Commercial Vendors: Sun, HP, IBM, SGI, DEC GNU: 1984 Richard Stallman, FSF POSIX: 1986 IEEE Portable Operating System unIX Minix: 1987 Andy Tannenbaum SVR4: 1989 AT&T and Sun Linux: 1991 Linus Torvalds Intel 386 (i386) Open Source: GPL, LGPL, Cathedral and the Bazaar
9
Unix History
1965 joint project between AT&T Bell Labs, GE(General Electric company), and MIT(Massachusetts Institute of Technology) to develop a new OS.
Goal : develop an OS that could provide computational power, data storage and the ability to share data among multiple users.
Result: Multiplexed Information & Computer Service - MULTICS.
10
Unix History
1969 Bell Labs withdraws from group.
Two Bell Lab scientists, Ken Thompson and Dennis Ritchie, continue research. They were still left without a “ Convenient interactive computing service” *.
* Ritchie, D.M. “ The Evolution of the Unix Time-sharing System” , AT&T Bell Laboratories Technical Journal, Oct. 1984, Vol 63, No.8, Part 2, pp. 1577-1594.
11
Unix History At the same time Ken Thompson
wrote a game “ space travel” in Fortran to run on GECOS OS (Honeywell 635).
The spaceship was hard to control and it was expensive to run. He was told to get the game off his work computer.
12
Unix History Thompson ported the game to a little used PDP-7
computer.
Unics (later Unix) was born as a pun on Multics.
13
Unix HistoryPDP-7
14
Unix History
Dennis Ritchie developed “ B” . Then wrote “ C” a compiled language.
In 1973 entire OS ported to “ C” .
15
Unix History➲ Because of a 1956 Consent Decree AT&T
could not market Unix so it provided it to academia.
➲ Late 70s : Thompson took a sabbatical to teach Unix at UC Berkley –
Birth of BSD Unix. Introduced many new features.
➲ AT&T Bell Labs realized the commercial potential and began distributing System V.
➲ Commercialization of Unix (70s / 80s) AT&T, Sun, SGI, HP, DEC, NCR, IBM.
16
Linus Torvalds
1991 Linux 0.02 is first released to the public. 1994 Linux 1.0 is released.
Three Definitions of Linux
Linux Kernel: The very low-level software that manages your computer hardware and provides a library (POSIX) interface for user-level software. The Linux kernel runs on many platforms (Intel x86 and IA-64, Alpha, MIPS, HP PA-RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.).
GNU/Linux OS: The Linux kernel plus utility software to provide a useful working environment.
Linux Distributions: The packaging of the Linux Kernel, the GNU/Linux OS and lots of other software to make Linux easy to install, configure, and use (at least for the target audience).
Tux, the Linux Mascot
What is UNIX?
Unix is a Operating System that popular since 1969. System divided into two parts First parts-programs and services such as shell, mail,
text processing packages and source code control systems
Second parts-system calls and hardware for supports to first parts
Example is UNIX system V produced by AT & T. Now Unix is maintained by Open Group.
Unix Features
High level language ‘ C’ Simple user interface Complex programs built from simpler programs Hierarchical file system Consistent format for files-byte streams Consistent interface to peripheral devices. Multi user, multi process system Hides machine architecture – portable Even though Unix is in ‘ C’ we can run other languages like cobol,
fortran, ada, lisp, prolog, basic, pascal, java with their Compiler or interpreter.
Linux Features
“ UNIX-like” operating system “ aims at” standards compliance “ all the features you would expect in a modern UNIX”
preemptive multitasking virtual memory (protected memory, paging) shared libraries demand loading, dynamic kernel modules shared copy-on-write executables TCP/IP networking
other features: SMP support, large memory, large files advanced networking, advanced filesystems efficient, stable, highly portable, supports most device hardware active development community, support, documentation, open source GUIs, applications
Architecture of Unix System
hardware
kernelsh who
date
ed
wc
grep
as
nroff
ldcc
cpp
emacs
Other apps
• OS interacts directly with the hardware • Such OS is called system kernel
Unix: Structure
• Operating System: a system that manages the resources of a computer.• Resources: CPUs, Memory, I/O devices, Network• Kernel: the memory resident portion of Unix system• File system and process control system are two major components of Unix Kernel.
Unix System Kernel
• Three major tasks of kernel: Process Management Device Management File Management
• Three additional Services for Kernel:✯ Virtual Memory✯ Networking✯ Network File Systems
• Experimental Kernel Features:✱ Multiprocessor support✱ Lightweight process (thread) support
Block Diagram of System Kernel
System Call Interface
File SubsystemInterprocess
communication
Scheduler
Memorymanagement
Process
control
subsystemDevice drivers
hardware control
hardware
LibrariesUser ProgramsUser LevelKernel Level
Hardware Level
Process Control Subsystem
• Process Synchronization• Interprocess communication• Memory management: • Scheduler: process scheduling
(allocate CPU to Processes)
File subsystem
• A file system is a collection of files and directories on a disk or tape in standard UNIX file system format. • Kernel’s file sybsystem regulates data flow between the kernel and secondary storage devices.
Hardware Control
• Hardware control is responsible for handling interrupts and for communicating with the machine.• Devices such as disks or terminals may interrupt the CPU while a process is executing.• The kernel may resume execution of the interrupted process after servicing the interrupt.
System Call
• A process accesses system resources through system call.• System call for
❂ Process Control: fork: create a new process wait: allow a parent process to synchronize its
execution with the exit of a child process. exec: invoke a new program. exit: terminate process execution❂ File system: File: open, read, write, lseek, close inode: chdir, chown chmod, stat fstat others: pipe dup, mount, unmount, link, unlink
User Perspective-File system
31
Architecture of Unix Unix file system
root (/)
bin
dev etc
tmpvar
home
spool
(users’ mail)
log
usr
X11R6
bin
(users’ accounts)
32
Typical Directories
➲ / : Root of the tree. Where it starts.
➲ bin, sbin, usr/bin: software for the shells and most common Unix commands.
➲ dev: short for devices, holds the files necessary to operate peripherals such as printers and terminals.
➲ home: contains the home directories of users (/export/home on sun computers).
33
Typical Directories
tmp: holds temporary files.
var: contains files that vary in size; (Mail directories, printer spool files, logs, etc.)
etc: administrative files such as lists of user names and passwords.
34
Typical Directories
usr: Contains application programs
lib: Contains libraries for programs
proc: a pseudo-filesystem used as an interface to kernel data structures.
Unix File System
File system is organised as a heirarchy of directories It starts from a single directory called root(represented
by a /).Types
Ordinary files Directories Special files Pipes
Directory
Directory is file containing list of files and subdirectories It has fixed size records of 16 bytes each which contains
- a filename(14 bytes)- an inode number (2 bytes) which acts as a pointer
to where the system can find info about the file.
Special Files
Special files are contained in the directory /dev. They are used to represent a real physical device such
as a printer, tape device etc. Ex: Special device - /dev/null(unwanted output can be
redirected).Pipes UNIX allows us to link commands together using a pipe. The pipe acts as a temporary file which only exists to
hold data from one command until its read by another.
ex: command1 | command2 | command3....
Processes
• A program is an executable file.• A process is an instance of the program in execution.• For example: create two active processes
$ emacs &$ emacs & $ psPID TTY TIME CMD12893 pts/4 0:00 tcsh12581 pts/4 0:01 emacs12582 pts/4 0:01 emacs$
Processes
• A program is an executable file.• A process is an instance of the program in execution.• For example: create two active processes
$ emacs &$ emacs & $ psPID TTY TIME CMD12893 pts/4 0:00 tcsh12581 pts/4 0:01 emacs12582 pts/4 0:01 emacs$
Processes
• A process has text: machine instructions
(may be shared by other processes) data stack
• Process may execute either in user mode and in kernel mode.• Process information are stored in two places:
❋ Process table ❋ User table
System call: fork()
• fork: the only way for a user to create a process in Unix operating system.• The process that invokes fork is called parent process and the newly created process is called child process.• The syntax of fork system call:
newpid = fork();• On return from fork system call, the two processes have identical copies of their userlevel context except for the return value pid. • In parent process, newpid = child process id• In child process, newpid = 0;
/* forkEx1.c */#include <stdio.h>
main(){ int fpid; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d\n", fpid); } else { printf("Parent Process fpid=%d\n", fpid); } printf("After forking fpid=%d\n", fpid); }
$ cc forkEx1.c o forkEx1$ forkEx1Before forking ...Child Process fpid=0After forking fpid=0Parent Process fpid=14707After forking fpid=14707$
/* forkEx1.c */#include <stdio.h>
main(){ int fpid; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d\n", fpid); } else { printf("Parent Process fpid=%d\n", fpid); } printf("After forking fpid=%d\n", fpid); }
$ cc forkEx1.c o forkEx1$ forkEx1Before forking ...Child Process fpid=0After forking fpid=0Parent Process fpid=14707After forking fpid=14707$
/* forkEx2.c */#include <stdio.h>
main(){ int fpid; printf("Before forking ...\n"); system("ps"); fpid = fork(); system("ps"); printf("After forking
fpid=%d\n", fpid);}
$ forkEx2Before forking ... PID TTY TIME CMD 14759 pts/9 0:00 tcsh 14778 pts/9 0:00 sh 14777 pts/9 0:00 forkEx2 PID TTY TIME CMD 14781 pts/9 0:00 sh 14759 pts/9 0:00 tcsh 14782 pts/9 0:00 sh 14780 pts/9 0:00 forkEx2 14777 pts/9 0:00 forkEx2After forking fpid=14780$ PID TTY TIME CMD 14781 pts/9 0:00 sh 14759 pts/9 0:00 tcsh 14780 pts/9 0:00 forkEx2After forking fpid=0
$ ps PID TTY TIME CMD 14759 pts/9 0:00 tcsh$
System Call: getpid() getppid()
• Each process has a unique process id (PID). • PID is an integer, typically in the range 0 through 30000.• Kernel assigns the PID when a new process is created.• Processes can obtain their PID by calling getpid().• Each process has a parent process and a corresponding parent process ID.• Processes can obtain their parent’s PID by calling getppid().
/* pid.c */#include <stdio.h>#include <sys/types.h>#include <unistd.h>
main(){ printf("pid=%d ppid=%d\n",getpid(), getppid());}
$ cc pid.c o pid$ pidpid=14935 ppid=14759$
/* forkEx3.c */#include <stdio.h>#include <sys/types.h>#include <unistd.h>main(){ int fpid; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d pid=%d ppid=%d\n", fpid, getpid(), getppid()); } else { printf("Parent Process fpid=%d pid=%d ppid=%d\n", fpid, getpid(), getppid()); } printf("After forking fpid=%d pid=%d ppid=%d\n", fpid, getpid(), getppid());}
$ cc forkEx3.c o forkEx3$ forkEx3Before forking ...Parent Process fpid=14942 pid=14941 ppid=14759After forking fpid=14942 pid=14941 ppid=14759$ Child Process fpid=0 pid=14942 ppid=1After forking fpid=0 pid=14942 ppid=1
$ ps PID TTY TIME CMD 14759 pts/9 0:00 tcsh
System Call: wait()
• wait system call allows a parent process to wait for the demise of a child process.
#include <stdio.h>#include <sys/types.h>#include <unistd.h>main(){ int fpid, status; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d pid=%d ppid=%d\n",
fpid, getpid(), getppid()); } else { printf("Parent Process fpid=%d pid=%d ppid=%d\n",
fpid, getpid(), getppid()); } wait(&status); printf("After forking fpid=%d pid=%d ppid=%d\n",
fpid, getpid(), getppid());}
$ cc forkEx4.c o forkEx4$ forkEx4Before forking ...Parent Process fpid=14980 pid=14979 ppid=14759Child Process fpid=0 pid=14980 ppid=14979After forking fpid=0 pid=14980 ppid=14979After forking fpid=14980 pid=14979 ppid=14759$
System Call: exec()
• exec() system call invokes another program by replacing the current process• No new process table entry is created for exec() program. Thus, the total number of processes in the system isn’t changed.• Six different exec functions: execlp, execvp, execl, execv, execle, execve, (see man page for more detail.)• exec system call allows a process to choose its successor.
/* execEx1.c */#include <stdio.h>#include <unistd.h>
main(){ printf("Before execing ...\n"); execl("/bin/date", "date", 0); printf("After exec\n"); }
$ execEx1Before execing ...Sun May 9 16:39:17 CST 1999$
/* execEx1.c */#include <stdio.h>#include <unistd.h>
main(){ printf("Before execing ...\n"); execl("/bin/date", "date", 0); printf("After exec\n"); }
$ execEx1Before execing ...Sun May 9 16:39:17 CST 1999$
/* execEx2.c */#include <sys/types.h>#include <unistd.h> #include <stdio.h>
main(){ int fpid; printf("Before execing ...\n"); fpid = fork(); if (fpid == 0) { execl("/bin/date", "date", 0); } printf("After exec and fpid=%d\n",fpid); }
$ execEx2Before execing ...After exec and fpid=14903$ Sun May 9 16:47:08 CST 1999$
/* execEx2.c */#include <sys/types.h>#include <unistd.h> #include <stdio.h>
main(){ int fpid; printf("Before execing ...\n"); fpid = fork(); if (fpid == 0) { execl("/bin/date", "date", 0); } printf("After exec and fpid=%d\n",fpid); }
$ execEx2Before execing ...After exec and fpid=14903$ Sun May 9 16:47:08 CST 1999$
Building Block Primitives
User to create complex program effectively. Users capability to (1)redirect I/O.
User much access three files Std input file Std output file Std error file
Example: ls, ls > ssn.txt, mail x < letter, 2>(err) (2)Pipe: mechanisms allows a stream of data to be
passed between reader and writer processes. Example: grep ssn file1.c file2.c file3.c file4.c | wc -l
< file - means open a file for reading and associate with STDIN. << token - Means use the current input stream as STDIN for
the program until token is seen. > file - means open a file for writing and truncate it and
associate it with STDOUT. >> file - means open a file for writing and seek to the end and
associate it with STDOUT. This is how we append to a file using a redirect.
n>&m means redirect FD n to the same places as FD m. Eg, 2>&1 means send STDERR to the same place that STDOUT is going to.
FD 0 – STDIN , FD 1 – STDOUT, FD 2 -STDERR
Operating System Services
Controlling Scheduling Memory Management
Allocating to main memory Allocating to secondary memory
Process management – controlled access to devices Transparent Service – Kernel hides the type of file from
user.
ASSUMPTIONS ABOUT H/W:User mode and Kernel mode
• At any given instant a computer running the Unix system is either executing a process or the kernel itself is running• The computer is in user mode when it is executing instructions in a user process and it is in kernel mode when it is executing instructions in the kernel.• Executing System call ==> User mode to Kernel mode perform I/O operations system clock interrupt
Difference between two modes
Process in user mode can access their own instruction not kernal instructions and data of other processes but in kernal mode can access both. Example: virtual address space access only in kernal mode.
Some machine instructions are privileged. Example: the process status register is not capable to access
from the user mode.
Kernel runs on behalf of the user processes. Part of each user process.
Interrupts and Exceptions
Interrupts: for multitasking CPU is shared by many processes
Blocks low priority interrupts but services high priority interrupts.
Exception refers to unexpected events caused by process.
Example: addressing illegal memory, executing privileged instructions and dividing by zero.
Restart the instruction after handling exception. Exceptions in middle of an instruction whereas interrupts
in between 2 instructions.
Processor Execution Levels
Kernal prevent the occurrence of interrupts during critical activity.
Manipulating linked list pointers High priority interrupts
Machine Errors Clock Disk
Lower Priority interrupts Network Devices Terminals Software Interrupts
Masking a level will block below and allow above.
Memory Management
Kernal resides in main memory permanently Compiler generates virtual address space Kernal Mapping virtual address to physical address