let's make linux together

8/8/2019 Let's Make Linux Together

1/183


2/183


3/183


4/183


5/183


6/183


7/183


8/183


9/183


10/183


11/183


12/183


13/183

Shell:Shell is a program which interprets commands given by user. These commands

can be given in two ways: either typed or in a file called shell script. A variety of shells supported by Unix are: Bourne shell, Korn shell, C shell.Features of Shell:

-

communication between user and Unix system takes place through the shell- Shell allows background processing- A sequence of commands can be collected in a file called shell script. The name

of the file can be used to execute the sequence of commands automatically.- Shell includes features like loop and conditional constructs.- Input of one command can be taken from the output of another command or

output of one command can be diverted to the input of a file or printer. Twocommands can be combined using pipe operator.

Multi User:Unix is a multi-user OS. Each user has a different terminal. A terminal consists of

keyboard and a monitor and is connected to main computer known as host computer. A

host consists of hard disks, memory, processor and printer etc. these resources areaccessible to all the users.

Time Sharing System: In time sharing system, CPU time is divided among all the users. Unix works on

the concept of time sharing system. Each users program is allocated a short period of CPU time, one by one. This short period of CPU time is called Time Slice/ time slot orquantum. It may be of 10-20 ms. In time sharing environment, CPU switches from oneuser to another so rapidly that each user has the illusion that he alone is using thecomputer. eg: let us assume that the time slice for a time sharing system is 10 ms that isin one cycle each user gets 10 ms to execute its task.

Command Line and Command Syntax:A command in Unix is a series of characters. These characters consist of words

separated by white spaces. The first work is the command itself and the rest are thecommands argument. Unix commands are case-sensitive. ie: cd is different from Cd orCD. Type all your commands in lower case. Commands are issued to the shell at thecommand line. A command line comprises commands, the line of instructions, optiosnand any command-line argument that you provide. eg: a command line $man cp .Commands are entered at the shell prompt ($,#). Prompt is merely a symbol that appearsat the start of a command line. Prompt means Unix is ready and waiting for yourcommand.Note: a command line may have more than one command separated by semicolons(;).Pipes(|) or ampersands(&).


14/183

Internal and External Commands: Internal commands are such programs that are built-in into the OS and reside in

the memory along with the kernel. They are loaded in the memory at the time of booting.eg: cp, mv, rmdir, ls etc. all are internal commands.

External commands are not built-in into the OS but loaded from other program

files which resides on HDD/Floppy. They are loaded as and when required.Help:

To seek help on a command, we use $ man eg: $man cpA manual page will be displayed on the screen, with a lot of information such as

commands used, its syntax, switches/options available, examples etc.


15/183

File System :- All information in Unix is treated as a file- A single disk can store thousand of file.- To manage these files, OS provides a file system.- Similar files can be grouped in a directory.

-

The file system of Unix is the main key to success of Unix system.File Type in Unix:

Everything in Unix is treated as a file. eg: it treats I/O devices as files. There arethree types of files in Unix.

a) Ordinary files: All files created by user are called ordinary/regular files. Itincludes data file, program file, object file and executable file.

b) Directory files: For each directory, there is a file by same name as thedirectory which contains information about files under that directory. eg: forthe directory abc, there will be a directory file called abc. This file containsthe information on all the files and sub-directories under the directory abc.

Some main points about this are:a. It is automatically created by Unix, whene ver a directory is created.b. A directory file contains 2 fields- name of file and identification

number(inode)c. Cannot be modified by the user. Only change is done by Unix when a

old file is deleted or new file is created.c) Device files: Each I/O device is associated with a special file called device

file. Any combination to I/O device is done through device file.

Text and Binary files:A text file is a file that consists of text characters. A text file can be read by any

editor or word processor.A binary file is a program or data file that contains binary information in a

machine readable form rather than in a human readable form.

Structure of File System:It follows tree or hierarchical directory structure. It starts with root directory. It is

represented by forward slash(/). In Unix forward slash is used as a separator. Nore thatwindows or DOS use backslash(\) as a separator. Under the root directory, there areseveral system and home directory. Brief descriptions of these are given here:

1. /bin: contains binary file, executable program files. In his directory we can findthe files for Unix commands. It is same as command.com file in DOS

2. /dev: This directory contains the device files. eg: printer file is known as prn,HDD file is had. Its first partition is hda0

3. /etc: This directory contains all the configuration information of the system.4. /lib: contains the library files. It contains the reusable functions and routines for

the programmers to use.5. /tmp:This directory contains all the temporary informatoins same as

c:\windows\temp\ directory in windows.


16/183

6. /mnt: This contains the directories where other mounted file system reside likefloppy, CD, other partitions of HDD.

7. /usr: This directory contains the home directories of the users. There is one homedirectory for each user.

8. /kernel: This directory contains all the kernel specific code.

File naming conventions:1. filename can be up to 255 characters long.2. may or may not have extensions.3. can contains alphabets, digits, dots(.), hyphen(-), underscore(_) any where.4. can have any no. of dots. eg: a.b.c5. can contain both upper and lower case characters6. file names are case-sensitive7. should not have a blank space or tab.

****************************************


17/183


18/183

Unit2:History of Linux:

- Linus Torvald, a student at the University of Helsinki, Finland introducedLinux in 1991

- Torvalds worked on Linux project and wrote the source code of Linux kernel.

He made Linux available on net.- Many programmers added to the code, change it and built in support for allkind of hardware.

- There are several versions of Linux available for different hardware platforms.o Linux version 0.02 of Linux kernel was released in 1991

- Torvalds and hundreds of developers from across the world worked on it andin march 1994, version 1.0 of Linux kernel was released.

o Red Har Linux 6.0 uses version 2.25 of all the Linux kernel- Linux is Unix clone and has been written from scratch by Torvalds. Torvalds

was working in minix, a miniature version of Unix which was used forteaching purposes in universities and colleges.

-

Linux follows the open development model. Torvalds made the source codeavailable for study and changes over the internet. (and still is)- Torvalds accepts modification to the kernel code. The result is that whenever

a new version of Linux having new functionality was released, people work on it to fix the bug if any.

- To assess whether they are using some version, the following scheme is mageeg: version 1.x.y if x is even it signifies stable version else changes to be donevery soon its not stable.

Main Features of Linux:1. Multitasking: Linux support true preemptive multitasking. All the processes

run entirely independent of each other. No process needs to be concerned withmaking processor time available to other processes.

2. Multi-user: Linux allows a number of users to work with the system at thesame time.

3. Multiprocessing: From version 2.0 upwards, Linux also run on multiprocessorarchitectures. This means that the OS can distribute several applicationsacross several processors

4. Architecture Independence: Linux run on almost all the platforms that are ableto process bit and bytes. The supported hardware, from embedded systems toIBM mainframes, is sufficient. This kind of hardware independence is notachieved by any other serious OS.

5. Demand load executables: Only those parts of a program actually required forexecution are loaded into memory.

6. Shared Libraries: Linux introduced the concept of shared libraries. Thoselibraries which are used by almost all the programs are put in shared librarieswhich when required are used from that place.

7. Support for POSIX: Linux support all of the POSIX standards.


19/183

8. Various executable formats: Linux supports various executable formats like.out was old extensions which were supported by Linux and now ELF alsosupported by Linux and for .exe files there are Dos Emulator.

9. Different File System: This is that point which makes Linux much popularover other all OS. Linux supports 13 different file systems (NTFS, vfat,

FAT16, FAT32, DOS, ext2, proc, swap ) by help of a virtual file systemnamely VFS.10. Cron Scheduler: Linux has a scheduler program, called Cron scheduler. It is

used to run commands, shell scripts or programs at scheduled time.11. Office suits12. Data archiving utilities: Linux provides utilities for basic data backup such as

tar, cpio and dd. Red Hat 5.0 onwards also provide a Backup and Restore Unit(BRU), which can be purchased. BRU offers automated backup andscheduling.

13. Licensing: Linux is copyright under GNU general public license. Thislicensing for Red Hat Linux states that a person can make any number of

copies of softwares and distributes it freely or charge a price for it. One canfreely download Linux from internet for use.

Advantages of Linux:1. Reliability: Linux is stable OS. Linux se rvers are not shut down for years

together. This means that users on Linux OS work consistently withoutreporting OS failure.

2. Backward compatibility: Linux has an excellent support for older hardware. Itcan run on different type of processors not just intel 386/ 486 but also onDECs alpha, sun sparc machine, powe r PC etc.

3. GUI interface: The graphical interface for Linux is X windows system. It isdevided into 2 subsystem consisting of server and client. Linux has a numberof graphical user interfaces called Desktop Environments such as K DesktopEnvironment and GNU Object Model Environment, both of which areversions of X windows system.

a. When we start KDE, desktop is organized into folder such as autostart, CDROM, printer and floppy drive in form of icons

b. GNOME can be configured in the way you want to use it. It supportsthe drag and drop mechanism. Gnome follows the common clientrequest bnker architecture (CORBA) standards to allow differentsoftware to communicate easily.

4. Excellent Security features: It support high security that is why many ISP(internet service providers) are replacing their current OS with Linux system.

5. Development libraries: Linux provides an excellent platform for manydevelopment languages like c++, c, Perl, java, PHP and many more.

6. Can support high user load: Linux can support a large number of usersworking simultaneously.

7. No known viruses: Linux is free from any viruses attacks so far, there are noknown viruses for Linux.

8. Multiple Distributors: there are many distributors of Linux


20/183

9. Simple upgrade: the installation procedure of most Linux version is menudriven and easy. It includes the ability to upgrade from prior version. Theupgrade process preserves the existing configuration files and maintains a listof its actions during installation.

Multiple Distributions: To install Linux, the user requires a distribution. This consists of a boot disketteand other diskettes or a CD-ROM. Installation scripts enable even in experienced users toinstall systems that can be run. It helps that many software packages are already adaptedto Linux and appropriately configures: this saves a lot of time. Discussions are constantlytaking place with in Linux community on the quality of various distribution of this sort isa very lengthy and complex task.

Internationally, the RedHat, S.u.S.E, Debian and slackware distributions arewidely used. Which of these distributions is used is a matter of taste. Distributions can beobtained from FTP servers, e-mail systems, public domains distributors and somebookshops. Sources of supply can be found by consulting specialist magazines or the

Linux newsgroup in usenet.Unix vs. Linux:

Unix is costly while Linux is freely ava ilable Source of Linux is available while source of Unix is not available Linux supports all of the network topologies while Unix support only star

topology Linux can run on different platforms while for different hardware there are

different versions of Unix. Linux is GUI while Unix is CUI Linux is paged system while Unix is swap system. Linux can support 78 GB and 2 GB RAM while Unix can support 2 GB

and 512 MB. More shells are available under Linux Linux support development libraries, scripting languages but Unix not.

Pattern Matching: 1. ls a.???2. ls a*.*3. ls a[1 2 4]4. ls a[1-5]5. ls a[!1-5]

What are these what are their functions and why they are used? Try these onsystem and find difference or you may ask any teacher of Linux. For further information regarding these put these points in Google search.


21/183

Basic Commands: These commands are executed on a terminal ( main menu > system tools > terminal)

$ useradd : to add a new user$ adduser : to add a new user

$

passwd : to change the password$ passwd rohit and press enter it will change the password of user rohit byasking you the new password.

$ passwd and press enter it will change the password of root/administrator byasking you the new and confirm password

$ exit : to exit the terminal$ cat filename : to display the contents of a file$ cat > filename: create a file and to save press ctrl + d$ cat firstfilename secondfilename : to display content of both files. Firstly

first file followed by second file$ cat Ifile > IIfile : it will copy the content of first file into second file

$

cat Ifile >> IIfile: it will append the contents of first file into second file$ cat >. filename: it will create the hidden file$ ls: to see the list of files under current directory$ ls l: to see the long details with descriptions as permissions, users$ ls a: to see all files hidden and normal$ ls x: to see files in multicolumn$ ls Fx: F is used for identifying directories and executable files. * for files ad /

for folders.$ ls [mv]? r* : to list files that begin with m or w following any one characters,

then r and then any string.$ cat a*>b : to send all files starting with a to b$ bc: basic calculator. 12+5 and press enter, it will show 17. you can use

multiple expressions separated by semicolon


22/183

Overview of commands:

access() Used to check the accessibility of files

int Access(pathname, access_mode)Char* pathname;int access-mode;The access modes are.04 read02 write01 execute (search)00 checks existence of a file

& operator execute a command as a background process.

banner prints the specified string in large letters. Each argument may be upto 10 characters long.break is used to break out of a loop. It does not exit from the program.

Cal Produces a calender of the current month as standard output. The month (1-12) and year(1-9999) must be specified in full numeric format.Cal [[ month] year]

Calendar Displays contents of the calendar file

case operatorThe case operator is used to validate multiple conditions.

Case $string in

Pattern 1)Command list;;Command list;;

Pattern 3)Command list;;easc

cat (for concatenate) command is used to display the contents of a file. Used withoutarguments it takes input from standard input is used to terminate input.

cat [filename(s)]cat > [filename]Data can be appended to a file using >>

Some of the available options are :Cat [-options] filename(S)


23/183

-s silent about files thatcannot be accessed-v enables display of non printinging characters (except tabs, new lines, form-

feeds)-t when used with - v, it causes tabs to be printed as ^Is

-e when used with -v, it causes $ to be printed at the end of each lineThe -t and -e options are ignored if the -v options is not specified.

cd Used to change directories

chgrp Changes the group that owns a file.Chgrp [grou -id] [filename]

chmod Allows file permissions to be changed for each user. File permissions can be changedonly by the owner (s).Chmod [+/-][rwx] [ugo] [filename]chown Used to change the owner of a file.The command takes a file(s) as source files and the login id of another user as the target.Chown [user-id] [filename]

cmp The cmp command compares two files (text or binary) byte-by-byte and displays the firstoccurrence where the files differ.Cmp [filename1] [filename2] -1 gives a long listing

comm. The comm command compares two sorted files and displays the instances that arecommon. The display is separated into 3 columns.Comm. filename1 filename2first displays what occurs in first files but not in the secondsecond displays what occurs in second file but not in firstthird displays what is common in both files

continue statement The rest of the commands in the loop are ignored. It moves out of the loop and moves onthe next cycle.

cp

The cp (copy) command is used to copy a file.Cp [filename1] [filename2]

cpio(copy input/output) Utility program used to take backups.Cpio operates in three modes:-o output-i input-p pass


24/183

creat()the system call creates a new file or prepares to rewrite an existing file. The file pointer isset to the beginning of file.#include#include

int creat(path, mode)char *path;int mode;

cut used to cut out parts of a file. It takes filenames as command line arguments or input fromstandard input. The command can cut columns as well as fields in a file. It however doesnot delete the selected parts of the file.Cut [-ef] [column/fie,d] filenameCut-d :-f1,2,3 filenameWhere -d indicates a delimite r specified within :

df used to find the number of free blocks available for all the mounted file systems.#/etc/df [filesystem]

diff the diff command compares text files. It gives an index of all the lines that differ in thetwo files along with the line numbers. It also displays what needs to be changed.Diff filename1 filename2

echo The echo command echoes arguments on the command line.echo [arguments]

env Displays the permanent environment variables associated with a users login id

exit command Used to stop the execution of a shell script.

expr command Expr (command) command is used for numeric computation.The operators + (add), -(subtract), *(multiplu), /(divide), (remainder) are allowed.Calculation are performed in order of normal numeric precedence.

find The find command searches through directories for files that match the specified criteria.It can take full pathnames and relative pathnames on the command line.To display the output on screen the -print option must be specified

for operator The for operator may be used in looping constructs where there is repetitive execution of a section of the shell program.For var in vall val2 val3 val4;

Do commnds; done


25/183

fsck Used to check the file system and repair damaged files. The command takes a devicename as an argument# /etc/fsck /dev/file-system-to-be-checked.

grave operator Used to store the standard the output of a command in an enviroment variable. () grep The grep (global regular expression and print) command can be used as a filter to searchfor strings in files. The pattern may be either a fixed character string or a regularexpression.Grep string filename(s)

HOME Users home directory

if operator The if operator allows conditional operator

If expression; then commands; fiif thenelse fi $ if; then

commandsefile; then

commandsfi

kill used to stop background processes

In used to link files. A duplicate of a file is created with another name

LOGNAME displays users login name

ls Lists the files in the current directory

Some of the available options are:-l gives a long listing-a displays all file{including hidden files

lp used to print data on the line printer.Lp [options] filename(s)

mesgThe mesg command controls messages received on a terminal.-n does not allow messages to be displayed on screen-y allows messages to be displayed on screen


26/183

mkdirused to create directories

more The more command is used to dispay data one screenful at a time.More [filename]

mv Mv (move) moves a file from one directory to another or simply changes filenames. Thecommand takes filename and pathnames as source names and a filename or exitingdirectory as target names.mv [source-file] [target-file]

news The news command allows a user to read news items published by the systemadministrator.

ni Displays the contents of a file with line numbers

passwd Changes the password

paste The paste command joins lines from two files and displays the output. It can take anumber of filenames as command line arguments.paste file1 file2

PATH The directories that the system searches to find commands

pgUsed to display data one page (screenful) at a time. The command can take a number of filenames as arguments.Pg [option] [filename] [filename2]..

pipe Operator (1) takes the output of one commands as input of another command.

ps Gives information about all the active processes.

PS1The system prompt

pwd (print working directory) displays the current directory.

rm The rm (remove) command is used to delete files from a directory. A number of files maybe deleted simultaneously. A file(s) once deleted cannot be retrieved.rm [filename 1] [filename 2]


27/183

sift commandUsing shift $1becomes the source string and other arguments are shifted. $2 is shifted to$1,$3to $2 and so on.

Sleep The sleep command is used to suspend the execution of a shell script for the specified

time. This is usually in seconds.sort Sort is a utility program that can be used to sort text files in numeric or alphabetical orderSort [filename]

split Used to split large file into smaller filesSplit-n filenameSplit can take a second filename on the command line.

su Used to switch to superuser or any other user.

sync Used to copy data in buffers to files

system0 Used to run a UNIX command from within a C program

tail The tail command may be used to view the end of a file.Tail [filename]

tarUsed to save and restore files to tapes or other removable media.Tar [function[modifier]] [filename(s)]

tee output that is being redirected to a file can also be viewed on standard output.

test commandIt compares strings and numeric values.The test command has two forms : test command itself If test ${variable} = value thenDo commands else do commands

File The test commands also uses special operators [ ]. These are operators following the of are interpreted by the shell as different from wildcard characters.Of [ -f ${variable} ]

ThenDo commandsElif [ -d ${variable} ]

thendo commands


28/183

elsedo commands

fimany different tests are possible for files. Comparing numbers, character strings, valuesof environment variables.

time Used to display the execution time of a program or a command. Time is reported inseconds.Time filename values

tr The tr command is used to translate characters.tr [-option] [string1 [string2]]

ttyDisplays the terminal pathnameumask Used to specify default permissions while creating files.

uniqThe uniq command is used to display the uniq(ue) lines in a sorted file.Sort filename uniq

until The operator executes the commands within a loop as long as the test condition is false.

wall Used to send a message to all users logged in.# /etc/wall message

waitthe command halts the execution of a script until all child processes, executed asbackground processes, are completed.

wc The wc command can be used to count the number of lines, words and characters in afine.wc [filename(s)]The available options are:wc -[options] [filename]-1-w-cwhile operatorthe while operator repeatedly performs an operation until the test condition proves false.

$ while do


29/183

commands done

whodisplays information about all the users currently logged onto the system. The user name,terminal number and the date and time that each user logged onto the system.

The syntax of the who command is who [options]write The write command allows inter-user communication. A user can send messages byaddressing the other users terminal or login id. write user-name [terminal number]


30/183


31/183

Linux architecture:Kernel:

The core of Linux system is the kernel the OS program. Kernel controls theresources of computer, allot them to different users and tasks. It interacts directly withh/w so making programs easy to write and portable across the different platforms

hardware. Since kernel communicates directly with hardware, the part of kernel must becustomized to the h/w features of each system. However kernel does not deal directlywith user. Instead, the login process starts up a separate interactive program called shellfor each user.

Shell:Linux has a simple user interface b/w user and kernel called shell that has a power

to provide the service that a user wants. It protects the user from having to now theintricate h/w details.

Linux utilities and application programs:

Linux utilizes or commands are a collection of programs that service day to dayprocessing requirements. These programs are invoked through the shell which is itself another utility. Apart from utilities provided as a part of Linux OS, more that a thousandLinux based application programs like DBMS, word processor and various otherprograms are available from independent software vendors.


32/183

Vi editor: General Startup

To use vi: vi filenameTo exit vi and save changes: ZZ or :wqTo exit vi without saving changes: :q!

To enter vi command mode: [esc] Counts

A number preceding any vi command tells vi to repeat that command that manytimes.

Cursor Movement

h move left (backspace)

j move downk move up

l move right (spacebar)

[return] move to the beginning of the next line

$ last column on the current line

0 move cursor to the first column on the current line

^ move cursor to first nonblank column on the current line

w move to the beginning of the next word or punctuation mark

W move past the next space

b move to the beginning of the previous word or punctuation mark

B move to the beginning of the previous word, ignores punctuation

e end of next word or punctuation mark E end of next word, ignoring punctuation

H move cursor to the top of the screen

M move cursor to the middle of the screen


33/183

L move cursor to the bottom of the screen

Screen Movement

G move to the last line in the file

xG move to line x

z+ move current line to top of screen

z move current line to the middle of screen

z- move current line to the bottom of screen

^F move forward one screen

^B move backward one line^D move forward one half screen

^U move backward one half screen

^R redraw screen ( does not work with VT100 type terminals )

^L redraw screen ( does not work with Televideo terminals )

Inserting

r replace character under cursor with next character typed

R keep replacing character until [esc] is hit

i insert before cursor

a append after cursor

A append at end of line

O open line above cursor and enter append mode


34/183

Deleting

x delete character under cursor

dd delete line under cursor

dw delete word under cursor

db delete word before cursor

Copying Code

yy (yank)'copies' line which may then be put by the p(put) command. Precedewith a count for multiple lines.

Put Command

brings back previous deletion or yank of lines, words, or characters

P bring back before cursor

p bring back after cursor

Find Commands

? finds a word going backwards

/ finds a word going forwards

f finds a character on the line under the cursor going forward

F finds a character on the line under the cursor going backwards

t find a character on the current line going forward and stop one characterbefore it

T find a character on the current line going backward and stop onecharacter before it

; repeat last f, F, t, T


35/183

Miscellaneous Commands

. repeat last command

u undoes last command issued

U undoes all commands on one line

xp deletes first character and inserts after second (swap)

J join current line with the next line

^G display current line number

% if at one parenthesis, will jump to its mate

mx mark current line with character x'x find line marked with character x

NOTE: Marks are internal and not written to the file.

Line editor mode

Any commands form the line editor ex can be issued

upon entering line mode.

To enter: type ':'

To exit: press [return] or [esc]

Reading files

copies (reads) filename after cursor in file currently editing

:r filename

Write file

:w saves the current file without quitting


36/183

Moving

:# move to line #

:$ move to last line of file


37/183

Starting ee

To edit a file simply type ee followed by the filename at your Unix prompt, for example:

ee mytext

If a file of that name exists, the start of the file is then displayed on the screen, otherwisean empty file of that name is created.

Full details of the ee command can be seen by typing

man ee

at your Unix prompt.

Text and Commands

Unlike some other editors, there are no special modes to worry about. Typing ordinarytext will insert it into the document at the position of the cursor, and commands areprovided by [Ctrl] key combinations, a list of which is shown permanently at the top of the screen as follows:

^[ (escape) menu ê search prompt ^y delete line û up ^p prevpageâ ascii code ^x search ^z undelete line ^d down ^n next page^b bottom of text ^g begin of line ^w delete word ^l left

^t top of text ô end of line ^v undelete word ^r right Note: ^means^c command ^k delete char ^f undelete char hold downCtrl

The caret symbol (^) indicates that the [Ctrl] key should be held down while pressing therelevant key. On most keyboards, the cursor arrow keys will work correctly as well as the[Ctrl] commands for cursor movement, and some keyboards also have [Page Up],[Page Down] and [Delete] keys which can be used.

Using Menus

You can call up a menu of additional commands by pressing [Ctrl+[] or [Esc]. This mainmenu appears as follows:


38/183

The cursor will be over the top menu item. leave editor in this case.

To select a menu item, move the cursor down to the required item using the cursormotion commands or arrow keys and press the Enter key. Some items call up a furthermenu.

Further Commands

Typing [Ctrl+C] will cause a prompt command: to appear at the bottom of the screen andthe panel at the top of the screen to be replaced by the following list of commands:

help : get help info |file : print file name |line : print line #

read : read a file |char : ascii code of char |0-9 : go to line "#"write: write a file |case : case sensitive search |exit : leave and save!cmd : shell "cmd" |nocase: ignore case in search |quit : leave, nosaveexpand: expand tabs |noexpand: do not expand tabs

Text and Commands

Unlike some other editors, there are no special modes to worryabout. Typing ordinary text will insert it into the document at the

position of the cursor, and commands are provided by [Ctrl] keycombinations, a list of which is shown permanently at the top of thescreen as follows:

command:

To obey one of these commands, simply type the command name at the prompt and pressthe Enter key. Most of the commands are also available via the main menu, so you canchoose your favourite way of doing some tasks. Commands may be abbreviated, as longas that abbreviation is unique. For example, as no other command starts with a q, you canquit the editor without saving your changes by typing the sequence [Ctrl+C] [q] [Enter].

Invoking ee from Mail and other Programs

A number of programs, including the pine > mailer, call a text editor directly. The defaulteditor on IS Unix Services is ee .

Tip for mail users: to incorporate a file into a mail message, you can use the read command on the [Ctrl+C] menu.


39/183

Quitting ee

Leaving the editor is achieved by selecting the top item (leave editor) in the main menu,which calls the following further menu:

Text and Commands +---------------------+| leave menu || |

Unlike some other editors, | save changes | s to worryabout. Typing ordinary text | no save | document at theposition of the cursor, and | | [Ctrl] keycombinations, a list of whi | press Esc to cancel | t the top of thescreen as follows: +---------------------+

So the usual method of finishing an editing session is to type the key sequence [Esc][Enter] [Enter]


40/183

ee

ee stands for "easy editor" ee is a very easy to use editor which always displays thekeybindings on line so that you don't have to know anything to use it. It is good forpeople who don't want to learn how to use a text editor.

Emacs

Emacs is a popular text editor. it has several features including extensibility,customisability, and several powerful features file comparison, spell checking, syntaxhilighting which is good but not as good as that on vim, an IDE, mail and news reading ...calling emacs an editor is a little misleading. Compared to vim , it seems fast for editingsmall regions of text but slower for editing large files , as the vim commands are morecompact, and in some cases more powerful. The nice use of emacs-style keybindings isfor interactive shells: the default line editor in bash resembles emacs.

JedJed is a versatile editor that can "impersonate" a number of well known text editorsincludiong emacs. For people looking for a wordstar clone or a lightweight version of emacs, jed is nice.

Joe

Joe is a lot like jed: it is a lightwieght editor that includes different modes including pico,wordstar and emacs.

mceditmcedit is the editor that ships with midnight commander. It is very dos like, it looks andfeels a lot like dos edit. Good for nostalgic windows refugees.

Nedit

Nedit is a GUI editor which should make windows users feel at home. It has someprogrammer-friendly features, and is essentially a superset of the classic windows texteditors (notepad) in functionality.

PicoPico ships with the popular email client pine pico is much like ee in a number of ways. Itis not terribly powerful (in fact quite the opposite) but easy to use. Good for people whodon't want to know how to use a text editor.


41/183

Process Management

Any application that runs on a Linux system is assigned a process ID or PID. This is anumerical representation of the instance of the application on the system. In mostsituations this information is only relevant to the system administrator who may have to

debug or terminate processes by referencing the PID. Process Management is the seriesof tasks a System Administrator completes to monitor, manage, and maintain instances of running applications.

Multitasking

Process Management beings with an understanding concept of Multitasking. Linux iswhat is referred to as a preemptive multitasking operating system. Preemptivemultitasking systems rely on a scheduler. The function of the scheduler is to control theprocess that is currently using the CPU. In contrast, symmetric multitasking systems suchas Windows 3.1 relied on each running process to voluntary relinquish control of the

processor. If an application in this system hung or stalled, the entire computer systemstalled. By making use of an additional component to pre-empt each process when itsturn is up, stalled programs do not affect the overall flow of the operating system.

Each turn is called a time slice, and each time slice is only a fraction of a second long.It is this rapid switching from process to process that allows a computer to appear to bedoing two things at once, in much the same way a movie appears to be a continuouspicture.

Types of Processes

There are generally two types of processes that run on Linux. Interactive processes arethose processes that are invoked by a user and can interact with the user. VI is anexample of an interactive process. Interactive processes can be classified into foregroundand background processes. The foreground process is the process that you are currentlyinteracting with, and is using the terminal as its stdin (standard input) and stdout(standard output). A background process is not interacting with the user and can be in oneof two states paused or running.

The following exercise will illustrate foreground and background processes.

1. Logon as root.

2. Run [cd \]

3. Run [vi]

4. Press [ctrl + z]. This will pause vi

5. Type [jobs]


42/183

6. Notice vi is running in the background

7. Type [fg %1]. This will bring the first background process to the foreground.

8. Close vi.

The second general type of process that runs on Linux is a system process or Daemon(day- mon). Daemon is the term used to refer to process that are running on the computer and provide services but do not interact with the console. Most server software isimplemented as a daemon. Apache, Samba, and inn are all examples of daemons.

Any process can become a daemon as long as it is run in the background, and does notinteract with the user. A simple example of this can be achieved using the [ls R]command. This will list all subdirectories on the computer, and is similar to the [dir /s]command on Windows. This command can be set to run in the background by typing [ls

R &], and although technically you have control over the shell prompt, you will be able

to do little work as the screen displays the output of the process that you have running inthe background. You will also notice that the standard pause (ctrl+z) and kill (ctrl+c)commands do little to help you.


43/183

System Call:System call is the services provided by Linux kernel. In C programming, it often usesfunctions defined in libcwhich provides a wrapper for many system calls. Manual page section 2 provides moreinformation about

system calls . To get an overview, use man 2 intro in a command shell. It is also possible to invoke syscall() function directly. Each system call has a functionnumber defined inor . Internally, system call is invokded by software interrupt 0x80to transfer control tothe kernel. System call table is defined in Linux kernel source filearch/i386/kernel/entry.S . System Call Example#include #include #include

#include int main(void) {long ID1, ID2;

/*-----------------------------*/ /* direct system call */ /* SYS_getpid (func no. is 20) */ /*-----------------------------*/ ID1 = syscall(SYS_getpid);printf ("syscall(SYS_getpid)=%ld\n", ID1);

/*-----------------------------*/ /* "libc" wrapped system call */ /* SYS_getpid (Func No. is 20) */ /*-----------------------------*/ ID2 = getpid();printf ("getpid()=%ld\n", ID2);return(0);}System Call Quick ReferenceNo Func Name DescriptionSource1exitterminate the current processkernel/exit.c 2fork create a child processarch/i386/kernel/process.c 3read


44/183


45/183

change ownership of a file fs/open.c 18 statget file status

fs/stat.c

19 lseek reposition read/write file offset fs/read_write.c 20 getpidget process identificationkernel/sched.c 21 mountmount filesystems

fs/super.c 22 umountunmount filesystems

fs/super.c 23 setuidset real user IDkernel/sys.c 24 getuidget real user IDkernel/sched.c 25 stimeset system time and datekernel/time.c 26 ptraceallows a parent process to control the execution of a child processarch/i386/kernel/ptrace.c 27 alarmset an alarm clock for delivery of a signalkernel/sched.c 28 fstatget file status

fs/stat.c 29 pausesuspend process until signalarch/i386/kernel/sys_i386.c 30 utimeset file access and modification times

fs/open.c 33 accesscheck user's permissions for a file

fs/open.c 34 nice


46/183

change process prioritykernel/sched.c 36 syncupdate the super block

fs/buffer.c

37 killsend signal to a processkernel/signal.c 38 renamechange the name or location of a file

fs/namei.c 39 mkdircreate a directory

fs/namei.c 40 rmdirremove a directory

fs/namei.c 41 dupduplicate an open file descriptor

fs/fcntl.c 42 pipecreate an interprocess channelarch/i386/kernel/sys_i386.c 43 timesget process timeskernel/sys.c 45 brk change the amount of space allocated for thecalling process's data segmentmm/mmap.c 46 setgidset real group IDkernel/sys.c 47 getgidget real group IDkernel/sched.c 48 sys_signalANSI C signal handlingkernel/signal.c 49 geteuidget effective user IDkernel/sched.c 50 getegidget effective group IDkernel/sched.c


47/183

51 acctenable or disable process accountingkernel/acct.c 52 umount2

unmount a file system fs/super.c 54 ioctlcontrol device

fs/ioctl.c 55 fcntlfile control

fs/fcntl.c 56 mpx(unimplemented)57 setpgid

set process group IDkernel/sys.c 58 ulimit(unimplemented)59 oldunameobsolete uname system callarch/i386/kernel/sys_i386.c 60 umask set file creation mask kernel/sys.c 61 chroot

change root directory fs/open.c 62 ustatget file system statistics

fs/super.c 63 dup2duplicate a file descriptor

fs/fcntl.c 64 getppidget parent process IDkernel/sched.c

65 getpgrpget the process group IDkernel/sys.c 66 setsidcreates a session and sets the process group IDkernel/sys.c 67 sigactionPOSIX signal handling functions


48/183

arch/i386/kernel/signal.c 68 sgetmask ANSI C signal handlingkernel/signal.c 69 ssetmask

ANSI C signal handlingkernel/signal.c 70 setreuidset real and effective user IDskernel/sys.c 71 setregidset real and effective group IDskernel/sys.c 72 sigsuspendinstall a signal mask and suspend caller untilsignal

arch/i386/kernel/signal.c 73 sigpendingexamine signals that are blocked and pendingkernel/signal.c 74 sethostnameset hostnamekernel/sys.c 75 setrlimitset maximum system resource con sumptionkernel/sys.c 76 getrlimitget maximum system resource con sumptionkernel/sys.c 77 getrusageget maximum system resource con sumptionkernel/sys.c 78 gettimeofdayget the date and timekernel/time.c 79 settimeofdayset the date and timekernel/time.c 80 getgroupsget list of supplementary group IDskernel/sys.c 81 setgroupsset list of supplementary group IDskernel/sys.c 82 old_selectsync. I/O multiplexing


49/183

arch/i386/kernel/sys_i386.c 83 symlink make a symbolic link to a file

fs/namei.c 84 lstat

get file status fs/stat.c 85 readlink read the contents of a symbolic link

fs/stat.c 86 uselibselect shared library

fs/exec.c 87 swaponstart swapping to file/devicemm/swapfile.c

88 rebootreboot or enable/disable Ctrl-Alt-Delkernel/sys.c 89 old_readdirread directory entry

fs/readdir.c 90 old_mmapmap pages of memoryarch/i386/kernel/sys_i386.c 91 munmapunmap pages of memorymm/mmap.c 92 truncateset a file to a specified length

fs/open.c 93 ftruncateset a file to a specified length

fs/open.c 94 fchmodchange access permission mode of file

fs/open.c 95 fchownchange owner and group of a file

fs/open.c 96 getpriorityget program scheduling prioritykernel/sys.c 97 setpriorityset program scheduling prioritykernel/sys.c


50/183

98 profilexecution time profile99 statfsget file system statistics

fs/open.c

100 fstatfsget file system statistics fs/open.c 101 iopermset port input/output permissionsarch/i386/kernel/ioport.c 102 socketcallsocket system callsnet/socket.c 103 syslogread and/or clear kernel message ring buffer

kernel/printk.c 104 setitimerset value of interval timerkernel/itimer.c 105 getitimerget value of interval timerkernel/itimer.c 106 sys_newstatget file status

fs/stat.c 107 sys_newlstatget file status

fs/stat.c 108 sys_newfstatget file status

fs/stat.c 109 oldunameget name and information about current kernelarch/i386/kernel/sys_i386.c 110 ioplchange I/O privilege levelarch/i386/kernel/ioport.c 111 vhangupvirtually hangup the current tty

fs/open.c 112 idlemake process 0 idlearch/i386/kernel/process.c 113 vm86oldenter virtual 8086 mode


51/183


52/183

initialize a loadable module entrykernel/module.c 129 delete_moduledelete a loadable module entrykernel/module.c

130 get_kernel_symsretrieve exported kernel and module symbolskernel/module.c 131 quotactlmanipulate disk quotas

fs/dquot.c 132 getpgidget process group IDkernel/sys.c

133 fchdirchange working directory

fs/open.c 134 bdflushstart, flush, or tune buffer-dirty-flush daemon

fs/buffer.c 135 sysfsget file system type information

fs/super.c 136 personalityset the process execution domain

kernel/exec_domain.c 137 afs_syscall(unimplemented)138 setfsuidset user identity used for file system checkskernel/sys.c 139 setfsgidset group identity used for file system checkskernel/sys.c 140 sys_llseek move extended read/write file pointer

fs/read_write.c 141 getdentsread directory entries

fs/readdir.c 142 selectsync. I/O multiplexing

fs/select.c 143 flock


53/183

apply or remove an advisory lock on an open file fs/locks.c 144 msyncsynchronize a file with a memory mapmm/filemap.c

145 readvread data into multiple buffers fs/read_write.c 146 writevwrite data into multiple buffers

fs/read_write.c 147 sys_getsidget process group ID of session leaderkernel/sys.c 148 fdatasyncsynchronize a file's in-core data with that on disk

fs/buffer.c 149 sysctlread/write system parameterskernel/sysctl.c 150 mlock lock pages in memorymm/mlock.c 151 munlock unlock pages in memorymm/mlock.c 152 mlockalldisable paging for calling processmm/mlock.c 153 munlockallreenable paging for calling processmm/mlock.c 154 sched_setparamset scheduling parameterskernel/sched.c 155 sched_getparamget scheduling parameterskernel/sched.c 156 sched_setscheduler set scheduling algorithm parameterskernel/sched.c 157 sched_getscheduler get scheduling algorithm parameterskernel/sched.c 158 sched_yieldyield the processorkernel/sched.c 159


54/183

sched_get_priority_maxget max static priority rangekernel/sched.c 160

sched_get_priority_minget min static priority rangekernel/sched.c 161sched_rr_get_intervalget the SCHED_RR interval for the named processkernel/sched.c 162 nanosleeppause execution for a specified time (nano seconds) kernel/sched.c

163 mremapre-map a virtual memory addressmm/mremap.c 164 setresuidset real, effective and saved user or group IDkernel/sys.c 165 getresuidget real, effective and saved user or group IDkernel/sys.c 166 vm86enter virtual 8086 modearch/i386/kernel/vm86.c 167 query_modulequery the kernel for various bits pertain ing tomoduleskernel/module.c 168 pollwait for some event on a file descriptor

fs/select.c 169 nfsservctlsyscall interface to kernel nfs daemon

fs/filesystems.c 170 setresgidset real, effective and saved user or group IDkernel/sys.c 171 getresgidget real, effective and saved user or group IDkernel/sys.c 172 prctloperations on a process


55/183

kernel/sys.c 173 rt_sigreturnarch/i386/kernel/signal.c 174 rt_sigactionkernel/signal.c

175 rt_sigprocmask kernel/signal.c 176 rt_sigpendingkernel/signal.c 177 rt_sigtimedwaitkernel/signal.c 178 rt_sigqueueinfokernel/signal.c 179 rt_sigsuspendarch/i386/kernel/signal.c 180 pread

read from a file descriptor at a given offset fs/read_write.c 181 sys_pwritewrite to a file descriptor at a given offset

fs/read_write.c 182 chownchange ownership of a file

fs/open.c 183 getcwdGet current working directory

fs/dcache.c 184 capgetget process capabilitieskernel/capability.c 185 capsetset process capabilitieskernel/capability.c 186 sigaltstack set/get signal stack contextarch/i386/kernel/signal.c 187 sendfiletransfer data between file descriptorsmm/filemap.c 188 getpmsg(unimplemented)189 putpmsg(unimplemented)190 vfork create a child process and block parentarch/i386/kernel/process.c


56/183

System Administration To look after everything about Linux system, called system administration which

includesa. monitoring disk space and taking backupb. handling system problems which are unexpected

c.

to handle every eventuality. System administrator must have thoroughpractical knowledge of every system componentd. he is responsible for installing all system peripheralse. he devise the scripts for automating some operations carried out regularlyf. he must also be able to configure the systems initialization script

logging into system administrator:login: root (press enter)passeord: _ (press enter after entering password)# (will appear not $ as for other users)

Acquiring super user mode:

$su (press enter)Password:_#

Change password:#passwd (will change password of root)$passwd rohit (will change password of us er rohit)

Date:#date MMDDhhmm (to set date, month,day,hour,minute)

Calendar:#calendar (to see the scheduler of all users)

Ulimit:To limit on file size#ulimit (press enter)2079151#ulimit 20791510 (press enter : this will change file size)

Shutdown:$shutdown g2 (shutdown after 2 minutes)$shutdown y g0 (shutdown immediately)$shutdown 17:30 (shutdown at 17:30)$shutdown r now (shutdown and reboot now)$shutdown h10 (halt after 10 minutes)

Diskfree#df (will show how much disk free in each main directories)

/ --- /home --- /root --- /stand ---

Disk usuage:#du /home/rohit (show in particular directory)

/home/rohit/one ---


57/183

File compression:$compress foldername$uncompress foldername.z

Gnu zip:

$gzip foldername/filename$gunzip gzipfileLocating files:

$find

#find / -name a.txt print (search file a.txt in / and then display)#find /home atime +365 mailroot (search file whose access time is more than

365 days and then mail it)#find / -size -2048 print (search file whose size is below 2 mb)#find / -size +1024 atime -365 rm

Copying diskettes and tapes

Copy floppy to temp folder:#dd if=/dev/rdsk/f0q18dt of=/temp bs=147456Input file name output file name bl ock size

Copy tape#dd if=/dev/rct0 of=/dev/rct1 bs=9K

CPIO (copy input output)#ls | cpio ov > /dev/rdsk/f0q18dt#cpio iv < /dev/rdsk/f0q18dtFor understanding: o- output i -input v- visually

Tar: tape archieve programtar vs cpio- tar can take directories as input- it copies entire directory tee- it can create several versions in single archive- it can append without overwriting# tar cvf /dev/fdsk/f0q18dt /home/rohit/*#tar xvfb /dev/fdsk/f0q18dtx- extractc- copy

mount and unmount:mount /dev/hda1 /mnt/flash (mount c:\ to /mnt/flash)unmount /dev/hda1 (for unmounting) orunmount /mnt/flash ( - - -)in later versions umount also works as unmount

creating partitions#fdisk (it will give list of commands to perform)


58/183

Ques1: write any five function of a Linux system administrator.

Ques2: how kernel access a fileAns:

1. kernel know inode of current directory, maintained in memory, with this, it first

search inode blocks and find inode block of this directory and then fetch addressof data block which contain directory file2. open directory file and search for file a.txt and go es to inode and find inode for

file a.txt, reads its detail like size, indirect nodes etc. 3. instruct disk driver to move disk head to respective blocks and count the number

of byte and match with file size and read till they match.


59/183

Shell Script1) written in any editor2) to run, we

i. firstly change its mode1. $chmod +x filename

ii.

then to run1. ./filename

a. to display: echo string nameb. to read: read variablenamec. to pick value of a variable: $variablenamed. to put values in a variable: $variablename=valuee. to solve expression: `expr 1+3`

example1:copy one file to other

#!/bin/sh#this is commentcp $1 $2 #cp will copy first command line argumented file to otherexit 0

example2:search a word in file#!/bin/sh#search a wordecho "Enter strings"echo "if you have done, press ctrl + d"cat > str_fileecho "e nter string to find"read strgrep -s $str "/root/shell_scripts/str_file"exit 0

example3:sort n numbers#!/bin/sh#sort n numbersecho "enter n numbers"echo "after you have done, press ctrl+d"cat > n_numsort -n n_numexit 0


60/183

example4:to display date, process status and calendar of year#!/bin/shdateps a

cal 2008exit 0

example5:multiplication table#!/bin/shecho enter a numberread ni=1while [$i le 10 ]do

echo $n X $i= èxpr $i \* $nì=èxpr $i + 1`

doneexit 0

example6:fibnoccii series#!/bin/sha=1b=0c=1i=1n=15while [ $i le $n ]do

c=èxpr $a + $bècho $ca=$bb=$ci=èxpr $i + 1`

doneexit 0

example7:count word, lines and characters#!/bin/shecho enter a stringread strecho $str > a.txtwc a.txt


61/183

example8:prime or not#!/bin/sh

echo enter a numberread num

i=2 #counterk=1 #flagn=èxpr $num / 2 `

echo n=$nwhile [ $i le $n ]do

echo counter : $ij=èxpr $num % $i`

echo remainder: $jif [ $j eq 0 ]then

echo not a prime numberk=0exit 0

fii=èxpr $i + 1`

doneif [ $k eq 1]then

echo primeelse

echo not primefiexit 0

example9:use of for loop#!/bin/shfor i in 1 2 3 4 5do

echo welcome $idone


62/183

example10:use of case#!/bin/shecho enter a numberread a

case $a in1) echo hello;;2) echo hi;;3) echo by;;*) echo else case

esacexit 0

shell script is a language that not ends here but it depends on your practice howmuch you can, try your c programs which you have done earlier.


63/183


64/183

Unit 4Data Structure in Linux Kernel*:

*- this unit is given as for crammers. If you want to understand this unit, you haveto study at your own yet I have tried to make understandable.

Intro to Linux kernel:Linux was not designed on drawing board, but developed in an evolutionary manner andis continued to develop. Every function of kernel has been repeatedly altered andexpanded to get rid of bugs and incorporates new features. Linux kernel is not in everyrespect of a good model of Structured programming. There are magic numbers instead of constant declaration in header files, inline expansion of functions instead of functioncalls, goto instead of break, assembler instructions instead of c code. Large part of Kernelis time-critical, so kernel code is optimized for good running time rather than easyreadability. This distinguishes Linux from Minix which was written as a TeachingOpera ting System and never designed for everyday use.

Data structure in Linux: Task Structure Process Table Files and Inodes Dynamic Memory Management Queue and Semaphores System time and timers

System time and timers:In the Linux system there is j ust one internal time base. It is measured in ticks

elapsed since the system was booted, with one tick equal to 10 millisecond. These ticksare generated by a timer chip in the hardware and counted by timer interrupt. Functionsused are:

void add_timer(struct timer_list *timer); int del_timer(struct timer_list *timer); int mod_timer(struct timer_list *timer, unsigned long expires) ;

add_timer : activates a timer by entering it to the global timer listdel_timer : removes it from global timer listMod_timer : modifies the expires time of an activated timer


65/183

Dynamic Memory Management:Under Linux, memory is managed on a page basis. One page contains 2 raised to

12 bytes. The basic operations to request a free page are the functions

struct page * _alloc_pages(int gfp_mask, unsigned long order);

unsigned long _get_free_pages(int gfp_mask,unsigned long order);(2) Fn gets free pages in memory and (1) Fn allocates pages to a process gfp_mask need to control the pages and behavior of functions

Like of C-functions, Kernel uses kmalloc() and kfree() for allocating small amountof memory and free(ing) that memory.void * kmalloc(size_t size,int flags)void kfree(void * object)

Process Table:Every process occupy exactly one entry in pro cess table. INIT_TASK macro

points at the first task in the system. It is initialized by starting the system usingINIT_TASK macro. After the system has been booted, that is only responsible for the useof unclaimed system time (idle process : System idle process (pid:0))

Even if the process table has a dynamic structure, the number of tasks is restrictedto max_threads in system.

int max_thread;however it can be change by sysctl interface.for working with each process kernel use for_each_task() macro#define for_each_task(p)

for (p=&init_task; p != &init_task; p=p->next_task)


66/183

Queue and Semaphores:struct list_head {

struct list_head *next,*prev; };struct _wait_queue {

struct task_struct *task;struct list_head task_list; };struct wait_queue_head {

struct list_head task_list; };add_wait_queue() and remove_wait_queue() are used for adding deleting in wait queueextern int sem->count=1;void down(struct semaphore *sem){

while(sem->count wait);sem->count -= 1; }

void up(struct semaphore *sem){

sem->count +=1;wake_up(&sem->wait);

}


67/183

Files and inodes:

struct file{

mode_t f_mode; //access mode in which file is opened loff_t f_pos; //pos of read/write pointer atomic_t f_count; //simple reference number(index)unsigned int f_flags; //additional flag for accessstruct dentry *fs_dentry; //reference to entry in directory cache

// to access inode

};struct inode{

kdev_t i_dev; //description of device (fd0,cdrom,hda,sda)unsigned long i_no; //identify file in device (fifth,100th) eg: index nooff_t i_size; //size of filetime_t i_mtime; //last modification timetime_t i_atime; //last access timetime_t i_ctime; //last modification to inode eg: file move, cut-paste.

};


68/183

Task Structure:One of the most important concept in a multitasking system such as Linux is the

task : the data structure and algorithms for process management form the central core of Linux

struct task_struct {volatile long state; //current state of process TASK_RUNNING

//TASK_INTERRUPTABLE, TASK_UNINTERRUPTABLE unsigned long flags; //bit mask of system status (sys. Working, stand

//by, logoff, switch user unsigned long ptrace; //process is monitored by another process/not long counter; //number of ticks assigned long nice; //priority default: {NZERO}unsigned long policy; //SCHED_FIFO,SCHED_RR,SCHED_OTHERunsigned long rt_priority; //real-time priority

};Process/ Task relations : * subpart of task structure

struct task_struct *prev_task,*next_task;//for previous and next task struct task_struct *p_opptr; //pointer to original parent struct task_struct *p_pptr; //pointer to current parent struct task_struct *p_cptr; //current process : youngest child struct task_struct *p_ysptr; //pointer to younger child/ sibling struct task_struct *p_osptr; //pointer to older child/ sibling


69/183

Memory management: The Architecture-independent Memory Model in LINUX The Virtual Address Space for a Process Block Device Caching Paging Under LINUX


70/183

The architecture-independent memory model Pages of Memory

Defined by the PAGE_SIZE macro in the asm/page.h For X86, the size is 4k bytes For Alpha uses 8K bytes

Virtual Address Space Given by reference to a segment selector and the offset within the segment C pointers hold the offsets Defined in asm/segment.h KERNERL_DS (segment selector for kernel data) USER_DS (segment selector for user data) By carrying out a conversion on the segment selector register, a system

function can be given pointers to the kernel segment. Used by UMSDOS file system to simulate a Unix file system MMU of an x86 processor converts the virtual address to a linear address 4 Gbytes by width of the linear address 3 Gbytes for user segment 1 Gbyte for kernel segment Alpha does not support segmentation Offset addresses for the user segm ent not permitted to overlap with the

offset addresses for the kernel segment Converting the Linear Address

The Page Directory The Page Middle Directory The Page Table


71/183

The virtual address space for a process The User Segment

o In user mode, access only in user segmento Individual page tables for different processeso system call fork

o

child and parent processes have different page directories and pagetableso however, in the kernel segment page tables are shared by all processeso system call clone o old and new threads share the memory fullyo Some explanation for shared libraries in the user segment

Originally, linked into one binary, lead to efficiency Drawback is the growth of the length Stored in separate files and loaded at program start Linked to static addresses With ELF, allowed shared libraries to be loaded during

program execution No absolute address references in t he compiled code Virtual Memory Areas

o Process not use all functions at any timeo Process can share codes if they are run by the same executable fileo Copy-on-write strategy used for memory management

The System Call brk o The brk field points to the end of the BSS segment for non-statically

initialized datao Used for allocating or releasing dynamic memoryo The system call brk can be used to find the current value of the pointer

or to set it to a new one under protection check o Rejected if the mem required exceeds the estimated sizeo function sys_brk() calls do_map() to map a private and anonymous

area between the old & new values of brk The Kernel Segment

o In x86 architecture, a system call is generally initiated by the softwareinterrupt 128 (0x80) being triggered.

o Any processes in system mode will encounter the same kernel segmento Kernel segment in alpha architecture cannot start at addr 0o A PAGE_OFFSET is provided between physical & virtual addrs

Static Memory Allocation in the Kernel Segmento Initialization routine for character-oriented devices is called as follows

o memory_start = console_init(memory_start, memory_end);o Reserves memory by returning a value higher than the parameter

memory_start o The memory between the return value and memory_start can be used

as desired by the initialized component Dynamic Memory Allocation in the Kernel Segment


72/183

o In LINUX kernel, kmalloc() and kfree() used for dynamic memoryallocation

o void * kmalloc(size_t size, int priority);o void kfree(void *obj);o To increase efficiency, the memory reserved is not initialized

o

In LINUX kernel 1.2, __get_free_pages() only to reserve contiguousareas of memory of 4, 8, 16, 32, 64, and 128 Kbytes in sizeo kmalloc() can reserve far smaller areas of memoryo Sizes[] contains descriptors for different for different sizes of memory

area one manages memory suitable for DMA the other is responsible for ordinary memory

o Kmalloc () and kfree () restricted to the size of one page of mem

o vmalloc () and vfree () improved to multiple of the size of one page of mem

o The max of value of size is limited by the amount of physical memoryavailable

o Memory reserved by vmalloc () wont be copied to external storage


73/183

Block Device Caching: Block Buffering

o Block size may be 512, 1024, 2048, or 4096 byteso Held in memory via a buffering systemo A special case applies for blocks taken from files opened with the flag

0_SYNC o Transferred to disk every time their contents are modifiedo Data is organized as frequently requested data lie every close together

& can be kept in the processor cache The update and bdflush Processes

o At periodic intervals, update process calls the system call bdflush withan parameter

o All modified buffer blocks are written back to disk with all superblock and inode information

o bdflush , writes back the number of blocks buffers ma rked dirtygiven in the bdflush parameter

o

Always activated when a block is released by means of brelse()o Also activated when new block buffers a re requested or the size of thebuffer cache needs to be reduced

List Structures for the Buffer Cacheo LINUX manages its block buffers via a number of different doubly

linked listso Block buffers in use are managed in a set of special LRU lists

LRU list(index) Description

BUF_CLEAN

Block buffers not managed in other lists -content matches relevant block on hard disk

BUF_UNSHARED Block buffers formerly (but no longer)managed in BUF_SHARED

BUF_LOCKED Locked block buffers (b_lock != 0 )BUF_LOCKED1 Locked block buffers for inodes and

superblocksBUF_DIRTY Block buffers with contents not matching the

relevant block on hard disk BUF_SHARED Block buffers situated in a page of memory

mapped to the user segment of a process

The various LRU lists Using the Buffer Cache

o Function bread () is called for block reado Variance of bread(), breada(), reads not the block requested into the buffer

cache but a number of following blocks

Paging under Linux: Page Cache and Management


74/183

LINUX can save pages to extenral media in 2 ways a complete block device as the external medium, typically a partition on a

hard disk fixed-length files on a file system for its external storage Data that belong together are stored in a cache line (16 bytes)

Finding a Free Page __get_free_pages() is called after physical pages of mem reserved unsigned long __get_free_pages(int priority, unsigned long order, int dma)

Priority Description

GFP_BUFFER

Free page to be returned only if free pages are still availablein physical mem

GFP_ATOMIC The function __get_free_page must not interrupt the currentprocess, but a page should be returned if possible

GFP_USER The current process may be interrupted to swap pagesGFP_KERNEL This para is the same as GFP_USERGFP_NOBUFFER The buffer cache wont be reduced by an attempt to find a

free page in memGFP_NFS The difference between this & GFP_USER is that the # of

pages reserved for GFP_ATOMIC is reduced frommin_free_pages to five. Will speed up NFS operations

Priorities for the function __get_free_page()

Page Errors and Reloading a Page do_page_fault() is called when there generates a page fault interrupt

void do_page_fault(struct pt_regs *regs, unsigned long error_code); do_no_page() or do_wp_page() is called when the address is in a virtualmemory area, the legality of the read or write operation is checked byreference to the flags for the virtual mem


75/183

Linux File System: In the PC field, variety in a file system is common: practically every OS has its

own file system and each of these of course claims to be faster, better and more securethan its predecessors. The large number of file systems supported by Linux is undoubtlyone of the main reasons why Linux has gained acceptance so quickly in its short life. Not

every user is in a position to put in the time and effort to convert his or her old data to anew file system.The range of file system supported is made possible by the unified interface of

Linux kernel. This is virtual file system switch (VFS). Note that it is not a file system onits own but an interface providing a clearly defined link between the OS kernel and thedifferent file system.

The virtual file system supplies the applications with the system calls for themanagement, maintains internal structures and passes tasks on to the appropriate actualfile system. Another important task of the VFS is the performance of default actions. As arule, no file system implementation will actually provide an lseek() function which isprovided by VFS as a default. So, VFS is commonly known as virtual file system.

A central demand of file system is purposeful structuring, speed of access andfacility for random access. Random access is made possible by block oriented devices,which are divided into a specific number of equal sized blocks. Using the functions of buffer cache to access any of the sequentially numbered blocks in a given device. The filesystem itself must be capable of ensuring unique allocation of the data to the hardwareblocks.

In Unix, the information required for management is kept strictly apart from thedata and collected in separate inode structures for each file. This information includesaccess time, access rights, pointers to data blocks, indirect pointers to data blocks, doubleindirect pointers, triplet indirect pointers, owner, size etc. Access to larger files isprovided via indirect blocks which also contain block numbers. Each file is representedby just one inode, which means that within a file system, each inode has a unique numberand the file itself and the file itself can also be accessed using this number.

Directories allow the file system to be given a hierarchical structure. These arealso implemented as files, but the kernel assumes them to contain pairs consisting of afile name and its inode number.

The basic structure is the same for all the different Unix file systems. Each filesystem starts with a boot block. This block is reserved for the code required for the coderequired to boot the operating system. The boot block will be present whether or not thecomputer is booted from the device or not.

All the information which is essential for managing the file system is held in thesuper block. This is followed by a number of inode blocks containing the inode structuresfor the file system. The remaining blocks for the device provide the space for the data.The data blocks thus contain ordinary files along with the directory entries and indirectblocks.

A new file system can be mounted onto any directory. This orginal directory isthen known as the mount point and is occupied by the root directory of new file systemalong with its subdirectories and files. Unmouting the file system releases the hiddendirectory structure again.


76/183

Representation of File System in kernel: Every file system need to be made known to the VFS via the following

register_filesystem()

int register_filesystem(struct file_system_type *fs){

int res=0;struct file_system_type **P;

write_lock(&file_system_lock);p=find_filesystem(fs->name);if(*p)

res=-EBUSY;else

*p=fs;write_unlock(&file_system_type);return res;

}

Mounting:Before a file can be accessed, t he file system containing the file must be mounted.

This can be done using either the system call mount or the function mount_root(). Everymounted file system is represented by a super_block structure. These structures areplaced in a dynamic table super_blocks held by the struct list_head type. The maximumlength of this list is limited by the max_super_blocks variable. The function read_super()of VFS is used to initialize the superblock. It creates an empty superblock, puts it in thesuperblock list and calls the function provided by every file system implementation tocreate the superblock.


77/183

Superblock:The file system specific function read_super() reads information from the

corresponding block device. This is also the reason why a process is necessary formounting file system.struct super_block

{ struct list_head s_list; //list of super blockskdev_t s_dev; //deviceunsigned long s_blocksize; //block sizeunsigned char s_dirt; //if superblock has been changedwait_queue_hed_t s_wait; //waiting queue

}; struct super_operations{

void (*read_inode)(struct inode *);//inode structure is initialized by this function

void (*write_inode)(struct inode *);//save info of inode

void (*write_super)(struct super_block *);//save info of super block

void (*delete_inode)(struct inode *);//delete inode

}; Inode:

struct inode {struct list_head i_list; //chainingunsigned long i_ino; //inode numberuid_t i_uid; //user idgid_t i_gid; //group idkdev_t i_rdev; //real devicetime_t i_atime; //last access timeoff_t i_size; //size of filetime_t i_mtime; //last modification timetime_t i_ctime; //last change to inode

};struct inode_operations {int (*create) ( . . .);

//create the inodeint (*mkdir)(. . .);

//creates a directoryint (*rmdir)(. . .);

//removes a directory


78/183

int (*rename)(. . .);//move a file or rename it

int (*set_attr)(. . .);//to set attributes of a inode

int (*get_attr)(. . .);

//to get attributes of a inode};

File Structure:

struct list_head {struct list_head f_list; //chainingmode_t f_mode; //access typeloff_t f_pos; //position in fileunsigned int f_uid,f_gid; //owner

};

struct file_operation{

loff_t (*llseek)( . . .);//to deal with positioning with in the file

ssize_t (*read)(. . .);//reads count bytes from the file and copies them into the buffer

ssize_t (*write)(. . .);//write count bytes

int (*open)( . . .);//to open a file

. . .};


79/183

Proc file system: Linux supports different filesystem. Each process in the system which is currently

running is assigned a directory /proc/pid where pid is the process identification number of the relevant process. This directory contains files holding information on certaincharacterstics of the process.

When proc file system is mounted, the VFS function read_super() is called bydo_mount() and in turn calls the function proc_read_super() for the proc file system inthe file_system list.

iget() generate the indoe for the proc root directory, which is entered in thesuperblock_parse_options() then processes the mount options data that have beenprovided and sets the owner of the root inode.

Accessing the file system is always carried out by accessing the root inode of thefile system. The first access is made by calling iget(). If the inode does not exists, thisfunction then calls the proc_read_inode() function entered in the proc_sops structure

The inode describe the directory with read and execute permissions for allprocesses. The proc_root_inode_operations only provides two functions: readdir in form

of proc_readroot() function and lookup as proc_lookuproot(). Both functions operateusing the table root_dir[], which contains the different entries for the root directory

struct proc_dir_entry{

const char *name; //name of entrymode_t mode; //modeuid_t uid; //user idgid_t gid; //group idunsigned long size; //size of filestruct proc_dir_entry *next,*parent; //chainingvoid *data;. . .

};


80/183

Ext2 file system: As Linux was initially developed under MINIX, it is hardly surprising that first

Linux file system was minix file system. However, this file system restricts partitions to64 MB and file names to 14 chars, so the search for a better file system was obvious.Although this allowed partitions of up to 2GB and filename up to 255 chars. It included

several signigicant extensions but offered unsatisfactory performance. The secondextended file system (Ext2) was introduced in 1994.

Features:i) Block Fragmentation: it allows different sized blocks to be allocated.ii) Access Control List: allows ACL to be associated with each fileiii) Handles compressed and encrypted filesiv) Logical Deletion: an undelete option will allow users to easily recover

removed files.Structure:

Ext2 file system has blocks and each block has

a.

a copy of file systems super block b. a copy of group of block group descriptorsc. a data block bitmapd. a group of inodee. an inode bitmapf. a chunk of data belonging to a file/ data block

An ext2 disk super block is stored in ext2_super_block structure which contains the totalnumber of inodes, file system size, number of reserved blocks, free blocks counter, freeinodes counter, block size, fragment size and other important information.

Directories:are maintained by singly link list.

struct ext2_dir_entry_2 {_u32 inode; //inode number_u16 rec_len; // length of directory entry_u8 name_len; //length of file name_u8 file_type; //type of fileChar name[EXT2_NAME_LEN]; //file name

};

Block allocation:a. Target oriented: This algo looks for target block if that not found then look with

in 32 blocks near target block if no one is free, then find block at least in sameblock group and even if that is not found search else where and allocate that.

b. Pre allocation: If a free block is found, up to eight following blocks are reserved.When the file is closed, the remaining blocks still reserved are released. This alsoguarantees that as many data blocks as possible are collected into one cluster.


81/183

Proc vs ext2:

Proc ext2

1. procedure oriented Access oriented2. file name length=14 chars 2553. max. partition=512MB 2 GB4. mounted on /proc on / 5. currently running process assigned all files stored on it

/proc/pid.


82/183


83/183

Unit5:Multiprocessing:More advance and faster processors are entering in market; there will always beapplications that require still more processor power. A multitasking system, solution is toemploy several processors in order to achieve true parallel processing of tasks.

Performance doesnt increase linearly with number of processors, rather, OS bears anincreased responsibility to distribute all task among processors in such a way that fewprocessors as possible hamper each other.

Intel Multiprocessor Specification:Pentium already has some internal function which supports multiprocessor operation suchas cache synchronization, interrupt handling and atomic operations for checking, settingand exchanging values in main memory. Cache synchronization facilitates symmetricmultiprocessing implementation in kernel.Intel multiprocessor specification version 1.4 defines the interaction between hardwareand software in order to facilitate the development of SMP capable OS and to create

possibility of these systems run on new hardware. It defines a highly symmetricarchitecture in term of Memory Symmetry: same main memory, same data/code , same OS and applicationI/O Symmetry: All processor share I/O subsystem to reduce possible I/O bottleneck.The following diagram shows a typical SMP system with two processors connected viaInterrupt Controller Communications (ICC) bus with one/more Advance ProgrammableInterrupt Controller (APIC) Pentium Processors also have Local APIC + I/O APICconstitute a unit which deals with distribution of incoming interrupts.

One processor is chosen by BIOS called Boot Processor (BSP) and used for systeminitialization. All other Application Processors (AP) are initially halted by BIOS. TheMulti Processor (MP) specification filled in BIOS and informs OS about existingmultiprocessor system. BIOS initially forwards all interrupts only to boot processor, sothat single processor system see no difference and run only on BSP.Problems with multiprocessor systems:For correct functioning of multitasking system, it is important that data in kernel can onlybe changed by one processor, so that identical resources cannot be allocated twice.


84/183

Unix system:a) Coarse granted locking: lock whole kernelb) Finer grained locking: reduce time that a lock must keep => reduce

particularly critical latency time

Linux system:Rules were establishedi. no process running in kernel mode interrupted by other process running in

kernel mode except when it releases control and stepsii. an interrupt handling can interrupt a process in kernel mode but in end

control return back to same processiii. interrupt handling will be processed completely and interrupt cannot be by

process in kernel mode and be by interrupt of higher priority

Initially, a semaphore was used by all processes to monitor transition to kernel

mode which obey all rules => low performance. Later, finer grained locking wasused. The transition can be carried out hierarchi cally by substituting onesemaphore with several others which cover smaller area of Linux kernel thisguarantees higher parallelism and higher system performance.

Symmetric Multiprocessing:There are two processors and kernel decides which process should be allocated to

the processor. Memory is shared b/w processors. When there is some updation in theprocess, then they will be stored in process only. This will not be reflected back whichwill create problem. We use shared memory because changes are reflected. SMP denote amultiprocessor architecture in which low CPU is selected as a Master CPU but rather allof these cooperate on an equal basis. When kernel is to be loaded, the basic processorCMOS is set which loads the kernel.Features of SMP:

1. Shared Memory2. Shared IO Port3. Hardware Cache Synchronization: Synchronization means providing a cache at

time to process one by one. Suppose both processes are working on same programand storing variables in different cache and they are doing same modifications atsame time, which is not safe and therefore only one cache is provided at a time.

4. SMP is provided by atomic operation like read/write back to disk, modification5. Distributed Interrupt Handling: Every process has been provided by its own

interrupt controller known as APIC


85/183

Changes to kernel:In order to implement SMP in Linux, changes have to be made to both portable

and processor specific implementations.

Kernel Initialization:

All processors must be started because BIOS has halted all APs and only bootprocessor is running. Only this processor enters start_kernel(), then smp_init() for normalinitialization then smp_boot_cpus(). This activates all other processors. Each processorreceives its own stack. Each processor execute code and jump to start_kernel(). Onceexception handling and interrupt handling are initialized, processors trapped bysmp_callin() inside start_secondary().asmlinkage void start_secondary(void){

trap_init();init_IRQ();smp_callin();

cpu_idle(NULL);}void smp_callin(void){

let's make linux together

Documents