unix from capgemini

146
UNIX Operating System Level I

Upload: naveen-reddy

Post on 01-Feb-2016

251 views

Category:

Documents


0 download

DESCRIPTION

unix for

TRANSCRIPT

Page 1: UNIX From Capgemini

UNIX Operating SystemLevel I

Page 2: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

2

Purpose

Provide understanding of UNIX operating system. Introduce various commands and shell scripting capability of UNIX.

Page 3: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

3

Product

Proper understanding of UNIX operating system architecture.

» Proper understanding of shell scripting abilities of UNIX.

Page 4: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

4

Process

Explain UNIX features. Hands on session.

Page 5: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

5

Agenda

»Understanding unix.»File system»Commands.»Exercise & Assignment

Page 6: UNIX From Capgemini

Part I

What is UNIX?

Page 7: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

7

What is Unix?

UNIX, in computer science, a powerful multi-user, multitasking operating system. Considered a very powerful operating system, UNIX is written in the C.

Designed with Flexibility in mind. Designed with Modularity in mind. Everything is a file !

Page 8: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

8

Structure

Page 9: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

9

Processing model

Centralized

Page 10: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

10

Components

Kernel Shell

Page 11: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

11

Components

Kernel Core of the UNIX system. Loaded at system start up (boot). Memory-resident control

program. Manages the entire resources of the system, presenting them to you and every other

user as a coherent system. Provides service to user applications such as device management, process scheduling, etc.

Example functions performed by the kernel are: • managing the machine's memory and allocating it to each process. • scheduling the work done by the CPU so that the work of each user is carried out as

efficiently as is possible. • accomplishing the transfer of data from one part of the machine to another.• interpreting and executing instructions from the shell.• enforcing file access permissions.

Page 12: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

12

Components

Shell Whenever you login to a Unix system you are placed in a shell program. The shell's prompt is

usually visible at the cursor's position on your screen. To get your work done, you enter commands at this prompt.

The shell is a command interpreter; it takes each command and passes it to the operating system kernel to be acted upon.

Several shells are usually available on any UNIX system, each with its own strengths and weaknesses.

Different users may use different shells. Initially, your system administrator will supply a default shell, which can be overridden or changed. The most commonly available shells are: • Bourne shell (sh) • C shell (csh) • Korn shell (ksh) • Bourne Again Shell (bash)

Each shell also includes its own programming language. Command files, called "shell scripts" are used to accomplish a series of tasks.

Page 13: UNIX From Capgemini

Part II

Log in

Page 14: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

14

Log in

you will need to have a valid username and a password Type your username at the login prompt

• UNIX is case sensitive• When the password prompt appears, type in your password. • Your password is never displayed on the screen as a security measure.

Page 15: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

15

Login screen

Page 16: UNIX From Capgemini

Part III

Working with Unix

Page 17: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

17

General commands

date - show date and time whoami - show your userid who - show who is logged onto the system w - show who is logged onto the system pwd - print the working directory's name clear - clear the screen finger

• The finger command displays information about users on a given host. The host can be either local or remote.

finger your userid

Page 18: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

18

General commands

ls -a - list all files in current directory ls -al - long list of current directory cat .bashrc - display contents of .bashrc file mkdir dir1 - make a directory called dir1 cd dir1 - change directory to dir1 cd .. - change to parent directory rmdir dir1 - remove directory dir1 cp .login new.login - copy the .login file to new.login wc new.login - count the lines, words and characters in the new.login file wc -l new.login - count just the lines rm new.login - remove the new.login file

Page 19: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

19

Changing your password

» passwd Old password: - enter your current password

New password: - enter your new password

Retype new password: - re-enter your new password

Page 20: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

20

Logging Off The System

To logout enter the command logout or exit. If this does not, work press Ctrl-d. If you have a .logout file in your home directory, the system will execute any commands contained there.

Page 21: UNIX From Capgemini

Part IV

File system

Page 22: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

22

Features of UNIX File system

All of the files in the UNIX file system are organized into a multi-leveled hierarchy called a

directory tree.

A family tree is an example of a hierarchical structure that represents how the UNIX file

system is organized. The UNIX file system might also be envisioned as an inverted tree or

the root system of plant.

At the very top of the file system is single directory called "root" which is represented by

a / (slash). All other files are "descendents" of root.

The number of levels is largely arbitrary, although most UNIX systems share some

organizational similarities.

Page 23: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

23

File types

Ordinary Files Directory Special Files Pipes

Page 24: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

24

File types

Ordinary Files • Used to store your information, such as some text you have written or an image you have drawn. This is the type of file that

you usually work with. • Always located within/under a directory file • Do not contain other files

Page 25: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

25

File types

»Directories • Branching points in the hierarchical tree • Used to organize groups of files • May contain ordinary files, special files or other directories • Never contain "real" information which you would work with (such as text). Basically, just used for organizing files. • All files are descendants of the root directory, ( named / ) located at the top of the tree.

Page 26: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

26

File types

»Special Files • Used to represent a real physical device such as a printer, tape drive or terminal, used

for Input/Ouput (I/O) operations • Unix considers any device attached to the system to be a file - including your terminal:

By default, a command treats your terminal as the standard input file (stdin) from which to read its input

Your terminal is also treated as the standard output file (stdout) to which a command's output is sent

Stdin and stdout• Two types of I/O: character and block • Usually only found under directories named /dev

Page 27: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

27

File types

»Pipes • UNIX allows you to link commands together using a pipe. The pipe acts a temporary file which only exists to hold data from

one command until it is read by another

Page 28: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

28

File Types

File type Meaning

- a normal file

d a directory

l symbolic link

b block device file

c character device file

p a fifo or named pipe

Page 29: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

29

File Names

UNIX permits file names to use most characters, but avoid spaces, tabs and characters that have a special meaning to the shell, such as: & ; ( ) | ? \ ' " ` [ ] { } < > $ - ! /

Case Sensitivity: uppercase and lowercase are not the same! These are three different files: NOVEMBER November november

Length: can be up to 256 characters Extensions: may be used to identify types of files

libc.a - archive, library file program.c - C language source file alpha2.f - Fortran source file xwd2ps.o - Object/executable code mygames.Z - Compressed file

Page 30: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

30

File Names

Hidden Files: have names that begin with a dot (.) For example: .cshrc .login .mailrc .mwmrc Uniqueness: as children in a family, no two files with the same parent directory can have the same name. Files

located in separate directories can have identical names. Reserved Filenames:

• / - the root directory (slash) • . - current directory (period) • .. - parent directory (double period) • ~ - your home directory (tilde)

Page 31: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

31

Pathnames

Specify where a file is located in the hierarchically organized file system Must know how to use pathnames to navigate the UNIX file system Absolute Pathname: tells how to reach a file begining from the root; always begins with / (slash). For

example: /usr/local/doc/training/sample.f Relative Pathname: tells how to reach a file from the directory you are currently in ( current or

working directory); never begins with / (slash). For example: • training/sample.f • ../bin • ~/projects/report.001

For example, if your current directory is /usr/home/johnson and you wanted to change to the directory /usr/home/quattro, you could use either of these commands: • cd ../quattro - relative pathname • cd /usr/home/quattro - absolute pathname

Page 32: UNIX From Capgemini

Automating

Page 33: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

33

Automating logon tasks.

• Naming: Depending upon the shell, you must name your initialization file(s) accordingly. • Executed during interactive login

• .login - csh, tcsh

• .profile - sh, ksh, bash

• .bash_profile - bash (alternative 1)

• .bash_login - bash (alternative 2)

Page 34: UNIX From Capgemini

Part V

Commands

Page 35: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

35

Commands

»cat • dumps the entire file to the screen without paging. This command is more useful for concatenating (hence the name "cat")

files together than it is for reading files. • Examples:

cat myprog.c - diplays entire file

cat -b myprog.c - shows line numbers

cat file1 file2 > file3 - adds file1 and file2 to make file3

Page 36: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

36

File and Directory Commands

‘ls’• ls - show contents of working directory • ls file - list file, if it exists in working directory • ls dir - show contents of the directory dir • ls -a - shows all your files, including hidden ones • ls -al - give detailed listing of contents • ls -F - mark directories with "/" and executable files with "*" • ls *.doc - show all files with suffix ".doc" • ls -lt - Time of last modification.

• ls -lut - Time of last access

Page 37: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

37

Commands

»more –• browses/displays files one screen at a time. • Use h for help, • spacebar to page, • b for back, • q to quit, • /string to search for string

Eg: more sample.f

Page 38: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

38

Commands

»head displays the first n lines of a file

• head sample.f - display first 10 lines (default)

• head -5 sample.f - display first 5 lines

• head -25 sample.f - display first 25 lines

Page 39: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

39

Commands

»tail • displays the last n lines or n characters of a file• Examples

tail sample.f - display last 10 lines (default)

tail -5 sample.f - display last 5 lines

tail -5c sample.f - display last 5 characters

tail -25 sample.f - display last 25 lines

Page 40: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

40

Commands

»cp • copies files. Will overwrite unless otherwise specified. Must also have write permission in the destination directory. • Examples:

cp sample.f sample2.f - copies sample.f to sample2.f

cp -R dir1 dir2 - copies contents of directory dir1 to dir2 cp -i file.1 file.new - prompts if file.new will be overwritten

cp *.txt chapt1 - copies all files with .txt suffix to directory chapt1

cp /usr/doc/README ~ - copies file to your home directory

cp ~betty/index . - copies the file "index" from user betty's home directory to current directory

Page 41: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

41

Commands

»mv• moves files. Will overwrite unless otherwise specified. Must also have write permission

in the destination directory. • Example:

mv sample.f sample2.f - moves sample.f to sample2.f mv dir1 newdir/dir2 - moves contents of directory dir1 to newdir/dir2 mv -i file.1 file.new - prompts if file.new will be overwritten mv *.txt chapt1 - moves all files with .txt suffix to directory chapt1

Page 42: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

42

Commands

»rm»deletes/removes files or directories if file permissions permit. »Example

• rm sample.f - deletes sample.f

• rm chap?.txt - deletes all files with chap as the first four characters of their name and with .txt as the last four characters of their name

• rm -i * - deletes all files in current directory but asks first for each file

• rm -r / olddir - recursively removes all files in the directory olddir, including the directory

Page 43: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

43

Commands

find • finds files. • The syntax of this command is: find pathname -name filename -print • The pathname defines the directory to start from. Each subdirectory of this directory will be

searched. The -print option must be used to display results. • You can define the filename using wildcards. If these are used, the filename must be placed in

'quotes'. • Example

find . -name mtg_jan92 -print - looks for the file mtg_jan92 in current directory find ~/ -name README -print - looks for files called README throughout your home directory find . -name '*.fm' -print - looks for all files with .fm suffix in current directory find /usr/local -name gnu -type d -print - looks for a directory called gnu within the /usr/local

directory

Page 44: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

44

Commands

»diff »comparing two files or directories. Indicates which lines need be added (a), deleted (d) or changed (c). Lines in file1 are identified with a (<) symbol: lines in file2 with a (>) symbol

• diff file1 file2 - compares file1 to file2 • diff -iw file1 file2 - compares two files ignoring letter case and spaces • diff dir1 dir2 - compares two directories showing files which are unique to each and also, line by line

differences between any files in common. »For example, if file1 and file2 are:

• John erpl08@ed John erpl08@ed • Joe [email protected] Joe [email protected] • Kim [email protected] Jean [email protected] • Keith keith@festival Jim jim@frolix8

Kim [email protected] Keith keith@festival

»Using the diff command: diff file1 file2 Yields the output: • 2a3,4 • > Jean [email protected] • > Jim jim@frolix8

Which means that to make these files match you need to add (a) lines 3 and 4 (3,4) of file2 (>) after line 2 in file1.

Page 45: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

45

Commands

pwd • print working directory. Tells you which directory you are currently in.

mkdir • make directory. Will create the new directory in your working directory by default. • mkdir /u/training/data mkdir data2

rmdir • remove directory. Directories must be empty before you remove them.

rmdir project1 • To recursively remove nested directories, use the rm command with the -r option:

rm -r directory_name

Page 46: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

46

Agenda

»File permissions»‘vi’ Editor»More commands

Page 47: UNIX From Capgemini

Part VI

Access Permissions

Page 48: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

48

Access Permissions

UNIX is a multi-user system. Every file and directory in your account can be protected from or made accessible to other users by changing its access permissions. Every user has responsibility for controlling access to their files.

Permissions for a file or directory may be any or all of: • r - read • w - write • x - execute = running a program

Each permission (rwx) can be controlled at three levels: • u - user = yourself • g - group = can be people in the same project • o - other = everyone on the system

Page 49: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

49

Access Permissions

File access permissions are displayed using the ls -l command. The output from the ls -l command shows all permissions for all levels as three groups of three according to the

scheme: • owner read (r) owner write (w) owner execute (x) • group read (r) group write (w) group execute (x)• public read (r) public write (w) public execute (x) • which are displayed as: -rwxrwxrwx • Note: a directory must have both r and x permissions if the files it contains are to be accessed.

Page 50: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

50

Access Permissions

The chmod command is used to change access permissions for files which you own. The syntax is: chmod permission_triads filename [who][action][permissions]

where: who action permissions u = user + = add r = read g = group - = remove w = write o = other x = execute a = all • chmod a+r sample.f - Adds read permission for all users to the file sample.f. • chmod o-r sample.f - Removes read permission for others to the file sample.f. • chmod og+rx prog* - Adds read and execute permissions for group and others to all files which

contain "prog" as the first four characters of their name. • chmod +w * - Adds write permission for user to all files in current directory.

Page 51: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

51

Access Permissions

File access permissions can also be changed by a numerical (octal) chmod specification. Read permission is given the value 4, write permission the value 2 and execute permission 1.

r w x

4 2 1

These values are added together for any one user category:

0 = no permissions

1 = execute only

2 = write only

3 = write and execute (1+2)

4 = read only

5 = read and execute (4+1)

6 = read and write (4+2)

7 = read and write and execute (4+2+1)

Page 52: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

52

Access Permissions

So access permissions can be expressed as three digits.

For example: user group others

chmod 640 file1 rw- r-- ---

chmod 754 file1 rwx r-x r–

chmod 664 file1 rw- rw- r–

Never set write permission for all other users on a file or directory which is in your home directory. If you do other users will be able to change its content. This can represent a serious security risk.

Page 53: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

53

Change file ownership

chown• Can be issued by owner of file ONLY.• Syntax: chown newowner filename• Ownership once given cannot be revoked.

Page 54: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

54

More on File permissions

Each user has default set of permissions which apply to all files/dir created by users, unless explicitly set.

By default:• For files permission is -rw-r--r—• For directory it is drwxr-xr-x

» Command umask -S

• Will show you present setting of mask.• You mention those permissions that you do not want to be given by default to newly

created files and directories.• umask 0027• Default 0022

Page 55: UNIX From Capgemini

Part VII

Editors

Page 56: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

56

‘vi’ editor

‘vi’ = visual vi supplies commands for:

• inserting and deleting text • replacing text • moving around the file • finding and substituting strings • cutting and pasting text • reading and writing to other files

Page 57: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

57

‘vi’ editor

vi uses a "buffer" While using vi to edit an existing file, you are actually working on a copy of the file that is

held in a temporary buffer in your computer's memory. If you invoked vi with a new filename, (or no file name) the contents of the file only exist in

this buffer. Saving a file writes the contents of this buffer to a disk file, replacing its contents. You can

write the buffer to a new file or to some other file. You can also decide not to write the contents of the buffer, and leave your original file

unchanged.

Page 58: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

58

‘vi’ editor

vi operates in two different "modes": Command mode

• vi starts up in this mode • Whatever you type is interpreted as a command - not text to be inserted into the file. • The mode you need to be in if you want to "move around" the file.

Insert mode • This is the mode you use to type (insert) text. • There are several commands that you can use to enter this mode. • Once in this mode, whatever you type is interpreted as text to be included in the file.

You can not "move around" the file in this mode. • Must press the ESC (escape) key to exit this mode and return to command mode.

Page 59: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

59

Starting ‘vi’ editor

vi filename • The filename can be the name of an existing file or the name of the file you want to create.

view filename • Starts vi in "read only" mode. Allows you to look at a file without the risk of altering its contents.

Page 60: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

60

‘vi’ editor

Exiting vi :q - quit - if you have made any changes, vi will warn you of this, and

you'll need to use one of the other quits. :w - write edit buffer to disk :w filename - write edit buffer to disk as filename :wq - write edit buffer to disk and quit :ZZ - write edit buffer to disk and quit :q! - quit without writing edit buffer to disk

Page 61: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

61

‘vi’ editor

»Positioning within text »By character

• left arrow - left one character • right arrow - right one character • backspace - left one character • space - right one character • h - left one character • l - right one character

By word • w - beginning of next word • nw - beginning of nth next word • b - back to previous word • nb - back to nth previous word • e - end of next word • ne - end of nth next word

Page 62: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

62

‘vi’ editor

By line • down arrow - down one line • up arrow - up one line • j - down one line • k - up one line • + - beginning of next line down - - - beginning of previous line up - 0 - first column of current line (zero) - ^ - first character of current line - $ - last character of current line

Page 63: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

63

‘vi’ editor

»By block • ( - beginning of sentence • ) - end of sentence • { - beginning of paragraph • } - end of paragraph

Page 64: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

64

‘vi’ editor

»By screen • CTRL-f - forward 1 screen

• CTRL-b - backward 1 screen

• CTRL-d - down 1/2 screen

• CTRL-u - up 1/2 screen

• H - top line on screen

• M - mid-screen

• L - last line on screen

Page 65: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

65

‘vi’ editor

Inserting text a - append text after cursor * A - append text at end of line * i - insert text before cursor * I - insert text at beginning of line * o - open a blank line after the current line for text input * O - open a blank line before the current line for text input * * Note: hit ESC (escape) key when finished inserting!

Page 66: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

66

‘vi’ editor

Deleting text x - delete character at cursor dh - delete character before cursor nx - delete n characters at cursor dw - delete next word db - delete previous word dnw - delete n words from cursor dnb - delete n words before cursor d0 - delete to beginning of line d$ - delete to end of line D - delete to end of line dd - delete current line d( - delete to beginning of sentence d) - delete to end of sentence d{ - delete to beginning of paragraph d} - delete to end of paragraph ndd - delete n lines (start at current line)

Page 67: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

67

‘vi’ editor

Copying lines

yy - "yank": copy 1 line into buffer nyy - "yank": copy n lines into buffer p - put contents of buffer after current line P - put contents of buffer before current line » Moving lines (cutting and pasting)

ndd - delete n lines (placed in buffer) p - put contents of buffer after current line P - put contents of buffer before current line

Page 68: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

68

‘vi’ editor

Searching / Substituting

/str - search forward for str ?str - search backward for str n - find next occurrence of current string N - repeat previous search in reverse direction The substitution command requires a line range specification. If it is omitted, the default is the current line only. The examples below show how to specify line ranges.

:s/old/new - substitute new for first occurrence of old in current line

Page 69: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

69

‘vi’ editor

:s/old/new/g - substitute new for all occurrences of old in current line :1,10s/old/new - substitute new for first occurrence of old in lines 1 - 10 :.,$s/old/new - substitute new for first occurrence of old in remainder of file :.,+5s/old/new - substitute new for first occurrence of old in current line and next 5 lines :.,-5s/old/new - substitute new for first occurrence of old in current line and previous 5 lines :%s/old/new/g - substitute new for all occurrences of old in the entire file :%s/old/new/gc - interactively substitute new for all occurrences of old - will prompt for y/n response for each substitution.

Page 70: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

70

vi Options

You can change the way vi operates by changing the value of certain options which control specific parts of the vi environment.

To set an option during a vi session, use one of the commands below as required by the option:

:set option_name :set option_name=value

Some examples of the more common options are described below.

:set all - shows all vi options in effect

:set ai - set autoindent - automatically indents each line of text

:set noai - turn autoindent off

Page 71: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

71

vi Options

:set nu - set line numbering on

:set nonu - turn line numbering off

:set scroll=n - sets number of lines to be scrolled to n. Used by screen scroll commands.

:set sw=n - set shiftwidth to n. Used by autoindent option.

:set wm=n - set wrapmargin to n. Specifies number of spaces to leave on right edge of the screen before wrapping words to next line.

Page 72: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

72

vi Options

:set showmode - reminds you when you are inserting

text.

:set ic - ignore case of characters when

performing a search. Options can be set permanently by putting them in a file called .exrc in your home directory.

Page 73: UNIX From Capgemini

Part VIII

More commands

Page 74: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

74

Processes

Whenever you enter a command at the shell prompt, it invokes a program. While this program is running it is called a process. Your login shell is also a process, created for you upon logging in and existing until you logout.

UNIX is a multi-tasking operating system. Any user can have multiple processes running simultaneously, including multiple login sessions. As you do your work within the login shell, each command creates at least one new process while it executes.

Process id: every process in a UNIX system has a unique PID - process identifier.

Page 75: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

75

Processes

ps - displays information about processes. Note that the ps command differs between different UNIX systems - see the local ps man page for details.

»To see your current shell's processes: $ ps PID TTY TIME CMD 26450 pts/9 0:00 ps 66801 pts/9 0:00 -csh

Page 76: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

76

Processes

To see a detailed list of all of your processes on a machine (current shell and all other shells):

$ ps uc USER PID %CPU %MEM SZ RSS TTY STAT STIME TIME COMMANDjsmith 26451 0.0 0.0 120 232 pts/9 R 21:01:14 0:00 psjsmith 43520 0.0 1.0 300 660 pts/76 S 19:18:31 0:00 elmjsmith 66801 0.0 1.0 348 640 pts/9 S 20:49:20 0:00 cshjsmith 112453 0.0 0.0 340 432 pts/76 S Mar 03 0:00 csh

To see a detailed list of every process on a machine:

$ ps ug USER PID %CPU %MEM SZ RSS TTY STAT STIME TIME COMMANDroot 0 0.0 0.0 8 8 - S Feb 08 32:57 swapperroot 1 0.1 0.0 252 188 - S Feb 08 39:16 /etc/init root 514 72.6 0.0 12 8 - R Feb 08 28984:05 kprocroot 771 0.2 0.0 16 16 - S Feb 08 65:14 kprocroot 1028 0.0 0.0 16 16 - S Feb 08 0:00 kprocroot 60010 0.0 0.0 1296 536 - S Mar 07 0:00 -ncd19:0kdr 60647 0.0 0.0 288 392 pts/87 S Mar 06 0:00 -ksh manfield 60968 0.0 0.0 268 200 - S 10:12:52 0:00 mwmkelly 61334 0.0 0.0 424 640 - S 08:18:10 0:00 twm sjw 61925 0.0 0.0 552 376 - S Mar 06 0:00 rlogin kanaha mkm 62357 0.0 0.0 460 240 - S Feb 08 0:00 xterm ishley 62637 0.0 0.0 324 152 pts/106 S Mar 06 0:00 xedit march2tusciora 62998 0.0 0.0 340 448 - S Mar 06 0:05 xterm -edilfeath 63564 0.0 0.0 200 268 - S 07:32:45 0:00 xclock tusciora 63878 0.0 0.0 548 412 - S Mar 06 0:41 twm S=Sleeping, R-Runnable, T-Stopped

Page 77: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

77

Processes

Terminating a process kill - use the kill command to send a signal to a process. In most cases, this will be a kill signal,

hence the command name. However, other types of signals are usually supported. Note that you can only kill processes which you own. The command syntax is:

» kill [-signal] process_identifier(PID)

Examples: » kill 63878 - kills process 63878» kill -9 1225 - kills (kills!) process 1225. Use if simple kill doesn't work.» kill -STOP 2339 - stops process 2339» kill -CONT 2339 - continues stopped process 2339» kill -l - list the supported kill signals You can also use CTRL-C to kill the currently running process.

Page 78: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

78

Processes

Suspend a process: Use CTRL-Z. Background a process: Normally, commands operate in the foreground - you can not do

additional work until the command completes. Backgrounding a command allows you to continue working at the shell prompt.

To start a job in the background, use an ampersand (&) when you invoke the command: myprog & To put an already running job in the background, first suspend it with CRTL-Z

and then use the "bg" command: myprog - execute a process CTRL-Z - suspend the process bg - put suspended process in background

Page 79: UNIX From Capgemini

Unix Level II

Page 80: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

80

Agenda

Shell Shell programming Introduction to awk

Page 81: UNIX From Capgemini

Part I

Shell

Page 82: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

82

Shells

Whenever you login to a Unix system you are placed in a program called the shell. All of your work is done within the shell.

The shell is your interface to the operating system. It acts as a command interpreter; it takes each command and passes it to the operating system. It then displays the results of this operation on your screen.

There are several shells in widespread use. The most common ones are described below. Bourne shell (sh)

• Original Unix shell written by Steve Bourne of Bell Labs. Available on all UNIX systems. Does not have the interactive facilites provided by modern shells such as the C shell and Korn shell. The Bourne shell does provide an easy to use language with which you can write shell scripts.

C shell (csh) • Written at the University of California, Berkley. As it name indicates, it provides a C like language with which

to write shell scripts. Korn shell (ksh)

• Written by David Korn of bell labs. It is now provided as the standard shell on Unix systems. Provides all the features of the C and TC shells together with a shell programming language similar to that of the original Bourne shell.

Page 83: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

83

Shells

TC Shell (tcsh) • Available in the public domain. It provides all the features of the C shell together with

EMACS style editing of the command line. Bourne Again Shell (bash)

• Public domain shell written by the Free Software Foundation under their GNU initiative. Ultimately it is intended to be a full implementation of the IEEE Posix Shell and Tools specification. Widely used within the academic commnity. Provides all the interactive features of the C shell (csh) and the Korn shell (ksh). Its programming language is compatible with the Bourne shell (sh).

Your login shell is usually established by the local System Administrator when your userid is created. You can determine your login shell with the command:

echo $SHELL

Page 84: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

84

Shells

Each shell has a default prompt. For the 5 most common shells:

$ (dollar sign) - sh, ksh, bash

% (percent sign) - csh, tcsh

Page 85: UNIX From Capgemini

Part II

Shell programming

Page 86: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

86

Shell programs

Contains sequence of commands Shell reads the file on request and executes commands within. Shell program is called as Shell Script. Each script executes in a shell environment.

Page 87: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

87

Scripts:

Besides regular Unix commands we can insert more commands which are program oriented and write interactive scripts also.

After creating a script, we have to change the mode of the file to ‘x’ so that it can run as any other Unix command.

If we have not done so then we have to submit the name of the script file to ‘sh’ command for executing. Script file can be created using ‘vi’ editor. It may or may not have an extension.

Page 88: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

88

Example 1

# This script demonstrates use of echo in original syntax

echo This is a demo of echo command.

echo Syntax is : echo argument

echo E.g.: echo Happy Learning.

# allows you to enter comments in a program. Anything to the right of # is ignored. ‘echo’ command allows you to echo/print the argument on screen.

Page 89: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

89

Handling special characters in script.

Character Name Action

' single quote the shell will ignore all special characters contained within a pair of single quotes

" double quote the shell will ignore all special characters EXCEPT $ ` \ contained within a pair of double quotes

\ backslash the shell ignores any special character immediately following a backslash

Page 90: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

90

Example 2

# This script explain how a command can be used in a shell script

echo Good Morning

echo Today is : `date`

` ` - is used to execute a command on the same line as echo.

Page 91: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

91

• In our school, we use to do this normally• A = 2• B = 2• Hence A + B = 4• Here A and B are ?? !!

Page 92: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

92

Variables

Variables gives a script ability to store data. Data can be predefined or entered by user during execution. ‘=‘ operator is used for to assign values to variable.

Page 93: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

93

Example 3

myname=Anup

echo Hello $myname

In the first line, variable myname is created and assigned a value i.e. Anup. In the second line we are using echo command to print Hello. Alongwith we are using $ to extract the value of

variable and print it on the same line.

Page 94: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

94

Readonly variables

readonly command is used to make a shell variable readonly. Once you execute a command likereadonly my_variable

The shell variable my_variable can no longer be modified. To get a list of the shell variables that are currently set to read only you run the readonly command without any

parameters.

Page 95: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

95

‘read’ command

Allows you to make your program interactive. Syntax : read variable1 [variable2]… A variable declared in a script is only valid till the script is in execution.

Page 96: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

96

Example 4

# This script demonstrates the use of read commandecho Please enter your name: #line 1read name #line 2echo Have a nice day, $name #line 3

Line 1 – Ask a question Line 2 – wait for user input Line 3 – give the message and print the variable’s value.

Page 97: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

97

Example 5

# This script will execute a command if it exists.

echo Please type a command which you want me to execute:

read cmdname

$cmdname

Page 98: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

98

Example 6

# This script is interactive.

echo I will help you in displaying contents of a file.echo Please enter a file whose contents you want to display:read fnameclearcat $fnameechoechoechoecho Thank you for using 6.script

» Only echo command will print a blank line.

Page 99: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

99

Example 7

# This script copies file

echo This program copies a fileecho What file do yo wish to copy \?read mainfileecho What name do you desire to give to the new file\?read newfilecp $mainfile $newfileecho $mainfile is copied as $newfileecho Thank YOU !

Page 100: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

100

Positional parameters

The most important concept in shell scripts is passing arguments to a script. The number after the dollar sign indicates the position on the command line. That is, "$1" indicates the first parameter, and "$2" indicates the second. Positional Parameters are $1, $2, ..., $9 $0 - Script name

Page 101: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

101

Example 8

# This program illustrate the use of positional parameters in conjunction with grep command.

clearecho Program name: $0 #prints the name of the program

echo The number of arguments specified are $# #prints the total number of arguments supplied to the program

echo The arguments are $* #Prints the actual arguments

grep $1 $2 $3

echoecho Program over.

Page 102: UNIX From Capgemini

Arithmetic !

Page 103: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

103

Arithmetic

UNIX shells do not support any notion of numeric data types such as integer or real. All shell variables are strings. Try this:

count=1 Rcount=$count+1

Page 104: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

104

‘expr’ command

The UNIX command expr is used to evaluate expressions. In particular it can be used to evaluate integer expressions.

Examples:expr 5 + 6expr 10 / 5expr 5 \* 10expr 5 + 6 * 10

Note that the shell special character * has to be quoted. The expr command only works with integer arithmetic. If you need to perform floating point arithmetic have a look at the bc and awk commands. the parameters for the expr command must be separated by spaces

Page 105: UNIX From Capgemini

Text handling

Page 106: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

106

Agenda

» grep» cut» paste» sort

Page 107: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

107

grep command

»grep• Searches through one or more files for lines containing a target and then prints all of the matching lines it finds.• $grep room resources

Will search pattern room in the file resources.

• $grep “North West” partners Here pattern is North West and is should be enclosed in quotes.

Page 108: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

108

grep - switches

$grep –v North partners• Will display those records which do not contain North.

$grep –i west partners• Will ignore case differences and display records.

$grep –n Wool partners• List line numbers on which pattern is found.

Page 109: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

109

fgrep• Similar as grep. • $fgrep “plastics

> jackets” partners

$fgrep –f patsource partners• -f allows you to input all the patterns in file ‘patsource’ and fgrep will read from patsource and search each pattern in

partners.

Page 110: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

110

$egrep “West|East|North” partners• Similar to grep/fgrep.• Allows to specific multiple search patterns on same command line.

Page 111: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

111

Columns & fields

West 2 56 25000

East 1 23 13000

North 3 55 50000

South 2 78 34000

North East 1 10 12000

South West 0 0 00000

North West 1 10 11000

South East 3 22 45000

Tab is delimiter.

Page 112: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

112

$cut –f1 resources• Will retreive field 1 from file resources.

$cut –c1 resources• Will retreive column 1 from file resources.

$cut –f1,4 resources > locationsales• Will retreive column 1 & 4 from file resources.

$cut –f2-3 resources > staff• Will retreive column 2 to 3 from file resources.

$cut –d: –f2 newstaff• Delimiter is : and field 2 is picked.

Page 113: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

113

paste• Joins files together line by line.• Use it to create new tables by gluing together fields or columns from two or more files.• $paste country country_abbrv >> countries• -s option will paste the lines of one file at a time rather than one line from each file.• $paste –s regions q3sales

Page 114: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

114

Sorting

Orders or reorders the lines of a file.• $sort <filename>• $sort –o

To replace the input file with the sorted version. $sort –o onefile onefile <E>

• $sort –f Sort while ignoring cases. $sort –f partners

• $sort –t To specify a new delimiter $sort –t: newstaff sort -k 2 partners

Sort second field. sort -t: -k 4 newstaff

Delimiter is : Sort field 4.

Page 115: UNIX From Capgemini

Conditional Statements

Page 116: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

116

If statement

» Purpose of if statement• It takes two way decision depending on fulfillment of a certain condition.• Syntax

if condition is truethen execute commandelse

execute commandfi

» Red – indicates not to be included.» Blue indicates – optional.» It is concerned with status of commands.

Page 117: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

117

Example 9

# This script illustrates if....else...fi statement

if grep 'Technology' catalog.txtthen echo Pattern found in file catalog.txt

elseecho Pattern not found

fi

Page 118: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

118

Example 10

# This script illustrates if....else...fi statement# Input is taken from command line.

clear

echo Give me a pattern to search:read patternecho Give me name of file where I should search:read fname

if grep $pattern $fnamethen echo Pattern found in file $fname

elseecho Pattern not found

fi

Page 119: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

119

‘test’ command

The test command allows you to:• test the length of a string• compare two strings• compare two numbers• check on a file's type• check on a file's permissions• combine conditions together

Syntax:• test an_expression

or• [ an_expression ]

Expression is a string that you want to evaluate.

Page 120: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

120

Example

if [ "$1" = "hello" ]then echo "hello to you too!"else echo "hello anyway"fi

Page 121: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

121

Operators with test command

-eq equal to

-ne not equal to

-gt greater than

-ge greater than equal to

-lt less than

-le less than or equal to

Page 122: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

122

File related operators with test command

-e file – True if file exists. -f file - True if file exists and is a regular file. -r file - True if file exists and is readable. -w file - True if file exists and is writeable -x file - True if file exists and is executable. -d file - True if file exists and is directory. -s file - True if file exists and its size is greater than zero.

Page 123: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

123

Page 124: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

124

Example 11

# llustrates the use of relational operator used by if

if test $# -eq 0

then echo Cannot continue as no filename was supplied.

else

cat $1fi

Page 125: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

125

Example 12

# Relational operator test

echo Your ageread mage

if test $mage = 18

then echo Vote

else

echo cannot vote

fi

Page 126: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

126

case

It matches an expression for more than one alternative and uses a compact construct to permit multi-way branching.

It handles string tests in more efficient manner than if. Syntax:

case expression in

pattern1) execute commands ; ;

pattern2) execute commands ; ;

. . .

esac

Page 127: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

127

Example 14

# Illustrates case statement

clear

echo " Command Menu"echo " 1. List files."echo " 2. Today's date."echo " 3. Who all are working on Unix"echo " 4. Exit"read choice

case "$choice" in

1) ls -l ;; 2) date ;;3) who ;;4) exit ;;

esac

Page 128: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

128

while

Loops let you perform a set of instructions repeatedly. As long as condition is true, commands are executed. Syntax

while condition is true

do

execute commands

done

Page 129: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

129

Example 15

# Illustrates whle loop

ans=y

while [ "$ans" = "y" ]

doecho "Enter the name and code of employee: "read ename ecode

echo "$ename | $ecode" >> emplistecho "More entry (y/n)? "read respons

case $respons in

y*|Y*) ans=y;;n*|N*) ans=n;; *) ans=y;;

esac

done

Page 130: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

130

‘for’ statement.

It checks for a value in a list. A list of commands is executed for each value in the list. Syntax

for variable in list

doexecute command # loop body

done

The loop body is executed as many times as there are items in the list.

Page 131: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

131

Example

for x in 1 2 4 5

do

echo “The value of x is $x”

done

Page 132: UNIX From Capgemini

Introduction to awk programming

Page 133: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

133

Pre-requisites

You should already be familiar with basic system commands, such as cat and ls, Basic shell facilities, such as Input/Output (I/O) redirection and pipes.

Page 134: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

134

There are three variations of AWK:

AWK - the original from AT&T NAWK - A newer, improved version from AT&T GAWK - The Free Software foundation's version

We will cover AWK.

Page 135: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

135

History of awk.

Three designers were involved :• Alfred V. Aho • Peter J. Weinberger • Brian W. Kernighan

The original version of awk was written in 1977 at AT&T Bell Laboratories. The GNU implementation, gawk, was written in 1986 by Paul Rubin and Jay Fenlason, with advice from Richard

Stallman.

Page 136: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

136

Uses of AWK.

The AWK text-processing language is useful for such tasks as: The basic function of awk is to search files for lines or other text units containing one or

more patterns. Tallying information from text files and creating reports from the results. Translating files from one format to another. When you run awk, you specify an awk program that tells awk what to do. Creating small databases. Performing mathematical operations on files of numeric data.

Page 137: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

137

Limitations

It is not really well suited for extremely large, complicated tasks. It is also an "interpreted" language -- that is, an Awk program cannot run on its own, it must be executed by the

Awk utility itself.

Page 138: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

138

Syntax:

awk PROGRAM inputfile(s)

awk -f PROGRAM-FILE inputfile(s)

Page 139: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

139

Initial examples:

$ ls -l | awk '{ print $5 $9 }‘• ls command output is 9 columns• awk is printing from col 5 to 9.

Page 140: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

140

Example 1

Coins.txtmetal weight in ounces date minted country of origin description

gold 1 1986 USA American Eaglegold 1 1908 Austria-Hungary Franz Koronasilver 10 1981 USA ingotgold 1 1984 Switzerland ingotgold 1 1979 RSA Krugerrandgold 0.5 1981 RSA Krugerrandgold 0.1 1986 PRC Pandasilver 1 1986 USA Liberty dollargold 0.25 1986 USA Liberty 5-dollar piecesilver 0.5 1986 USA Liberty 50-cent piecesilver 1 1987 USA Constitution dollargold 0.25 1987 USA Constitution 5-dollar piecegold 1 1988 Canada Maple Leaf

Page 141: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

141

Example 1

awk '/gold/' coins.txt • This tells Awk to search through the file for lines of text that contain the string "gold", and print them out.

awk '/gold/ {print $5}' coins.txt• Line containing gold will printed. $5 means fifth field i.e description.

Page 142: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

142

simplest general form of an Awk program: • awk <search pattern> {<program actions>}

Awk, makes no strong distinction between strings and numbers. Not "strongly-typed“. All the fields are regarded as strings, but if that string also happens to represent a number, numeric

operations can be performed on it. We can perform an arithmetic comparison on the date field.

Page 143: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

143

Format of AWK

awk 'BEGIN {<initializations>}

<search pattern 1> {<program actions>}

<search pattern 2> {<program actions>}

...

END {<final actions>}' • The BEGIN clause performs any initializations required before Awk starts scanning the input file. • The subsequent body of the Awk program consists of a series of search patterns, each with its own program action. • Awk scans each line of the input file for each search pattern, and performs the appropriate actions for each string found. • Once the file has been scanned, an END clause can be used to perform any final actions required.

Page 144: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

144

Example 2

awk 'END {print NR,"coins"}' coins.txt

Explanation:This example doesn't perform any processing on the input lines themselves. Its scans through the file and perform a final action: print the number of lines in the file, which is given by the "NR" variable. NR stands for “Number of Records” awk 'END {print NF,"coins"}' coins.txt

• NF gives the number of fields in a line

Page 145: UNIX From Capgemini

The information contained in this presentation is proprietary.Copyright ©2011 Capgemini. All rights reserved.

145

Example 3

Data file: revenues

Program file : total.awk.

{ total=total + $5 } { print "Send bill for " $5 " dollar to " $4 } END { print "---------------------------------\nTotal revenue: " total }

$ awk –f total.awk revenues

Page 146: UNIX From Capgemini

Thank you.