unix shell scripting

1

Unix Shell Scripting

2

Course Objectives

The participant will learn Linux Introduction Linux file related commands Linux text manipulation

commands Environment variables The vi-editor Shell programming Awk programming

3

Introduction to Linux Features of Linux System:

1. Simple design, organization and functioning2. Portability3. Background processing4. Hierarchical File System5. Multi-user6. Multi-tasking 7. Security8. Interactive Operating System 9. Communication facilities10. Utilities

4

Introduction to Linux

Utility Software:1. Text Manipulation tools: cut, grep, tr, etc.2. Advanced filters: sed, awk, etc.3. Document Formatting tools: troff, nroff, etc.4. Various Programming Languages: C, C++, Java, etc.5. Interactive Calculators : bc, dc, etc.6. Advanced tools : lex, yacc.

5

Linux File System

The Linux file system is the structure in which all the information on your computer is stored.

It has hierarchical file structure, where each directory can contain files, as well as other directories.

It looks like an upside down tree. Linux looks at everything as a file. At the top is the root directory, represented by a

forward slash (/). At the children node of it is a set of common

directories in the Linux system, such as /bin, /dev, /home, etc.

6

Linux File System

The diagram shows the Linux file structure /

bin/ home/ etc/ dev/ …

itp1/ itp2/ … test1.sh test2.sh …

7

Linux File System

In Windows file systems, drive letters represent different storage devices like A: is a floppy drive, C: is a hard disk, etc. In Linux, all storage devices are in the same file system hierarchy.

Windows file systems uses backslash (\) to separate directory names, whereas Linux uses forward slash (/) for the same.

Every file and directory in a Linux system has permissions and ownership associated with it.

File names have suffixes in Windows, whereas in Linux you can use them as conventions.

8

Linux File System

The chunk of the disk is divided into blocks (1024-bytes block, or 4096-bytes block, etc.) in multiple of 512 bytes in size.

These blocks are organized into four groupings- boot block, superblock, inode blocks and data blocks.

FS1

FS2FS3

Bootblock

Superblock inode list Data blocks …..

9

Linux Users

Users

1. Super user2. Owner3. Group4. Others

10

Introduction to Linux

Logging on and Logging off.Step 1: telnet <System Address>Step 2: login : ravindra

Step 3: password : ****** Login incorrect Step 4: login : ravindra Step 5: password : ******

Step 6: $ Command prompt to execute commandsStep 7: $ logout

11

Unix Shells

The Bourne shell /bin/sh (written by S. R. Bourne). Along came the people from UCB and the C-shell

/bin/csh was born. Into this shell they put several concepts which were new, (the majority of these being job control and aliasing) and managed to produce a shell that was much better for interactive use.

Eventually David Korn from AT&T had the bright idea to sort out this mess and the Korn shell /bin/ksh made its appearance. The Korn shell became part of System V but had one major problem; unlike the rest of the UNIX shells it wasn't free, you had to pay AT&T for it.

12

Unix Shells

Also at about this time the GNU project was underway and they decided that they needed a free shell, they also decided that they wanted to make this new shell POSIX compatible, thus bash (the Bourne again shell) was born.

Like the Korn shell bash was based upon the Bourne shells language and like the Korn shell, it also pinched features from the C shell and other operating systems.

13

Unix Shells

The Bourne Shell The Bourne shell is the original UNIX shell program.

It is very widely used. You can start the Bourne shell—if it hasn't been set as your default startup shell—by typing "sh" or "/bin/sh" at the command prompt. This will not spawn a new shell window, but rather will just change your current shell to the Bourne shell.

The Bourne shell supports conditional branching in the form of if/then/else statements. In addition, the Bourne shell supports case statements and loops (for, while, and until).

The Bourne shell uses the $ as a prompt.

14

Unix Shells

The Korn shell The Korn shell is a much newer variation of the

Bourne shell. It supports everything the Bourne shell does, and adds features not available in the Bourne shell. The Korn shell is not a standard offering in UNIX installations. If you have the Korn shell, you can run it by typing ksh or /bin/ksh at the shell prompt. A public-domain version of the Korn shell, called pdksh.

The Korn shell was originally written by David Korn and is copyrighted by AT&T.

The programming structure of the Korn shell is very similar to that of the Bourne shell. The Korn shell, however, is more interactive.

15

Unix Shells

The C shell The C shell is a very commonly used shell. Its

programming structure closely resembles that of the programming language "C."

The C shell uses the "%" as a prompt. The C shell supports all of the features that the

Bourne shell supports, and has a more natural syntax for programming.

The C shell is more interactive than the Bourne shell, with additional features that aren't available in older shells.

The configuration of the C shell is controlled by the .rc and the .login files.

16

Unix Shells

The tc shell The tc shell is a more modern variation of the

C shell. It reads the same configuration files that the C

shell uses. Tcsh contains command line editing

keystrokes that the C shell is missing, and has more "modern" conveniences that the C shell lacks.

17

Unix Shells

The Bourne-Again shell The Bourne-Again shell is a variation of the

Bourne shell. It is commonly used in Linux, but is widely

available in other standard UNIX distributions. The Bourne Again shell is another modification of

the Bourne shell, and uses the $ as a prompt. To start the Bourne Again shell, type "bash" at the

shell prompt. The behavior and environment of the Bourne

Again shell is controlled by the .bashrc file, which you'll find as a hidden file in your home directory.

18

Unix Shells

Unix Shell application comparison table Application sh csh ksh bash tcsh Job control N Y Y Y Y Aliases N Y Y Y Y Input/Output redirection Y N Y Y N Command history N Y Y Y Y Command line editing N N Y Y Y Vi Command line editing N N Y Y Y Underlying Syntax sh csh ksh sh csh

19

Basic commands in Linux

$ who The command shows who is logged on

$ who am I The command shows who are you

$ who \> am \> i \

We can type a command over two or more lines. A backslash character before the end of the line followed by a new line is taken to be continuation of the line.

20

Commands In Linux

$ man The command formats and displays on-line

manual pages. The manual pages are divided into logical

grouping of the commands called the sections. Sections are numbered from 1 through 9. For example, commands are 1, system calls are 2, library function are 3, file formats are 5, & management commands are 8.

If section is specified, man only looks in that section of the manual.

The command man 2 open will display the manual for the system call open.

21

Commands In Linux

Option Description –k searches a set of database files

containing short descriptions of system commands for keywords and displays the result on

the standard output. -f gives one line introduction to the

command (only complete word matches are displayed).

22


$ date The command prints or sets the system date and

time $ date –r TestFile

It displays the last modification time of the file “TestFile”

$ date string The super user can invoke the date command with

a numeric argument to set the system date. This argument is usually an eight-character string of

the form MMDDhhmm (month, day, hour in 24-hour format, minutes), and optionally followed by a two-digit year string.

23


The command can be used with suitable format specifiers as arguments. Each format is preceded by a + symbol, followed by the % operator, and a single character describing the format.

Sequence Interpretation +%a abbreviated weekday name (Mon .. Sun) +%b abbreviated month name (Jan .. Dec) +%d day of month (01 .. 31) +%r time (12- hour) +%T time (24- hour) +%y last two digits of year (00 .. 99) +%Ddate (mm/dd/yy)

24

Commands In Linux

$ ls The command lists contents of directories

Option Description -l list in long format -C multicolumn output

-F indicates type of file by /, * -R recursive listing of all subdirectories

encountered -a list all files including hidden files

25

Commands In Linux

$ cat The command concatenates files and prints

on the standard output $ cat file1 file2 …

It displays contents of all files specified on the command line one below the other

$ cat > file It creates a new file by accepting text from

the standard input Press CTRL-d to save and exit the file

26

Commands In Linux

$ mkdir [-p] dirname The command makes a directory dirname. The dirname can contain the full path prefix of

the directory to be created. When executed with –p option, it won’t give

any error if the directory dirname already exists.

When executed with option –p, it makes parent directories in the path if needed (if any parent directory in the path is not available) .

27

Commands In Linux

cd [directory] change working directory to the directory if

specified or to the home directory otherwise

28

Commands In Linux

$ pwd The command prints name of your current

working directory

29

Commands In Linux

$cp Copies files and directories

$ cp file1 file2 It copies file1 to file2

$ cp file1 file2 … dest_directory It copies multiple files in the specified

existing directory $ cp -r directory1 directory2 … dest_directory

Recursively copies files from directory1, directory2 etc. to the dest_directory

30

Commands In Linux

$ mv The command changes name of your file or

moves your file to the specified destination path $ mv file1 new-file

It renames file1 as new-file $ mv file1 file2 … dest_directory

It moves multiple files to the specified existing directory

$ mv directory1 directory2 … dest_directory moves one or more directory subtrees to an

existing or new dest_directory

31

Commands In Linux

$ rm The command is used to get rid of

unwanted files/directories $ rm [-i] file …

It is interactive removal (option –i) of specified files.

$ rm -r directory … It is recursive deletion of all files within the

specified directories and also the directories themselves.

32

Commands In Linux

Wild characters ^ beginning of the line $ end of the line ? Matches any character [ ] set of characters character hyphen(-)

represents range { } a number enclosed in it specifies the

number of times the preceding expression is to be repeated

! represents negation

33

Linux Users

Users

1. Super user2. Owner3. Group4. Others

34

Commands In Linux

$ chmod The command changes file permission.

Using absolute value. 4 - Read, 2 - Write, 1 – Execute

$ chmod 740 zz

35

Commands In Linux

Using Symbolic Mode Code Meaning a all

u userg groupo other+ add- remove= assign

36

Commands In Linux $ umask

The command changes initial Permission of newly created file.

The value of the argument can be calculated by subtracting the mode you want as default from the current default mode.

Assume the current default mode is 0666 and you want it as 0644 then 666 – 644 = 022 will be the parameter which we have to pass with umask command.

$umask 0 – sets default mode which is 0666.

37

Text Manipulation Commands

Filters : Filters : A filter is a shell command which takes

input from the standard input, processes it, and sends its output to the standard output.

At run time, the system supplies data to the filter as standard input. This std input file can not be altered by the program.

38


An important characteristic of filter is that all the input, output and error channels have the same structure. They are all unstructured byte streams data delimited by an end-of-data marker.

Some widely used filters in Unix are: sort, grep, and wc.

Filters allows the users to create the complex programs from the simpler programs.

The commands cp, mv and cd are not filter commands.

39


wc : word count. $ wc -[wlc] [filename] head : displays first ‘n’ lines $ head -[n] [filename] tail : displays last ‘n’ lines $ tail -[n] [filename] split : divides files horizontally $ split -[n] [filename] we get m subfiles of size n (xaa, xab,…).

40


$ cut -[cfd] [filename]

-c characters-f field no-d field separator

$ cut -c2-5 samplecuts characters from 2 to 5 from file sample. continued...

41


$ cut -c1,2 file cuts 1st and 2nd characters from file.

$ cut -d” “ -f1-2 file cuts first 2 fields of the file.

$ cut -c2- names cuts from 2nd char to the end of line.

42


paste: join files column-wise $ paste -d[field separator] [list of files]

uniq: removes duplicate lines from a sorted file $ uniq –[dcf] [file]

-d only print duplicate lines-c prefix lines with number

of occurences

-f2 avoid comparing first two fields.

43


sort: Sorting text files Self study. Try to find out different options for this

command using man pages.

44


tee: read from standard input and write to standard output and files

$ls –l | tee dircontents displays the directory contents on the

standard output as well as stores them to file dircontents.

nl: number lines of files nl –[options] [files]

45


tr : Translating Characters.$ tr [options] < [file]-d : deletes specified characters.-cd : do not delete specified characters.-s : substitute characters.

$ tr “abc” “ABC” < samp : replaces all occurrences of a with A, b with B, c with C.

46

Commands In Linux grep: it is called as global regular

expression pattern. It searches for a given pattern in the file(s)

$ grep -[cvnl] [pattern] [files] Option Description

-c counts the total no of lines containing that pattern. -v displays all the lines not containing that

pattern. -n displays lines with line no. -l displays file names containing the pattern.

47

Commands In Linux

fgrep: fast searching for fixed strings It handles fixed character strings as text patterns. It cannot process wild-card matches, character

classes. egrep: used to search with full regular

expressions. It is called as extended grep.

egrep ‘ (Ravi|Ravindra) Joshi ‘ employee.txt This will search for Ravi Joshi as well as for

Ravindra Joshi

48

Commands In Linux

$ ps It shows process status.

Display the attribute of a process. Options

$ ps -f --> Full option $ ps -f -u itp5 --> gives processes of user itp5 $ ps -a --> all users processes. $ ps -e --> all processes on the

system including system processes.

49

Commands In Linux

$ kill The command is used to terminate a

process. The command uses one or more PIDs as its arguments.

$ Kill 105 It will terminate job with PID 105.

$ kill -9 121 The option –9 indicates sure kill signal.

$ kill $! The system variable $! Stores the PID of the

last background job.

50

Commands In Linux

To execute any process in background just use & at the end of the command. $ sh test &

To bring the process in foreground use fg To check out jobs currently running in

background use command --> jobs $ jobs

51

The Environment

The Unix system is controlled by a number of shell variables that are separately set by the system some during boot sequence, and some after logging in. These variables are called system variables or environment variables.

The set statement displays the complete list of all these variables. Built-in variable names are defined in uppercase.

52

The Environment

The PATH : is a variable that instructs the shell about the route it should follow to locate any executable command.

The HOME : when you log in, UNIX normally places you in a directory named after your login name.

The MAIL : determines where all incoming mail addressed to the user is to be stored.

The PS1 and PS2 : PS1 - your command prompt and PS2-Multi-line command string.

The SHELL: determines the type of shell that a user sees on logging in.

53

The Environment

.bash_profile : the script executed during login time. Every time you make changes to it, you should log out and log in again.

The .bash_profile must be located in your home directory, and it is executed after /etc/profile, the universal profile for all users. Universal environment settings are kept by the administrator in /etc/profile so that they are available to all users.

54

The Environment

ALIASES : it allows you to assign short-hand names for commands you may be using quite frequently. This is done with the alias statement. Consider following ex. $ alias l=‘ls -l’

Aliases are listed when the alias statement is used without argument.

The alias feature also allows you to incorporate positional parameters as variables in an alias. For ex. $ alias showdir=‘cd $1 ; ls -l’

When you want to see the contents of the directory /home/itp5 use the alias as $ showdir /home/itp5

55

The Environment

Setting the Environment variable: Environment variables can provide a way of

storing information that you need to use often from the shell.

Assigning a value to a variable will set an environment variable temporarily. For example:

$ x=50 This example will set value 50 to the variable x.

Use the export (discussed latter) command to export the value to the shell so that it can be propagated to other shells you may open.

56

The Environment

The variable will not be available if you exit the shell. Add these variables to configuration files (discussed earlier) to set them permanently.

If you no longer want a variable to be set, you can use the unset command to erase its value.

For example $ unset x This command will cause x to have no value set.

57

The Environment

By default,the values stored in shell variables are local to the shell, i.e., they are available only in the shell in which they are defined. They are not passed on to a child shell. But the shell can also export those variables recursively to all child processes so that, once defined, they are available globally. This is done with the export command.

For example, consider a simple script $vi exporttest.sh

– echo the value of x is $x– x=50– echo the new value of x is $x

58

The Environment

Now at the command prompt, first assign value 25 to x, and then execute the above script and observe, we will get following output

$ x=25 $ ./exporttest.sh

the value of x is the new value of x is 50

$ echo $x $ 25 Because x is a local variable in the login shell, its

value can’t be accessed in the script exporttest.sh.

59

The Environment We will use the export command and observe the

change. $ x=25 $ export x $ ./exporttest.sh

the value of x is 25 the new value of x is 50

$ echo $x 25

When x is exported, its assigned value 25 is available in the script. A reassignment (x=50) in the script which is a sub-shell, is not seen in the parent shell which executed the script.

60

The Environment

Executing commands from a file: . (dot) If we have a list of commands in a file sample

then we can execute it using command . sample

It is like executing a shell script, but with following difference: Standard shell scripts cause a new sub shell

to be created to run the script. The dot command, uses the same shell and it just needs redirection to take the commands from the file instead of from the keyboard.

61

The Environment

The dot command can change the value of a shell variable in the current shell.

The export command lets you pass a variable’s value from parent shell to child shell, but there is no comparable mechanism for passing a value from child to parent.

The dot command, however, creates no child process, so any changes it produces apply to the original shell.

It also doesn’t require the script to have executable permission.

62

The Environment

For example, consider the simple script $vi dottest.sh

– echo the value of x is $x– x=50– echo the new value of x is $x

Now we will observe the change by exporting variable x and executing the script using dot command.

63

The Environment

$ x=25 $ export x $ . dottest.sh $ the value of x is 25 $ the new value of x is 50 $ echo $x $ 50

When x is exported, its assigned value 25 is available in the script. A reassignment (x=50) in the script has also changed the value of x in the parent shell which executed the script.

64

The vi Editor

A vi session begins by invoking the command “vi” with or without a filename.

$ viAfter executing this command you will get a full empty screen, each line beginning with a ~(tilde). The last line in the screen is reserved for some commands that you can enter to act on the text. This line is also used by the system to display messages.

65

Modes of operation The three different modes of operations are:1. Command mode : This is the mode where you

can pass the commands to act on the text, using most of the keys of the keyboard.

Example: Key “x” is used to delete the character at the cursor position.

You can switch to this mode using “ESC” key.

contd..

66

Modes of operation2. Insert (Input) mode: To enter the text, you have

to enter into input mode. Press key “i” to enter into insert mode from command mode.

3. ex mode or line mode : You have to save your file or switch to another file or make a global substitution in the file. You then have to use ex mode, where you can enter the instruction in the last line of the screen. To enter into this mode press “ESC” & “:” Example: ex mode command “:wq” will save the current file and will quit from the editor.

67

Text insertion commands

Command Description

1. i inserts text at cursor position2. a inserts text after cursor position3. I inserts text at beginning of line4. A inserts text after end of line5. o opens line below current line to

insert text6. O opens line above current line to

insert text

68

Cursor movement commands

Command Description

1. h left by one character2. l right by one character3. k up by one line4. j down by one line5. w right by one word6. b left by one word7. 0 beginning of line8. $ end of line

69

Text deletion commands

Command Description1. x character under cursor2. X character before cursor3. dw word4. d0 beginning to cursor position5. d$ cursor position to end of line6. [n]dd n lines from current line7. [n]dd p p will paste deleted lines to

current cursor position. Equivalent to Ctrl-X and Ctrl-V [in Windows]

70

Text copy commands

Command Description1. y character2. y0 beginning to cursor position3. y$ cursor position to end of line4. yw word5. [n]yy n lines from current line in to

the buffer 6. [n]yy p p will paste copied lines to

current cursor position. Equivalent to Ctrl-

C and Ctrl-V [in windows]

71

Text modification commands

Command Description1. nc [space] overwrites next n characters 2. c0 overwrites the portion

between beginning of line to cursor position

3. c$ overwrites the portion between cursor to end of line

4. cw overwrites current word5. :[addr1,addr2]s/pattern1/pattern2[/g]

globally replaces pattern1 with pattern2 on the specified lines

72

File related commandsCommand Description

1. ZZ or :wq save and exit2. :w save & continue editing3. :q! quit without saving4. :r filnam insert file filnam 5. :[addr1,addr2]w filname write the lines between line

number addr1 and line number addr2 in the file filname

contd..

73

File related commands Command Description

1. “a4yy copy 4 lines into buffer a2. “ap paste contents of buffer a to

current cursor position.Maximum 26 buffers are

available buffer having names “a” to “z”

1. ctrl-v select particular columns 2. “byy copy selected into buffer b 3. “bp paste contents of buffer b to

current cursor position. contd..

74

File related commands Command Description1. 1,$s/source/target/ substitute string source by

string target from line number 1 to last line

2. u undo last change on the line3. U undo last changes on the line4. Ctrl-R redo the undone changes.5. e edit multiple files 6. e# return to original file7. rew back to first file in command line

contd..

75

Visual mode commands

Command Description

1. v_u converts the character to lower case

2. v_U converts the character to upper case

3. sp splitting window 4. Ctrl-w toggle between windows5. <Ctrl-w>j moves to lower window6. <Ctrl-w>k moves to upper window

76

Customizing vi

The set command When the string “no” is prefixed to any option,

it indicates that the option is inoperative.Command Description

1. :Set all displays all set option2. :set autoindent does automatic indentation3. :set number shows all line duly

numbered4. :set showmatchhelps to locate matching

brackets5. :set tabstop=5 sets tab=5 for display

77

Advanced Commands The .exrc File and exinit : all sets, maps and

abbreviations can be conveniently stored in the file $HOME/.exrc

Vi looks for this file on startup and executes the instructions as ex mode command

78

Shell Programming

When a group of commands has to be executed regularly, they are stored in a file. All such files are called shell scripts, shell programs, or shell procedures.

There is no restrictions on extension of these files, but conventionally extension .sh is used for a shell program.

You can use the vi editor to create the shell script.

79

Shell Programming

You can execute the shell scripts using either command sh or by just typing shell script name at the prompt (make sure that you have execute permission).

$ sh test.sh $ test.sh $ ./test.sh

80

Shell Programming User-created Shell Variables :

variable=value => will assign value to variable.$variable => will print value of the variable.

Command echo : it displays message on the screen. Consider following ex.$ var=hello$ echo $var “$var” ‘$var’with this the output will be $ hello hello $var

2. read var => will read value from keyboard to var.$ read var

81

Shell Programming Operators

Arithmetic : + - / * %Relational : -lt -le -gt -ge –eq –ne Logical : -a -o

Expressionexpr 10 + 20

Assignment statement.x=77

Command substitution: when a command is enclosed in back quotes, the command is replaced by the output it produces. Ex. The command

x=`expr $x + 1` will set x=78

82

Shell Programming

Shell Metacharacters: Wildcard substitution: * ? [] Redirection: > >> < 2> << Piping: | Command substitution: ` ` Sequential commands: semicolon (;) Inserting comment: # Command grouping: () parenthesis

Ex. (date; cat employee.txt) > Report This will copy contents of employee.txt

with current date to a file Report.

83

Shell Programming

Creating a here document (<< operator): It allows a shell script to get its input from the

same file that holds the script. Data used this way is said to be in a here

document. To create a here document, we use the <<

operator. Any command using standard input can also have

the input from a here document. The here document symbol (<<) followed by the

data, and a delimiter (the termination string).

84

Shell Programming

The example for here document $ wc << EOF >This is for testing the here document. It will

>give us the count of lines, words and characters >till the terminator given after here document >symbol, the terminator which is the EOF.

We will get output of the command as number of lines, words, characters till first occurrence of the EOF symbol in the script

We can use any string as a terminator.

85

Shell Programming

Debugging the scripts Command set is used for debugging purpose The –v option of sh causes the shell to echo each

command before it is executed. Command Action1. set –v sets the debugging

mode on2. sh –v sample.sh echoes each command of

script sample.sh before it is executed

3. set +v turns off –v option of sh We can also use –x option. What is difference?

86

Shell Programming

Conditional Statement – if then else Syntax:

if test <condition> then

command1command2

elsecommand3

fi The if statement takes two-way decisions

depending on the condition .

87

Shell Programming

String Tests used by the command test String Comparison True if

string1 = string2 strings equal string1 != string2 strings not equal-n string string not null-z string string is null

88

Shell Programming

Arithmetic expression comparisons used by the command testArithmetic comparison True ifexpr1 -eq expr2 expressions equalexpr1 -ne expr2 expressions not equalexpr1 -gt expr2 expr1 > expr2expr1 -ge expr2 expr1 >= expr2expr1 -lt expr2 expr1 < expr2expr1 -le expr2 expr1 <=expr2! expression expression is false

89

Shell Programming

String Tests used by the command testTest True if-d file file exists and is a directory-e file file exists-f file file exists and is a regular file-r file file exists and is a readable-s file file exists and has a size > 0-w file file exists and is a writable-x file file exists and is a executable

90

Shell Programming

Conditional Statement - case Syntax:

case $value inval1) command1 command2;;val2) command3;;….*) command4;;

esac The statement matches an expression for more than

one alternative, and permits multi-way branching.

91

Shell Programming

Example of case statement:echo “Enter the color”read colorcase $color in

Red | red) echo “You have selected red color”;;Blue | blue) echo “You have selected blue color”;;*) echo “Sorry! Yet to add this color in our list”

esac

92

Shell Programming

Loop Statement - while Syntax:

while test <condition>do

command1command2

done While statement repeatedly performs a set of

instructions till the control command returns a true exit status.

93

Shell Programming

Loop Statement – until Syntax:

until test <condition>do

command1command2

done The set of instructions is executed repeatedly as

long as the condition remains false. The until statement complements the while

statement.

94

Shell Programming

Unconditional loop – for Syntax

for <var> in <val1 val2 …>do

command1command2

done The loop body (do-done) is executed as many

times as there are items in the list. It doesn’t test condition but uses a list.

95

Shell Programming

Example of for statement Assume file “Users.txt” contains names of

users.for name in `cat Users.txt`do

echo Hello, $namedone

This statement when executed, will print the message “Hello, user-name” for each user name from the file.

96

Shell Programming

Command Line parameters (positional parameters)

When arguments are specified with a shell script, they are assigned to variables called positional parameters.

The first argument is read by the shell into the parameter $1, the second into the parameter $2, and so on.

The $# represents total no of arguments passed to the script.

The command is assigned to a variable $0. You can use these variables up to $9.

97

Shell Programming

The $* indicates all arguments, in a single variable, separated by the first character in the environment variable IFS

The $@ is same as $* except when enclosed in double quotes.

The “$@” works with string input. The parameter $? stores the exit status of the

last command. On success, the command stores value 0 in $?, and a non-zero value if it fails.

98

Shell Programming

set : Assigning values to positional parameters.

$ set 23 532 The command assigns value 23 to the

positional parameter $1, value 532 to $2. It also sets $#, $*.

99

Shell Programming

shift : shifting arguments to left. The command shift copies the contents of a

positional parameter to its immediate lower numbered positional parameter. When called once, contents of $2 are copied to $1, $3 to $2 and so on. $ shift 2

The command does two shifts i.e. $1=$3, $2=$4, and so on.

Using shift command we can use more than 9 command line parameters.

100

Shell Programming

Finding the length of the string: The regular expression .* signifies to command

expr that it has to print the number of characters matching the pattern, i.e., the length of the entire string.

We can also use keyword length with command expr

$ expr “abcd” : “.*” $ 4 $ expr length “abcd” $ 4

101

Shell Programming

Command grouping using parenthesis () Commands are grouped using parenthesis () Using parenthesis, we can collectively redirect

the standard output of two commands with a single redirection symbol.

For example: $ (date; cat employee.txt) > Report.txt

This will redirect the collective output which is current date and employee.txt file contents to Report.txt file.

102

Shell Programming

Command grouping using curly braces {} The curly braces {} are also used to group

commands. The difference between () and {} is that parenthesis

() executes the commands group in a sub-shell, while the curly braces {} uses the current shell only.

The closing curly brace must be on a separate line by itself, or simply terminate the last command with a semi-colon.

For example, observe output of the following: $ pwd ; ( cd /home/ravindra/test ; pwd) ; pwd $ pwd ; { cd /home/ravindra/test ; pwd; } ; pwd

103

Shell Programming

Shell Functions: A function consists of a group of statements which

are executed together as a bunch. For function we need function definition (body of

function) and function call. A shell function must precede the statements that

call it. The return statement, when present, returns a

value representing the success or failure of the function.

The return statement is optional.

104

Shell Programming

The syntax for function definition: function_test() {

Command1 Command2 Command3 return value

} When function is invoked it executes all the

commands enclosed by the curly braces. Use only function name (function_test ) to

invoke it.

105

Shell Programming

Shell functions can be defined at a number of places: At the beginning of every script In a separate file, so that other scripts can also

use them In the .bash_profile, so that they are available

in the current session If we store the functions in a file called

function_library then you must include this file in your script using dot command as follows:

$ . function_library

106

Shell Programming

The positional parameters made available to shell scripts externally are not available directly to a shell function.

We have to store these parameters in the shell variables and then pass them to the function.

The parameters are passed on the function call statement itself.

These parameters are accessed in the function using system variables $1, $2, etc.

107

Advanced Filter - awk

AWK Programming. Aho, Weinberger, Kernighan A typical and complete awk command

specifies address and an action. $ awk ‘/unix/ { print }’ filename

the address section /unix/ selects lines that are processed in the action section { print }

the print statement, when used without any field specifiers, prints entire line.

108


Splitting a line into fields : awk uses the special “variable” $0 to indicate the entire line. It also identifies fields by $1, $2, $3, etc.

awk uses -F option for indicating the field separator.

For ex. password file contains colon (:) separated fields.

So if you have to use awk you can use as $ awk -F”:” ‘/itp/ { print $1, $3, $2 }’

passwd

109


The logical and relational operators. $ awk -F”:” ‘$1 == “itp3” || $1 == “itp5”

{ printf “%-20s %-12s “ , $2, $5}’ passwd The awk can also handle numbers, both integer

and floating type, and all the relational tests can be handled by awk. (ex. <,<=,==, etc)

awk has certain built-in variables, like NR- cumulative record no and $0 (whole record) $ awk ‘NR == 3, NR ==5 { print $3, $4 }’ tex

110


Storing awk Programs in a File Large awk programs should be stored in files.

The extension should be preferable .awk to distinguish it from other files.No quotes are used to enclose the awk program. $ vi test.awk NR == 3, NR ==5 {print $3, $4}

you can use awk with -f to get result as $ awk -f test.awk text

111


The BEGIN and END sections these are optional, and take the form BEGIN { action } END { action } you can use them to print a suitable heading

at the beginning and the average salary at the end. Always start the opening brace in the same line the section (BEGIN or END) begins.

112


Built-In variables: NR cumulative no of records read FS input field separator OFS output field separator NF number of fields in current record FILENAME current input file

113


FUNCTIONS : awk has several built-in functions for arithmetic and string operation

int(x) returns integer value of x sqrt(x) returns sqare root of x length returns length of complete record length(x) returns length of x substr(s1,s2,s3) returns portion of string of

length s3, starting from position s2 in str s1

114


CONTROL FLOW : awk has practically all the features of a modern programming language.

If statement : if($5 >20000) interest = 0.10 * $5 else interest = 0.05 * $5

115


Looping with for for( i=1; i<10; i++) Looping with while while ( i < 10) print $2 * i

116

Thanks!Contact - Sharda Centre - ext. 5160

unix shell scripting

Documents

c shell

shell prompt

free shell

used shell

current shell

bourne shell binsh

korn shell binksh

korn shell bash