unix - shell scripting

Unix Shell Programming - Advanced

Vinodh K Nair

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 1

Introduction: Name Short ID Role Responsibility Experience Area of interest Any previous experience in shell scripting Current rating on shell scripting knowledge (1-10) Expectations from the program?

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 2

Ground Rules In order to ensure the productivity of our training, we will need to be.Pagers and mobile phones off Full participation One speaker at a time Respect the views of others Silence indicates agreement Keep to the break times agreed

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 3

Course Agenda:Shell Basics1. Introduction to Shell and types of Shells 2. Variables and Keywords 3. Metacharacters 4. Regular Expressions 5. Using grep and egrep 6. Using Quotes

Introduction Shell Programming1. Shell Programming Step 1 2. Expression Handling 3. Positional Parameters 4. Conditional Statement 5. Case Statement 6. Loops in Shell 7. IO Redirection

Advance Shell Programming1. Shell Functions 2. Advanced tools a) The find utility b) Cutting the outputs with cut 3. Fundamentals of awk 4. Fundamentals of sed 5. Miscellaneous tools 6. Debugging the Shell scripts 7. Dealing with Signals8/23/2012 7:52:52 AM 5864_ER_FED_ALT 4

Pre Quiz Write your name and Employee ID without fail. 25 questions, multiple choices in 15 minutes Use the pre-quiz columns for writing your answers Do not discuss A question can have more than one correct answer, but choose the best one. Please encircle your guessed answers to find how good you are at guessing Chocolates will be distributed for: Pre-quiz highest scorer Post-quiz highest scorer One with maximum difference8/23/2012 7:52:52 AM 5864_ER_FED_ALT 5

1. 2. 3.

Unix BasicsStructure of Unix

Applications

Shell Kernel

H/W

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 6

ShellShell is a command interpreter and acts as an interface between a user and kernel

Types of Shells

Bourne Shell /usr/bin/sh Korn Shell /usr/bin/ksh C Shell /usr/bin/csh bash /bin/sh POSIX /usr/bin/sh

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 7

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 8

HPUX Directory Structure / (root)

stand

bin

lib

dev

usr

tmp

opt

etc

home

sbin

bin

lib

newconfig

user1

user2

user3

user4

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 9

Unix Shell ProgrammingBREAK

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 10

Permissions --- --- ---

owner

group

others

Permission r w x

Description read write execute

Weight 4 2 1

Ex: - rw- r- -r- - 1 user1 group1 12 Oct 10:45 file1

Use chmod to change the attributes

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 11

Absolute path

Absolute path represents the location of a file or directory starting from the root directory

Ex: /> /home/user1/scripts/check.sh

Relative pathAccessing files and directories from the current directory

Ex: /home/user/scripts> ./check.sh

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 12

VI Editor

bottom8/23/2012 7:52:52 AM 5864_ER_FED_ALT 13

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 14

Shell BasicsVariablesVariables are the general words which are used to store and manipulate information within a shell program. Variables are fully under user control. These can be created and destroyed at anytime. Ex: = TOWN=Delhi ; TOTAL=0 ; DATE= 28Nov05

Local Variables Local Variables are the variables that are presented within the current instance of the Shell. These are not available to the any child processes that are started by the current shell. Local variables are defined by users. Global Variables Global Variables are the Variables that are presented within the current shell and also available to any child processes that are started by the current shell.

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 15

Keywords

Keywords are the words which have standard predefined meaning in the shell already. Keywords can not be used as variable names. Keywords are also called as Reserve words.

List of Keywords

echo export done continue sleep

read if for exit test

set else until return

unset fi case trap

readonly while esac wait

shift do break exec

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 16

Environment VariablesEnvironment Variables are the special variables that are set by the shell and is required by the shell in order to function correctly. These variables are also called as shell variables.

List of Environment Variables

PATH HOME LOGNAME SHELL TERM PWD PS1 IFS

Defines the path which the shell must search in Stores the default working directory for the user Stores the login name of the user Defines the name of default working shell Defines the name of the terminal Stores the Present working Directory Defines the system prompt Defines the Internal Field Separator

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 17

Unix Shell ProgrammingBREAK

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 18

Shell Meta Characters. * ? [] [!...] Matches any one character Represents any combination of any characters Represents any one character Matches any one character from the enclosed list Matches any one character except those in the list

Ex: $ ls a* $ ls a?b? $ ls [aeiou]* $ ls [A-Fv-z]* $ ls [!A-Z]* Lists all files beginning with character a Lists all 4 character filenames whose first character is a and third character is b Lists all the files whose first character is a, e, i, o, u Lists all files whose first character is in the range A to F or v to z Lists all files whose first character is anything other than in the range of A to Z

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 19

Regular ExpressionsA regular expression is a string that can be used to describe several sequences of characters. ^ $ \ ^$ \< \> \ Matches the beginning of a line Matches the end of a line Used to specify patterns that contains wild cards Matches a blank line Matches at the beginning of the word Matches at the end of the word Matches a complete word

8/23/2012 7:52:52 AM 5864_ER_FED_ALT 20

Basic Shell commands man - Very important command known in UNIX cp [-ir] file1 file2 cp [-ir] file-list directory i for interactive. prompt use whenever a file will be overwritten r for recursive. copy a directory tree

ls [-alRF] file-list a for listing all files including the dot files l for long format R for recrusive. list the all subdirectories. F for listing directories with a trailing /

date [+format] %date +%h %d, 19%y Oct 1, 19968/23/2012 7:52:53 AM 5864_ER_FED_ALT 21

Basic Shell commands (Cont) wc file-listDisplay the number of lines, words and characters

more file-listBrowse through text files one page at a time.

head [-n ] file-listDisplay the first n lines of the files (default=10)

tail [+n|-n| -f| ] Display the last few lines in the files (default = 10) Example: # tail +5 foo # display the last parf of foo starting from line 5 # tail -5 foo # display the last five lines of foo # tail +30 foo | head -15 | more #display line 30-45 of foo # tail -f foo # wait and display the new lines appended to foo

8/23/2012 7:52:53 AM 5864_ER_FED_ALT 22

Basic Shell commands (Cont) cut -c list filecut -f list [-dChar] file Cut out selected charatcers or fields from each line of a file Examples: # cut -c 1-5,15-20 foo # cut -f 1,3 -d moo

# extract chars 1-5 and 15-20 from each line of foo. # extract field 1 and 3 from each line of moo.

paste file1 file2 Concatenate corresponding lines of the given input files Example (reverse two fields of the file abc) # cut -f1 abc > abc1 # cut -f2 abc > abc2 # paste abc2 abc1 > xyz

8/23/2012 7:52:53 AM 5864_ER_FED_ALT 23

Basic Shell commands (Cont)sort [-tC] [-o outfile] [field-list] [file-list]sort the files # sort +1 list1 # sort +2 -3 list2 # sort -n -o list4 list3 # sort list 1 starting from field 2 to the end of the line # sort list2 on the third field # sort list3 numerically and place the output in list4

diff file1 file 2 Display different lines that are found when comparing two files It prints a message that users ed-line notation (a - append, c - change, d -delete) to describe how a group of lines has changed. It also describes what changes need to be made to the first file to make it the same as the second file. Example file1 file2 file3 apples apples oranges oranges oranges bananas bananas kumquats peaches8/23/2012 7:52:53 AM 5864_ER_FED_ALT 24

Basic Shell commands (Cont)Tr Command Translate input character to output character based on the input and output patterns Example # tr [A-Z] [a-z] out # translate all letters to lower case. # tr -s \012\011\040 \012\012\012 < in > out # translate blank, tab and new line chars to new line chars and squeeze (-s) consecutive newline char into one # tr -cs [a-z][A-Z] [\012*] < in > out # change all non-alphabetic (-c) chars to new line chars and squeeze consecutive new line char into one. # tr -d \040 < in > out # delete all blanks.

8/23/2012 7:52:53 AM 5864_ER_FED_ALT 25

Unix Shell ProgrammingBREAK

8/23/2012 7:52:53 AM 5864_ER_FED_ALT 26

Basic Shell commands (Cont)Uniq Display a file, removing all successive repeated lines

Example:file1: apple banana banana apple banana # sort fruit | uniq -c apple 2 banana 3 # tr -cs [a-z][A-Z] [\012*] < fileA | sort | uniq # show a list of distinct words in fileA.8/23/2012 7:52:5