unix shell scripting4n2 - sigmanet guntis/unix/unix_shell_  · shell scripting...

Shell Scripting

Opertjsistma UNIX (DatZ6007)Pasniedzji: prof. Guntis Brzdi, doc. Normunds Grztis

Ldzautori: irts Folkmanis, Juris Krmi, Kristaps Donsons, Leo Trukns, Artrs Lavrenovs

Latvijas Universitte

Outline

w Shell featuresw Helper utilities: introw Connecting utilities with shell scriptingw Helper utilities: detailsw Piping, advanced examplesw Shell scripts as files, programming constructs

Bash

w We will talk about bash; there might be differences for other shellsn bash GNU Bourne-Again Shelln Authors: Brian Fox and Chet Ramey (1989)

l Free Software Foundation; a replacement for the Bourne shell

w Popular (default) in different distributionsw To find your current (login) shell, type the command

echo $SHELL/bin/bash

Shell Featuresw The shell command language (sh) is defined in SUS / POSIX

n The language interpreted by the shelln The standard utility programs (150+)

w bash is rather a dialect of the POSIX shell languagen Has acquired many extensions that may change the behavior of valid

POSIX shell scriptsn Supports a --posix switch which makes it POSIX-compliantn Conforms to the POSIX standard if invoked as sh

l /bin/sh

w Few systems are fully SUS/POSIX compliantn e.g. Debian GNU/Linux, OpenBSD, NetBSD are not w.r.t. the utility section

l Many commands are missing on all systemsl Others require special installation

Shell features

w Two types of usage:n Command line interactiven Shell script, usually non-interactive

w Shell scripts:Series of commands written in plain text fileLike batch file is MS-DOS but much more powerful

Shell features

w Two types of commands:n Internal commands built in the shell interpreter

l e.g. type -a cd

n External commands calling other executable filesl e.g. type -a ls

w Almost everything applies to both command line usage and shell scripts

External commands

External commands

w The environment variable $PATH determines where to search for external programsn $ echo $PATH/bin:/usr/bin:/usr/local/bin:/opt/bin

n : as a separatorn export PATH=$HOME/bin:$PATH

w The current directory . is usually not included in $PATHfor security reasons

Shell configuration files

Shell configuration files

Aliasingw Assigning a command to a shorter alias

n Type a shorter command instead of the longer onew Aliasing is useful for options that you want all of the time

n alias rm rm i

w Aliasing is similar to shell function definitionsn dos2unix() { cat $1 | perl -pe 's/\r\n$/\n/g'; }n unix2dos() { cat $1 | perl -pe 's/\n$/\r\n/g'; }

w Both can be specified in a shell profile file

Internal commands

w A list of built in commands, that are handled internally without running an external commandn Does not require forking off a separate processn Needs direct access to the shell internals

w Most commonly used internal command is cd, used to change the current working directory

Internal commands$ helpGNU bash, version 2.05b.0(1)-release (i686-pc-linux-gnu)

These shell commands are defined internally.

Type `help name' to find out more about the function `name'.Use `info bash' to find out more about the shell in general.

A star (*) next to a name means that the command is disabled.

%[DIGITS | WORD] [&] (( expression ))

. filename :

[ arg... ] [[ expression ]]

alias [-p] [name[=value] ... ] bg [job_spec]bind [-lpvsPVS] [-m keymap] [-f fi break [n]builtin [shell-builtin [arg ...]] case WORD in [PATTERN]cd [-L|-P] [dir] command [-pVv] ......

SUS: shell grammar%token WORD%token ASSIGNMENT_WORD%token NAME%token NEWLINE%token IO_NUMBER

%token AND_IF OR_IF DSEMI/* '&&' '||' ';;' */

%token DLESS DGREAT LESSAND GREATAND LESSGREAT DLESSDASH/* '' '&' '' '|' */

/* The following are the reserved words. */

%token If Then Else Elif Fi Do Done/* 'if' 'then' 'else' 'elif' 'fi' 'do' 'done' */

%token Case Esac While Until For/* 'case' 'esac' 'while' 'until' 'for' */

...

SUS: shell grammarcomplete_command : list separator

| list;

list : list separator_op and_or| and_or;

and_or : pipeline| and_or AND_IF linebreak pipeline| and_or OR_IF linebreak pipeline;

pipeline : pipe_sequence| Bang pipe_sequence;

pipe_sequence : command| pipe_sequence '|' linebreak command;

command : simple_command| compound_command| compound_command redirect_list| function_definition;

compound_command : brace_group| subshell| for_clause| case_clause| if_clause| while_clause| until_clause

Helper utilities

w Various small external programsw Helpful when working with shell scripts or the

command linew Called from shell (scripts or command line)w Somehow transforms input into output, based on

the parameters

Helper utilities

cat: concatenates files and prints on stdout Syntax: cat [file1] [file2] [fileN]

$ cat /etc/*release DISTRIB_ID=UbuntuDISTRIB_RELEASE=14.04DISTRIB_CODENAME=trustyDISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"NAME="Ubuntu"VERSION="14.04.5 LTS, Trusty Tahr"ID=ubuntuID_LIKE=debianPRETTY_NAME="Ubuntu 14.04.5 LTS"VERSION_ID="14.04"

Helper utilities

echo displays a line of text Besides the program /bin/echo, also usually

built in the shell (takes precedence) Syntax: echo [STRING]$ echo quick brown foxquick brown fox

Can be used to display environment variables$ echo $HOME/home/girtsf

Helper utilities

wc prints the number of newlines, words or bytes in files wc [options] [file1] [file2] [fileN]

By default, newlines, words and byte counts are displayed

Options -c : print only byte count -w : print only word count -l : print only line count

Helper utilities

Example use of wc:$ wc /etc/passwd50 76 2257 /etc/passwd

$ wc -l /etc/passwd50 /etc/passwd

lines words bytes

lines only

Helper utilities

grep prints lines matching a pattern grep PATTERN [file1] [file2] [fileN]

The lines that contain PATTERN are printed to the standard output

If no files are specified, input is taken from thestandard input

Advanced versions of grep allow using regular expressions in PATTERN

Helper utilities

File testfile contains the following lines:$ cat testfilethe quick brown

fox jumped overthe lazy dog

We search for the:$ grep the testfilethe quick brown

the lazy dog

Helper utilities

Some useful parameters for grep: -i : ignore case (the finds the, The, THE,) -l : output filenames with matches, not the contents -B : output also n lines before the matching line -A : output also n lines after the matching line

See the man page (man grep) for all parameters

Helper utilities

w tee reads from stdin and writes to stdout and filesn Syntax: tee [File1] [File2] .. [FileN]

w Example of tee taking users input from terminal and writing to 3 files:$ tee a b csome string^Dsome string$ cat asome string$ cat bsome string$ cat csome string

Helper utilities

w Any program can be used as a helper programw More examples later

Connecting utilities with shell scripting

w Standard I/O streamsw I/O redirection to/from filew I/O redirection using a pipew Command substitution

Standard I/O

w Every process, when run, has 3 already open data streams (file descriptors, fd):n Standard inputn Standard outputn Standard error

Standard I/O

w When run interactively (from command line), these streams are attached to the terminal:n Standard input attached to users keyboard inputn Standard output attached to users terminal outputn Standard error attached to users terminal output

w Usually referred to as stdin, stdout, stderr

Standard output & error

w ls command does not use stdin, but stdout and stderrn Sample stdout from ls :$ ls -l /etc/passwd-rw-r--r-- 1 root root 2257 Oct 22 13:35 /etc/passwd

n Sample stderr from ls :$ ls -l /etc/asdfasdfls: /etc/asdfasdf: No such file or directory

n Both stdout and stderr simultaneously:$ ls -l /etc/passwd /etc/asdfasdfls: /etc/asdfasdf: No such file or directory

-rw-r--r-- 1 root root 2257 Oct 22 13:35 /etc/passwd

I/O Redirection to/from file

w By default, the 3 streams are attached to the terminalw This can be overridden when executing a command

n > specifies that stdout is redirected to a filen specifies that stderr is redirected to a filen &> specifies that both stdout and stderr are redirected to

the same file

I/O Redirection to/from file

w Syntax:n [ > ] [ < ] [ 2> ]

w For redirections which are specified, the respective stream will be attached to the specified filen None, one, two or all three streams can be specified

w If output file exists:n > - replace the filen >> - append to the file

I/O Redirection to file

w Sample stdout redirection to a file$ ls -l /lib/ > direktorijas_saraksts$ cat direktorijas_sarakststotal 4035

-rwxr-xr-x 1 root root 7488 Oct 6 12:33 cpp

drwxr-xr-x 13 root root 1024 Oct 25 15:57 dev-statedrwxr-xr-x 2 root root 1024 Jun 28 09:53 evms