unix shell scripting4n2 - sigmanet guntis/unix/unix_shell_  · shell scripting...

Download unix shell scripting4n2 - SigmaNet guntis/unix/unix_shell_  · Shell Scripting OperētājsistēmaUNIX (DatZ6007) Pasniedzēji: prof. GuntisBārzdiņš, doc. Normunds Grūzītis Līdzautori:

Post on 01-Feb-2018




0 download

Embed Size (px)


  • Shell Scripting

    Opertjsistma UNIX (DatZ6007)Pasniedzji: prof. Guntis Brzdi, doc. Normunds Grztis

    Ldzautori: irts Folkmanis, Juris Krmi, Kristaps Donsons, Leo Trukns, Artrs Lavrenovs

    Latvijas Universitte

  • Outline

    w Shell featuresw Helper utilities: introw Connecting utilities with shell scriptingw Helper utilities: detailsw Piping, advanced examplesw Shell scripts as files, programming constructs

  • Bash

    w We will talk about bash; there might be differences for other shellsn bash GNU Bourne-Again Shelln Authors: Brian Fox and Chet Ramey (1989)

    l Free Software Foundation; a replacement for the Bourne shell

    w Popular (default) in different distributionsw To find your current (login) shell, type the command

    echo $SHELL/bin/bash

  • Shell Featuresw The shell command language (sh) is defined in SUS / POSIX

    n The language interpreted by the shelln The standard utility programs (150+)

    w bash is rather a dialect of the POSIX shell languagen Has acquired many extensions that may change the behavior of valid

    POSIX shell scriptsn Supports a --posix switch which makes it POSIX-compliantn Conforms to the POSIX standard if invoked as sh

    l /bin/sh

    w Few systems are fully SUS/POSIX compliantn e.g. Debian GNU/Linux, OpenBSD, NetBSD are not w.r.t. the utility section

    l Many commands are missing on all systemsl Others require special installation

  • Shell features

    w Two types of usage:n Command line interactiven Shell script, usually non-interactive

    w Shell scripts:Series of commands written in plain text fileLike batch file is MS-DOS but much more powerful

  • Shell features

    w Two types of commands:n Internal commands built in the shell interpreter

    l e.g. type -a cd

    n External commands calling other executable filesl e.g. type -a ls

    w Almost everything applies to both command line usage and shell scripts

  • External commands

  • External commands

    w The environment variable $PATH determines where to search for external programsn $ echo $PATH/bin:/usr/bin:/usr/local/bin:/opt/bin

    n : as a separatorn export PATH=$HOME/bin:$PATH

    w The current directory . is usually not included in $PATHfor security reasons

  • Shell configuration files

  • Shell configuration files

  • Aliasingw Assigning a command to a shorter alias

    n Type a shorter command instead of the longer onew Aliasing is useful for options that you want all of the time

    n alias rm rm i

    w Aliasing is similar to shell function definitionsn dos2unix() { cat $1 | perl -pe 's/\r\n$/\n/g'; }n unix2dos() { cat $1 | perl -pe 's/\n$/\r\n/g'; }

    w Both can be specified in a shell profile file

  • Internal commands

    w A list of built in commands, that are handled internally without running an external commandn Does not require forking off a separate processn Needs direct access to the shell internals

    w Most commonly used internal command is cd, used to change the current working directory

  • Internal commands$ helpGNU bash, version 2.05b.0(1)-release (i686-pc-linux-gnu)

    These shell commands are defined internally.

    Type `help name' to find out more about the function `name'.Use `info bash' to find out more about the shell in general.

    A star (*) next to a name means that the command is disabled.

    %[DIGITS | WORD] [&] (( expression ))

    . filename :

    [ arg... ] [[ expression ]]

    alias [-p] [name[=value] ... ] bg [job_spec]bind [-lpvsPVS] [-m keymap] [-f fi break [n]builtin [shell-builtin [arg ...]] case WORD in [PATTERN]cd [-L|-P] [dir] command [-pVv] ......

  • SUS: shell grammar%token WORD%token ASSIGNMENT_WORD%token NAME%token NEWLINE%token IO_NUMBER

    %token AND_IF OR_IF DSEMI/* '&&' '||' ';;' */


    /* The following are the reserved words. */

    %token If Then Else Elif Fi Do Done/* 'if' 'then' 'else' 'elif' 'fi' 'do' 'done' */

    %token Case Esac While Until For/* 'case' 'esac' 'while' 'until' 'for' */


  • SUS: shell grammarcomplete_command : list separator

    | list;

    list : list separator_op and_or| and_or;

    and_or : pipeline| and_or AND_IF linebreak pipeline| and_or OR_IF linebreak pipeline;

    pipeline : pipe_sequence| Bang pipe_sequence;

    pipe_sequence : command| pipe_sequence '|' linebreak command;

    command : simple_command| compound_command| compound_command redirect_list| function_definition;

    compound_command : brace_group| subshell| for_clause| case_clause| if_clause| while_clause| until_clause

  • Helper utilities

    w Various small external programsw Helpful when working with shell scripts or the

    command linew Called from shell (scripts or command line)w Somehow transforms input into output, based on

    the parameters

  • Helper utilities

    cat: concatenates files and prints on stdout Syntax: cat [file1] [file2] [fileN]

    $ cat /etc/*release DISTRIB_ID=UbuntuDISTRIB_RELEASE=14.04DISTRIB_CODENAME=trustyDISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"NAME="Ubuntu"VERSION="14.04.5 LTS, Trusty Tahr"ID=ubuntuID_LIKE=debianPRETTY_NAME="Ubuntu 14.04.5 LTS"VERSION_ID="14.04"

  • Helper utilities

    echo displays a line of text Besides the program /bin/echo, also usually

    built in the shell (takes precedence) Syntax: echo [STRING]$ echo quick brown foxquick brown fox

    Can be used to display environment variables$ echo $HOME/home/girtsf

  • Helper utilities

    wc prints the number of newlines, words or bytes in files wc [options] [file1] [file2] [fileN]

    By default, newlines, words and byte counts are displayed

    Options -c : print only byte count -w : print only word count -l : print only line count

  • Helper utilities

    Example use of wc:$ wc /etc/passwd50 76 2257 /etc/passwd

    $ wc -l /etc/passwd50 /etc/passwd

    lines words bytes

    lines only

  • Helper utilities

    grep prints lines matching a pattern grep PATTERN [file1] [file2] [fileN]

    The lines that contain PATTERN are printed to the standard output

    If no files are specified, input is taken from thestandard input

    Advanced versions of grep allow using regular expressions in PATTERN

  • Helper utilities

    File testfile contains the following lines:$ cat testfilethe quick brown

    fox jumped overthe lazy dog

    We search for the:$ grep the testfilethe quick brown

    the lazy dog

  • Helper utilities

    Some useful parameters for grep: -i : ignore case (the finds the, The, THE,) -l : output filenames with matches, not the contents -B : output also n lines before the matching line -A : output also n lines after the matching line

    See the man page (man grep) for all parameters

  • Helper utilities

    w tee reads from stdin and writes to stdout and filesn Syntax: tee [File1] [File2] .. [FileN]

    w Example of tee taking users input from terminal and writing to 3 files:$ tee a b csome string^Dsome string$ cat asome string$ cat bsome string$ cat csome string

  • Helper utilities

    w Any program can be used as a helper programw More examples later

  • Connecting utilities with shell scripting

    w Standard I/O streamsw I/O redirection to/from filew I/O redirection using a pipew Command substitution

  • Standard I/O

    w Every process, when run, has 3 already open data streams (file descriptors, fd):n Standard inputn Standard outputn Standard error

  • Standard I/O

    w When run interactively (from command line), these streams are attached to the terminal:n Standard input attached to users keyboard inputn Standard output attached to users terminal outputn Standard error attached to users terminal output

    w Usually referred to as stdin, stdout, stderr

  • Standard output & error

    w ls command does not use stdin, but stdout and stderrn Sample stdout from ls :$ ls -l /etc/passwd-rw-r--r-- 1 root root 2257 Oct 22 13:35 /etc/passwd

    n Sample stderr from ls :$ ls -l /etc/asdfasdfls: /etc/asdfasdf: No such file or directory

    n Both stdout and stderr simultaneously:$ ls -l /etc/passwd /etc/asdfasdfls: /etc/asdfasdf: No such file or directory

    -rw-r--r-- 1 root root 2257 Oct 22 13:35 /etc/passwd

  • I/O Redirection to/from file

    w By default, the 3 streams are attached to the terminalw This can be overridden when executing a command

    n > specifies that stdout is redirected to a filen specifies that stderr is redirected to a filen &> specifies that both stdout and stderr are redirected to

    the same file

  • I/O Redirection to/from file

    w Syntax:n [ > ] [ < ] [ 2> ]

    w For redirections which are specified, the respective stream will be attached to the specified filen None, one, two or all three streams can be specified

    w If output file exists:n > - replace the filen >> - append to the file

  • I/O Redirection to file

    w Sample stdout redirection to a file$ ls -l /lib/ > direktorijas_saraksts$ cat direktorijas_sarakststotal 4035

    -rwxr-xr-x 1 root root 7488 Oct 6 12:33 cpp

    drwxr-xr-x 13 root root 1024 Oct 25 15:57 dev-statedrwxr-xr-x 2 root root 1024 Jun 28 09:53 evms


View more >