Shell ScriptingLab 2Advanced Operating SystemSpring 2013

1

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Today Objectives

• Review on Shell interpreter

• Know basic shell scripting (Tcl)

• Know cons and pros of shell scripting

2

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Shell (review)• Is a utility program with the Linux system that serves as an

interface between the user and the kernel • Plays an important role of command interpretation for the kernel

3

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Remember: How the Shell works?

Shell (review) … cont.• The shell as a Command Interpreter :

• Reads the command

• Locates the file in the directories containing utilities

• Loads the utility into memory

• Executes the utility

5

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

• The shell creates a child shell for the execution of a utility

• The shell requests the kernel for any hardware interaction

Shell (review) … cont.

6

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Unix Shell

• There are many shells , some of them are

• Bourne Shell (sh/bash): sh is the executable filename for

this shell

• C Shell (csh/tcsh ): csh is the executable filename for

this shell

• Korn Shell: The executable filename is ksh

• Restricted Shell: Is typically used for guest logins• To know the Type of your Shell write “echo $SHELL”

• The command to change your shell is ……… ?!

7

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Shell Scripts• Stores a sequence of frequently used Linux commands in a file

• Enables the shell to read the file and execute the commands in it

• Allows manipulation of variables, flow-of-control and iteration

constructs that make programming possible

8

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

• To run the script save it with any name (i.e. hello)• Give it execute permission (chmod …)• Then run it (i.e. ./hello)Hint : to know the path to your shell write on the shell which shellname (i.e. which tclsh)

Comment

Commands …..

Hello World Example

#!/bin/tclsh

# My first script

puts "Hello World!”

Directory for the shell that will interpret the script

9

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 1 ... (7 mins )• Open server.c and client.c , take 2 mins to understand what they do • Check the IPC resources used (ipcs ..)

• take notes how many message queues there

• Run server program in the background ( … &)

• Run the client program

• Enter the following three messages separately

(message1 , message2 , message3)

• Repeat last 2 steps

• What happened ? And why ?

• Remove message queues (ipcrm) 10

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 1 ... cont.• Imagine that you repeat this task every time you run your

program or your program crashes …. Boring and Timing Consuming isn’t it !!

Don't you think a quick script could make our life easier :)

11

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

So Why Script Programming?• A working knowledge of shell scripting is essential to

understand the behavior of your system . Consider that as a Linux machine boots up, it executes

the shell scripts in /etc/rc.d to restore the system configuration and set up services.

• Writing shell scripts is easy to learn, simple to make , easier to

debug and faster (no compilation, write and run ).

• A shell script is a "quick and dirty" method of prototyping a complex application.

• Whenever you find yourself doing the same task over and over again you should use shell scripting

12

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Scripting Language• Scripting language like any other language, it has

Conditional statements (if …) Loops (for , while , until , …) Functions Variables Can do arithmetic operation Very easy string handling ...etc

• As mentioned before it is much easier to learn , to use to debug but Take Care all comes at a cost

• Scripts are interpreted while normal language are pre-compiled which make them much more faster

• Also Scripts in general doesn't have such structures • A good reference for bash scripting is http://tldp.org/HOWTO/Bash-

Prog-Intro-HOWTO.html 13

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Tcl (Tool Command Interpreter)

• Extensible You can easily add new commands and libraries to a Tcl interpreter.

• Embeddable You can use Tcl as a scripting language within another larger application.

• OrthogonalThe syntax is simple and consistent - Tcl code is easy to write and also easy to read.

• Modern All the modern programming constructs are supported, including simple objects and namespaces.

• Multiplatform The Tcl interpreter has been ported to more platforms that Java. The same scripts can be run on everything from IBM Big-Iron to real-time embedded kernels.

• Powerful The Tcl architects have provided powerful and well-thought generalizations that make it fast and easy to write what would otherwise be complex applications.

14

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

15

Basic operations• print to screen (puts)puts –nonewline "Hello, world!"

puts "!!"

• assignment (set)set income 32000

puts "income is $income"

(using '$' to get the value of a variable)

• mathematical expressions (expr)set a 10.0

expr $a + 5

expr int($a/3)

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

16

Some useful commands• unset: destroy a variable unset num

• info: check whether the named variable has been defined if {![info exists num]} {

set num 0

}

incr num

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

17

Special characters# : single-line comments, similar to "//" in C;# : in-line comments, just like "//" in C\ : escape character, same function as in C : also used to break a long line of code to two lines$ : get the value of a variable

• var : name of variable• $var : value of variable

[] : evaluate command inside brackets

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

18

Control structures (1)• if then else set income 32000 if {$income > 30000} {

puts "$income -- high"

} elseif {$income > 20000} {

puts "$income -- middle"

} else {

puts "$income -- low"

}

• while loops set i 0 while {$i < 100} {

puts "I am at count $i"

incr i

}

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

19

Control structures (2)• for loops for {set i 0} {$i < 100} {incr i} { puts "I am at count $i and going up"

}

for {set i 100} {$i > 0} {set i [expr $i - 1]} {

puts "I am at count $i and going down"

}

• foreach loops set lstColors {red orange yellow green blue purple} foreach c $lstColors {

puts $c

}

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

20

Control structures (3)• foreach loops (con't) set lstColors {red orange yellow green blue purple} foreach {a b c} $lstColors {

puts "$c--$b--$a"

}

set lstFoods {apple orange banana lime berry grape}

foreach f $lstFoods c $lstColors {

puts "a $f is usually $c"

}

foreach {a b} $lstFoods c $lstColors {

puts "$a & $b are foods. $c is a color."

}

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

21

Procedures(1)• procedure calls (embedded commands)set b [expr $a + 5]

puts "The value of b is $b"

• create your own procedure (called by value only)proc foo {a b c} {

return [expr $a * $b - $c]

}

puts [expr [foo 2 3 4] + 5]

proc bar { } {

puts "I'm in the bar procedure"

}

bar

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

22

Procedures (2)• procedure calls (variable number of arguments)proc sum {args} {

set s 0

foreach arg $args {incr s $arg

}

return $s

}

puts [sum 1 2 3 4]

puts [sum 1 2]

puts [sum 4]

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

23

Procedures (3)• procedure calls (implicit variable)proc power {a {b 2}} {

if {$b == 2} {return [expr $a * $a]

}

set value 1

for {set i 0} {$i<$b} {incr i} {set value [expr $value * $a]

}

return $value

}

set v1 [power 5]

set v2 [power 5 4]

puts "5^2 is $v1"

puts "5^4 is $v2"

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

24

Variable scope

local and global variablesset a 5

set b 6

set c 7

proc var_scope { } {

global a

set a 3

set b 2

set ::c 1

}

var_scope

puts "The value for a b c is: $a $b $c"

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

25

Variable scope

local and global variablesset a 5

set b 6

set c 7

proc var_scope { } {

upvar a x

set x 3

set b 2

set ::c 1

}

var_scope

puts "The value for a b c is: $a $b $c"

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 2 ... (30 min) 1. Write the function change, which takes a whole number and set of coins (integer numbers too) as its arguments and prints a list that represents how to "make change" for that number using the set of coins (or notes) for any decimal currency. first number is the whole number then followed by the set of coins . your function should have the minimum number of coins possible . (10 mins)

proc change {n args} {...}

Examples :

change 75 25 10 5 1=> 3 0 0 0change 75 10 25 1 5=> 3 0 0 0change 75 50 25 10 5 1=> 1 1 0 0 0

26

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 2 ...

2. Write the command vardefault, which takes two arguments, the name of a variable, and a default value. If the variable is defined in the calling scope, vardefault returns the variable's value. If it is not defined, vardefault returns the default value. (10 min)

proc vardefault {var default} {...}

Examples :

set x 45vardefault x 0=> 45proc foo {} {Vardefault x 0}foo=> 0

To write this function, you'll need to use upvar to achieve call by name, and you'll need a way to test whether or not a variable is defined (use info exists).

27

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 2 ...

3. Write the iota function, which takes a numeric argument n and returns a list of numbers of length n which are the numbers from 0 to n-1. the iota function can take an optional second argument which is an increment to separate the integer values. The default increment is 1. iota always returns a list whose length is equal to its first argument. (10 min)

proc iota {n {inc 1}} {...}

Examples :

iota 10=> 0 1 2 3 4 5 6 7 8 9iota 10 30 3 6 9 12 15 18 21 24 27

28

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 2 …. (optional)4. Write the function bits which takes two parameters, both numbers, and returns a binary

representation of the first number as a list of bits (each bit is a 0 or a 1), from most significant at the left to least siginifcant at the right. The result list should contain as many bits as specified by the second number, which should be an optional parameter, using leading zero bits if necessary (assume that the bit length requested is long enough to represent the number). If the second parameter isn't specified, use the minimal number of bits (no leading zeros).

You'll probably want to use the expr command's bit-fiddling operators for this.

proc bits {howmany num} {...}

Examples bits 0777 1 1 1 1 1 1 1 1 1

bits 0644 1 1 0 1 0 0 1 0 0

bits 19 1 0 0 1 1

bits 1 3 0 0 1

29

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

30

Lists in Tcl• Everything is a list!• Many ways to create a list set myList [list a b c] set myList "a b c" set myList {a b c} set myList [list $a $b $c] set myList {$a $b $c} set myList [list a b c] set myList "a b c"

set s Hello puts "The length of $s is [string length $s]." => The length of Hello is 5. puts {The length of $s is [string length $s].} => The length of $s is [string length $s].

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

31

List operationsset lstStudents [list "Fan" "Kristy" "Susan"]

puts [lindex $lstStudents 0]

puts [lindex $lstStudents end]

puts [llength lstStudents] (unexpected result!)puts [llength $lstStudents]

lappend $lstStudents "Peter" (wrong!)lappend lstStudents "Peter"

puts [linsert lstStudents 2 "Tom"] (wrong!)puts [linsert $lstStudents 2 "Tom"]

set lstStudents [linsert $lstStudents 2 "Tom"]

set lstStudents [lreplace $lstStudents 3 3 "Rachel"]

set lstStudents [lreplace $lstStudents end end]

set lstStudents [lsort –ascii $lstStudents]

puts [lsearch $lstStudents "Peter"]

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

32

Lists of lists (of lists…)set a [list [list x y z]]

puts [lindex $a 0]

puts [lindex [lindex $a 0] 1]

puts [lindex [lindex $a 1] 0] (unexpected result)set a [list x [list [list y] [list z]]]

=> How to get to the z?

set arg1 [list g [list f [list h [list i X]]] [list r Y] k]

set arg2 [list g [list f [list h [list i Y]]] [list r b] L]

set both [list $arg1 $arg2]

puts $both Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

33

Array operationsAssociative arrays (string as index)set color(rose) redset color(sky) blueset color(medal) goldset color(leaves) greenset color(blackboard) blackputs [array exists color]

(tests if an array with the name "color" exists)puts [array exists colour] puts [array names color] (returns a list of the index strings)foreach item [array names color] { puts "$item is $color($item)"} (iterating through array)set lstColor [array get color] (convert array to list)array set color $lstColor (convert list to array)

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

34

Regular expressions• regsubset stmt "Fan is one of Shania’s fans"regsub –nocase "fan" $stmt "Kristy" newStmt ?switches? exp string subSpec ?varName?puts "$newStmt"regsub –nocase –all "fan" $stmt "Kristy" newStmtputs "$newStmt"

• regexp(returns 1 if the regular expression matches the string, else returns 0)

puts [regexp –nocase "fan" $stmt] ?switches? regexp string

• format puts [format "%s is a %d-year-old" Fan 26] formatString ?arg arg ...?

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

35

String operationsset statement " Fan is a student "

set statement [string trim $statement]

puts [string length $statement]

puts [string length statement]

puts [string index $statement 4]

puts [string index $statement end]

puts [string first "is" $statement]

(string last)

puts [string first $statement "is"]

puts [string range $statement 4 end]

puts [string replace $statement 9 end "professor"]

puts [string match "*student" $statement] (* ? [])

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

36

File operationsset fRead [open source.txt r]

set fWrite [open target.txt w]

while {![eof $fRead]} {

set strLine [gets $fRead] ;#or gets $fRead strLine

regsub –nocase –all "fan" $strLine "kristy" strLine

puts $fWrite $strLine

}

close $fRead

close $fWrite

################ source.txt ################Fan is a CSE student.Fan is also one of Shania’s fans.Kristy and Fan are classmates.

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

37

Miscellaneous commands• eval: execute a command dynamically built up in your

program set Script { set Number1 17

set Number2 25

set Result [expr $Number1 + $Number2]

}

eval $Script

• exec: execute external programs

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

38

Common pitfalls• Missing $ or extraneous $• Using {a} vs "a" vs [list a]• Creating list items that are empty lists

a b {} d

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

When not to use shell scripts• need data structures, such as linked lists or trees• need to generate or manipulate graphics or GUIs• need direct access to system hardware.• procedures involving heavy-duty math operations.• resource-intensive tasks, especially where speed is a factor

(sorting, hashing, etc.)• complex applications, where structured programming is a

necessity (need typechecking of variables, function prototypes, etc.)

• situations where security is important

39

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 3 …. (60 min)1. Write the function strlenlist, which takes one argument, a list of strings, and returns a list of

equal length as a result, in which each element is the string length of the corresponding element of the argument list.

(10 mins )

proc strlenlist {L} {...}

Examples :

strlenlist {34 987 1 567 -23 8} => 2 3 1 3 3 1 strlenlist {foo bar antidisestablishmentarianism} => 3 3 28

40

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 3 …. (60 min)2. Write the function choose, which takes three parameters: three equal-length lists, the first

two of arbitrary values, and the third a list of boolean values (zeros and ones). choose returns a list (also of the same length) where each element is chosen from one of the first two lists, depending on the corresponding boolean value. Elements of the first parameter are chosen for 1's (true), while elements of the second parameter are chosen for 0's (false). choose is one of many useful functions that operate on bit vectors.

(10 mins )

proc choose {trues falses bools} {...}

Examples : choose [list a b c d] [list A B C D] [list 0 1 1 0] => A b c D

41

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 3 …. (60 min)3. Write a function to find all lines containing a certain word in a file . It takes filename and

the word to search for as an argument , and returns line number : the line this word occurs at

proc findword { filename word} { …}

Examples :################ source.txt ################Fan is a CSE student.Fan is also one of Shania’s fans.Kristy and Fan are classmates.

findword source.txt is 1 : Fan is a CSE student 2 : Fan is also on of Shania’s fans.

(Hint : it could be done with one line of code ) 42

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Exercise 3 …. (60 min)A sequence of strings is generated as follows: the first two strings are given and each successive string is made by

joining together the previous two strings (in their original order). For example, if the first string is abc and the second string is cde, the third string will be abccde and the fourth will be cdeabccde. The task for this question is to determine the ith character in the sequence of strings (not the ith string). In the previous example the 2nd character is b, the 5th is d, as is the 11th. The 13th character, which happens to be the 1st character of the 4th term, is c. (bonus)Input

The first line will contain the first string, of between 1 and 10 characters, using lower case letters. The second line will contain the second string, again between 1 and 10 lower case letters. The third line will be a single integer i (1 <= i <= 2^30). Please note that there can be any number of sets of input, each consisting of the specified three lines of input.Sample input

fibonaccirevenge1000000

Output

Output the ith character in each input sequence, separated by lines.Sample output

e

43

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

44

Tcl references and resources• Help file

http://www.tcl.tk/scripting/http://www.msen.com/~clif/TclTutor.html

• Search site:http://xyresix.com/nwsbook/search.html

• List of Tcl commands with man pageshttp://tcl.activestate.com/man/tcl8.4/TclCmd/contents.htm

• Tcl examples:http://www.beedub.com/book/2nd/tclintro.doc.html#2668

• All code in this tutorial (plus some) and expected outputftp://cslu.ece.ogi.edu/pub/kristina/tutorial.tcl, tutorial_output.txt, source.txt A

dva

nce

d O

pe

ratin

g S

yste

m -

S

prin

g 2

01

3 -

La

b 2

45

Tcl editors• Any editor• nedit• emacs• TextPad

• http://www.textpad.com/• http://www.textpad.com/add-ons/synn2t.html - TCL/TK (5)

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2

Other References • Linux Shell Scripting Tutorial - A Beginner's handbook - Vivek G.

Gite• Advanced Bash-Scripting Guide: A complete guide to shell

scripting, using Bash by Mendel Cooper• CSCI6303 – Principles of I.T.• http://tmml.sourceforge.net/doc/tcl/• http://en.wikipedia.org/wiki/Tcl

46

Ad

van

ced

Op

era

ting

Sys

tem

-

Sp

ring

20

13

- L

ab

2