Download - Unix Shell Scripting
Unix Shell Scripting
Instructor
Mustafa Qasim
Certified Ethical Hacker (CEH)
Oracle Certified Expert Solaris Security Administrator
Oracle Certified Professional Solaris System Administrator
Oracle Certified Associate Solaris 10 Operating System
Oracle Certified Associate MySQL 5.5/5
Cisco Certified Network Associate (CCNA)
Catch Me
LinkedIn pk.linkedin.com/in/mustafaqasim
Google+ gplus.to/mustu
Twitter twitter.com/mustafaqasim
Introduction● Name● Job Description● Professional Experience● Expectations from Unix Shell Scripting course
Ideal Participants
Hardworking?
Ideal Participants
Hardworking?ŊÖ
Ideal Participants
LaaazzZZzzyyy....
Let's Start...
Unix● AT&T Employees at Bell Labs in 1969● Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy,
Michael Lesk and Joe Ossanna● Unix Family OS
– University of California, Berkeley's BSD– Solaris– HP-UX– AIX– Sequent and as well as Darwin
GNU● GNU Not Unix● Richard Stallman, 1983● GNU General Public License (GPL)● GNU Compiler Collection (GCC), the GNU C library (glibc), and
GNU Core Utilities (coreutils),[1] but also the GNU Debugger (GDB), GNU Binary Utilities (binutils),[18] and the bash shell.
● GNU Hurd Kernel● Free Software Foundation (FSF)
GNU/Linux● In April 1991, Linus Torvalds, a 21-year-old
student at the University of Helsinki, Finland started working on some simple ideas for an operating system
● Kernel vs OS
Linus Torvalds
To Do
Make yourself familiar with Unix, Unix Derivatives, GNU, Linux and software license GPL and Free & Open Source Software (FOSS).
http://en.wikipedia.org/wiki/Unix
http://www.gnu.org/philosophy/philosophy.html
http://en.wikipedia.org/wiki/GNU
http://www.gnu.org/gnu/why-gnu-linux.html
http://en.wikipedia.org/wiki/Linux
To Do
Make yourself familiar with Unix, Unix Derivatives, GNU, Linux and software license GPL and Free & Open Source Software (FOSS).
http://en.wikipedia.org/wiki/Unix
http://www.gnu.org/philosophy/philosophy.html
http://en.wikipedia.org/wiki/GNU
http://www.gnu.org/gnu/why-gnu-linux.html
http://en.wikipedia.org/wiki/Linux
So, Where comes the Shell?
Shell?
Shell?
Compiler vs Interpreted
Complete Program > Compiler > Executable
Program Fragment > Interpreter > Instant Output
Compiler vs Interpreted (Contd.)● Compiled Languages
– C/C++– Pascal– FORTRAN
● Interpreted Languages– Unix Shell– Perl– Tcl/Tk
Unix Shell● First shell for Version 6 Unix was written by Ken Thompson
(Bell Labs) in 1971● It was only Interactive Shell
Bourne Shell● Created by Stephen Bourne at AT&T Bell Labs for V7 UNIX● Goals
– Interpreted Language– Scripting
● New Features– Control Flows, Loops, Variables
● Lacked Functions
Bourne Shell (Contd.)● Bourne Shell Derivatives
– Korn Shell (ksh)– Almquist shell (ash)– Bourne Again Shell (Bash)
C Shell (csh)● Bill Joy in 1978 for BSD● Create a scripting language similar to C● Prominent Feature
– Command History● Tenex Enhancements (tcsh)
– Command completion– Command line editing
Korn Shell (ksh)● By David Korn● Backward compatibility with Bourne Shell (sh)● Derivative of Bourne Shell (sh)
Bourne Again Shell (Bash)● GNU Project● Superset of sh and ksh
POSIX● Institute of Electrical and Electronics Engineering (IEEE)● International Organization for Standardization (ISO)● POSIX Compliance● Bash, gawk
Responsibilities of Shell
1. Reading input and parsing the command line
2. Evaluating special characters, such as wildcards and the history character
3. Setting up pipes, redirection, and background processing
4. Handling signals
5. Setting up programs for execution
To Do● Perform Login and Logout on a Unix shell● Perform and understand the following commands
– ls, dir, touch, mkdir, cd, rm, mv, cat, less, more, which, pwd, man, info
● Use vi editor to create and edit files● Understand Unix File System hierarchy especially the
following directories– /bin, /sbin, /home, /root, /var
Unix Startup● OS is booted and very first process is born
– Init, PID 1● OS authenticates the user and provides access to shell
– /bin/login● Data Streams are set
– Stdout, stderr, stdin● A process is forked to run Graphical Desktop
Parsing Commands● Lexical Analysis
– The process of breaking the line up into tokens is called lexical analysis.
● Tokens● Command?
– Built-in command => Execute Internally– Executable command => Fork a Child Shell
● Forking Child Shel
Types of Commands
1. Aliases
2. Keywords
3. Functions
4. Built-in commands
5. Executable programs
Shell Processes● PID● ps● Linux: pstree● Solaris: ptree
System Calls
Kernel
fork
exec
wait
exit }
fork System Call● Parent/Child● Inheritance of environment, open files, real and user
identifications, umask, current working directory, and signals.
fork System Call
wait System Call● Waits for Child's PID & Exit Code● Zombie Process● Orphan Zombie Process
exec System Call● exec call for executable● Executable becomes the child
exit System Call● sigchild signal to parent● Exit code 0 - 255
To Do● Perform and Understand the following commands
– top, ps, pgrep, pstree/ptree, kill● Learn to hunt a specific process and kill it● Learn to hunt a specific instance of a binary running multiple
times simultaneous
Environment and Inheritance● UID, GID● Ownership● Permission● Variables● Working Directory
User & Group IDs
$ id
uid=502(mustu) gid=502(mustu)
File Creation Mask● Umask
777 (Directory) 666 (File)
–022 (umask value) –022 (umask value)
------- ---------
755 644
drwxr-xr-x -rw-r--r--
Set Ownership & Permission● chown● chmod
Variables● Local Variables● Environment Variables
– $env
File Descriptor● 0 stdin● 1 stdout● 2 stderr
I/O Redirection
I/O Redirection
I/O Redirection
Pipe
Signals
To Do● Understand & Practice the following
– Bash Environmental variables– Check your current environment variables (hint: $env)– Check your user's UID & GID– Check a file's permission set for owner, group and others– Change a file/folder permission using both numerical and character format– Change a file/folder owner and group– Redirect stdout and stderr into a file– Pipe or Redirect output of programs
Let's write a Bash script ;-)
Writing Bash Script● Shebang
– #!/bin/bash● Comments
– #Getting input for value of age● Make script executable
– Chmod +x myscript.sh● Execute Script
– ./myscript.sh
Simple Bash Script
#!/bin/bash
# This is a comment
echo “Khamoshi Ka Boycot”
Use Variable
#!/bin/bash
name=”Telenor”
echo “$name Khamoshi Ka Boycot”
Pass Arguments
./myscript ali 15 Lahore
#!/bin/bash
echo “Mr. $1 is $2 years old and belongs to $3”
Get Input
#!/bin/bash
echo "What is your name?"
read uname
echo ” Welcome $uname”
Arithmetic Operators
== Equal to
!= Not equal to
> Greater then
>= Greater then, equal to
< Less then
<= Less then, equalt o
Logical Operators
&& and
|| or
! not
Conditional Statements
if Construct:
if command
then
block of statements
fi
Conditional Statements
if/else Construct:
if command
then
block of statements
else
block of statements
fi
Conditional Statementsif/else if/else Construct:
if command
then
block of statements
elif command
then
block of statements
else
block of statements
fi
Iterative Statements
for variable in word_list
do
block of statements
done
Schedule Script● Cron
– crontan -l #List Exiting Crons– crontab -e #Edit Cron Scheduling File
REMEMBER● Environment Variable Differ in Cron● Use the MAIL parameter to receive cron errors● Use the PATH parameter to set environment variables in cron● If your program needs to use neighbor files to run properly
then in cron script first change directory (cd) to your program's path then run the program.
Tip :)
Your best buddy
http://www.corntab.com/
Regular Expressions● Pattern of characters used to match the same characters in a
search● Enclosed by forward slash /telenor/
Regular Expressions
Hi Asad,
We've a upcoming training on Unix shell Programming later this week. As you've to deal with programming tasks frequently I would highly recommend you to participate in this training. It will improve your programming skills and help you perform daily tasks efficiently.
Thanks
Regular Expressions
:1,$s/programming/scripting/g
Regular Expression Metacharacters
:1,$s/\<[Pp]rogramming\>/scripting/g
1,$ First Line to Last Line
s Substitute
\<, \> Beginning and End of Word
[Pp] Match one of the word
<[Pp]rogramming\> Programming OR programming
g Globally
Regular Expression MetacharactersMeta Character
^
Function
Beginning of line anchor
Example
/^lahore/
What It Matched
Matches all lines beginning with lahore
Regular Expression MetacharactersMeta Character
$
Function
End of line anchor
Example
/lahore$/
What It Matched
Matches all lines ending with lahore
Regular Expression MetacharactersMeta Character
.
Function
Matches one character
Example
/la...e/
What It Matched
Matches all starting with la, followed by three characters, followed by e
Regular Expression MetacharactersMeta Character
*
Function
Matches zero or more of the preceding characters
Example
/ *lahore/
What It Matched
Matches lines with zero or more spaces, followed by the pattern lahore
Regular Expression MetacharactersMeta Character
[]
Function
Matches one in the set
Example
/[Ll]ahore/
What It Matched
Matches lines containing Lahore or lahore
Regular Expression MetacharactersMeta Character
[x-y]
Function
Matches one character within a range in the set
Example
/[A-Z]ahore/
What It Matched
Matches letters from A through Z followed by ahore
Regular Expression MetacharactersMeta Character
[^]
Function
Matches one character not in the set
Example
/[^A-Z]ahore/
What It Matched
Matches any character not in the range between A and Z
Regular Expression MetacharactersMeta Character
\
Function
Used to escape a metacharacter
Example
/lahore\./
What It Matched
Matches lines containing lahore, followed by a literal period; Normally the period matches one of any character
Regular Expression MetacharactersMeta Character
\<
Function
Beginning-of-word anchor
Example
/\<lahore/
What It Matched
Matches lines containing a word that begins with lahore (supported by vi and grep)
Regular Expression Metacharacters Additional
Meta Character
\>
Function
Ending-of-word anchor
Example
/lahore\>/
What It Matched
Matches lines containing a word that ends with lahore (supported by vi and grep)
Regular Expression Metacharacters Additional
Meta Character
\(..\)
Function
Tags match characters to be used later
Example
/\(love\)able \1er/
What It Matched
May use up to nine tags, starting with the first tag at the leftmost part of the pattern. For example, the pattern love is saved as tag 1, to be referenced later as \1. In this example, the search pattern consists of lovable followed by lover (supported by sed, vi, and grep)
Regular Expression Metacharacters Additional
Meta Character
\(..\)
Function
Tags match characters to be used later
Example
/\(love\)able \1er/
What It Matched
May use up to nine tags, starting with the first tag at the leftmost part of the pattern. For example, the pattern love is saved as tag 1, to be referenced later as \1. In this example, the search pattern consists of lovable followed by lover (supported by sed, vi, and grep)
Regular Expression Metacharacters Additional
Meta Character
x\{m\} or x\{m,\} or x\{m,n\}
Function
Repetition of character x, m times, at least m times, at least m and not more than n times
Example
o\{5,10\}
What It Matched
Matches if line contains between 5 and 10 consecutive occurrences of the letter o (supported by vi and grep)
Sed● Streamlined, non-interactive editor
How Sed Works
abc
Text File
abc
Sed Buffer
abc
Sed Output
Awk● UNIX programming language used for manipulating data and
generating reports
How Awk Works
Ahmad 33 LHRShakir 35 ISBQasim 28 KHI
Text File
Ahmad 33 LHR
Awk takes a line and put in an internal variable $0
Ahmad 33 LHR
Line is broken into fields separated by spaces and stored into internal numbered variables starting from $1
$0
$1 $2 $3
How Awk Works
Ahmad 33 LHRShakir 35 ISBQasim 28 KHI
data.txt
#awk '{print $1, $3}' data.txt
Ahmad LHRShakir ISBQasim KHI
To Do● Practice Regular Expressions with grep, sed and awk.