Essential Unix and Linux!Perl for Bioinformatics, 140.636!

F. Pineda

Generic computer architecture

Fig. 1.2 From “Designing Embedded Hardware”, 2nd Ed. by John Catsoulis

Memory

Storage

OS concepts

The file system

Devicese.g. Keyboard,

printerscreen, ethernet,

mouse, etc.

Operating system (e.g. Linux, Unix, MacOS

Shellinterprets typed commands

GUIinterprets mouse !

and keyboard events

Processese.g. editors,

word processors, spreadsheets,scripts, etc.

ApplicationsExcel, R, Perl scripts

n  Unixn  Linuxn  OS X, MacOSn  Windowsn  CPM

The Operating System (OS) orchestrates and coordinates computing on the machine

Unix Phylogeny

Essential unix (operating system) concepts

1. A processEverything that “runs” under unix is a process, e.g. commands, spreadsheets, terminal windows, etc.Relevant commands: ps, top

2. A file & the hierarchical file systemdirectories(folders), data files, applications. On the disk, everything is a file. Even directories! Files are not processes!Relevant commands: ls, df

3. A shell A shell is a process. The shell accepts command lines and feed

them to the unix kernel. Command lines consist of commands or programs to execute along with information needed to execute them

ProcessesWhat processes am I running from my shells?!

ps

Which processes are using the most cpu?!top

What processes is everyone running from shells?!ps -a

What processes of all kinds are running?!ps -ax

File system concepts

/

bootbin dev etc home2 lib mnt opt proc root sbin tmp usr var vmlinux

140.636F08 faculty

stu08-01 stu08-02stu08-00 Stu08-03

bin local

fernando

private_htmlpublic_html

home

public

example.txt

The hierarchical file systemThe file system is a tree

/home2/140.636F08/stu08-00/private_html

Navigating the hierarchical file system

1.  Navigation commandsn  ls -- list directory contents n  pwd -- prints the absolute path to the working directory n  cd -- change directory to a directory specified by a pathn  df -- free disk spacen  du -- disk usage statistics

2.  Pathsn  Absolute path

n  A fully qualified path must be specified from the root directoryn  / is the symbol for the root directory

n  Relative pathn  Always defined relative to the working directoryn  . is the symbol that means the working directoryn  .. is the symbol that means the directory above the working directory

n  Executing an executable that resides in the current working directoryn  ./filename

Review: commands to get around the file system

n  pwd the command that tells you where you are, i.e. print working directoryn  ls the command that lists the contents !

of the current directoryn  cd the command that changes the !

current directory. In other words ! it is how you move around the ! file system

Memorize these three navigation commands or die!

Files permissionsn  to see what permissions a file has use ls

command with -l option:n  ls -l filename

n  to change file permissions use the chmod command (see linux quickstart notes)

read write execute

user

read write execute

group

read write execute

otherfiletype

d r w x r - x r - x

Basic Bash shell commands

simple commandsn  Commands

n  examples: clear, cd, ls, man, wcn  Syntax: command [options] parameters !

n  Optionsn  An option is a way of telling Linux to perform a command in a

particular way (like pressing alt-key while clicking a mouse)n  Typically a minus sign followed by one or more characters!

and sometimes an argument !

n  Parametersn  Often filenames or arguments for the command!

n  Wildcardsn  used to represent filenamesn  ? matches any single charactern  * matches any number of characters

some example commands

wc helloworld.plwc -l helloworld.plwc -l *.plls *.*

On-line help for commands

n  man commandn  command -helpn  apropos keywords

Standard input & output

two very important Unix concepts

that you absolutely must understand

Standard I/On  Commands are processes, so they have a default standard input and

a default standard output !!!!

n  For commands executed in shell window:n  standard input is connected to the keyboardn  standard output is connected to the screen

processstandard input standard output

shellprocess

STDIN STDOUT

Keyboard device graphics device

When you invoke the Perl interpreter it is a process!

perlprocess

STDIN STDOUT

Keyboard device graphics device

Redirection and pipes

two more very important and related ideas

that you absolutely must understand

Standard i/o streams can be Redirected

n  The standard output of a command can be redirected to a file by the “>” redirector

ls process

STDIN

Storage device

STDOUT

graphics device

ls -l

results.txt

STDOUT

file

ls -l > results.txt

Standard i/o streams can be Redirectedn  a file can be redirected to the standard input of a command by the

“<“ redirector.

wcprocess

STDOUT

graphics device

STDINresults.txt

file

wc -l < results.txt

Keyboard device

STDIN

wc -l

a pipeline of processes

process1

file1.txt

STDOUT process2STDIN

file2.txt

STDOUT STDIN process3

a “pipeline” of processes!(without the intermediate files)

process1 process2STDIN process3STDOUT STDIN STDOUT STDIN

PipesThe standard output stream of one command can be piped to the �

standard input of another command by using thepipe “|” symbol. No intermediate file is required.

Example: What does this do?

ls -l | wc -l

ls -l wc -lSTDINSTDOUT STDIN

Let’s use perl to illustrate pipes!1st recall that the Perl interpreter is a process

perlprocess

STDIN STDOUT

Keyboard device graphics device

In the shell start the perl interpreter and feed it a text file of perl statements(examine the file of perl statements)

perl helloworld.txt

In the shell start the perl interpreter and feed it another text file of perl statements (examine the file of perl statements)

perl helloperl.txt

What does this command line do?

perl hello.txt | perl helloperl.txt

perl process

perl processSTDINSTDOUT STDIN STDOUT

demo…

Execution of unix commands and scripts

How the shell executes commands and applications

1.  If the shell does not recognize the command name as one of it’s built-in commands, it starts a search for an executable file with the same name.

2.  An executable file is either an application binary or a special kind of text file called a script

3.  The shell searches through all the directories on the disk specified by the PATH shell variable (demo)

4.  And executes the first executable that it finds with the specified name.

5.  The file must have “execute permission in order to be executed, otherwise you will see a “permission error”

How the shell executes a “script”

1.  If the executable is a text file, the shell treats it as a “script”. A script must have a special first line

1.  The first line must start with #!, the rest of the first line is the path to an executable

2.  The executable is launched3.  The file itself is passed to the standard input (STDIN) of the

the executable. 2.  Examples

helloworld.plhelloperl.pl

Equivalent command lines

./helloworld.pl | ./helloperl.pl

perl hello.txt | perl helloperl.txt

Perl scripts are the standard way of invoking the perl interpreter. You almost never see perl invoked “by hand”, we did it just to

illustrate what is going on “under the hood.”