A Gentler Introduction to Unix

Zhizhang Shen ∗

Dept. of Computer Science and Technology

Plymouth State University

April 21, 2020

Abstract

Since this course is to run C/C++ in a Unix like environment, we start with abrief introduction to Unix, based on [1], to learn some of the basic Unix commands,

the directory structure of the Unix system, and how to compile and run a C programconsisting of either one file, or multiple files.

In the rest of this course, we will then proceed to learn now to write programs in Cto solve various problems, and then go through the same process as mentioned aboveto compile, and run, such programs.

Later on, we will also look at a variety of Unix system calls, and shell scriptsconsisting of such calls, and see a comprehensive example as how C is used to write

programs that make use of system information as kept by Unix.

Contents

1 An introduction 3

1.1 How to log into the system? . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Simple Unix commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Unix file system 6

2.1 What is in a name? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Work with files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Work with directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4 Background processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.5 Miscellaneous commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

∗Address correspondence to Dr. Zhizhang Shen, Dept. of Computer Science and Technology, PlymouthState University, Plymouth, NH 03264, USA. Email address: zshen@plymouth.edu.

1

3 A Unix shell 19

3.1 How does a shell work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.2 Options and arguments of commands . . . . . . . . . . . . . . . . . . . . . . 203.3 Redirection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.4 Group commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.5 Pipes and tees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.6 Quoting special characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 The editors 23

5 C programming in Unix 23

5.1 First thing first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.2 Are you bored with it? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.3 Speak mathematically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.4 Cut it into pieces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

6 A debugger 28

6.1 Set break points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.2 How to proceed from a break point? . . . . . . . . . . . . . . . . . . . . . . . 316.3 Check out the values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.4 A final example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7 shell scripts 36

7.1 Positional variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387.2 User-defined variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8 Unix system interface 42

8.1 Look at one call in details . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428.1.1 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428.1.2 Another example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448.1.3 Scripts again? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

8.2 An example: Listing directories . . . . . . . . . . . . . . . . . . . . . . . . . 478.2.1 Given a standard interface... . . . . . . . . . . . . . . . . . . . . . . . 488.2.2 ... and an implementation of the interface . . . . . . . . . . . . . . . 518.2.3 A little summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2

1 An introduction

Unix is a popular, and powerful operating system 1, which supports multitasking and time-sharing (Think about how Turing is operating, where multiple users can use it “at the sametime”.). It consists of a kernel, a file system, a shell, and various utilities. As the core ofthe system, the kernel controls everything inside the computer, and manages all kinds ofresources, in particular, it implements various queues to support the multitasking ability,which we will further explore in CS 4310 Operating Systems.

With the support of the file system, Unix organizes all kinds of data into a bunch of files,categorized into directories (Referred to as folders in Windows platform).

The Unix shell interprets user commands and passes their requests to the kernel so thatthey can be further carried out.

A utility is just a useful piece of software that makes the system more user friendly. Forexample, we can print out a calendar, list all the files out in various forms, find a word in afile, etc..

There are various versions of Unix, such as System V Unix, BSD Unix, and Unix-likesystems, such as Linux. Turing is a Linux based system. As mentioned earlier, even AppleiOS is built on top of Unix.

Since Unix is a multiuser system, we need a user account/password to get into the system,which is just the usual myPlymouth account credential.

1.1 How to log into the system?

We can get into Turing via puTTY, an SSH and telnet client, with the above credential.The puTTY program can be found in all the programming labs in Memorial, and can

also be obtained via https://www.putty.org/.If you use a MAC, try https://www.ssh.com/ssh/putty/mac/.Once you start puTTY, you should enter turing.plymouth.edu into the “Host Name”

box 2, then click “Open”, as shown in Figure 1. You will then be prompted for your credential.If everything works out, it should look like Figure 2. You are now ready to play with it. ,

1.2 Simple Unix commands

Once getting into the system, we can immediately try out a few simple things.

• Change your password.

% passwd

Changing password for user zshen.

Current passworE: 1234567

1Both Apple iOS and Adnroid are based on Unix.2You can use turing later on.

3

Figure 1: What does PuTTy look like?

Figure 2: How to log into turing?

New passworE: 7654321

Re-enter passworE: 7654321

passwE: all authentication tokens updated successfully.

%

Here, by ‘%’, we mean the command line prompt, or simply prompt. It is often associatedwith a user path, which we will explain further later on. For example, in my case, theactual prompt looks like the following (Cf. Figure 2):

/home/zshen >

Question: What is the path in your case?

• What is the date today?

4

% date

Tue Jan 7 08:06:31 EST 2020

• Who just logged on?

% who am i

zshen pts/0 2020-01-07 08:06 (vpnuser-1-20.plymouth.edu)

• What is the command that you want to use? Here “man” certainly refers to “manual”.

% man date

DATE(1) User Commands DATE(1)

NAME

date - print or set the system date and time

...

• I am done.

% logout

Labwork 1.2: Unix commands are case sensitive. Try out the following commands withturing to see what happens: WHO, CAL 2018, DATE, Who Am I, e.g.,

% WHO

You might also want to use the manual command 3 man to check if they are validate Unix

commands, e.g.,

% man WHO

Collect what you have done with the above in a ‘.txt’ file, using, e.g., Notepad++, andsend it in as part of your report for this lab 4. (Check out the course page for the wholeassignment for Lab 1.)

3Notice that you can exit the manual mode by entering ‘q’, standing for “quit”?.4An effective and efficient way to send in a lab report for this course is to copy the stuff that you have

done in turing, and paste them into a .txt file. Make sure that you will clearly label your solutions to theproblems. For example, collect all the stuff you just did for Labwork1.2 after a label “Labwork1.2”.

5

2 Unix file system

There is nothing special in terms of the file system, compared with what we have beendealing with every day with the Windows system, except perhaps a folder in Windows isreferred to as a directory in Unix.

An ordinary file can be either a text file or a binary file. A text file is usually representedas ASCII codes; and a binary file, on the other hand, is composed as a sequence of binarybits, intended for a computer to use. A special file contains information necessary to workwith a certain device.

There are lots of files kept in a computer. To facilitate a mundane task of looking for afile, a bunch of related files are organized into a directory, and a bunch of related directoriesare organized into a parent directory, ..., until everything is collected under a root directory.From an operating system’s perspective, a directory is also regarded as a file. Thus, thewhole thing is organized as a tree, which you should know very well after taking CS2381Data Structure and Intermediate Programming. Below shows an example of such a filesystem, based on a tree.

Figure 3: An example of a Unix tree

To dig a bit deeply, the very reason to use a tree structure to organize all these directoriesand files is that, as you should have learned in MA2200 Finite Mathematics, MA2250Mathematics for Computer Scientists, or MA3200 Discrete mathematics, that, in such astructure, the path between any two nodes, especially that between the root directory and anyother directory and/or file, is unique. This simple fact allows us to use the same name touniquely identify different entities in such a structure, which will be made clear later whenwe discuss the naming of such entities as directories and files.

Another fact is that it is relatively fast for use to look for a specific file in such a treestructure, which will be made clear when you take CS3221 Algorithm Analysis.

6

More specifically, at the top level, a Unix file tree contains the following directories:bin, containing the software for the shell and most of other commands; dev, containing allthe special files, especially drivers, associated with various devices; etc, containing variousadministrative files, such as the list of the authorized users; home, containing the homedirectories of various users; tmp, containing you-know-what; usr, containing somethinguseful; and var, containing something that varies, e.g., users’ mailboxes.

When just logging into the system, you are in your working directory. For example, whenI log into the system, I am in my working directory /home/zshen, as shown in Figure 2. Youcan always find out your current working directory by using the “print working directory”command pwd.

% pwd

/home/zshen

%

If you want to have a look at the tree structure rooted at the current working directory,you can use the tree command. For example, a sub-tree rooted at the working directory ofcounting is shown in Figure 4, where ls lists all the stuff in a working directory.

Figure 4: Usage of the tree command

Once you know where you are, you can change to any directory you want to work with,by using the “directory change” command, cd. The issue is certainly how do you know whereto go, and how to tell the system about where you want to go.

We will explain how to change to the desired directory after understanding how to identifythe location of such directories in the following Section.

7

2.1 What is in a name?

Every file and directory is identified by a name. On most of the systems, a name containsbetween 1 and 255 characters out of the following: upper case letters, lowercase letters,digits, period, underscore and comma. You should avoid some of the special symbols suchas ‘&’, ‘*’, ‘\’, various brackets, ‘%’, etc., when coming up with a name, and you should notuse various reserved words 5 or special characters 6, as the names of your directories/files.

To refer to a file located in your current directory, you only need to use its name. But,if you want to use a file located in another directory, you have to use its pathname, whichuniquely identifies its location in the whole file structure, which is a tree.

For example, the root directory is almost always labeled with ‘/’, and the absolute path-names for the root’s child directories starts with the root directory, i.e., ‘/’. In general, toform the absolute pathname of a node name in a tree structure, we start with ‘/’, the nameof the root, and collect the names of all the nodes along the unique path from the root toname in a sequence, and use ‘/’ to separate all such names.

For example, the absolute name of the directory home is /home as shown in Figure 3,where, on the other hand, the directory ‘home’ contains two sub-directories: jack and jill.

Their absolute path names 7 are ‘/home/jack’ and ‘/home/jill’, respectively. Similarly, theabsolute pathname for the file kangaroo is ‘/home/jill/Marsupials/kangaroo’.

Figure 5: What does a marsupial look like?

Although the absolute path name uniquely identifies a file, and allows us to give the samename to different files, it is certainly an overkill, since we have to specify all the intermediatenames /, e.g., as shown in Figure 4, the absolute name of the directory counting is

5Check out some of the Unix reserved words from the course page.6Check out some of the Unix special characters from the course page.7In ros, a robotic operating system, which some of you will study in the EMTR program, an absolute

pathname is called a global name [3, Sec. 5.2].

8

/home/zshen/Courses/CS2470/programs/Core02/counting

Moreover, we usually don’t have access rights to some of these directories, especially,the root directory. Furthermore, the host computer most likely won’t have the directorystructure that you will use. Thus, we almost always use relative pathname 8, a pathnamerelative to the current working directory, to refer to a certain node in a file structure .

This notion is also based on the key structural property of a tree: there is a unique pathbetween any two nodes, going through the root of the minimum tree, which includes bothnodes.

For example, in Figure 3, if we are in the directory jack, and want to refer to thedirectory Marsupials, we notice that home is the root of the minimum tree that containsboth of these two nodes. We thus 1) move up to home, the parent of jack, as representedwith ..; 2) then move down to jill; and 3) move further down to Marsupials, whileusing ‘/’ to separate all these three names. As a result, the name of Marsupials relative tojack is “../jill/Marsupials”.

Note: It is easy to tell apart from an absolute path name from a relative one: the formerstarts with a slash ‘/’, while the latter doesn’t.

We can also use ‘.’ to refer to the current working directory, as in, e.g., ‘./Oceans’,assuming the current working directory is still “jack”. But we prefer to use “Oceans” as itsaves us two key strokes. , On the other hand, because of an access privilege issue, whenwe want to execute a program in the current working directory, e.g., a.out, we have to say

%./a.out

Labwork 2.1:

1. Which of the following are valid names for ordinary Unix files, and why 9?

foo, guess?, book.chap1, BOOK.chap2, 2good2Btrue, {2bad}, rank*, serial#

2. Which of the following are valid names for directories, and why?

Dir2, Directory.3, *Hook, |Line1|, "Sinker", money.$, .hideNseek, 777

3. In Figure 3, what are the absolute path names for root, bin, jill, and kangaroo,respectively?

4. Again, in Figure 3, suppose that Marsupials is now the working directory, what arethe relative pathnames of root, bin, jill, and kangaroo, respectively?

8It is called relative graph resource name, or simply relative name in ros, including a special case ofprivate name, relative to a running node [3, Sec. 5.3].

9You might want to talk about them now, and try them out later on. Things might have changed, duringthe last thirty years.

9

5. Still in Figure 3, directory jack has two subdirectories, Continents and Oceans, whatare their absolute pathnames?

Collect what you have done in a ‘.txt’ file and send it in as part of your report for this lab,together with what you have done for Labwork 1.2. (Check out the course page for the wholeassignment for Lab 1.)

2.2 Work with files

Once a directory is identified, we can list all the files included in that directory using the ls

command, as you already saw in Figure 4.

1. List all the files in the current directory 10

% ls

a.out first.c hello lucky second.c

basicExamples format.c hello.c pow.c sqroot.c

char formatFirst.c longestLine power.c temperature

counting getline.c lsOutput printd.c

2. A hidden file, i.e., a file with its name starting with a ‘.’, will not be listed, unless weuse the list all command.

% ls -a

. char formatFirst.c longestLine power.c temperature

.. counting getline.c lsOutput printd.c

a.out first.c hello lucky second.c

basicExamples format.c hello.c pow.c sqroot.c

3. The command ls shows us the names of files and directories. We can have a look atthe content of a text file by using more, e.g.,

% more first.c

#include <stdio.h>

main(){

printf("I feel lucky today

\n");

}

10What you will get is certainly different from mine ,.

10

4. To create a “new file”, you can either 1) redirect the “standard input” into a “newfile”; 2) copy an existing file into a “new file”; 3) use a text editor to create a “newfile”, which we will do a lot in this course; or 4) use a computer program to generatea “new file”.

(a) It is easy to create a file, e.g., “sampleRedirect”, by redirecting the output ofthe standard input device into such a file with ‘>’ as follows: Use the output ofthe “ls” command as the content of the file sampleRedirect.txt.

% ls

output.txt

% ls > sampleRedirect.txt

% more sampleRedirect.txt

output.txt

sampleRedirect.txt

In particular, the following creates an empty file (?).

% > testFile

% more testFile

%

(b) You can also copy an existing file with the cp command. For example, in thefollowing, you copy “output.txt” into “anotherOutput.txt”

% ls

output.txt sampleRedirect.txt

% cp output.txt anotherOutput.txt

% ls

anotherOutput.txt output.txt sampleRedirect.txt

(c) If you want to redirect the output of the standard input and append it to anexisting file, you can use the ‘>>’ operator. In the following, you add the outputof the program “ls” at the end of the the file “output.txt”.

% more output.txt

This is a test

% ls

anotherOutput.txt output.txt sampleRedirect.txt

% ls >> output.txt

% more output.txt

This is a test

anotherOutput.txt

output.txt

sampleRedirect.txt

11

(d) What we will do mostly in this course, just as we did in the programming courses,is to use an editor to come up with a program, actually a text file. We will talka lot more about this approach in Section 4.

5. We can change the name of a file, or even move a file to another place, with the movecommand ‘mv’. For example, the following renames “output.txt” to “result.txt”.

% ls

anotherOutput.txt output.txt sampleRedirect.txt

% mv output.txt result.txt

% ls

anotherOutput.txt result.txt sampleRedirect.txt

In general, you can also move a file somewhere else by changing its absolute path name.Here the command “mkdir” creates a directory sampleDir within the current workingdirectory. We will further discuss “mkdir” in Section 2.3.

% mkdir sampleDir

% ls

anotherOutput.txt result.txt sampleDir sampleRedirect.txt

% ls sampleDir

%

The following moves the file result.txt into the just created directory sampleDir.

% mv result.txt sampleDir

% ls

anotherOutput.txt sampleDir sampleRedirect.txt

% ls sampleDir

result.txt

You should use the man(ual) command to check out the other formats, option, etc. ofthis “mv” command.

6. To delete a file, you use “rm”. Go through the following process step by step andunderstand what happens in each step.

% ls

anotherOutput.txt sampleDir sampleRedirect.txt

% mv sampleDir/result.txt result.txt

% ls

anotherOutput.txt result.txt sampleDir sampleRedirect.txt

% cp result.txt output.txt

12

% ls

anotherOutput.txt output.txt result.txt sampleDir sampleRedirect.txt

% rm output.txt

% ls

anotherOutput.txt result.txt sampleDir sampleRedirect.txt

7. With the “ls” command, the ‘l’ option shows all the information of the items in adirectory.

% ls -l

total 16

...

-rw-r--r-- 1 zshen domain^users 15 Mar 27 2013 test1.txt

...

Look at the above long listing, you might be wondering what does, e.g., “rw-r--r--”,mean. They deal with the access privilege of a file or a directory in three groups:u(owner of this file), g (other users in this file group), o (everyone else), and a (everybody,i.e., ugo). For each group, it tells its access privileges regarding whether it can be read,written or executed.

Thus, “rw-r--r--” means that the owner of the file can read and update, but cannotexecute, this file; all the members of her group, as well as the public, can only readthis file.

8. Only the owner of a file, or a directory, can change such privileges with the chmod

command. Go through the following process step by step and understand what happensin each step.

% ls -l test1.txt

-rw-r--r-- 1 zshen domain^users 15 Mar 27 2013 test1.txt

% more test1.txt

-4 2 + 4 5 * -

% chmod u-r test1.txt

% more test1.txt

test1.txt: Permission denied

% ls -l test1.txt

--w-r--r-- 1 zshen domain^users 15 Mar 27 2013 test1.txt

% chmod u+r test1.txt

% ls -l test1.txt

-rw-r--r-- 1 zshen domain^users 15 Mar 27 2013 test1.txt

% more test1.txt

-4 2 + 4 5 * -

13

The following gives everyone the read access privilege to the file.

% chmod a=r test1.txt

% ls -l test1.txt

-r--r--r-- 1 zshen domain^users 15 Mar 27 2013 test1.txt

2.3 Work with directories

It is quite common to develop a tree like directory structure to manage the files more effi-ciently by using various directory related commands.

1. When working with directory, the most important thing is to know where you are, i.e.,what is your working directory, with the “pwd” command.

% pwd

/home/zshen/testDir/parent

% ls

anotherOutput.txt result.txt sampleDir sampleRedirect.txt

2. To create another directory within the current directory, you use “mkdir”.

% mkdir anotherDir

% ls

anotherDir anotherOutput.txt result.txt sampleDir sampleRedirect.txt

3. Use “cd” to change the working directory to this newly created directory anotherDir.

% cd anotherDir

% ls

4. Come back to the previous working directory. Here is where we are at this moment.

% pwd

/home/zshen/testDir/parent/anotherDir

Remember “..” stands for the parent directory of the current working directory.

% cd ..

% pwd

/home/zshen/testDir/parent

% ls

anotherDir anotherOutput.txt result.txt sampleDir sampleRedirect.txt

14

5. To remove a directory, we use “rmdir”.

% rmdir anotherDir

% ls

anotherOutput.txt result.txt sampleDir sampleRedirect.txt

6. What have we got?

Use some of the previous commands, we can find out the contents of the currentworking directory.

% pwd

/home/zshen/Courses/CS2470/programs/Core02/basicExamples

% ls

a.out recip sqroot.c testScanf.c

hello.c recip.c testAnotherScanf.c tripExample

The tree command provides a more structural perspective of the content, as we sawin Figure 4.

Labwork 2.3: When sending in a lab report, write down as much detail as you could forthe assignments, and label them clearly.

1. Create the directory structure as shown in Figure 3, notice that “kangaroo” is the nameof a file, but not a directory. Since “/” is a special character, for the real root directory,which you don’t have access right, use “root” as the name of the root directory in yourstructure.

Use the tree command to demonstrate what you have done.

2. Set up some additional subdirectories for jack as follows: Continents now holdsAfrica, Antarctica, Asia, Australia, Europe, NAmerica, and SAmerica; andeach of these directories holds some countries and regions of their own.

(a) User tree command to decomonstrate this expanded file structure.

(b) List the absolute path names for the following countries: Norway, India, Egyptand Argentina 11.

(c) Assume that your current working directory is USA, accomplish the followingusing a single command line with the help of relative pathnames.

i. List the content of Marsupials.

ii. List the content of Australia.

iii. Place a copy of the kangaroo file in the Australia directory under Continents.

11Make sure you will put these countries in the right continent.

15

(d) Repeat the above using absolute pathnames.

3. The echo command takes a line of input that you type in through the keyboard andkicks it right back on the screen. Try out the following:

% echo try this out.

try this out.

Create a file “fun” by redirecting the output with the echo command, with the helpof the redirection mechanism (Cf. Step 4a of Sec.2.2)

4. Use the commands who, who am i, and date (Cf. Sec.1.2) to append more data tothe fun file.

5. Come up with a file “stuff” and place it in your working directory. Then do thefollowing:

(a) Give everyone permission to read stuff, but do not change any other accessprivilege.

(b) Permit the owner and group members to read and write the file; remove theseprivileges from everyone else.

(c) Give the owner and group members permission to execute stuff while giving theowner sole permission to read or write it.

Confirm what you have done (Cf. Step 8 of Sec.2.2)

6. A hidden file has a name starting with ‘.’. (Cf. Step 2 of Sec.2.2) Use the “cal” utility(What does it do? How to find it out?) and the redirection operator ‘>’ to create a.hidden file. How to tell that you have created this file? How to show the content ofthis file?

7. Create a new directory “Misc” and move the file fun into Misc.

8. Without leaving your current working directory (What is it?), create a directory“Vacations” under Misc. Then create calendars for the summer months (June, Julyand August) of 2020 12, and move them into the Vacations directory.

9. Redo Labworks 2.1.1 and 2.1.2 by testing them out, using appropriate commands. Tellme what happens when you try to create a file with its name being, e.g., “{2bad}”,etc..

12They will be here soon... ,.

16

2.4 Background processing

As we mentioned earlier, Unix is a powerful operating system that supports multitaskingin terms of multiple processes identified with PID. Since there is only one processor in themachine 13, at any time, one process is running, while the others are simply waiting.

The command ps will show the PIDs of all the processes that have been running.

% ps

PID TTY TIME CMD

21847 pts/0 00:00:00 bash

22164 pts/0 00:00:00 man

22167 pts/0 00:00:00 sh

22168 pts/0 00:00:00 sh

22173 pts/0 00:00:00 less

25002 pts/0 00:00:00 ps

We can use the kill command to terminate a process, stop to suspend a process, bgbrings it to the background, and fg to bring a process to the foreground. The ‘\&’ symbolat the end of a line puts it into the background.

% (sleep 60; echo stop this command) &

[2] 25128

% ps

PID TTY TIME CMD

21847 pts/0 00:00:00 bash

22164 pts/0 00:00:00 man

22167 pts/0 00:00:00 sh

22168 pts/0 00:00:00 sh

22173 pts/0 00:00:00 less

25128 pts/0 00:00:00 bash

25129 pts/0 00:00:00 sleep

25130 pts/0 00:00:00 ps

% kill 25129

% -bash: line 42: 25129 Terminated sleep 60

stop this command

Question: What does “sleep 60” do? You might want to check out the syntax of this“sleep” command. Also change the 60 to other values to see what happens.

13There are perhaps multiple processing cores, but sill limited, as compared with a large number ofprocesses, and threads. A lot more details along this line will be discussed in CS 4310 Operating Systems.

17

2.5 Miscellaneous commands

There are also quite a few other commands, such as comm, crypt, cut, diff, etc.. Tocheck them out for details, you can use the manual command “man” to find out their respec-tive syntax and semantics. For example, “grep” essentially does the “search” of a targetstring in a file, or pattern recognition, which we will explore later on in the Function chapter.

% man grep

GREP(1) GREP(1)

NAME

grep, egrep, fgrep - print lines matching a pattern

SYNOPSIS

grep [options] PATTERN [FILE...]

grep [options] [-e PATTERN | -f FILE] [FILE...]

DESCRIPTION

Grep searches the named input FILEs (or standard input if no files are

named, or the file name - is given) for lines containing a match to

the given PATTERN. By default, grep prints the matching lines.

In addition, two variant programs egrep and fgrep are available.

Egrep is the same as grep -E. Fgrep is the same as grep -F.

OPTIONS

-A NUM, --after-context=NUM

Print NUM lines of trailing context after matching lines.

Places a line containing -- between contiguous groups of

matches.

Below shows an application.

% more first.c

#include <stdio.h>

main(){

printf("I feel lucky today\n");

}

% grep printf first.c

printf("I feel lucky today\n");

18

Labwork 2.5: How should we display, with grep, two lines after a matching of “printf”is found in the above file, first.c, starting with each matching?

3 A Unix shell

As we mentioned earlier that a Unix shell interprets and executes user commands.

3.1 How does a shell work?

The general process looks like the following:

1. The shell displays a prompt such as ‘% ’.

%

2. You type in a command such as cal.

% cal

3. The shell interprets your command by looking for an appropriate software. If it can’tfind it, it will print out an error message.

% wrong

-bash: wrong: command not found

4. If it finds the command, the kernel runs the requested software to give you the output.

% cal 2 2020

February 2020

Su Mo Tu We Th Fr Sa

1

2 3 4 5 6 7 8

9 10 11 12 13 14 15

16 17 18 19 20 21 22

23 24 25 26 27 28 29

5. The shell then prints another prompt and waits for the next command.

%

Note: The above provides a general user interface of a Unix Shell, which you will work onas your final project for this course.

Such a user interface is also the core of a GUI (Graphic User Interface) interface of acontemporary operating system, where we just use mouse and window to provide the needinformation to an operating system for its execution.

19

3.2 Options and arguments of commands

For the “cal” command, you can specify which month and year you want. Here both themonth and the year are the arguments of this command.

Sometimes, we have options for a command. For example, we have -a, -F, -l, -r,

-s, -t, and -u for the listing command ls, where, e.g., the -F option classifies all the objectswithin a directory by showing a ‘/’ after each directory and a ‘*’ after each executable file.

% ls

parent testFile testFile.txt

% ls -F

parent/ testFile* testFile.txt

Labwork 3.2:

1. What does “-a” stand for? Find it out and do the same for all the other options ofls, the listing command.

2. Give an example for each of the aforementioned option about its usage.

3.3 Redirection

We already talked about this stuff earlier, in Step 3 in Sec. 2.2, e.g.,

% more result.txt

This is a test

anotherOutput.txt

output.txt

sampleRedirect.txt

The following gets the input from result.txt for the utility “wc”, which then sends itsoutput to aother file “wcOutput.txt.

% wc < result.txt > wcOutput.txt

% more wcOutput.txt

5 7 64

%

Question: What does “wc” do? We will check out its implementation details in the C Coreunit later, also in the Function chapter.

Labwork 3.3:

1. Use man to dig out all the details of this wc utility.

2. Run wc to find out only the number of characters in the calendars for all the summermonths, i.e., June through August, of 2020.

20

3.4 Group commands

Besides typing one command at a time, we can also type in several at a time and place themin the same line, which will then be executed sequentially. For example,

% who am i; cal 2 2020

zshen pts/1 2019-11-19 06:46 (vpnuser-1-11.plymouth.edu)

February 2020

Su Mo Tu We Th Fr Sa

1

2 3 4 5 6 7 8

9 10 11 12 13 14 15

16 17 18 19 20 21 22

23 24 25 26 27 28 29

3.5 Pipes and tees

The pipe command, ‘|’, sends the output of one command as the input to another command.For example,

% ls | more

input

output

output.txt

sampleRedirect

Here, the command ls throws its output, the content of your working directory, into atemporary file, which is passed as input to more, which then displays such a content.

On the other hand, “|tee” does one more thing: besides passing the output of onecommand to another as input, it also saves the output in a file. In the following, besidespassing the output of ls to more to display, it also saves such a content into the file IsOut.

% ls |tee lsOut |more

input

output

output.txt

sampleRedirect

% ls

input lsOut output output.txt sampleRedirect

% more lsOut

input

output

output.txt

sampleRedirect

21

Labwork 3.5:

1. Use “|tee” to list the content of your working directory, and also save them in aseparate file.

3.6 Quoting special characters

To prevent confusion, a file name should not contain any of the special characters that Unix

uses for some specific purpose. But, when we have a need to use any of the special characters,we have to tell the system, using the escape character ‘\’. For example, ‘*’ stands for a wildcard, i.e., everything, while “\*” stands for itself.

% echo *

anotherDir anotherOutput.txt a.out hello.c result.txt sampleDir wcOutput.txt

% echo \*

*

We use grave accent, ‘‘’, i.e., the key above the “Tab” key in a standard keyboard 14, toindicate a command that we want the shell to run. For example,*

% date

Thu Jan 9 07:22:12 EST 2020

Nothing leads to nothing.

% echo It is not date

It is not date

If we use ‘’’, apostrophe (Decimal code 39), by pressing the key to the left of the Enter key,we get the following:

% echo It is not ’date’

It is not date

On the other hand, we will get the following if we use, ‘‘’, the grave accent.

% echo It is not ‘date‘

It is not Thu Jan 9 07:25:48 EST 2020

Labwork 3.6:

1. What does each of the following do, and why?

14In the standard ASCII code chart, the binary, Octadecimal, Decimal, and Hexadecimal code of graveaccent is 110 0000, 140, 96 and 60, respectively.

22

% echo *

% echo /*

% echo \*

% echo "\*"

% echo "/*"

% echo

% echo */*

2. Investigate the wild cards such as ‘*’, ‘?’, and use “cat”, another utility (What doesit do?), to show the contents of all the files with their names ending in “ing”. How tolist all the files with their names containing either ‘x’ or ‘X’?

4 The editors

There are a few editors available in Unix that we can use to enter a program, such as vi(m),pico and emacs. We once all used pico 15 to edit emails. It is pretty easy to use, but youcan only work with one line at a time. /.

% pico newFile

Unix does come with a full-screen editor, vim.

% vim newFile

Although it is not as easy, you are strongly urged to use vim as the editor. Check out thecourse page for a vim tutorial.

Labwork 4: Make sure that you know vim reasonably well, before proceeding to the nextunit. ,

5 C programming in Unix

We will now go over the process that, once you have saved a program somewhere, how tocompile, and run this program.

This compilation, link, and execution, process is the same for all the other programs thatwe will come up in the rest of this course.

15Well, pico is not available in turing, but nano is.

23

5.1 First thing first

It is pretty easy to compile a c program in Unix, using the cc, or gcc 16 facility. For example,after editing and saving your program, “hello.c” in your working directory, you can checkout its content as follows:

% more hello.c

/* the first program in c */

#include <stdio.h>

int main(void){

printf("Hello, world\n");

return 0;

}

The following line compiles this program 17:

% cc hello.c

You now can run it 18

% ./a.out

Hello, world

5.2 Are you bored with it?

You don’t need to always call the executable a.out, for example

% cc hello.c -o hello

% ./hello

I feel lucky today

Here is another example which calculates the reciprocal of an integer.

% more recip.c

% more recip.c

/* Compute reciprocals */

#include <stdio.h>

16Later on, when we work with C++ programs in Unit 9, we use the g++ command.17Technically, you might also want to go through the following section on debugger. But, it might serve

the purpose better if you get some C programming experience first. ,

18Revisit Sec.2.1 as why we have to prefix “a.out” with “./”.

24

int main(void){

int n;

float recip;

printf("This is a program computes reciprocals.\n");

printf("Enter a number: ");

scanf("%d", &n);

recip=1/(float) n;

printf("The reciprocal of %d is %f.\n", n, recip);

return 0;

}

% cc recip.c -o recip

% ./recip

This is a program computes reciprocals.

Enter a number: 34

The reciprocal of 34 is 0.029412.

5.3 Speak mathematically

When using a mathematical function, we have to add on the -lm option to link to themathematics library. For example,

% more sqroot.c

/* Compute square roots */

#include <stdio.h>

#include <math.h>

int main(void){

float n, x, root;

printf("This program computes square root.\n");

printf("Enter a number: ");

//printf("%3.1f", sqrt(32));

scanf("%f", &n);

x=n;

printf("The number you entered is %f: ", x);

printf("The square root of %f is %f.\n", n, sqrt(n));

return 0;

}

We now bring in “lm”, a library of the math functions 19, including that of sqrt(n), to servethe comiling purpose.

19Check out the relevant link in the course page.

25

% cc sqroot.c -lm

Ready to run?

% ./a.out

This program computes square root.

Enter a number: 34

The number you entered is 34.000000: The square root of 34.000000 is 5.830952.

5.4 Cut it into pieces

We can also work with a program consisting of multiple files, just as we did with Java. Forexample,

% more main.c

/* Illustrate multiple source files */

void chicago(void);

void indiana(void);

int main(void){

chicago();

indiana();

chicago();

return 0;

}

% more chicago.c

/* The chicago() function */

#include <stdio.h>

void chicago(){

printf("\nI’m waiting at O’Hare International,\n ");

printf("Airport, a fun place.\n");

}

% more indiana.c

/* The indiana() function */

#include <stdio.h>

void indianapolis(void);

void indiana(void){

26

printf("Back home again, Indiana.\n");

indianapolis();

printf("Wander Indiana--come back soon!\n ");

}

% more indy.c

/* The indianapolis() function */

#define POP2000 1.6

#include <stdio.h>

void indianapolis(void){

printf("\nWelcome to Indianapolis, Indiana.\n");

printf("Population: %f million.\n", POP2000);

}

Compile all the pieces individually. Notice the “-c” option 20.

% cc -c main.c

% cc -c chicago.c

% cc -c indiana.c

% cc -c indy.c

Link all the object codes, in “.o”.

% cc *.o

Run the executable.

% ./a.out

I’m waiting at O’Hare International,

Airport, a fun place

Back home again, Indiana.

Welcome to Indianapolis, Indiana.

Population: 1.600000 million.

Wander Indiana--come back soon!

I’m waiting at O’Hare International,

Airport, a fun place

%

Notice that, at the end, the printer head returns to its original place.

20Check out the other popular “cc” options in the course page.

27

Labwork 5:

1. To help you better organize the fifty or so programs that you will soon develop in thiscourse, come up with a directory structure under your working directory, for all thechapters that we will cover in this course. Each chapter, e.g., Core, is assigned its owndirectory. And some of them, such as unixNotes, must have several sub-directories,e.g., one for the current section, editor.

A sample, and partial structure, is given as follows for your reference:

% pwd

/home/zshen/Courses/CS2470

% tree

.

+-- Core

+-- Function

+-- Type

+-- unixNotes

+-- editor

+-- hello.c

+-- multi

+-- chicago.c

+-- indiana.c

+-- indy.c

+-- main.c

6 directories, 5 files

2. Use an editor, e.g., vim, to add in all the programs that we have gone through inthis section, and put them under editor. The above structure shows hello.c, anda group of files, as shown in Section 5.4, collected under the sub-directory, multi.

3. Test out all these program samples, make sure they all run smoothly.

4. Let me know that “I’m done.”. You don’t need to send in anything else.

6 A debugger

A C debugger, gdb, is a very useful tool for a C programmer, similar to what jdb could dofor you if you program in Java.

To help us to find out programming errors, it allows us to check out various snapshotsof the execution of a program, namely, values of various variables, and determine whethera particular line in a program will be the next one to be executed, allowing us to find out

28

logic errors. We can also use it to execute one line at a time, instead of letting the wholething go through in a single shot. We can also use it to observe how a program behaves witha collection of choice points. All these features help us to diagnose whether the program iswhat we have in mind, and if it does the right thing.

We will now check out this many possibilities through a particular debugger gdb thataccompanies cc. Let’s use the following code as a running example, which prints out thevalue of sum at the end of its execution.

1. #include <stdio.h>

2.

3. main(){

4. int i, sum;

5.

6. sum=0;

7. for(i=0; i<10; i++)

8. if(i < 5)

9. sum=sum+1;

10. else

11. sum=sum+((i-3)/2+(i/3)); //line 11

12.

13. printf("sum=%d\n", sum);

}

We can certainly run it as follows.

% cc sum.c

% ./a.out

sum=24

Let’s send it through the debugger gdb:

% cc -g sum.c

% gdb a.out

GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /home/zshen/Courses/CS2470/programs/Debugger/a.out...done.

(gdb)

29

In the above, to facilitate the debugger, we use the “-g” switch to come up with additionalinformation, such as the symbolic table, built during the compilation process 21. Moreover,when this switch is used, no optimization is done when compiling the codes, which makes itless complicated to do the debugging.

After compiling, we run the debugger on the executable a.out, which simply loads theexecutable into the debugger environment. We are now ready to execute various debuggercommands. Once any such command is done, it comes back to the debugger prompt

(gdb)

Before we get into something, we alwasy want to know how to get out... .

Question: How to get out of gdb?

(gdb) quit

%

6.1 Set break points

It is often useful to execute a program partially, i.e., through part of a program, which canbe done by setting a break point. It tells the debugger to run the program until that breakpoint is reached, when the program just paused. We can then do other things to the pausedprogram. For example,

(gdb) break 11

Breakpoint 1 at 0x400541: file sum.c, line 11.

(gdb) run

Starting program: /home/zshen/Courses/CS2470/programs/Debugger/a.out

Breakpoint 1, main () at sum.c:11

11 sum=sum+((i-3)/2+(i/3));

Missing separate debuginfos, use: debuginfo-install glibc-2.17-260.el7_6.3.x86_64

(gdb)

Question: What should be value of i at this point?Answer: 5. Since if i is less than 5, it would go through the then part.

Question: How do we find it out?Answer: We can certainly check this out using another debugger command display:

(gdb) display i

1: i = 5

(gdb)

21If you take CS 3221 with me, this is an example that I gave as an application of the Hashing table.

30

Question: What should be value of sum at this point?Answer: Also 5. Since we went through from i=0 through i=4.

Question: How do we find it out?Answer:

(gdb) display sum

2: sum = 5

We can also check out where the program is paused.

(gdb) where

#0 main () at sum.c:11

6.2 How to proceed from a break point?

There are three ways: step, next and continue.

1. The step command executes the next line in the logic flow and then pause again. Sortof “one step at a time”.

Question: What is the next line at this point in the logic flow?

Answer: Since line 11 is the last line of the loop body, the control has to go back tothe for line, i.e., line 7.

Question: How do we find it out?

Answer: Use step.

(gdb) step

7 for(i=0; i<10; i++)

1: i = 5

Question: Then what?

(gdb) step

8 if(i < 5)

2: sum = 7

1: i = 6

(gdb)

The first step executes line 11, and brings the control to the loop at line 7, when thevalue of i is still 5. The next step does the increment of i and brings the control toline 8, where the value of i turns into 6.

31

2. The next command does the same, except if the next line is a function call, it willcomplete everything that call does.

(gdb) next

Breakpoint 1, main () at sum.c:11

11 sum=sum+((i-3)/2+(i/3));

1: i = 6

(gdb) next

7 for(i=0; i<10; i++)

1: i = 6

(gdb) next

8 if(i < 5)

1: i = 7

The next command might be more preferable over step.

3. The command continue allows the execution continue until either a breakpoint isreached, the program exits normally, or it tries to do something illegal.

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:11

11 sum=sum+((i-3)/2+(i/3));

1: i = 7

6.3 Check out the values

There are two ways to observe the values of a variable: print and display.

1. The print command is a one-time request to see the value. The debugger shows itscurrent value then forgets about it.

(gdb) print i

$1 = 7

2. As we have already seen, the display command shows it on an ongoing basis: Thedebugger will show its value every time the program is paused.

32

6.4 A final example

The program sum.c generates different values of sum for different value of i. We might wantto see the big picture as what specific values of sum will be generated for what specific valuesof i.

What we could do is to set break points at lines 8 and 9, and print out values of sum andi in that context.

% gdb a.out

GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /home/zshen/Courses/CS2470/programs/Debugger/a.out...done.

(gdb) break 8

Breakpoint 1 at 0x400535: file sum.c, line 8.

(gdb) break 9

Breakpoint 2 at 0x40053b: file sum.c, line 9.

(gdb) run

Starting program: /home/zshen/Courses/CS2470/programs/Debugger/a.out

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

Missing separate debuginfos, use: debuginfo-install glibc-2.17-260.el7_6.3.x86_64

(gdb) display i

1: i = 0

(gdb) step

Breakpoint 2, main () at sum.c:9

9 sum=sum+1;

1: i = 0

(gdb) display sum

2: sum = 0

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 1

1: i = 1

(gdb) step

33

Breakpoint 2, main () at sum.c:9

9 sum=sum+1;

2: sum = 1

1: i = 1

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 2

1: i = 2

(gdb) continue

Continuing.

Breakpoint 2, main () at sum.c:9

9 sum=sum+1;

2: sum = 2

1: i = 2

(gdb) step

7 for(i=0; i<10; i++)

2: sum = 3

1: i = 2

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 3

1: i = 3

(gdb) continue

Continuing.

Breakpoint 2, main () at sum.c:9

9 sum=sum+1;

2: sum = 3

1: i = 3

(gdb) continue’

Unmatched single quote.

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 4

34

1: i = 4

(gdb) continue

Continuing.

Breakpoint 2, main () at sum.c:9

9 sum=sum+1;

2: sum = 4

1: i = 4

(gdb) continue

Continuing.

//When i<5, we just add 1 to sum--zs

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 5

1: i = 5

(gdb) continue

Continuing.

//When i>=5, the rule changes, and the

//first such value for sum becomes 7--zs

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 7

1: i = 6

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 10

1: i = 7

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 14

1: i = 8

(gdb) continue

Continuing.

Breakpoint 1, main () at sum.c:8

8 if(i < 5)

2: sum = 18

35

1: i = 9

(gdb) continue

Continuing.

sum=24

[Inferior 1 (process 102194) exited with code 07]

(gdb)

The order of the break points does not matter, the debugger will pause whenever a breakpoint is reached.

Break points can be removed using the clear command.

(gdb) clear 9

Deleted breakpoint 2

(gdb) continue

Continuing.

As we just saw, once we are done, we simply quit.

(gdb) quit

A debugging session is active.

Inferior 1 [process 18704] will be killed.

Quit anyway? (y or n) y

%

Incidentally, this debugger also works with g++, i.e., the c++ stuff.

Labwork 6

1. Play with the debugger with the sum.c program, and familiarize with its features.

2. Find a non-trivial program in C that you did this semester. Identify one or more aspectsthat you want to identify, particularly, a malfunctioning feature of your program, i.e.,something that you did wrong. Then use various features of this gdb debugger toidentify the bugs, and come up with a correct program.

7 shell scripts

A very powerful feature of Unix is that we can group together various Unix commands tocome up with a script, or a little program, that will execute. For example, the followingscript, simpleShell, containing the following lines:

36

% more simpleShell

#A simple shell script

cal

date

who

Here ‘#’ indicates a comment. We can then run the program by using a redirection:

%more simpleShell

#A simple shell script

cal

date

who

% sh < simpleShell

April 2020

Su Mo Tu We Th Fr Sa

1 2 3 4

5 6 7 8 9 10 11

12 13 14 15 16 17 18

19 20 21 22 23 24 25

26 27 28 29 30

Wed Apr 8 19:57:45 EDT 2020

zshen pts/11 2020-04-08 19:39 (vpnuser-1-119.plymouth.edu)

%

We can also make this file executable by changing its access privilege using chmod, whichwas discussed in Item 8 in Section 2.2:

% chmod u+x simpleShell

% ./simpleShell

April 2020

Su Mo Tu We Th Fr Sa

1 2 3 4

5 6 7 8 9 10 11

12 13 14 15 16 17 18

19 20 21 22 23 24 25

26 27 28 29 30

Wed Apr 8 19:59:20 EDT 2020

zshen pts/11 2020-04-08 19:39 (vpnuser-1-119.plymouth.edu)

%

37

In this sense, shell becomes a system programming language, where the control structuresare pretty much the same stuff; but the variables are different: besides the user-definedvariables, there are two other kinds: environment variables and positional variables.

Environment variables are used to keep values for special shell variables such as HOME,

TERM, PATH, referring to your home directory, the type of the terminal you use, and wherethe system should look for stuff, respectively. They set the scene for everything, but we willnot touch on them too much here to avoid a mess /.

7.1 Positional variables

Positional variables are used to capture the command-line arguments used by the shell script,numbered 0, 1, 2, ..., 9. Similar to the argv[] usage (Cf. Page 35, notes of Pointers andArrays Chapter), $0 keeps the name of this very program, S1 the first argument, etc.. Forexample, the following is the echo.args script.

% more echo.args

#!/bin/sh

#Illustrate the use of positional variables

echo $0 $1 $2 $3 $4 $5 $6 $7 $8 $9

Notice that the very first line

#!/bin/sh

specifies that we will always use a particular shell, sh, to run the scripts. Thus,

% ./echo.args I love Spring, but is it coming soon?

./echo.args I love Spring, but is it coming soon?

%

Positional variables $1 through $9 are also collected as $*. Their number is denoted as$#. For example,

% more echoRead.args

#!/bin/sh

#Illustrate the use of positional parameters, used defined variables and

#the read command.

echo ’What is your name?’

read name

echo "Well, $name, you typed $# arguments:"

echo $0 $*

Thus,

38

% ./echoRead.args This is great!

What is your name?

Zhizhang

Well, Zhizhang, you typed 3 arguments:

./echoRead.args This is great!

You can also set up those variables using the set command, which is very useful. Forexample,

% more weeklyReminder

#!/bin/sh

# A daily reminder service

set ‘date‘

echo "Remember for today:"

case $1 in

Mon) echo "Plan the week.";;

Tue) echo "Take clothes to the cleaners.";;

Wed) echo "Attend group meeting.";;

Thu) echo "Make plans for the weekend.";

echo "Pick up clothes at the cleaners.";;

Fri) echo "Answer e-mail";;

Sat) echo "You should not be here working.";

echo "Finish your work and log off.";;

Sun) echo "Call Grandma and Grandpa.";;

esac

Let’s try to run this piece out....

% date

Wed Apr 8 20:01:23 EDT 2020

% chmod u+x weeklyReminder

% ./weeklyReminder

Remember for today:

Attend group meeting.

%

Yes, I have to attend group meeting Wednesday ,

Just one more example, setDate,

% more setDate

#!/bin/sh

#Demonstrate the set command

set ‘date‘

echo "Time: $4 $5"

39

echo "Day: $1"

echo "Date: $3 $2 $6"

Note that when you type this ‘date‘ stuff, you have to use the grave accent, i.e., thekey above the Tab key and to the left of 1.

% chmod u+x setDate

% ./setDate

Time: 20:03:57 EDT

Day: Wed

Date: 8 Apr 2020

%

Question: What is going on?

Answer: The grace accented ‘date‘ run the date command, which sends back the following

Wed Apr 8 20:03:57 EDT 2020

as values of positional parameters: $1, $2, $3, $4, $5 and $6.

Labwork 7.1:

1. Play with all the scripts and really understand them.

2. Write a script to come up with a daily reminder, e.g., for the week days,

(a) If it is about 6 a.m., get up

(b) If it is about 7 a.m., have breakfast

(c) If it is about 8 a.m., start the first class

(d) If it is about 10 p.m., time to do homework

If it is either Saturday or Sunday, do something appropriate for the weekend, e.g.,cleaning the backyard.

Note: For the “about” part, you might put in a difference, say, “ about 6 a.m.” is thesegment between 5:55 and 6:05 in the morning. You might also use a wild card, e.g.,“05:5*” stands for a period between 5:50 through 5:59. Do some research with WEBto dig out more relevant expressions.

In either case, you certainly need to investigate the ancient syntax for the conditionalstructure as used in shell.

40

7.2 User-defined variables

Please don’t try the following delFile, if you don’t want to accept the consequence.

% more delFile

#!/bin/sh

#Delete a file interactively

filename=$1

if test ! -f $filename

then

echo "There is no file \"$filename\"."

else

echo "Do you want to delete \"$filename\"?"

read choice

if test $choice = y

then

rm $filename

echo \"$filename\" deleted

else

echo \"$filename\" not deleted.

fi

fi

% ./delFile fakedir

There is no file "fakedir".

% ls testFile

testFile

% ./delFile testFile

Do you want to delete "testFile"?

y

"testFile" deleted

% ls testFile

ls: cannot access testFile: No such file or directory

Labwork 7.2: Modify the delFile script so that it will test if the one to be deleted is adirectory, in this case, it should call rmdir instead of rm to delete it. Otherwise, it doesnothing.

Notice that the following line tests if dirname is a directory, which sends back True ifthe value of $dirname holds a directory’s name.

if test ! -d $dirname

41

8 Unix system interface

The Unix provides its services via a set of system calls, which can be used by various userprograms. We now look at a few important and useful system calls. Since most of the Unix

are written in C, such system calls also provide a way to us to have an insight look of C.

8.1 Look at one call in details

We will experiment with an important Unix system call, fork(), which creates a newprocess. When a process creates another child process, it simply creates a copy of the callingprocess (parent process), except that the child process has its own identification number, pid,and its own pointers to shared kernel entities 22. After fork() is called, two processes willexecute the next statement after this fork() call with their respective 23 data space.

If the fork() call succeeds in the parent process, it returns the pid of the newly createdchild process. For the child process, fork() returns a 0. This tells apart the execution oftwo processes, as we will see in the later demos.

It may seem like an overkill by creating another process to carry out certain computation,but it is actually essential from at least a reliability consideration: If the newly created childprocess does anything wrong, it might crash; while the parent process will still run normally.

This technique is used extensively. For example, after the Unix system is booted, aprocess 0 is created, which is the ancestor of every process created during that session.Whenever a command is to be executed; or whenever a user is to log on, a new process iscreated. For those who have taken, or are taking, CS3221 Algorithms, we discussed its usagein spawning processes to potentially take care of parallel tasks. This important tool will bediscussed in much more details through projects as assigned in CS4310 Operating Systems.

You are also expected to use this technique when taking on the final project on shell forthis course.

8.1.1 An example

Below is a simple example, fork.c, involving the fork() call.

/* Slight modification of Richard Stevens example from "Advanced Programming

in the UNIX Environment"

*/

#include <stdio.h>

#include <sys/types.h>

#include <unistd.h>

#include <stdlib.h>

22On the other hand, the system call vfork() creates a child process that shares its data space with itsparent.

23This is not true with the vfork() call, which we will work with in CS4310 Operating Systems.

42

int globalVar = 6; /* external variable in the initialized data */

char buf[] = "Write to the standard output\n";

int main(void){

int localVar; /* local variable in the initialized data */

pid_t pid;

localVar = 88;

if (write(STDOUT_FILENO, buf, sizeof(buf)-1) != sizeof(buf)-1){

printf("write error");

exit(1);

}

printf("Now, the fork starts\n");

if ( (pid = fork()) < 0){

printf("fork error\n");

exit(1);

}

else if (pid == 0) {

globalVar++;

localVar++;

}

else sleep(2);

printf("pid = %d, globalVar = %d, localVar = %d\n", getpid(), globalVar, localVar);

exit(0);

}

Labwork 8.1.1: Run the program, then check it out carefully so that you can answer thefollowing questions:

1. What are the process ids for the child and the parent processes?

2. Which parts are done by both processes, by the child only, and by the parent only?

3. Why does only the child process increment the two variables, i.e., globalVar andlocalVar? Why are both of them affected?

4. Which process(es), child and/or parent, execute(s) the sleep command? Why is that?

43

8.1.2 Another example

More realistically, a parent wants to create a process to get something done, and will actuallywait until the newly created process to complete.

This is quite similar to the final project, where the parent process, might want to forkoff a child process to do, e.g., the ls command, and will wait there, until it is done.

The following code, consisting of two files, demonstrates this situation.

/* The parent.c file */

#include <stdio.h>

#include <stdlib.h>

#include <sys/wait.h>

int main(){

if(fork()==0){

execve("child", NULL, NULL);

exit(0);

}

printf("Process[%d]: Parent in execution...\n", getpid());

sleep(2);

if(wait(NULL)>0)

printf("Process[%d]: Parent detects terminating child \n", getpid());

printf("Process[%d]: parent terminating...\n", getpid());

}

/* The child.c file */

#include <stdio.h>

int main(){

printf("Process[%d]: child in execution...\n", getpid());

sleep(2);

printf("Process[%d]: child terminating...\n", getpid());

}

Notice that we have to name the executable file associated with child.c as child.

% cc child.c -o child

We can then compile and run parent.c and get the following:

% cc parent.c

% ./a.out

Process[9715]: Parent in execution...

Process[9716]: child in execution...

44

Process[9716]: child terminating...

Process[9715]: Parent detects terminating child

Process[9715]: parent terminating...

The parent program will create a process, which executes the child program using thesystem function execve(3), as contained in child.c. The parent then calls the systemfunction wait to block itself until the OS kernel signals the process to continue again, becauseone of its child processes has just terminated.

Note: Check out the link regarding another example of using execve(3) on the coursepage.

8.1.3 Scripts again?

Anything executable, including those shell scripts as discussed in Section 7, can be used tostart a child process in this forking context. We start with the following parent1.c file.

/* parent1.c */

#include <stdio.h>

#include <stdlib.h>

#include <sys/wait.h>

int main(int argc, char * argv[]){

if(fork()==0){

char *cmd[] = { "delFile", argv[1], NULL };

//How many parameters do you see?

if(argc==1)

execve("child", NULL, NULL);

else if(argc==2)

execve("delFile", cmd, NULL);

else exit(1);

exit(0);

}

printf("Process[%d]: Parent in execution...\n", getpid());

sleep(2);

if(wait(NULL)> 0)

printf("Process[%d]: Parent detects terminating child \n", getpid());

printf("Process[%d]: parent terminating...\n", getpid());

}

Below is the original child.c file.

% more child.c

45

#include <stdio.h>

int main(){

printf("Process[%d]: child in execution...\n", getpid());

sleep(2);

printf("Process[%d]: child terminating...\n", getpid());

}

We now have the delFile script that we saw earlier.

% more delDir

% more delFile

#!/bin/sh

#Delete a file interactively

filename=$1

if test ! -f $filename

then

echo "There is no file \"$filename\"."

else

echo "Do you want to delete \"$filename\"?"

read choice

if test $choice = y

then

rm $filename

echo \"$filename\" deleted

else

echo \"$filename\" not deleted.

fi

fi

%

We compile the child.c into an executable child,

% cc child.c -o child

then compile the parent1.c program.

% cc parent1.c

If we call it without giving it any parameter, thus argc=1, the executable of parent1.cexecutes the child program.

46

% ./a.out

Process[31193]: Parent in execution...

Process[31194]: child in execution...

Process[31194]: child terminating...

Process[31193]: Parent detects terminating child

Process[31193]: parent terminating...

On the other hand, if we call it with a parameter, child.c, thus argc=2, it calls mySpellto check the spelling of the words as contained in this file.

% ./a.out fakeFile

Process[31221]: Parent in execution...

Do you want to delete "fakeFile"?

y

"fakeFile" deleted

Process[31221]: Parent detects terminating child

Process[31221]: parent terminating...

%

This above situation is pretty similar to what you have to do in the final project as well,as different commands need different number of parameters. Even the same command mightneed different parameters to do different things. For example, ls will list all the stuff in thecurrent directory, while ls fake will list the content of the fake subdirectory.

Labwork 8.1.3:

1. Play with the programs as shown in this section. You may change some of the stuff,such as the time that the child process waits, to see if there is any change of theprintout.

2. Find out the general format, and discuss the various usage, of those “strange” functions,particularly, the exec family, fork(), getpid(), and wait().

You may need these stuff when completing the project....

You now should be ready to work on the very profitable project.

8.2 An example: Listing directories

After writing some thing simple, let’s now read something that is a lot more complicated,which is taken from the textbook[2, §8].

What we have discussed is something of a file, i.e., the content of a file. Sometimes, wealso need to know something about a file. For example, a Unix service, ls, which we oftenuse, lists the names of files in a directory, together with other information such as their size,permissions, and so on.

47

There are two aspects of this issue: Since a directory is just a file, we only need to getaccess to this file and print out its content. But, it is necessary to use a system call toget other information such as the sizes of files. We will show how to write such a program,fsize, that is to find out the size of a file, and when the argument is a directory, it willrecursively finds out the sizes of all the files as contained in such a directory.

As we mentioned in § 2.1, a directory is a file that consists of a list of filenames withsome other information, including the location of those files. Technically, a location of a fileis given as an index into another table called the inode list. An inode for a file tells where allthe information about that file, except its file name is kept. Thus, an entry in a directoryabout a file contains only two pieces: a file name and an index number.

However, the format and precise contents of a directory vary with system. Thus, wehave to specify a standard interface to deal with such a variety by dividing the directorylisting task into two pieces. The outer level defines a structure, Direnet, and three routinesopendir, readdir and closedir to provide system-independent access to the file name andthe associated inode number in a directory entry. We will write fsize with this interface inmind.

Then, we will show how to implement such a structure and routines on the directorystructures of Version 7 and System V Unix.

The advantage for such an approach is that, when given a different implementation, weonly need to revise the implementation of such an interface, rather than changing everything.

8.2.1 Given a standard interface...

The Dirent structure contains the name of a file and its associated inode index number.The declaration of this structure and other routines are collected in a file dirent.h as thestandard interface we can work with.

#define NAME_MAX 14 /* longest file name */

typedef struct } /* portable directory structure */

long ino;

char name[NAME_MAX];

} Dirent;

typedef struct { /* a minimal directory structure with no buffer */

int fd; /* file descriptor for a directory */

Dirent d; /* the Dirent for this directory */

} DIR;

DIR *opendir(char *dirname);

Dirent *readdir(DIR *dfd);

void closedir(DIR *dfd);

The system call stat takes a file name and returns all of the information in the inodefor that file, or -1 if there is an error.

48

char *name;

struct stat stbuf;

int stat(char *, struct stat *);

With the above declaration, the way to call it is certainly the following:

stat(name, &stbuf);

which fills the structure stbuf with the inode information for the file with its name beingname. The structure stat, as declared in <sys/stat.h> typically contains the followinginformation:

struct stat {/*inode information returned by stat */

dev_t st_dev; /* divide of inode */

ino_t st_ino; /* inode number */

short st_mode; /* mode bits */

short st_nlinks; /* number of links to file */

short st_uid; /* owner’s user id */

short st_gid; /* owner’s group id */

dev_t st_rdev; /* for special files */

off_t st_size; /* file size i characters */

time_t st_atime; /* time last accessed */

time_t st_mtime; /* time last modified */

time_t st_ctime /* time inode last changed */

};

Now, we have some ideas as where the information for file Properties is kept; and also howdoes the OS know who has what access rights to a file.

The types dev_t and ino_t are defined in <sys/types.h> , which should be includedas will.

The st_mode contains a bunch of flags to further describe the nature of the file in question.The following is just a part of such description, which is also defined in <sys/types.h> .

#define S_IFMT 0160000 /* type of file */

#define S_IFDIR 0040000 /* directory */

#define S_IFCHR 0020000 /* character special */

#define S_IFBLK 0060000 /* block special */

#define S_IFREG 0100000 /* regular */

/* ... */

We are now ready to write the program fsize. If the mode st_mode obtained tells usthat it is a file, then its size st_size can be immediately printed. Otherwise, we have toprocess that directory one file at a time. The process can be recursive since such a directorymight contain subdirectory. Let’s look at it.

49

#include <stdio.h>

#include <string.h>

#include "syscalls.h"

#include <fcntl.h>

#include <sys/stat.h>

#include "dirent.h"

void fsize(char *);

main(int argc, char *argv[]){

if (argc==1)

fsize(".");

else

while (--argc> 0)

fsize(*++argv);

return 0;

}

The function fsize prints the size of the file. If the file is a directory, it calls a functiondirwalk to process all the files within such a directory.

int stat(char *, struct stat *);

void dirwalk(char *, void (*fcn)(char *));

void fsize(char *name){

struct stat stbuf;

if(stat(name, &stbuf)==-1){

fprintf(stderr, "fsize: can’t access %s\n", name);

return;

}

if(stbuf.st_mode & S_IFMT)==S_IFDIR)

/* check the st_mode structure */

dirwalk(name, sfize);

/* go ahead with the guts first */

printf("%8ld %s\n", stbuf.st-size, name);

/* prints out information for this directory file */

}

The function dirwalk applies a function, as given in terms of a pointer, to each file (subdirectory)in a directory. The general flow is clear: it opens the directory, goes through all the fileswithin with the function, then closes the directory and returns. To find out the size of a file,it calls fsize indirectly.

50

#define MAX_PATH 1024

void dirwalk(char *dir, void (*fcn)(char *)){

char name[MAX_PATH];

Dirent *dp; /* portable interface */

DIR *dfd;

/* calls opendir to fill the fstat structure */

if((dfd=opendir(dir))==NULL){

fprintf(stderr, "fsize: can’t access %s\n", name);

return;

}

while((dp=readdir(dfd))!=NULL){

if(strcmp(dp-> name, ".")==0||strcmp(dp-> name, "..")==0)

continue; /* skip itself and parent, see Section 2.1 */

if(strlen(dir)+strlen(dp-> name)+2> sizeof(name))

/* dir/dp-> name/ contains more than MAX_PATH characters /*

fprintf(stderr, "dirwalk: name %s %s too long\n", dir, dp-> name);

else {

sprintf(name, "%s%s", dir, dp-> name);

/* fsize is applied in dirwalk in fsize declaration */

(*fcn)(name);

}

}

/* Done when no more files are left */

fclose(dfd);

}

8.2.2 ... and an implementation of the interface

We are done with the standard interface, and now have to get down to the next level toprovide a minimal implementation for this standard interface.

The following one is for Version 7 and System V Unix, two of the more popular versionsof Unix, which uses the following directory information as declared in <sys/dir.h> .

/* See Chapter 4 notes

#ifndef DIRSIZE

#define DIRSIZ 14

#endif

struct direct {/* directory entry */

ino_t d_ino; /* inode number */

char d_name[DIRSIZ]; /* long name does not have ’\0’ */

51

}

It is clear that in those two versions, the names are rather short. For some other versions,much longer names are allowed which could lead to a more complicated directory structure.

The type ino_t as occurred in the above direct and stat structures is a typedef thatdescribes the index into the inode list. We regularly implement it as a unsigned short

type, but it may change with a different system.Now, we will check out the three basic routines: opendir, readdir and closedir.

The routine opendir opens the directory as a file, verifies its status as a directory,allocates a directory structure for it, and then records the information.

/* fstat is similar to stat, but applies to a file descriptor

rather than a file name */

int fstat(int fd, struct state *);

DIR *openfir(char *dirname){

int fd;

struct stat stbuf;

/* DIR is part of the standard interface */

DIR *dp;

if((fd=open(dirname, O_RDONLY, 0))==-1 ||fstat(fd, &stbuf)==-1

||(stbuf.st_mode & S_IFMT)!=S_IMDIR ||(dp=(DIR*) malloc(sizeof(DIR)))==NULL)

return NULL;

dp-> fd=fd;

return dp;

}

The routine closedir simply closes a directory opened by an invocation of opendir.

void closedir(DIR *dp){

if(dp){

/* for close system call, see Section 6.3 of this lab notes */

close(dp-> fd);

/* for the free function, see the address arithmetic unit in Chapter 5 notes */

free(dp);

}

}

The important(:-)) routine readdir uses the read system call to reach each directoryentry. If a directory slot is not currently in use, the inode number is 0, and this position isskipped. Otherwise, the inode number and the associated name will be placed in a static

structure as called for the standard interface, then return a pointer to such a structure.

52

#include <sys/dir.h>

Dirent *readdir(DIR *dp){

struct direct dirbuf;

static Dirent d;

while(read(dp-fd, (char *)&dirbuf, sizeof(dirbuf))==sizeof(difbuf)){

if(dirbuf.d_info==0) /* slot not in use */

continue;

d.ino=dirbuf.d_ino;

strcpy(d.name, dirbuf.d_name, DIRSIZ);

d.name[DIRSIZ]=’\0’;

return &d;

}

return NULL

}

8.2.3 A little summary

The function fsize is one of those programs that they are not “system programs” in thesense that they are not exclusively written for the system, but they merely make use of theinformation provided by the system, through the stat structure in this case. For such pro-grams, all the system information should be included as a head file, such as the sys/stat.h

file, but not embedded into these programs. There are at least two advantages for this ap-proach: one is to share all such information consistently among all the files that have a needto use them; and the other is to remove as much redundancy as possible.

The division of the design in terms of a standard interface and a system dependentimplementation is also an important point since this makes the fsize function specificsystem independent.

Another important point for this example is that it demonstrates how to use C, a high-level programming language, to write programs at a system level.

It also demonstrates a few applications of bitwise operators, space allocations, etc..

References

[1] Anderson, P., Just Enough Unix (Fourth Ed.), McGraw-Hill, Boson, MA, 2003.

[2] Kernighan, B., and Ritche, D., The C Programming Language (Second Ed.), PrenticeHall, Englewood Cliffs, NJ, 1988.

[3] O’Kane, J. M., A Gentel Introduction to ROS. Available athttps://www.cse.sc.edu/~jokane/agitr/

53