2015. 09. 24.

Project 2.“Linux Fundamental”

procfs

By Dong Jae Shin

Computer Engineering Research Labora-tory

Introduction to Projects Project 1: Setting up Environment for Project Project 2: Linux Fundamental Project 3: System Call and Synchronization Project 4: File System Project 5: Device Driver for Embedded H/W

Especially, including “BeagleBoard Development”

2

Computer Engineering Research Labora-tory

Contents Follow-on Tasks

1. Linux System Administration 2. File System & Software Management 3. Build your own Linux Kernel

Project Tasks Write a System Monitoring Tools

1. Analyze the given skeleton program (10%) 2. Traverse Process tasklist (20%) 3. per Process Memory Usage (20%) 4. per Process I/O Usage (20%) 5. Process CPU Utilization (10% point Bonus)

Report (30%)------------------------------------------------------ Max. 100%

Computer Engineering Research Labora-tory

Account Management Add user

# useradd <user_name> Delete user

# userdel <user_name> Settings per user

/etc/passwd Change or create user password

# passwd # passwd username

4

Computer Engineering Research Labora-tory

File System Management Add a Virtual Disk

Virtual Machine Settings -> Add -> Hard Disk -> SCSI -> Create a new virtual disk -> 20GB

You need 20GB of free space in your HDD(or SSD)

5

Computer Engineering Research Labora-tory

File System Management Disk initialization

Format : Create a empty disk with underlying file system

File system How store and retrieve data in the storage as a logical

unit(file and directory) File system manages your files in their own structure

We’ll learn file system in our lectures later ex) ext2, ext3, ext4, FAT, NTFS

Commands mkfs.XXX : format with XXX file system

We will use ext4

6

Computer Engineering Research Labora-tory

File System Management Format disk we added

# ls /dev/sd*

/dev/sdb is our new disk # mkfs.ext4 /dev/sdb

Mount new disk # cd /root && mkdir mnt # mount /dev/sdb ./mnt

Check mounted disk partition # df -hT

Unmount disk # umount /dev/sdb

we can also use directory name # umount /root/mnt

7

# gnome-disks

Computer Engineering Research Labora-tory

Software Install and Unin-stall

Ubuntu use “Advanced Packaging Tool” Install : apt-get install package-name

ex) # apt-get install vim Uninstall : apt-get remove package-name Search : apt-cache search search-string

ex) # apt-cache search gcc

Redhat distribution family use “Yellowdog Updater Modified” # yum install vim # yum erase vim # yum search gcc

8

Computer Engineering Research Labora-tory

Kernel Kernel

Core of Operating System

9

User ProgramUser Interface

System CallsMemory Man-age-ment

Process Man-age-ment

I/OFile Sys-

temNetwork Security

Device Drivers

Hardware

User Space

Kernel Space

Computer Engineering Research Labora-tory

Linux Distributions Linux Distributions

Combine & Maintain features

10

LinuxKernel

System Software- Compiler- Libraries- User Interface

(GUI, CLI)

User Pro-gram

- Office- Data-

base- Browse

r- Server

Pro-gramLibraries

&Android Runtime

Apps- Camera- Browser- SMS

Applica-tion

Frame-work

Computer Engineering Research Labora-tory

Linux Kernel Download Download Linux Kernel Source Code

https://www.kernel.org/

or http://core.kaist.ac.kr/~EE516/resources/linux-3.18.21.tar.xz

11

Computer Engineering Research Labora-tory

Compile Linux Kernel change to root account

# su - root Install libraries for compile

# sudo apt-get install build-essential libncurses5-dev

Unzip # cd /root/mnt # tar -xvf linux-3.18.21.tar.gz # cd linux-3.18.21

Copy original configuration file # cp /boot/config-3.19.0-25-generic .config ./

12

Kernel compile needs large disk space about

11GB.We can use new disk we

already mounted.

Computer Engineering Research Labora-tory

Compile Linux Kernel Kernel build configuration

# make menuconfig

just save & exit (we already copy .config file)

13

Computer Engineering Research Labora-tory

Compile Linux Kernel Linux Kernel Build

Check your # of cores # cat /proc/cpuinfo |grep processor

Build with multiple processors if you have 4 processors # make -j4

It takes long time

Copy compiled kernel & modules # make modules_install # make install

Check /boot directory

14

Computer Engineering Research Labora-tory

Booting with new Kernel # reboot Keep “shift key down” while rebooting! Boot menu -> Advanced options -> Select 3.18.21

15

Check your new kernel # uname -a

Compile Time of Kernel

Computer Engineering Research Labora-tory

(1)Kernel Debugging Print Kernel-level Message

printk Kernel version of printf usage) printk(“%s %d %lu”, str, i, j );

Kernel Log Message tail -f /var/log/kern.log

See also (optional) for advanced debugging http://lwn.net/images/pdf/LDD3/ch04.pdf

16

procfs

Computer Engineering Research Labora-tory

(2)/proc File System /proc

a special file system which displays the present of the system

pseudo and virtual file system which resides in the RAM

provides a method of communication between kernel space and user space

Various information provided by proc https://

www.kernel.org/doc/Documentation/filesystems/proc.txt

17

procfs

Computer Engineering Research Labora-tory

(3)/proc File System CPU Information

cat /proc/cpuinfo

18

Main Memory Infor-mation cat /proc/meminfo

Kernel Debugging

Computer Engineering Research Labora-tory

(4)Create a proc entry Download Skeleton Files

# wget http://core.kaist.ac.kr/~EE516/Projects/Project2/Makefile

# wget http://core.kaist.ac.kr/~EE516/Projects/Project2/proc_sample.c

Compile # make

Useful ref) http://linux.die.net/lkmpg/c708.html

19

procfs

Computer Engineering Research Labora-tory

(5)Test proc entry Kernel Log

# tail -f /var/log/kern.log Insert Module

insmod proc_sample.ko Write Proc File

# echo blahblah > /proc/proc_sample Read Proc File

# cat /proc/proc_sample Remove Module

# rmmod proc_sample

20

procfs

Computer Engineering Research Labora-tory

Project Tasks Make a System Monitoring Tools

System monitoring is an important job especially on the embed-ded system

Low computing power, Less memory, Limited resources Monitoring tools

top, ps, netstat, gnome-system-monitor

Tasks 1. Analyze the given skeleton of promon.c.(10%) 2. Traverse Process tasklist (20%) 3. per Process Memory Usage (20%) 4. per Process I/O Usage (20%) 5. Process CPU Utilization (10% point Bonus Task)

Report (30%)

21

Project Tasks

Computer Engineering Research Labora-tory

Task 1 Analyze the given skeleton of

proc_sample.c Find the functions that you do not know

and explain the operations of those func-tions referring to references and Google.

22

Computer Engineering Research Labora-tory

Task2. Traverse Task List Every Linux Tasks are managed by doubly linked list

23

Print every task’s PID and Process Name in your proc file system Textbook Chapter 3. Understanding the Linux Kernel, 3rd

free e-book version : http://gauss.ececs.uc.edu/Courses/c4022/code/memory/understanding.pdf

Project Tasks

Computer Engineering Research Labora-tory

task_struct task_struct (=Process Descriptor)

24

KERNEL/include/linux/sched.h

Useful materials : Textbook Chapter 3. of ULK

Project Tasks

Computer Engineering Research Labora-tory

task_struct

25

………

Computer Engineering Research Labora-tory

Task3. Memory Usage Memory Mapping

RSS : Resident Set Size VIRT : Virtual Memory

Size

Print every task’s VIRT and RSS Mem-ory Size

26

VIRT RSS

RSSVIRT

Project Tasks

Hint) KERNEL/include/linux/mm_types.h struct mm_struct { … unsigned long total_vm; struct mm_rss_stat rss_stat; …}

Computer Engineering Research Labora-tory

Task3. Memory Usage

27

Project Tasks

task_struct

…*active_m

m…

mm_struct

…total_vm

…rss_stat

include/linux/mm_types.h

VIRT Memory

rss_stat.count[MM_FILEPAGES]rss_stat.count[MM_ANONPAGES]

RSS Memory

MM_FILEPAGES : Type of File Mapped PageMM_ANONPAGES : Type of Anonymous Page (Stack, Heap …)

include/linux/sched.h

Computer Engineering Research Labora-tory

Task4. Process I/O Stat I/O Stat is stored in struct task_io_accounting

include/linux/task_io_accounting.h

Hint)

# cat /proc/PID/io rchar : read bytes by process wchar : written bytes by process syscr : # of read system calls syscw : # of write system calls read_bytes : read bytes from disk write_bytes : written bytes to disk

28

write_bytes

read_bytes

Disk Cache in DRAM

Process

wcharrchar

Project Tasks

Computer Engineering Research Labora-tory

Task4. Process I/O Stat Hint 1)

fs/proc/base.c print function of /proc/pid/io

Hint 2) include/linux/task_io_accounting_ops.h

task_io_accounting_add() function

Hint 3) task_struct->ioac task_struct->signal->ioac for each thread

task_struct->ioac

29

Project Tasks

write_bytes

read_bytes

Disk Cache in DRAM

Process

wcharrchar

Computer Engineering Research Labora-tory

Task5. CPU Utilization “top” command

shows the CPU utilization per process

30

Challenging Task (Bonus Point 10%) Kernel manages CPU ticks consumed by each

process in struct task_cputime in include/linux/sched.h utime + stime

5. Project Tasks

Computer Engineering Research Labora-tory

Example of Output

31

5. Project Tasks

Computer Engineering Research Labora-tory

Check your results Check the correctness of your results Task 2.

# ps -ef Task 3.

# top -b -n 1 Task 4.

Browser is a good test case (# firefox) Get pid

ps -ef |grep firefox # cat /proc/PID/io

Task 5. # top

Project Tasks

Computer Engineering Research Labora-tory

Problems Problem1.

Describe similarity and difference Process and LWP(Lightweight Process) in the Linux

from the view point of task_struct, PID manage-ment, data sharing and scheduling

Problem2. Describe why we use copy_to_user(),

copy_from_user() functions in the proc file system

33

Computer Engineering Research Labora-tory

Submission Contents

Source Code Report

Project Tasks1~5 Key point of your Source Code Final Screenshots Page Limit : about 5 pages for Project Tasks1~5. (except fig-

ures)

Submission Due Date : Oct. 8, PM 23:59 Delay Penalty

10%/day (AM 00:00) E-mail : djshin@core.kaist.ac.kr [EE516 Project2] student_number.zip

(various compression format is allowed)