Presentation on the Ubuntu Operating System

Colorado Technical University

CS 340 – Operating Systems

October 2010

Loren Karl Schwappach

Overview

Security Hacking Networking Robustness Interface Basic Commands System Hardening Summary

Introduction History Design Principles Process Management Memory Management Process Communication Deadlock Handling File System

Ubuntu 10.10• Secure, Super-fast, and great-looking• Open source, free, & widely distributed• Suitable for server and desktop use.• Includes > 1000 pieces of software• Ubuntu One Personal Cloud• Linux kernel version 2.6 w/ Gnome 2.16Support for:• Intel x86 (IBM-compatible PC)• AMD64 (Hammer)• PowerPC (iBook, Powerbook, G4, G5)http://www.ubuntu.com/desktop/features

Introduction

• 1991, Linus Torvalds (a Finnish student) writes a open source small, self-contained kernel (Linux) for the 80386 processor. • Version 0.01 - no network support, basic VM subsystem, support for Minix fs and limited device-driver support. [1]• 1994, Version 1.0 - networking (TCP/IP protocols, new file system, SCSI, floppy disks, CD-ROM devices, sound cards, mice, keyboards, floating point emulation, UNIX-style IPC and extended VM subsystem. [1]• 1996, Version 2.0 - PCI support, 80386 CPU virtual 8086 mode , memory management improvements, ISDN, internal kernel threading, automatic module loading and multiprocessor support. [1]• 1999, Version 2.2 - firewalling, routing and traffic management, improved symmetric multiprocessor (SMP) performance and (Acorn, Apple, and NT) disk support. [1]http://en.wikipedia.org/wiki/Linux_kernel

History

Ubuntu - Debian GNU/Linux distribution.• Originally released under name “no-name-yet.com.” • Ubuntu - African word that means “humaneness.”• 2004, 1st publicly released version 4.10http://en.wikipedia.org/wiki/List_of_Ubuntu_releasesVariants include :• Edubuntu (designed for school environments)• Kubuntu (Uses KDE GUI environment)• Mythbuntu (Home Theater TVs)• Ubuntu Studio (Professional AV editing software)• Xubuntu (uses XFCE desktop environment)• 2010, Latest version 10.10 Maverick Meerkat:• Linux Kernel version 2.6.35Software:• OpenOffice, Firefox, Empathy IM, Transmission (BT client), Gimp, games, and more.• Desktop version supports Intel x86 and AMD64 .http://www.ubuntu.com/desktop/why-use-ubuntu

History

History

Linux kernel size as an indicator of complexity

Ubuntu’s Linux kernel (2.6):Multiuser, multitasking system w/ complete set of UNIX-compatible tools. Linux supports the Portable Operating System Interface [for Unix] (POSIX) threading extensions (Pthreads and a subset of the POSIX extensions for real time process control. [1]The Linux system three bodies of code:

Kernel – Maintains OS abstractions, virtual memory and process management. [1]System Libraries – Standard set of functions for applications to interact with the kernel. [1]System utilities – Programs that perform individual, specialized management tasks. [1]

Design Principles

Kernel• Ability to load/unload sections of code on demand.Three components to the module support :• Module management (allows loading modules in mem)• Driver registration (allows modules to tell kernel when new drivers are available)• Conflict-resolution.

Driver Registration tables include the following:• Device Drivers – Character devices (printers, terminals, mice), block devices (disk drives), and network interface devices. [1]• File Systems – Anything that implements Linux’s virtual-file-system call routines. [1]• Network protocols – IPX, firewall packet filtering rules, etc. [1]• Binary format – specifies way for recognizing/loading an executable file. [1]

Design Principles

Linux process properties generally fall into three groups:

Process Identity consists of:Process IDCredentials (associated UID/GIDs)Personality (modifies semantics of sys calls (unique to Linux)).

Process environment (inherited from parent) is composed of:Argument vector (lists command line arguments) Environment vector (list of NAME=VALUE pairs).

Process context (state of program in time) includes:Scheduling context – Information to suspend and restart the process. Includes: process registers, floating point registers, scheduling priority, and process’s kernel stack)Accounting – Information about resource usageFile table – array of pointers to kernel file structuresSignal-handler table – Asynchronous handling of external events.Virtual memory context – describes content of address space

Process Management

Linux kernel 2.6 Scheduling:• 2 Algorithms Used: Time Scheduling and Real Time• Uses FIFO & round robin real time scheduling• Real Time Priorities are from 1-100

Process Management

Linux avoids segmentation and separates physical memory into three zones: ZONE_DMA (lower 16MB of physical memory), ZONE_NORMAL (normally used 16-896 MB), ZONE_HIGHMEM (Not mapped into kernel address space > 896MB). [1]

Page tables keep track of the physical pages of memory that are used by a process, and they map the virtual pages to the physical pages.

Linux kernel 2.6 uses reverse page mapping

Memory Management

Virtual memory manager:• Responsible for maintaining the address space visible to each process.• Creates pages of virtual memory on demand and manages loading of pages to/from disk.• Views a process address space as independent separate regions and as independent separate pages.• Reserves its own internal use, architecture dependant region of virtual address space for every process.

Paging system divided into two sections:• Policy algorithm (decides which pages to write to disk and when to write them)• Paging mechanism (carries out transfers and pages data back into physical memory when needed).

Memory Management

MMU (Memory Management Unit) – allows software to reference physical memory by aliased addresses (using pages and page tables)

Zoned Buddy Allocator – responsible for page allocations. Manages list of physical contiguous pages and maps them to MMU page tables & manages memory zones.

Slab Allocator – Allows flexible memory sizes (Than standard 4kb page) by creating cache objects.

Kernel Threads – Recovery and management of memory (kscand, kswaped, kupdated, bdflush)

Memory Management

Memory Management

VM Page State Machine

High Level Overview of VM System

Signals - Sent by other processes or the kernel to a specific process to indicate various conditions.

Pipes - Unnamed pipes set up by the shell normally with the "|" character to route output from one program to the input of another.

FIFOS - Named pipes operating on the basis of first data in, first data out.

Message queues - Message queues are a mechanism set up to allow one or more processes to write messages that can be read by one or more other processes.

Semaphores - Counters that are used to control access to shared resources. These counters are used as a locking mechanism to prevent more than one process from using the resource at a time.

Shared memory - The mapping of a memory area to be shared by multiple processes.

Process Communication

Deadlock - a condition where one or more executing threads and one or more resources, such that each thread is waiting for one of the resources, but all the resources are already held. [1]

In essence the threads are all waiting for each other, but they will never make any progress toward releasing the resources that they already hold.

Unlike some Operating Systems: “There is no deadlock detection for applications or threads by the Linux kernel”.

Linux requires the use of semaphores (sleeping locks), spin locks (a lock that can be held by at most one thread of execution), and the Big Kernel Lock (BKL) to attempt deadlock prevention.

Deadlock Handling

Linux kernel version 2.6 used by Ubuntu 10.10 introduces:

A new type of lock called a seq lock (sequence lock).

• Works by maintaining a sequence counter..

• Whenever the in question data is written to, a lock is obtained and a sequence number is incremented.

• Prior to and after reading the data, the sequence number is read. If the values are the same, then a write did not begin in the middle of the read.

Deadlock Handling

Linux 2.6 retains UNIX’s standard file-system module. • Allows Linux files to be anything capable of handling the input or output of a stream of data.• Device drivers, interprocess communication channels and network connections all look like files on Linux.• The Linux kernel hides the implementation details of any single file type behind a layer of software called the virtual file system (VFS). [1]

The VFS defines four major types of objects:• inode object – represents an individual file.• file object – represents an open file.• superblock object – represents the entire file system.• dentry object – represents an individual directory entry.

File System

XFS was a file system:Developed by Silicon Graphics and designed to handle files as large as a million terabytes, in 32 bit Linux systems it can handle files up to 16 terabytes.

Ext4 is currently the default choice of Ubuntu 10.10 and performs much better than ext2, ext3, and XFS. A new file system btrfs is also supported in Ubuntu version 10.10 but is not yet stable [9]

Ext4 (forth extended file system) - journaling (keeps track of changes in a journal before changing in main file) Linux file system that became stable in 2008.

Ext4 :• Supports volumes up to 1 Exabyte (1018) and files with sizes up to 16 terabytes (16x1012). Uses extents (range of contiguous physical blocks) to replace the traditional block mapping used by ext2/3, performs pre-allocation of on-disk space, delayed allocation (using an allocate-on-flush technique), breaks the ext3 32,000 subdirectory limit (now 64,000), performs journal check-summing (improved reliability), faster file system checking, multi-block allocation, and improved timestamps.

File System

File System

The Linux security module is closely tied to UNIX security mechanisms.Security concerns can be classified into two groups: Authentication (Ensuring that only authorized individuals have access to files that they are authorized.)Access control (mechanism for validating whether a user has the right to access a particular object and preventing unauthorized access as necessary). [1]

Historically Linux suffered from the same security vulnerabilities (such as eight character passwords) as UNIX. However, a newer security mechanism known as the Pluggable authentication Module (PAM) system based on shared libraries is now available to Linux users. [1]

Security

Every object in a UNIX system under user and group access control has a single UID and GID associated with it. User processes have a single UID but may have more than one GID.

Linux performs access control by assigning objects a protection mask that specifies which access modes (read, write, or execute) can be granted to the user. The only exception is the root UID which is granted automatic access to any object in the system.

Linux allows use of the setuid command to run programs with different user privledges(for example lpr has access to the print queue even if the user does not).

This can be useful but can also pose security concerns for the operating system.

Security

Brute-force (password guessing) attacks are the most common form of attack on any operating system. In Linux, the most common types of services that can be brute-force attacked are: [4]

• Telnet• File Transfer Protocol (FTP)• The “r” commands (rlogin, rsh, and so on)• Secure Shell (ssh)• SNMP community names• Post Office Protocol (POP) and Internet Message Access Protocol (IMAP)• Hypertext Transport Protocol (HTTP/HTTPS)• Concurrent Version System (CVS) and Subversion (SVN)

Hacking

Hydra is one of the most popular and versatile brute force utilities available. However pop.c and SNMPbrute are also popular and can be downloaded at the following locations: [4]

THC – Hydra http://freeworld.thc.org/thc-hydra/pop.c http://packetstormsecurity.org/groups/ADM/ADM-pop.cSNMPbrute http://packetstormsecurity.org/Crackers/snmpbrute-fixedup.c

Hacking

According to Hacking Exposed 6: Network Secrets & Solutions [4] there are some freeware tools that can help prevent brute force attacks. For brevity I will not list the download locations (use Google search). These tools listed in the book follow:

Hacking

cracklib

npasswd

Secure Remote Password

OpenSSH

pam_passwdqc

pam_lockout

Password composition tool.

A replacement for the passwd command.

A new mechanism for performing secure password based authentication and key exchange over any type of network.

A telnet/ftp/rsh/login communication replacement with encryption and RSA authentication.

PAM module for password strength checking.

PAM module for account lockout.

Linux supports the entire standard internet protocols used for most UNIX to UNIX communications as well as many of the protocols native to non-UNIX operating systems.

Internally, networking in the Linux kernel is implemented in three layers of software: The socket interface, protocol drivers, and network-device drivers.

The most important set of protocols in Linux is the TCP/IP protocol suite containing (IP, UDP, TCP, and ICMP) protocols.

Linux also performs firewall management of IP traffic.

Networking

Networking

A comparison of operating system robustness was completed for the 19th International Symposium on Software Reliability Engineering by Shanghai Jiao Tong University [8].

They observe that Windows XP achieves better robustness performance than Ubuntu in their experiment.

Ubuntu had higher restart and abort rates than Windows OSs in general robustness test.

In particular, the robustness of Ubuntu rose drastically from 85.70% to 97.24% in CINT (surpassing Windows Vista in this generic compute intensive operational profile) and 97.37% in CFP.” [8]

Robustness

Robustness

Ubuntu 10.10 uses the GNU Network Object Model Environment (GNOME) graphical user interface. It has the look and appeal of Windows Vista without many of the problems.

It also uses a command line interface.

Interface

Basic Commands

Some Important Directories Found in Ubuntu

Directory Description

/bin Frequently used system binaries

/dev Special drivers for I/O devices

/etc Miscellaneous system administration parameters

/lib Frequently used libraries

/tmp Temporary files once stored here

/usr Contains all user files in this part of the tree

/usr/include System-provided header files

/usr/man On-line manuals

/usr/spool Spooling directories for printers, e-mail, and other daemons

Basic Commands

Some Popular Utility Programs Found in Ubuntu

Command Description

awk A pattern matching language

basename Strip off prefixes or suffixes from a file name

cat Link file(s) and write them to standard output

cc Compile a C program

chmod Change protection mode for file(s)

comm. Print lines common to two sorted files

cp Make a copy of a file

cut Make each column in a document into a separate file

date Print the date and time

diff Print all the differences between two files

echo Print the arguments (used mostly in shell scripts)

find Find all the files meeting a given condition

grep Search file(s) for lines containing a given pattern

head Print the first few lines of file(s)

kill Send a signal to a process

lp Print a file on a printer

ls List files and directories

make Recompile those parts of a large program that have changed

mkdir Make a directory

mv Rename a file or move file(s)

paste Combine multiple files as columns in a single file

pwd Print the working directory

rm Remove file(s)

rmdir Remove one or more directory

sed A stream (i.e., noninteractive) editor

sty Set terminal options such as the characters for line editing

sort Sort a file consisting of ASCII lines

tail Print the last few lines of a file

tr Translate character codes

uniq Delete consecutive identical lines in a file

wc Count characters, words, and lines in a file

Most Linux systems use two boot loaders, the Linux Loader (LILO) or Grand Unified Boot Loader (GRUB). Ubuntu uses GRUB. The boot loader controls your boot image and determines what kernel is booted when the system is started or rebooted. [3]

Both boot loaders are insecure if a hacker has physical system access and Turnbull recommends that users only keep the current and previous versions of the kernel.

Both boot loaders can easily be secured with passwords. To accomplish this Ubuntu’s GRUB do the following: *3+

superman# grubgrub> md5cryptPassword: ********Encrypted: $1$2FXKzQ0$I6k7iy22wB27CrkzdVPe70grub> quit

Now copy the encrypted output and add the password to your grub.conf configuration file. You can also add the --md5 option prior to the password in grub.conf to ensure interaction with GRUB can only occur when you type p and enter the required password.

System Hardening

Most Linux systems use two boot loaders, the Linux Loader (LILO) or Grand Unified Boot Loader (GRUB). Ubuntu uses GRUB. The boot loader controls your boot image and determines what kernel is booted when the system is started or rebooted. [3]

Both boot loaders are insecure if a hacker has physical system access and Turnbull recommends that users only keep the current and previous versions of the kernel.

Both boot loaders can easily be secured with passwords. To accomplish this Ubuntu’s GRUB do the following: *3+

superman# grubgrub> md5cryptPassword: ********Encrypted: $1$2FXKzQ0$I6k7iy22wB27CrkzdVPe70grub> quit

Now copy the encrypted output and add the password to your grub.conf configuration file. You can also add the --md5 option prior to the password in grub.conf to ensure interaction with GRUB can only occur when you type p and enter the required password.

System Hardening

There are also several services that may start at boot, of which many can pose as security risks. Turnbull [3] recommends disabling the following:

anacron, apmd, atd, autofs, cups, gpm, irda (unless used), isdn (unless used), kudzu, lpd, netfs, nfs, nfslock, pcmcia, portmap, rawdevices, rhnsd, snmpd, snmtptrap,winbind, xfs, ypbind.

Delete the following user accounts:adm, desktop, ftp, games, gdm, gnats, gopher, identd, irc, list (if not using

mailman), lp & lpd (if no printing) mailnull (if no Sendmail), news, nfsnobody, nscd, operator, postgres (if no Postgres), proxy, rpc, rpcuser, sync, telnetd, uucp, www-data (if not Web server).

Delete the following group accounts:lp, news, uucp, proxy, postgres, www-data, backup, operator, list, irc, src, gnats,

staff, games, users, telnetd, gdm, telnetd, gopher, ftp, nscd, rpc, rpcuser, nfsnobody, xfs, desktop

System Hardening

Summary

This briefing looked into the history and features of the Ubuntu Operating system as well as exploring the mechanics that make the Ubuntu operating system unique.

This briefing further dug deep into:Design principles (specifically the Kernel, System libraries, and System utilities)Process management (process context includes: scheduling context, accounting, file table, signal-handler table, and virtual memory context)Memory management (physical memory zones, allocation of memory using pages, and memory mapped to the address space)Process communication (signaling, use of semaphores, piping, and shared-memory)Deadlock handling (no detection for applications or threads by the Linux kernel, uses locks for prevention)File system (VFS, inodes, file objects, superblocks, dentry objects, extfs, and ext4) Security (authentication and access control)Networking (socket interface, protocol drivers, and network-device drivers)Program interface (Gnome GUI, and applications)And more.. (Hardening, Robustness, Common Commands, etc..)

Questions?

References

1. Silberschatz A., Galvin P., Gagne G. (2009). Operating System Concepts (pp. 801-843). 8th edition. Hoboken, NJ. John Wiley & Sons.

2. Michael S., (2001). The Linux Codebook: Tips and Techniques for Everyday Use. San Francisco. No Starch Press, Inc.

3. Turnbull J., (2005). Hardening Linux. New York, NY. Springer-Verlag New York, Inc.

4. McClure S., Scambray J., Kurtz G., (2009). Hacking Exposed 6: Network Security Secrets & Solutions (pp. 223-315). New York, NY. McGraw

Hill.

5. Saur K., Grizzard J., Locating x86 Paging Structures In Memory Images. “Digital Investigation” Volume 7 (2010): pages 29-30. SciDirect

Database. Accessed 7 Dec 2010.

6. Lien Y., 4: Operating Systems. Academic Press (2005): pages 355-366. SciDirect Database. Accessed 8 Dec 2010.

7. Narayan S., Shang P., Fan N., Performance Evaluation of IPv4 and IPv6 on Windows Vista and Linux Ubuntu. (2009) International Conference

on Networks Security, Wireless Communications and Trusted Computing. IEEE Database. Accessed 9 Dec 2010.

8. Ju X., Zou H., Operating System Robustness Forcast and Selection. 19th International Symposium on Software Reliability Engineering. IEEE

Database (2008). Accessed 10 Dec 2010.

9. Tozzi C., 2010, Ubuntu 10.10’s New File System. The Var Guy Retrieved 10 Dec 2010 from the website:

http://www.thevarguy.com/2010/08/02/ubuntu-10-10s-new-file-system-btrfs/