Intro to Linux Systems Administration

Systems AdministrationAdministering the system?Keep the system up in a consistent stateMonitor performanceBabysit users, make changes on their behalfInstall, configure, upgrade, maintainBackup, restore, disaster recovery

SysadminsSystem administration handled by various peopleFull time dedicated sysadmins on siteRemote servicesGeneric IT personnelThat user that seems to know what theyre doingCan be a skill set central to a career path, or a means to an end

Privilege HierarchyWant to divide system privilege by accountFirst step is file level permissionsDefault permissions limit end users in what configuration files they can read and which programs they can runNext level is within system programsLimit certain functions to only users with elevated privileges

The SuperuserBy default, one account has elevated privileges to issue any command, access any file, and perform every functionSuperuser, a.k.a. rootTechnically, can change to anything but dontUser and group number 0

The Superuser, contMust limit use of rootInexperienced users can cause serious harmUse of root for non-privileged tasks unnecessary and can be open to attackSecurity and privacy violations root can look at anyones filesLimit what root can do remotelyEnsure a strong password

Superuser PrivilegesWhat usually works best is short periods of superuser privilege, only when necessaryObtain privileges, complete task, relenquish privilegesMost common ways are su and sudoCan also use the setuid/setgid method (Ch. 4), but not recommended

suShort for substitute or switch userSyntax: su [options] [username]If username is omitted, root is assumedAfter issuing command, prompted for that users passwordA new shell opened with the privileges of that userOnce done issuing commands, must type exit

sudoAllows you to issue a single command as another userSyntax: sudo [options] [-u user] commandAgain, if no user specified, root assumedNew shell opened with users privilegesSpecified command executedShell exited

sudoersMust configure a user to run commands as another user when using sudoPermissions stored in /etc/sudoersUse utility visudo to edit this file (run as root)Permissions granted to users or groups, to certain commands or all, and with or without password being required

Other permissions modelsSome Linux distributions such as Ubuntu obscure away the root account altogetherBy default the end user doesnt know the root passwordCant login as rootCant suMust rely on sudo (and the graphical gksudo) to obtain privilege, along with Unlock functions in GUI

System OperationBooting the systemRunlevelsModesShutting down the system

Booting the SystemPower on, POST, hardware initializationBoot device selected by BIOS/user interactionMaster boot record of boot device readInitializes the bootloaderlilo (LInux LOader)grub (GRand Unified Bootloader)

Booting, contBoot loader selects and loads an OS kernelKernel stored as an compiled image fileKernel loads modules for hardware and software functionsInterrupts, device management, memory management, pagingLast thing kernel does is call init

initFirst non-kernel code loadedProcess number 1Acts as parent to all other processes on systemHandles starting services and programsBased on runlevel, runs the appropriate scripts

RunlevelsA set of defined system states that init can bring the system into (varies on distro)0: Halt/shutdown1: Single user mode2: Multiuser mode3: Multiuser mode with networking4: Not used5: Multiuser mode with networking and GUI6: Reboot

Runlevels, contOn boot, init checks /etc/inittab to see what runlevel to bring system toTo change runlevel after boottelinit runlevelshutdown/halt/rebootAny time the runlevel changes, init consults a set of scripts to determine what to stop/start

ScriptsInit works with run command (rc) scriptsFound in /etc/rc.dAll scripts housed in /etc/rc.d/init.dEach script takes a parameter for changing operation (start/stop/halt/reboot)Each runlevel has its own directory/etc/rc.d/rcN.d

Scripts, contIn each runlevel directory, there are symbolic links to scripts in /etc/rc.d/init.dThe name of the link is crucialStarting with S means start in this runlevelStarting with K means kill in this runlevelAfter S/K, there is an order numberStart ascendingKill descending

NotesWhat weve described is the traditional Linux init/boot processDifferent distros do things differentlylaunchd in Mac OS XUpstart in Ubuntu LinuxInitng in Debian, Gentoo, othersThe classic init is called System V init

Single User ModeRunlevel 1Console only no terminalsVery minimal environmentSome filesystems might not be mountedMaintenance of filesystemsFixing configuration errorsDisaster recovery

Multiuser ModeRunlevels 2-5Runlevel 2 allows terminal loginsRunlevel 3 allows remote terminal loginsRunlevel 5 enable X11 graphical environmentRunlevels 3 and 5 are the most common levels for day-to-day operations

Shutting Down the SystemSyntax: shutdown [options] time [message]Time: XX:XX or +X or NOW-k: dont really shutdown, just send message-r: reboot-h: halt-c: cancel a shutdownhalt: calls shutdown hreboot: calls shutdown -r

SchedulingLinux systems uses the Cron system for time-based job schedulingAllows users to schedule jobs to runAllows sysadmins to run jobs and batch processesDifferent distros implement the structures differentlyMost use /etc/crontab as primary set of instructionsSometimes other files are used, like /var/spool/cron/*

crontabEach line schedules a jobSyntax: * * * * * commandFirst field is minutes (0-59)Second field is hours (0-23)Third is day of the month (1-31)Fourth is month of year (1-12)Fifth is day of week (0-6, starting with Sun)

Filesystem ManagementA Linux installation can be comprised of many different filesystemsEach filesystem (except for swap) is connected to the filesystem hierarchy at a specific point in the treeThis is referred to as the mount pointA sysadmin uses mount, umount and /etc/fstab to manage these mounts

mountSyntax (most commonly): mount t type device directoryAssociates a device (partition, CD-ROM, etc) formatted with a particular type of filesystem with a specified directory in the hierarchyRequires root privileges to mount in most casesmount with no arguments displays list of mounted filesystems

umountSyntax: umount directory | deviceRemoves that associationCannot umount if device is still being accessed (i.e. open files)Again, most likely requires root privileges

fstabFor filesystems that should be mounted on boot every time, put them in /etc/fstabBasically a tab delimited file that contains the command line parameters youd give to mountDeviceMount point (directory)FS typeOptions (Readonly, attributes, etc)

Creating New FilesystemsFirst use fdisk device to create a partitionSimilar in function to old fdisk from DOSUse ? to display commands, p to display partition infoOnce partition created, must be formattedmkfs t type filesystemOnce formatted, you can mount it

Filesystem IntegrityFilesystem problems? Corrupt files? Forced into single user mode to fix errors?fsckSyntax: fsck [options] t type filesystemAgain, usually need root permissionsAlso, filesystem should NOT be mounted while running fsck can cause damage

Monitoring Disk Usagedu disk usage on files and directoriesdf reports filesystem utilizationlsof list open file handlesquota configure and display user quotasquotactlquotacheckquotaonedquota

Installing SoftwareThe open source movement has provided an enormous volume of freely available programsTwo primary methods of installing programsBy sourceBy package manager

Installing by SourceDownload source codeUsually comes in a compressed tar archive (.tar.gz or similar)Extract source codeConfigure the installation (usually ./configure)Then compile (make)Then copy into filesystem (make install)

Package ManagersThere are a wide variety of package managers available for different Linux distributionsIn turn, there are several different types of packages available for each of these managersPackages are an archived version of the source codeOften tailored to a specific architecture or distribution

RPMRed Hat Package ManagerPackage format and manager created by Red Hat developersUsed widely by Red Hat, Red Hat-based distros, and many othersSystem maintains a local RPM database to maintain consistency and track installs

RPM, contMany different utilities for managing RPMsrpm: command line package manager for installing/removing/configuring packagesup2date: command line package manager that fetches packages from internet and resolves dependenciesyum, yast: similar to up2dateMany GUI frontends available to these utilities

debDebian package formatUsed in Debian Linux and its derivatives such as Ubuntu and KnoppixContains compressed binary data and metadataAgain, usually specific to a distro and an architecture

deb contdpkg: Debian package manager, for installing/removing/configuring packagesapt: Advanced Package Tool, for installing and configuring packages from online sources. Also does dependency resolutionAgain, graphical front ends available for each of these

User AdministrationUser configuration stored in /etc/passwdFile got its name because it originally contained passwords as wellSecurity problem too many processes need to read passwdA shadow file used now instead (more in a sec)Each line contains info for one user

passwdjsmith:x:1001:1001:Joe Smith,Rm27,(234)555-8910,(234)555-0044,email:/home/jsmith:/bin/bashFirst field is usernameSecond was password now a dummy charThird is userid (uid)Fourth is groupid (gid)Fifth is GECOS fieldFull name, contact infoGen. Elec. Comprehensive OSSixth is users home directorySeventh is users default shell

passwd, contOriginally passwd contained a users password informationHow it worksUser picks a passwordA random number is generated (called the salt)The salt and the password is passed into a hash function (a one-way cryptographic algorithm)The salt and result are stored in ASCII

passwd, contProblem user-level programs need to read passwdGet user name, locationHome directory, shellSo passwd was world readableSo anyone on system could see a users salted hashIts encrypted whats the big deal???

passwd, contOriginal salt was 12-bit ... 4096 possibilitiesMany early users used bad passwordsDictionary wordsEven with 1970s computing, it wouldnt take very long to try all combinations of salts and passwords through the hash functionJust wait for a matchBrute force crack

shadowWasnt acceptable to have passwd world readable if it contained hashesSo salted hashes moved to a new file/etc/shadowFormat similar to passwd, one user per lineReadable only by root

shadow, contjsmith:$1$CzzxUSse$bKJL9wAns39vlxQlBZ8wd/:13744:0:99999:7:::First field is usernameSecond is the salted hash or account statusNP or ! or null for blank passwordLK or * for locked/disabled account!! for account with expired passwordThird is days since last password changeMeasured from epoch (midnight UTC 1/1/1970)

shadow, contFourth is days until password is eligible to be changedFifth is days before change is requiredSixth is days before expiration to warnSeventh is days before account expiresEighth is days since epoch when account expiresNinth is unused/reserved

Adding UsersIf you really wanted to, edit /etc/passwd by handSome distributions have graphical or simplified ways to add usersMost widely available however is command line utility useradd

Adding Users, contSyntax: useradd [options] [-g group] [-d home] \ [-s shell] username-g to define users initial group-d to define users home directory-s to define users default shellOther options for expiration, using defaults, etc

Deleting UsersAgain, could just hack /etc/passwdMore elegant:Syntax: userdel [-r] username-r to delete home directory and its contents

Modifying UsersSyntax: usermod [options] usernameOptions are pretty much identical to those of useraddAlso, -l to change the users login nameAnd G to list additional groups to add user to

Group ManagementGroup info housed in /etc/groupSimilar to user managementgroupaddgroupdelgroupmod

Daemons as UsersFor the most part, Linux daemons (services) each run as a unique user accountProvides additional security by segregating processes and filesRunning daemons as root usually a bad ideaAccounts usually created automatically and assigned passwordsUsually disabled from logging into system

NetworkingLinux is a powerful networking operating systemMuch of it developed in tandem with the InternetAbility to work as a client, server, or network deviceProxies, firewalls, routers, bridges, etc

Networking, contOverall networking usually governed by /etc/rc.d/init.d/networkInvoked in runlevels 3 and 5 usuallyNetwork device/interface configurations in either /etc/sysconfig/networking or in /etc/sysconfig/network-scriptsCan either edit manually, or use utilities to manage

ifconfigDisplays or alters network device configsSyntax: ifconfig interface [options]With no options, shows interfaces configIf interface omitted as well, show all configsOptions include flags, IP address, subnet mask, etc

routeDisplay or change routingIn simple configurations, mostly used to set default gatewaySyntax: route [options] [add/delete] [target]With no arguments, show route table

hostnameUsed to set/display the computers network nameDepending on what protocols your network uses, may also need to look atdomainnamednsdomainnameEspecially important for Internet-accessible systemsCan be defined in /etc/sysconfig/network

InterfacesBy default, wired ethernet interfaces are found as ethX, with X starting at 0These are aliases to the actual physical adapter and driverTo enable an interface:ifup interfaceTo disable an interface:ifdown interface

Interfaces, contOther types of interfaces existppp, slip, atm, etcManagement of them work similarlyWireless interfaces a bit differentUse iwconfig to manage these and display infoHas the additional options for frequency, encryption, channel, passphrases, etc

NetworkingAs with most things, GUI tools availableSimilar to TCP/IP configuration in WindowsMore advanced operations (bridging, NAT/IP Masquerading, advanced routing) take a little more configurationDefault firewall software is iptables or ipchains

Network SharesSamba SMB/CIFSCUPSNFS

Kernel ModificationVast majority of Linux kernel releases incredibly stableNew features/improvementsBug fixesModules vs. in kernelWe need to recompile

Kernel Mods, contIf we just want to upgrade to a newer kernel release, there are a couple of optionsCan download and install new kernel packages (RPM, deb, etc)Pre-compiled, and most package managers do all the workOr the manual way Necessary to do any real customization

Kernel CompilationFirst, need to get kernel source codewww.kernel.orgCurrent mainline branch is 2.6For legacy systems/apps, 2.4 is still availableUsually a tar.gz or tar.bz2Copy to either a temp location, or maybe /usr/src/kernel/

Kernel Compilation, contOnce you have the compressed archive, uncompress and extract contentsShould make a directory named after the kernel releasei.e. linux-2.6.31.6/Now go into that directoryShould see lots of directories for different aspects of the system, and a Makefile

Kernel Compilation, contNow we need to configure kernelSelect optionsChoose which items should be modules vs. in kernel itselfTo import in the previous system configmake oldconfigThe config is stored in the .config file

Kernel Compilation, contWant to configure from scratch? Or further customize?A few different methodsmake menuconfig (ncurses)make xconfig (X11 Qt)make gconfig (X11 Gtk)All basically do the same thing make selections

Kernel Compilation, contOnce youve done the config and saved it, time to compilemakeWill take a whileLots of info will scroll byDont worry about warnings, its coolErrors would be bad though

Kernel Compilation, contOnce kernel itself is compiled, must compile the kernel modulesmake modulesOnce thats done, we need to install the modules into the correct location in the filesystemmake modules_install

Kernel Compilation, contNow we need to install the kernel into the right spotmake installThis moves three things to /bootThe system map (symbol lookup in memory)The configThe kernel image (vmlinuz)vm = virtual memory support (from UNIX days)z = compressed

Kernel CompilationNow we have the kernel in placeBut we need the info necessary to launch initWe need an initial filesystem loaded so that init has what is necessary to load devices and other filesystems (including /)So we use a temporary, memory contained filesystem a RAM disk

Kernel Compilation, contSo we need to create an initrd a RAM Disk for init to work with before the real filesystems is mountedSo go to /bootmkinitrd o initrd.img-Makes an image of the necessary filesystem components for that version of the kernel

Kernel Compilation, contNow all the pieces are in placeOne last step tell the bootloader about itEdit /boot/grub/menu.lstBasically just copy the block from the current running kernel, change the version info, and youre doneIn most cases, you can usually instead just issue update-grub, but should still check

Kernel Compilation, contExample grub block

title Red Hat Enterprise Linux ES (2.6.9-5.ELsmp)root (hd0,0) kernel /boot/vmlinuz root=/dev/hdb1 ro initrd /boot/initrd.img-2.6.25savedefaultboot

Kernel Compilation, contNow you can reboot and try it outCheck the grub menu for the new kernel you installed and select itSystem should boot fine and everything should workPanic? Reboot, select old kernel, boot into itRetrace your steps, debug kernel, etc