linux forensics

Linux ForensicsLinux Forensics

SANTOSH KHADSARE

WindowsWindowsWindowsWindows

Advantages:• Easy to understand.• GUI interface.

Disadvantages:• Less Secure.• More vulnerable.

LinuxLinuxLinuxLinux

Advantages:• High security.• More tools and automation.

Disadvantages:• Hard to understand interface.• Difficult to administrate.

Windows versus LinuxWindows versus LinuxWindows versus LinuxWindows versus Linux

Server Type Windows Linux

Web Server IIS Apache

Database Server Spl Mysql,Postgresql

DNS Server Own Bind

FTP Server Own PROFTPD

Email Server MS exchange Sendmail, Qmail

Before LinuxBefore LinuxBefore LinuxBefore Linux

• In 80’s, Microsoft’s DOS was the dominated OS for PC

• Apple MAC was better, but expensive• UNIX was much better, but much, much more

expensive. Only for minicomputer for commercial applications

• People was looking for a UNIX based system, which is cheaper and can run on PC

• Both DOS, MAC and UNIX were proprietary, i.e., the source code of their kernel is protected

• No modification is possible without paying high license fees

GNU projectGNU projectGNU projectGNU project

– Established in 1984 by Richard Stallman, who believes that software should be free from restrictions against copying or modification in order to make better and efficient computer programs

GNU is a recursive acronym for “GNU's Not Unix”

Aim at developing a complete Unix-like operating system which is free for copying and modification

Companies make their money by maintaining and distributing the software, e.g. optimally packaging the software with different tools (Redhat, Slackware, Mandrake, SuSE, etc)

Stallman built the first free GNU C Compiler in 1991. But still, an OS was yet to be developed

Beginning of LinuxBeginning of LinuxBeginning of LinuxBeginning of Linux

– A famous professor Andrew Tanenbaum developed Minix, a simplified version of UNIX that runs on PC

– Minix is for class teaching only. No intention for commercial use

– In Sept 1991, Linus Torvalds, a second year student of Computer Science at the University of Helsinki, developed the preliminary kernel of Linux, known as Linux version 0.0.1

Beginning of LinuxBeginning of LinuxBeginning of LinuxBeginning of Linux

What is good about usingLinux for forensics?

What is good about usingLinux for forensics?

– Most tools are free of charge .– Most tools are open source (Allows you to

know exactly what the tool is doing).– Environment is very flexible.– When tools are used properly, evidence should

stand up in court.

limitations of using Linux for forensics?


– Most tools are free - you don’t get any tech support and may not be able to call the author in to court if necessary .

– Most tools are open source, which allows those who oppose you in court to scour the code for bugs and try to call the tool into question.

– Environment is very flexible - it is often difficult to decide what is the best way to do something and there are no universally accepted standard procedures.



– Free tools are not as full featured ascommercial packages.

– Tools are significantly more complicated touse than commercial packages and mostinvolve using the command line.

– In many operations, especially when imaging, you can destroy your work or even your original evidence with a typo.

Linux is Growing in PopularityLinux is Growing in Popularity

• Each new version becoming more user friendly.– Disk installation no longer confusing.– Installation interface more intuitive.– Graphical environment becoming much more mature.

• More and more companies are embracing & supporting Linux.– IBM has teams of developers working on it.– Apple’s OS now has a UNIX-like core.– Novell is now in the Linux business.

• More and more devices are now running Linux– Personal Devices: Cell Phones & PDA’s.– Electronics: Video Recorders, MP3 Players.

Why LinuxWhy Linux• Linux is a powerful operating system.

– Many web sites use Linux as the operating system.– Even Steve Ballmer of Microsoft said Linux has 60% of the server market in 2008.– Tolerant of a range of hardware platforms without special configuration.

• Computer Forensics need to be able to consider server forensics.– Forensic issues can happen on server platforms too.

• Host-Based forensic tools often run on linux platforms.– Free platform– Flexible and reliable– Easier to access low-level interfaces– Good forensic qualities.– Will consider Caine (a Linux live cd) for host-based forensics, which runs The

Forensic Toolkit and Autopsy.

Linux ShellLinux ShellLinux ShellLinux Shell

Shell interprets the command and request service from kernel

Similar to DOS but DOS has only one set of interface while Linux can select different shell

– Bourne Again shell (Bash), TC shell (Tcsh), Z shell (Zsh)

Kernel

Bash, Tcsh, Zsh

ls pwdwhoami

Different shell has similar but different functionality Bash is the default for Linux Graphical user interface of Linux is in fact an application program work on

the shell

GRUBGRUB

• GNU GRUB (short for GNU GRand Unified Bootloader) is a boot loader package from the GNU Project.

• GRUB is the reference implementation of the Multiboot Specification, which provides a user the choice to boot one of multiple operating systems installed on a computer or select a specific kernel configuration available on a particular operating system's partitions.

http://en.wikipedia.org/wiki/Boot_loader

http://en.wikipedia.org/wiki/GNU_Project

http://en.wikipedia.org/wiki/Reference_implementation

http://en.wikipedia.org/wiki/Multiboot_Specification

http://en.wikipedia.org/wiki/Operating_system

http://en.wikipedia.org/wiki/Kernel_(computer_science)

KERNELKERNEL

KERNELKERNEL

• Typically, a kernel (or any comparable center of an operating system) includes an interrupt handler that handles all requests or completed I/O operations that compete for the kernel's services, a scheduler that determines which programs share the kernel's processing time in what order, and a supervisor that actually gives use of the computer to each process when it is scheduled.

• A kernel may also include a manager of the operating system's address spaces in memory or storage, sharing these among all components and other users of the kernel's services. A kernel's services are requested by other parts of the operating system or by application programs through a specified set of program interfaces sometimes known as system calls.

http://searchsoftwarequality.techtarget.com/definition/application-program

KERNELKERNEL

KERNEL VERSIONKERNEL VERSION

• UBUNTU 11.0 running

• What is kernel version?

KERNEL VERSIONKERNEL VERSION

• UBUNTU 11.0 –say company name

• TYPE umane –r to find the kernel .

• UBUNTU 11.0 has 2.6.38 kernel version.

• We can also kind kernel when we start a linux machine. (GRUB)

KernelKernel

KDE and GNOMEKDE and GNOME

KDE and GNOMEKDE and GNOME• KDE and GNOME are two desktop environments

(collection of software that provides certain functionality and a look and feel for operating systems) that run on operating systems that use X Window System (mostly Unix, Linux, Solaris, FreeBSD, and Mac OS X).

• KDE’s main programming language is C++. The main reason for this is that the main functionality of KDE is coded using QT, which is written in C++. It takes approximately 210MBs to install the base system of KDE.

http://www.differencebetween.com/category/technology/it/programming/

KDE and GNOMEKDE and GNOME• GNOME’s main programming language is C, because

the tool kit used to write GNOME is GTK+ and it is written in C. Approximately 180 MB is required to install the base system of GNOME .

After the recent rebranding,“KDE” actually refers to the whole collection of applications including the desktop environment while GNOME refers to a desktop environment alone.

File managementFile management

Directory TreeDirectory Tree

Directory TreeDirectory TreeDirectory TreeDirectory Tree(root)

When you log on the the Linux OS using your username you are automatically located in your home directory.

• /bin System binaries, including the command shell

• /boot Boot-up routines

• /dev Device files for all your peripherals

• /etc System configuration files

• /home User directories

• /lib Shared libraries and modules

• /lost+found Lost-cluster files, recovered from a disk-check

• /mnt Mounted file-systems

• /opt Optional software

•/proc Kernel-processes pseudo file-system

• /root Administrator’s home directory

• /sbin System administration binaries

•/usr User-oriented software

• /var Various other files: mail, spooling and logging

The most important subdirectories The most important subdirectories inside the root directory are: inside the root directory are:

The most important subdirectories The most important subdirectories inside the root directory are: inside the root directory are:

• /bin : Important Linux commands available to the average user. • /boot : The files necessary for the system to boot. Not all Linux

distributions use this one. Fedora does. • /dev : All device drivers. Device drivers are the files that your

Linux system uses to talk to your hardware. For example, there's a file in the /dev directory for your particular make and model of monitor, and all of your Linux computer's communications with the monitor go through that file.

• /etc : System configuration files. • /home : Every user except root gets her own folder in here,

named for her login account. So, the user who logs in with linda has the directory /home/linda, where all of her personal files are kept.

• /lib : System libraries. Libraries are just bunches of programming code that the programs on your system use to get things done.

The most important subdirectories The most important subdirectories inside the root directory are:inside the root directory are:

The most important subdirectories The most important subdirectories inside the root directory are:inside the root directory are:

• /mnt : Mount points. When you temporarily load the contents of a CD-ROM or USB drive, you typically use a special name under /mnt. For example, many distributions (including Fedora) come, by default, with the directory /mnt/cdrom, which is where your CD-ROM drive's contents are made accessible.

• /root : The root user's home directory. • /sbin : Essential commands that are only for the system

administrator. • /tmp : Temporary files and storage space. Don't put anything in here

that you want to keep. Most Linux distributions (including Fedora) are set up to delete any file that's been in this directory longer than three days.

• /usr : Programs and data that can be shared across many systems and don't need to be changed.

• /var : Data that changes constantly (log files that contain information about what's happening on your system, data on its way to the printer, and so on).

Important Subdirectories inside the Important Subdirectories inside the root directory are:root directory are:

Important Subdirectories inside the Important Subdirectories inside the root directory are:root directory are:

• /proc : virtual file sys(imp for forensics). • /etc/passwd: contains all user specific info. • /etc/shadow: stores password.

Home directoryHome directoryHome directoryHome directory

• You can see what your home directory is called by entering

• pwd (print current working directory)

Useful commands:Useful commands:

Command Description

ls List dir and files

dir See directory

Ifconfig See IP address config

Useradd Add new user

Reboot Reboot machine

Su Switch user

Wget To download any file for internet

Top See process activity

Ps Display processes


Command Description

Vmstat Sys activity, hardware and sys info

Uptime Tell how long sys is running

Free Memory usages

Df –h See hard disk space

Cp Copy file or folder

Lostat Average CPU load, disk activity

Sar Collect and report sys activity

Ping Test contv

Netstat Network statistics


Command Description

Iptraf Eal time network statistics

Tcpdump Detailed network traffic analysis

Cat Displays contents of the file

vim Opens text editor

Mount Mounts dir

Cd Change dir

Mkdir Make dir

Rmdir Remove dir

Pwdshow Show present working dir

The PROMPTThe PROMPT

• Once you log into your machine, you are at the prompt. Here you can perform your commands.

• Everything on linux is either a file or a directory.• A file which is executed becomes a process.• Processes can be seen as files too.• Devices, such as scanners and hard drives are

also files.

●Windows uses letters of the alphabet to represent different devices and different hard disk partitions. Under Windows, you need to know what volume (C:, D:,...) a file resides on to select it, the file's physical location is part of it's name.

●In Linux all directories are attached to the root directory, which is identified by a forward-slash, "/". - root.

● For example, below are some second-level directories:

File System

●# - shell command.●# fdisk -l /*list partitions*/●/dev/sda1

/dev – device /sda1 or /hda1

• sd – SATA /*SATA- tech to read/write data*/

• hd – IDE

• a

File System

Sd ..a/b/c/d.....1/2/3.....

• a – primary master

• b – primary slave

• c – secondary master

• d – secondary slave

• 1/2/3... – first/second/third partition

File System

●# fdisk /dev/sda /*’m’ for help*/●(if type ‘l’ it will list all available file sys with their

Id e.g. Windows -7 and Linux -83)●(‘q’ to quit)●/dev/sda1 - sys reserve ●/dev/sda2 - is C:\●(windows makes 2 partitions: 100 Mb(from 100 GB)

for sys reserve and remaining C:\ (100GB))

File System

File System

RedirectionRedirection

• If you end a command with “>”, its output goes to a file.

• If you end a command with “<“, its input comes from a file.

●man ●info ●command –help●Forums.

Linux Help

Man -kMan -k

• You can keyword search for commands

• For instance, what commands show a calendar?

$ man -k calendarcal (1) - displays a calendarcal (1p) - print a calendardifftime (3p) - compute the difference…

– The “-k” option●man –k print

– Manual pages are divided in 8 sections:

User commands System calls Libc calls Devices File formats and protocols Games Conventions, macro packages and so

forth System administation

– To select correct section, add section number:

●man 1 passwd, man 5 passwd

man command

info command

●A program for reading documentation, sometimes a replacement for manual pages

●Example : info ls

Linux Boot SequenceLinux Boot Sequence

B a sh S h e ll S ta rted

In it P ro g ram

L in ux K ern e l In it

L inu x B o o t Lo ad er

B IO S P o st

• Start your computer that has Linux installed.

RUN LEVELSRUN LEVELS

• They initiate sys call which interact with kernel. • Run level 0 : shutdown.• Run level 1 : single user interface.• Run level 2 : multi user without NFS(new file sys) support.• Run level 3 : multi user with NFS(new file sys) support.• Run level 4 GUI.• Run level 5 restart.

The Linux EnvironmentThe Linux Environment

What exactly is Linux?• "Linux" is just an OS kernel• the rest is additional Open Source Software• together they are a "Linux Distribution"• [Knoppix, Ubuntu, Redhat, Novell/SuSe]

Large choice of GUI and/or commandlineenvironments• most popular are KDE and Gnome• Unix-like, Mac-like, MSWindows-like, NeXT-like• advanced shell environments• web front-ends, GUI front-ends• [KDE, Gnome, Windowmaker, bash, zsh, emacs, mc]

The Linux EnvironmentThe Linux Environment

Forensic boot CDs• fully installed Linux environment on bootable CD or DVD• non-mount booting• large pre-installed forensic toolset• Knoppix based

Most up to date (at the moment)• FCCU GNU/Linux Forensic Boot CD (Belgian FederalComputer Crime Unit)• Helix (US e-fense Inc.)

Full installation:• learning Linux -> Ubuntu, doing forensics -> Debian• must strip down automount services

Imaging and Evidence AcquisitionImaging and Evidence Acquisition

Wide range of supported technologies and media• ATA, SATA, SCSI, USB, Firewire• cd, dvd, USB sticks, tapes, floppies, etc

Forensically sound acquisition• typically any sector-based storage medium accessible as adevice can be safely imaged• can acquire an image without mounting drive• hardware write-blocker not needed for unmounted devices• support for handling errors, bad blocks• [dd, dcfldd, dd_rescue, sdd, AIR, sleuthkit, adepto, grab]

Floppy Disk AnalysisFloppy Disk Analysis

• Insert Floppy• Obtain SHA hash

Create Floppy Disk ImageCreate Floppy Disk Image

• Use dd to create forensic image.• Compare SHA hash of image against floppy

to confirm good image.

Identify File SystemIdentify File System

• Use file utility to identify the file system of floppy disk image.

• file utility can identify more than 30 different file system types & many more standard file types.

Mount the Image for AnalysisMount the Image for Analysis

• Create a directory to mount the image against.• Use mount utility to mount the image, using loop

back to trick OS into thinking this is a physical device.

Obtain SHA Hash of ContentsObtain SHA Hash of Contents

• Obtain SHA hash of each file on the floppy disk.• Check file to confirm all looks as expected.

Identify File ContentsIdentify File Contents

• ls to view all the files on floppy.• file utility to identify file header.

– Tells us that this is actually a Microsoft Office Document.

View File ContentsView File Contents

• strings utility to extract raw text from a binary file.

Evidential Search CriteriaEvidential Search Criteria

• Put together keyword list to use in search applied against evidence.– This screenshot shows the use of the vi editor.

Apply Search ListApply Search List

• Apply the search list against the entire image of the floppy by using the grep utility.

View Search ResultsView Search Results

• Viewing the Search Results File with cat shows binary, so we use strings instead to just view the ASCII text from the file. HEX location is shown in [numeric]: I.e. “49189:”

Search Hit ExampleSearch Hit Example• xxd utility used to perform a hex dump of data.• # xxd -s 49189 /evidence/floppy1.img | less• Note the match giving an address to the Boston Crackdlr of 11 Clarendon

Apt 6 in Boston’s Back Bay

FILE SYSTEMFILE SYSTEM

SECOND EXTENDED FILE SYSTEM(EXT2)SECOND EXTENDED FILE SYSTEM(EXT2)

• Commonly used in Linux.• Building blocks known as data blocks(4 Kb in

size) : similar to FAT32 file sys.• Allocation of blocks similar to FAT32.Thus

physical sizes and logical sizes in EXT2 differ as they do in FAT32.

• Main components:Directories. Inodes.Data blocks.


• Directories store the filenames in the file sys and the inodes associated with the files.

• In EXT2, files are represented by inodes. An inode contains attributes like file type, size, access rights, timestamps address of data blocks.

• An Inode is similar to a directory entry in FAT32. Each inode has a unique no. identifying it.


• File sys made up of “block groups” a sequential arrangement of a group of data blocks. The entire file sys is thus managed as a series of block groups.

• Block group meta data is provided by a ‘block group descriptor’. It contains a copy of super block(size of inode table and file sys), Block bitmap(tracks allocation of each data block), part of inode table and data blocks.

ext2 Characteristicsext2 Characteristics

• Complicated internal structure to enhance performance, but on-disk structure is straightforward

From: Understanding the Linux Kernel (Bovet/Cesati)

ext2 Directory Structureext2 Directory Structure

From: Understanding the Linux Kernel (Bovet/Cesati)

deleted

ext2/3: inodeext2/3: inode

Fro

m: U

nder

stan

ding

the

Linu

x K

erne

l (B

ovet

/Ces

ati)

ext3 Characteristicsext3 Characteristics

• Binary compatible with ext2 on-disk• Reason for existence: huge disks == huge

amounts of time to restore filesystem consistency after improper shutdown

• (Must check inodes, etc.)• Major improvement over ext2: log that stores

info about in-progress file operations• On boot, can check log and quickly restore

filesystem consistency• Journaling filesystems: DANGER!

File Deletion: LinuxFile Deletion: Linux• ext2 file deletion

– Adjust previous directory entry length to obscure deleted record– No reorganization to make space in directories– “first fit” for new directory entries, based on real name length– Directory entry’s inode # is cleared

• ext3 file deletion– Same as for ext2, but…– inode is wiped on file deletion, so block numbers are lost– Major anti-forensics issue!– But directory entry’s inode # isn’t cleared…

ToolsTools• Sleuthkit• Deft 7.0 • Helix• BackTrack

SleuthkitSleuthkit

• Linux toolkit for forensics written by Brian Carrier

• http://www.sleuthkit.org/

• Command line tools for forensic analysis under Linux (and now, Windows…)

• Graphical interface: Autopsy Forensic Browser

Sleuthkit (2)Sleuthkit (2)• Data Unit:

– dls – copy individual disk blocks– dcat – copy ranges of disk “chunks”– dstat – determine whether block is allocated

• Meta Data:– ils – list inodes (e.g., ones corr. to deleted files!)– icat – copy data associated with inode– istat – dump detailed inode info– ifind – find inode associated with a disk block

• File Name:– fls – list files/directories in image– ffind – find filename associated with inode

• File System: fsstat – Detailed filesystem information

“Next Generation” Digital Forensics

Meets Linux

“Next Generation” Digital Forensics

Meets Linux

Filesystem in Userspace (FUSE)Filesystem in Userspace (FUSE)• (FUSE) is a loadable kernel module for Unix-

like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a "bridge" to the actual kernel interfaces.

• FUSE is particularly useful for writing virtual file systems. Unlike traditional file systems that essentially save data to and retrieve data from disk, virtual filesystems do not actually store data themselves. They act as a view or translation of an existing file system or storage device.

• In principle, any resource available to a FUSE implementation can be exported as a file system.

FUSE (Filesystem in User Space)

FUSE (Filesystem in User Space)

user space

kernel space

LinuxVirtual File System

Interface(VFS)

C library

dd if=/evidence/DEC/img.dd of=copy.dd

read()

FUSE

ext3

reiserFS

C library

FUSE library

Filesystem Implementation

File carvingFile carving• Is the process of reassembling computer files from

fragments in the absence of filesystem metadata.

• The carving process makes use of knowledge of common file structures, information contained in files, and heuristics regarding how filesystems fragment data.

• Fusing these three sources of information, a file carving system infers which fragments belong together.

• File carving is a highly complex task, with a potentially huge number of permutations to try.

• To make this task tractable, carving software typically makes extensive use of models and heuristics. This is necessary not only from a standpoint of execution time, but also for the accuracy of the results.

In-Place File CarvingIn-Place File Carving

preview databaseFUSE

scalpel_fs

client applications

nbd server

nbd client

networklocal drive

remote drive

G. G. Richard III, V. Roussev, V. Marziale, “In-Place File Carving,” submitted to the Third Annual IFIP WG 11.9 International Conference on Digital Forensics, 2007.

Scalpel

Better AuditingBetter Auditing

Want: Digital Evidence Bags

See: P. Turner, “Unification of Digital Evidence from Disparate Sources (Digital Evidence Bags),” DFRWS 2005See: Common Digital Evidence Storage Format (CDESF) working group, http://www.dfrws.org/CDESF/.

Better Auditing (2)Better Auditing (2)

DEC

(DEB, AFF,Gfzip …)

FDAM

dd scalpel FTK…

VFS Interface

TSK

EvidenceData

Audit Log

Import/Export

Applications(User space)

(Kernel)

OperatingSystem

Block-levelData Access

FilesystemData Access

FDAM Block Device

G. G. Richard III, V. Roussev, "Toward Secure, Audited Processing of Digital Evidence: Filesystem Support for Digital Evidence Bags," Research Advances in Digital Forensics, Springer, 2006.

DigitalEvidenceContainer

Bluepipe: On the Spot Digital Forensics

Bluepipe: On the Spot Digital Forensics

Cu Bootable Bluepipe CD Removable media

Target

Bluetooth or 802.11dongle 3G/VPN

Remote investigator(s)

Handheld Bluepipe client

Y. Gao, G. G. Richard III, V. Roussev, “Bluepipe: An Architecture for On-the-Spot Digital Forensics,” International Journal of Digital Evidence (IJDE), 3(1), 2004.

Bluepipe Patterns

Distributed Digital ForensicsDistributed Digital Forensics

V. Roussev, G. G. Richard III, "Breaking the Performance Wall: The Case for Distributed Digital Forensics,“ Proceedings of the 2004 Digital Forensics Research Workshop (DFRWS 2004), Baltimore, MD

750GB750GB

Distributed Digital ForensicsDistributed Digital Forensics

• Scalable– Want to support at least IMAGE SIZE / RAM_PER_NODE nodes

• Platform independent– Want to be able to incorporate any (reasonable) machine that’s

available• Lightweight

– Horsepower is for forensics, not the framework—less fat• Highly interactive• Extensible

– Allow incorporation of existing sequential tools– e.g., stegdetect, image thumbnailing, file classification, hashing,

…• Robust

– Must handle failed nodes smoothly

Distributed Digital Forensics (2)


SHUTDOWN STARTUP

CACHE

FETCH

JOIN / LEAVE

CANCEL / EXITHASH / GREP / COMMAND/ …

DONE / REPORT / ERROR

CoordinatorWorker Worker

Common Store

Local Store

LOAD

STORE

SHUTDOWN STARTUP

CACHE

FETCH

JOIN / LEAVE

CANCEL / EXITHASH / GREP / COMMAND/ …

DONE / REPORT / ERROR

CoordinatorWorker Worker

Common Store

Local Store

LOAD

STORE



SCSIRAID: 504GBFile Server

CPU: 2x1.4GHz XeonRAM: 2GB

Switch96-port, 10/100/1000 Mb24 Gb Backplane

1Gb

NodeCPU: 2.4 GHz Pentium 4RAM: 1 GB

SCSIRAID: 504GBFile Server

CPU: 2x1.4GHz XeonRAM: 2GB

Switch96-port, 10/100/1000 Mb24 Gb Backplane

1Gb

NodeCPU: 2.4 GHz 4RAM: 1 GB

Thank you

linux forensics

Education

linux linux

linux version

linux platforms

supporting linux

linux business

linux forforensics

linux shellwhoamilspwd

free tools