CSE 513Introduction to Operating Systems

Class 10 - Security

Jonathan WalpoleDept. of Comp. Sci. and Eng.

Oregon Health and Science University

Overview

q Intro to cryptography toolsv one-way functions, public vs private key encryption,

hash functions, and digital signaturesq Protection domains and protection mechanismsq User authenticationq Internal attacks

v Trojan horses, spoofing, logic bombs, trap doors,buffer overflow attacks

q External attacksv Viruses, worms, mobile code, sand boxing,

interpretation

Security overview

q Security flavorsv Confidentiality - Ability to protect secretsv Integrity -Ability to protect the data contentsv Availability - Ability to continue to operate

q Know thy enemy!v User stupidity (bad default settings from companies)v Insider snoopingv Outsider snoopingv Blatant attacks (viruses and worms)v Bots!

Accidental data loss

q Acts of God- fires, floods, wars

q Hardware or software errors- CPU malfunction, bad disk, program bugs

q Human errors- data entry, wrong tape mounted- “you” are probably the biggest threat you’ll ever face

Introduction toCryptography Tools

Basics of Cryptography

Relationship between the plaintext and the ciphertext

Cryptography: confidentiality and integrity

q Example: mono-alphabetic substitutionPlaintext: ABCDEFGHIJKLMNOPQRSTUVWXYZCyphertext: QWERTYUIOPASDFGHJKLZXCVBNM

q Given the encryption key (QWERTYUIOPASDFGHJKLZXCVBNM),v easy to find decryption key using statistical

properties of natural language (common letters anddigrams)

v … despite size of search space of 26! possible keys

q Function should be more complex and searchspace very large.

Secret-key cryptography

Symmetric cryptography: DES

q DES operates on 64-bit blocks of datav initial permutationv 16 rounds of transformations each using a different encryption key

Manglerfunction

Per-round key generation in DES

q Each key derived from a 56-bit master by mangling functionbased on splitting, rotating, bit extraction and combination

Symmetric (secret) key cryptography

q Fast for encryption and decryptionq Difficult to break analyticallyq Subject to brute force attacks

v as computers get faster must increase the numberof rounds and length of keys

q Main problemv how to distribute the keys in the first place?

Public-key cryptography

q Use different keys for encryption and decryptionq Knowing the encryption key doesn’t help you decrypt

v the encryption key can be made publicv encryption key is given to senderv decryption key is held privately by the receiver

q But how does it work?

Public-key cryptography

q Asymmetric (one-way) functionsv given function f it is easy to evaluate y = f(x)v but given y its computationally infeasible to find x

q Trivial example of an asymmetric functionencryption: y = x2

decryption: x = squareroot (y)

q Challengev finding a function with strong security properties but

efficient encryption and decryption

Public-key cryptography: RSA

q RSA (Rivest, Shamir, Adleman)v encryption involves multiplying large prime numbersv cracking involves finding prime factors of a large number

q Steps to generate encryption key (e ) and decryptionkey (d )

v Choose two very large prime numbers, p and qv Compute n = p x q and z = (p – 1) x (q – 1)v Choose a number d that is relatively prime to zv Compute the number e such that e x d = 1 mod z

Public-key cryptography: RSA

q Messages split into fixed length blocks of bitsv interpreted as numbers with value 0 <= mi < n

q Encryptionci = mi

e (mod n)v requires that you have n and encryption key e

q Decryptionmi = ci

d (mod n)v requires that you have n and decryption key d

RSA vs DES

q RSA is more secure than DESq RSA requires 100-1000 times more computation

than DES to encrypt and decryptq RSA can be used to exchange private DES keysq DES can be used for message contents

Secure hash functions

q Hash functions h = H(m) are one way functionsv can’t find input m from output hv easy to compute h from m

q Weak collision resistancev given m and h = H(m) difficult to find different

input m’ such that H(m) = H(m’)

q Strong collision resistancev given H it is difficult to find any two different input

values m and m’ such that H(m) = H(m’)

q They typically generate a short fixed lengthoutput string from arbitrary length input string

Example secure hash functions

q MD5 - (Message Digest)v produces a 16 byte result

q SHA - (Secure Hash Algorithm)v produces a 20 byte result

Secure hash functions : MD5

q The structure of MD5v produces a 128-bit digest from a set of 512-bit blocksv k block digests require k phases of processing each with

four rounds of processing to produce one message digest

Per phase processing in MD5

q Each phase involves for rounds of processing

F (x,y,z) = (x AND y) OR ((NOT x) AND z)G (x,y,z) = (x AND z) OR (y AND (NOT z))H (x,y,z) = x XOR y XOR zI (x,y,z) = y XOR (x OR (NOT z))

Per round processing in MD5

q The 16 iterations during the first round in a phase ofMD5 using function F

What can you use a hash function for?

q To verify the integrity of datav if the data has changed the hash will change (weak

and strong collision resistance properties)

q To “sign” or “certify” data or software

Digital signatures

q Computing a signature blockq What the receiver gets

(b)

Digital signatures using a message digest

Private key of APublic key of ASecret key shared by A and B KA, B

DescriptionNotation

K A+

K A−

Digital signatures with public-key cryptography

Private key of APublic key of ASecret key shared by A and B KA, B

DescriptionNotation

K A+

K A−

Protection Domains

Protection domains

q Every process executes in some protection domainv determined by its creator, authenticated at login time

q OS mechanisms for switching protection domainsv system callsv set UID capability on executable filev re-authenticating user

A protection matrix

Protection matrix with domains as objects

Domain

Protection Mechanisms

Access control lists (ACLs)

Domain

q Domain matrix is typically large and sparsev inefficient to store the whole thingv store occupied columns only, with the resource? - ACLsv store occupied rows only, with the domain? - Capabilities

Access control lists for file access

Access Control Lists (2)

q Two access control lists with user names androles (groups)

Capabilities

Domain

q Domain matrix is typically large and sparsev inefficient to store the whole thingv store occupied columns only, with the resource? - ACLsv store occupied rows only, with the domain? - Capabilities

Capabilities associated with processes

q Each process has a capability list

q Cryptographically-protected capability can beheld in user space

q Generic Rightsv Copy capabilityv Copy objectv Remove capabilityv Destroy object

Cryptographically-protected capabilities

f(Objects, Rights, Check)RightsObjectServer

User Authentication

User authentication

q Basic Principles. Authentication must identify:v Something the user knowsv Something the user hasv Something the user is

q This is done before user can use the system !

Authentication using passwords

(a) A successful login(b) Login rejected after name entered (easier to crack)(c) Login rejected after name and password typed

Problems with pre-set values

q How a cracker broke into LBLv a U.S. Dept. of Energy research lab

Authentication using passwords and salt

q The use of salt to defeat precomputation ofencrypted passwords

v salt changes each time password changesv increases the size of the search space

Salt Password

,,

,,

Authentication using a physical object

q Magnetic cardsv magnetic stripe cardsv chip cards: stored value cards, smart cards

Authentication using biometrics

A device for measuring finger length.

Attacks on the authentication process

q Authentication - making sure the user is the user

q Attacks includev Placement of passwords in the clear

• Written on desk, included in a network packet etc…v Network packet sniffers

• Listen to the network and record login sessionsv Snooping

• observing key strokesv Automated bots

• Try a password every minute (don’t get greedy)

Counter-measures to combat attackers

q Limiting times when someone can log inq Automatic callback at number prespecifiedq Limited number of login triesq Keep a database of all loginsq Honey pot

v leave simple login name/password as a trapv security personnel notified when attacker bites

More counter-measures

q Better passwordsv No dictionary words, special characters, longer

q Don’t give up informationv Login prompts or any other time

q One time passwordsv Satellite driven security cards

q Limited-time passwordsv Annoying but effective

q Challenge-response pairsv Ask questions

q Physical authentication combined with passwords

Verifying the user is a person

Internal Attacks

Login spoofing

(a) Correct login screen(b) Phony login screen

Which would you rather log into?

Trojan horses

q Free program made available to unsuspecting userv Actually contains code to do harm

q Place altered version of utility program on victim'scomputerv trick user into running that programv example, ls attack

q Trick the user into executing something theyshouldn’t

Logic bombs

q Revenge driven attackq Company programmer writes program

v potential to do harmv OK as long as he/she enters password dailyv if programmer fired, no password and bomb “explodes”

Trap doors

(a) Normal code.(b) Code with a trapdoor inserted

Buffer overflow attacks

q (a) Situation when main program is runningq (b) After program A calledq (c) Buffer overflow shown in gray

Buffer overflow attacks

q The basic ideav exploit lack of bounds checking to overwrite return

address and to insert new return address and codeat that address

v exploit lack of separation between stack and code(ability to execute both)

v allows user (attacker) code to be placed in a setUID root process and hence executed in a moreprivileged protection domain

Other generic security attacks

q Request memory, disk space, tapes and just readq Try illegal system callsq Start a login and hit DEL, RUBOUT, or BREAKq Try modifying complex OS structuresq Try to do specified DO NOTsq Convince a system programmer to add a trap doorq Beg someone with access to help a poor user who

forgot their password

Famous security flaws

(a) (b) (c)q The TENEX password problem

v requires 128n tries instead of 128n

Design principles for security

q System design should be publicq Default should be no accessq Check for current authorityq Give each process least privilege possibleq Protection mechanism should be

- simple- uniform- in lowest layers of system

q Scheme should be psychologically acceptable

And … keep it simple!

External Attacks

External threats and viruses

q External threatv code transmitted to target machinev code executed there, doing damagev may utilize an internal attack to gain more privilege

(ie. Buffer overflow)

q Goals of virus writerv quickly spreading virusv difficult to detectv hard to get rid of

q Virus = program that can reproduce itselfv attach its code to another program

Virus damage scenarios

q Blackmailq Denial of service as long as virus runsq Permanently damage hardwareq Target a competitor's computer

v do harmv espionage

q Intra-corporate dirty tricksv sabotage another corporate officer's files

How viruses work

q Virus written in assembly languageq Inserted into another program

v use tool called a “dropper”q Virus dormant until program executed

v then infects other programsv eventually executes its “payload”

Searching for executable files to infect

Recursiveprocedurethat findsexecutablefiles on aUNIX system

Virus couldinfect them all

How viruses hide

q An executable programq Virus at the front (program shifted, size increased)q Virus at the end (size increased)q With a virus spread over free space within program

v less easy to spot, size may not increase

Viruses that capture interrupt vectors

q After virus has captured interrupt, trap vectorsq After OS has retaken printer interrupt vectorq After virus has noticed loss of printer interrupt vector and

recaptured it

How viruses spread

q Virus placed where likely to be copied orexecuted

q When it arrives at a new machinev infects programs on hard drive, floppyv may try to spread over LAN

q Attach to innocent looking emailv when it runs, use mailing list to replicate further

Antivirus and anti-antivirus techniques

(a) A program(b) Infected program(c) Compressed infected program(d) Encrypted virus(e) Compressed virus with encrypted compression code

Anti-antivirus techniques

q Examples of a polymorphic virusv All of these examples do the same thing

Antivirus software

q Integrity checkersv use checksums on executable filesv hide checksums to prevent tampering?v encrypt checksums and keep key private

q Behavioral checkersv catch system calls and check for suspicious activityv what does “normal” activity look like?

Virus avoidance and recovery

q Virus avoidancev good OSv install only shrink-wrapped softwarev use antivirus softwarev do not click on attachments to emailv frequent backups

q Recovery from virus attackv halt computer, reboot from safe disk, run antivirus

The Internet worm

q Robert Morris constructed the first Internetwormv Consisted of two programs

• bootstrap to upload worm and the worm itself

v Worm first hid its existence then replicated itselfon new machines

v Focused on three flaws in UNIX• rsh – exploit local trusted machines• fingerd – buffer overflow attack• sendmail – debug problem

q It was too aggressive and he was caught

Availability and denial of service attacks

q Denial of service (DoS) attacksv Examples of known attacks

• Breaking end systems– Ping of death – large ping packets– Teardrop – overlapping IP segments

• SYN floods• UDP floods• Window bombs (in browsers)

q Usually prevented by some sort of firewall butnot always effective

Security Approachesfor Mobile Code

Sandboxing

(a) Memory divided into 1-MB sandboxesv each applet has two sandboxed for code and datav some static checking of addresses

(b) Code inserted for runtime checking of dynamic target addresses

Interpretation

Applets can be interpreted by a Web browser

Code signing

How code signing works

Type safe languages

q A type safe languagev compiler rejects attempts to misuse variables

q Checks include …• Attempts to forge pointers• Violation of access restrictions on private class

members• Misuse of variables by type• Generation of stack over/underflows• Illegal conversion of variables to another type

Java security

Examples of specified protection with JDK 1.2

Covert Channels

Covert channels

Client, server andcollaborator processes

Encapsulated server can stillleak to collaborator via

covert channels

Locking as a covert channel

A covert channel using file locking

Covert channels

q Pictures appear the sameq Picture on right has text of 5 Shakespeare plays

v encrypted, inserted into low order bits of color values

Zebras Hamlet, Macbeth, Julius CaesarMerchant of Venice, King Lear

Spare Slides

Trusted Systems and Formal Models

Trusted SystemsTrusted Computing Base

A reference monitor

Formal Models of Secure Systems

(a) An authorized state(b) An unauthorized state

Multilevel Security (1)

The Bell-La Padula multilevel security model

Multilevel Security (2)

The Biba Model

q Principles to guarantee integrity of data

q Simple integrity principle• process can write only objects at its security level or lower

q The integrity * property• process can read only objects at its security level or higher

Orange Book Security (1)

q Symbol X means new requirementsq Symbol -> requirements from next lower category apply

here also

Orange Book Security (2)