Processes and Schedulers

What is a Process

• Process: An execution stream and its associated state

• Execution Stream– Set of instructions– “Thread of control”

• Process State– Hardware state

• Privilege level, segments, page tables

– OS State• Priority, I/O Buffers, Heap, memory map

– Resource state• I/O requests

• An abstraction to make it easier to program both OS and applications– Encapsulate state into manageable unit

Programs and Processes

• A process is not a program• Program: Static code and static data

int foo() { return 0;}

int main() { foo(); return 0;}

Programint foo() { return 0;}

int main() { foo(); return 0;}

Heap

Stack

Registers

Process

• OS can host multiple processes of same program– E.g. many users can run ‘ls’ at the same time

• Once program can invoke multiple processes– E.g. make runs many processes to compile code

• No one-to-one mapping between programs and processes

Threads and Processes

• A process is different than a thread– Conceptually (On Linux it is more complicated)

• Thread: Separate execution streams in same address space– “Lightweight process”

int foo() { return 0;}

int main() { foo(); return 0;}

Heap

Stack

Registers

Process

• Can have multiple threads within a process

int foo() { return 0;}

int main() { foo(); return 0;}

Heap

Threads

Stack

Registers

Stack

Registers

System Classification

• Uniprogramming: Only one process at a time– Examples: Original systems and older PC Oses

• DOS

– Advantages: Easier for OS designer– Disadvantages: Terrible utilization, poor usability

• Multiprogramming: Multiple processes at a time– Examples: Every modern OS you use– Note: Multiprogramming is different from multiprocessing

• Multiprocessing: Systems with multiple processors

– Advantages: Better utilization and usability– Disadvantages: Complex OS design

Multiprogramming

• OS requirements for multiprogramming– Policy to determine process to run– Mechanism to switch between processes– Methods to protect processes from one another

• Memory management system

• Separation of policy and mechanism– Recurring theme in OS design– Policy: Decision maker based on some metric

• Scheduler

– Mechanism: Low level code that implements the decision• Dispatcher/Context Switch

Multiprogramming and Memory

• Many OSes didn’t do a very good job of combining these

• Early PC OSes didn’t protect memory – MacOS and Windows– Each process could access all of memory

• Same address space• Basically a giant multithreaded environment

• All modern OSes not include memory map in PCB– Processes cannot access each other memory

Dispatch Mechanism

• OS maintains list of all processes• Each process has a mode– Running: Executing on the CPU– Ready: Waiting to execute on CPU– Blocked: Waiting for I/O or synchronization with

another thread• Dispatch Loop while (1) {

run process for a while; stop process and save its state; load state of another process;}

How does dispatcher gain control?What execution state must be saved/restored?

How does dispatcher gain control?• Must change from user to system mode

– Problem: Only one CPU, and CPU can only do one thing at a time– A user process running means the dispatcher isn’t

• Two ways OS gains control

• Traps: Events caused by process execution– System calls, page faults, Exceptions (segfault, etc)

• Hardware interrupts: Events external to process– Typing at keyboard, network packet arrivals– Control switch to OS via Interrupt Service Routine (ISR)

• How does OS guarantee it will regain control?

Approaches to dispatcher• Option 1: Cooperative multitasking

– Trust process to invoke dispatcher– Linux: Default for kernel code

• schedule()

– Disadvantage: A mistake in one part of the code can lock up entire system

• Option 2: True multitasking– Configure hardware to periodically invoke dispatcher– Hardware generated timer interrupt

• Timer ISR invokes dispatcher

– Linux: Enabled for user processes• 100-1000HZ

– Processes run for some multiple of timer “ticks” (interrupts)• Process time slice

What state must be saved?• OS must track state of processes

– On every trap/interrupt save process state in “process control block” (PCB)• Why on every trap/interrupt?

• Data structure problem: How to manages all PCBs

• Information stored in PCB– Execution state

• General registers, control registers, CPU flags, RSP, RIP, page tables– OS state

• Memory map, heap space– I/O status

• Open files and sockets– Scheduling information

• Execution mode, priority– Accounting information

• Owner, PID– Plus lots more

Context Switch implementation

• Machine dependent code (Assembly!)– Different for MIPS, ARM, x86, etc.– Save process state to PCB

• Tricky: OS must save state without changing state

• Requires special hardware support– Save process state on each trap/interrupt– Very nasty

• x86 has TSS (PCB) that most OSes avoid

Process Creation• Two ways to create a process

– Build one from scratch– Clone an existing one

• Option 1: From scratch (Windows – CreateProcess(…))– Load specified code and data into memory– Create empty call stack– Create and initialize PCB (make it look like a context switch)– Add process to ready list

• Option 2: Cloning (UNIX – fork())– Stop current process and save its state– Copy code, data, stack and PCB– Add new Process PCB to ready list

– Do we really need to copy everything?

Creating a process (Windows and UNIX)

BOOL WINAPI CreateProcess( _In_opt_    LPCTSTR lpApplicationName, _Inout_opt_ LPTSTR lpCommandLine,

_In_opt_  LPSECURITY_ATTRIBUTES lpProcessAttributes, _In_opt_    LPSECURITY_ATTRIBUTES lpThreadAttributes, _In_        BOOL bInheritHandles, _In_        DWORD dwCreationFlags, _In_opt_    LPVOID lpEnvironment, _In_opt_    LPCTSTR lpCurrentDirectory, _In_        LPSTARTUPINFO lpStartupInfo, _Out_       LPPROCESS_INFORMATION lpProcessInformation );

int fork();

Windows

UNIX

Creating processes in UNIX

• Combination of fork() and exec(…)– fork(): Clone current process– exec(…): copy new program on top of current process

int main() { int pid; char * cmd = “/bin/sh”;

pid = fork(); if (pid == 0) {

// child processexec(cmd);// exec does not return, WE NEVER GET HERE

} else {// parent process – wait for child to finishwait(pid);

}}

• Advantage: Flexible, clean, simple

Process Abstraction

• Processes are a low level component– Provide an abstraction to build on

• Fundamental OS design– Provide abstract units (resources) that high level policies can act

on

• Resources– Resources are high level units managed by OS– CPU time, memory, disk space, I/O bandwidth

• How does the OS manage resources?

Resources

• Preemptible– Resource can be taken away and used by somebody else– Example: CPU

• Non-preemptible– One a resource is assigned it can only be returned

voluntarily– Example: Disk space

• OS must balance set of resources and requests for those resources– OS management depends on type of resource

Decisions about Resources

• Allocation: Which process gets which resource– Which resources should each process get?– Space sharing: Control concurrent access to resource– Implication: Resources are not easily preemptible– Example: disk space

• Scheduling: How long process keeps resource– In which order should requests be serviced– Time sharing: More resources requested than exist– Implication: Resource is preemtible– Example: CPU time

Role of Dispatcher vs. Scheduler• Dispatcher

– Low-level mechanism– Responsibility: Context-switch

• Change mode of old process to either WAITING or BLOCKED• Save execution state of old process in PCB• Load state of new process from PCB• Change mode of new processes to RUNNING• Switch to user mode privilege• Jump to process instruction

• Scheduler– Higher level policy– Responsibility: Decide which process to dispatch to

• CPU could be allocated– Parallel and Distributed Systems

Scheduling Performance Metrics

• Minimize response time– Increase interactivity (responsiveness of user interfaces)

• Maximize resource utilization– Keep CPU and disks busy

• Minimize overhead– Reduce context switches (number and cost)

• Distributed resources fairly– Give each user/process same percentage of CPU

Scheduling Algorithms• Process (job) model

– Process alternates between CPU and I/O bursts– CPU bound job: long CPU bursts– I/O bound job: short CPU bursts

• Don’t know before execution– Need to handle full range of possible workloads

• Scheduling Algorithms– First-Come-First-Served (FCFS)– Shortest Job First (SJF) or Shortest-Time-Completion-First (STCF)– Round-Robin (RR)– Priority Scheduling– Other scheduling algorithms that are actually used

First Come First Served (FCFS)

• Simplest Scheduling algorithm– First job that requests CPU is allocated CPU– Nonpreemptive

• Advantage: Simple Implementation with FIFO queue• Disadvantage: Response time depends on arrival

order– Unfair to later jobs (especially if the system has long jobs)

Job A Job B Job C

Time

CPU

• Uniprogramming: Run job to completion• Multiprogramming: Put job at back of queue when performing I/O

Convoy Effect

• Short running jobs stuck waiting for long jobs– Example: 1 CPU bound job, 3 I/O bound jobs

• Problems– Reduces utilization of I/O devices– Hurts response time of short jobs

A B C

Time

CPU C A B C C

A B C C A B C CIdle IdleDisk

Shortest Job First

• Minimizes average response time

Job AJob B Job C

Time

CPU

• FCFS if simultaneous arrival• Provably optimal (given no preemption) to reduce

response time– Short job improved more than long job is hurt

• Not practical: Cannot know burst lengths (I/O + CPU)– Can only use past behavior to predict future behavior

Shortest Time to Completion First (STCF)

• SJF with preemption– New process arrives w/ short CPU burst– Shorter than remaining time of current job

AB D

Time

CPU A C

B D

Time

CPU A C

• SJF without preemption

Job Submission: A at time 0, B/C/D at time t

t0

Shortest Remaining Processing Time(SRPT)

• STCF for batch workloads

• Used in distributed systems– Provides maximum throughput (transactions/sec)– Minor risk of starvation

• Very popular in web servers and similar systems

Round Robin (RR)• Practical approach to support time-sharing

– Run job for a time-slice and then go to back of queue– Preempted if still running at end of time-slice

• Advantages– Fair allocation of CPU across jobs– Low average response time when job lengths vary widely

• Avoids worst case scenarios and starvation

AACPU B C A B C A B

Time

Disadvantages of Round Robin

• Poor average response time when job sizes are identical– E.g. 10 jobs each require 10 time slices– All complete after 100 time slices– Even FCFS is better

• How large should the time slice be?– Depends on the workload!– Tradeoff between throughput and responsiveness

• Batch vs. interactive workloads

Priority based scheduling

• Priorities assigned to each process– Run highest priority job in system (that is ready)– Round robin among equal priority levels

• Aside: How to parse priority numbers– Is low high or is high high? (Depends on system)

• Static vs. Dynamic priorities– Some jobs have static priority assignments (kernel threads)– Others need to be dynamic (user applications)– Should priorities change as a result of scheduling decisions?

Real world scheduling

• Current schedulers exhibit combinations of above approaches– Include priorities, round robin queues, queue reordering, and much

more– There is no single good algorithm– Nor is there a way to even measure “goodness”

• Most schedulers designed based on heuristics and best guesses of potential workloads– Linux has a lot of complexity to detect batch vs. interactive processes

(can change during execution)

• Many times a scheduler will work great 99% of the time but completely fall over for 1% of workloads