Concurrency

What is Concurrency

• Ability to execute two operations at the same time

• Physical concurrency– multiple processors on the same machine– distributing across networked machines

• Logical concurrency– illusion or partial parallelism

• Designer/programmer doesn’t care which!

Real or Apparentdepends on your point of view

• Multiple computers in distributed computing or multiprocessors on a computer

• Multiple clients on same or multiple computers

• Multiple servers on one or many machines

• Where does the complexity lie?– It varies on how you design the system

Why is concurrency important?

• One machine is only capable of a limited speed

• Multiple machines/processors– share workload and gain better utilization– optimize responsibility/requirements to each

machine’s ability– place secure processes in secure environments– parallel processors are tailored to the problem domain

• Single machine– OS can give limited parallelism through scheduling

Concurrency considerations

• What level of concurrency to consider

• How it is handled on a single processor– understand process scheduling

• How to share resources

• How to synchronize activity

• How to synchronize I/O specifically

Process Scheduling

Process

• The activation of a program.

• Can have multiple processes of a program

• Entities associated with a process– Instruction pointer– user who owns it– memory location of user data areas– own run-time stack

Process Scheduling

• Order of process execution is unpredictable

• Duration of execution is unpredictable

• Whether there is an appearance or real concurrency is not relevant to the designer.

• There will generally be a need to resynchronize activity between cooperating processes

Context switch

• When OS switches running process, it manipulates internal process tables

• Loading/storing registers must be done• Threads minimize that effort. Same

process but a different stack.

• Necessary overhead but must balance overhead with advantage of concurrency

Operating Systems Scheduling

Ready205198201

Running200

Blocked177206180185

Process 200 blocks when reading from the disk

Ready198201

Running205

Blocked177206180185200

What other activities are important in scheduling?

• Jobs go from RUNNING to READY when they lose their time slot

• Jobs go from blocked to READY when the I/O operation they are waiting for completes

• Jobs go from RUNNING to being removed completely upon exit

Concurrency at many levels

• Process level .. – Unix fork command…. (review example)– network client-server level

• Subprocess level (like a procedure) .. thread

• Statement level

• Scheduling level

How do you get limited parallelism from the OS?

time

ProcessA runsonCPU;blockson read

Disk reads for B whileblocked

Disk readsfor A whileA blocked

Process C runs forit’s timeslice

Process Bruns

cpudiskcontroller There are times

when both processors(CPU and controller)are busy so real parallelismdoes occur but not at the level of the CPU

What if YOU have to create the concurrency?

High-level Concurrency

Unix and 95/98/NTUnix

•process concurrency•use fork and exec•fork

•clone self•parent and child differ by ppid•two processes

•exec•new process replaces original•single different process

95/98/NT•thread part of same process•own copy of locals•shared copy of globals•shared resources like file descriptors•each has own activation record•parent and child do not always begin/continue at same spot as in fork•thread ceases execution on return•_beginthread() needs procedure•Createprocess() like fork/exec

Unix fork()process level

void main(){ fork(); cout << “Hi”;}

producesHiHi

Question is… who “Hi”ed first?

void main(){ fork(); cout << “Hi”;}

Process 1

timevoid main(){ fork(); cout << “Hi”;}

Process 1

void main(){ fork(); cout << “Hi”;}

Process 2

void main(){ fork(); cout << “Hi”;}

Process 2

OR

Output

HiHi

HiHi

OR

Another Example

fork();cout << “a”;fork();cout << “b”;

How many processes are generated?How many possible outputs can you see?

/bin/sh /bin/sh

ls

forks a new

shell

STEP 1:

STEP 2:execs (replaces self)a new program

STEP 3:exit to theparent

A more common unix Example

Use in servers (and clients)(not important for us)

cid = fork();if (cid > 0) { // this is parent code … }else { // this is child code … }

While (1) { talksock = accept (listsock...

Use listsockclose talksockrepeat loop

close listsockuse talksock…exit()

}

Threads(windows style)

Non-threaded

void main(){ count(4);}

void count(int i){int j; for (j=1; j<=i; j++) cout << j<<endl;}

1234

Threaded

void count(int i){int j; for (j=1; j<=i; j++) cout << j<<endl;}

12123344

void main(){ _beginthread(( (void(*) void()) count, 0 , (void*) 5); count(4);}

12123344

The Synchronization Problem

Concurrency frequently requires synchronization!

Cooperation Synchronization

A is working on somethingB must wait for A to finish

x = f + g;h = x + y;B does this

A does this

Competition Synchronization

A needs to read a streamB needs to read the streamOnly one can read at a time

instr >> a;instr >> b;B does this

A does this

We’ll see how to do this later!

The synchronization problem

time

Shared Memory Task A Task B

T=3

T = T + 1 T = T * 2

fetch T(3)

incr T(4)

lose CPU(time)

fetch T(3)

double T(6)

store TT=6get CPU

store TT=4

TRY THIS: What other combinations could occur?

The essence of the problem

• There are times during which exclusive access must be granted

• These areas of our program are called critical sections

• Sometimes this is handled by disabling interrupts so process keeps processor

• Most often through a more controlled mechanism like a semaphore

Where do we see it?

• EVERYWHERE

• Database access

• Any data structures

• File/Printer/Memory resources

• Any intersection of need between processing entities for data

Synchronization Solutionsfor c/c++

(java later)

Semaphore

• One means of synchronizing activity

• Managed by the operating system– implementing yourself will not work (mutual exclusion)

• Typical wait and release functions called by applications

• Count associated – 0/1 binary implies only a single resource to manage– larger value means multiple resources

• Queue for waiting processes (blocked)

wait and release

wait ( semA ) { if semA>0 then decr semA; else put in semA queue; (block it) }

release ( semA ) {if semA queue empty incr semA; else remove job in semA queue (unblock it) }

This represents what the operating systemdoes when an application asks for access to the resource by calling wait or release on the semaphore

Standard exampleproducer-consumer

semaphore fullspots, emptyspots;fullspots.count=0;emptyspots.count= BUFLEN;

task producer; loop wait (emptyspots); DEPOSIT(VALUE); release (fullspots); end loop;end producer;

task consumer; loop wait (fullspots); FETCH(VALUE); release (emptyspots); end loop;end producer;

shared resource

Why do you need TWOsemaphores? Are addingand removing the same?

Competition SynchronizationWhat if multiple processes want to put objects in the buffer?

We might have a similar synchronization problem.Use BINARY semaphore for accessCOUNTING semaphores for slots

semaphore access, fullspots, emptyspots;access.count=1; // BINARYfullspots.count=0;emptyspots.count= BUFLEN;

task producer; loop wait (emptyspots); wait (access); DEPOSIT(VALUE); release (access); release (fullspots); end loop;end producer;

task consumer; loop wait (fullspots); wait (access); FETCH(VALUE); release (access); release (emptyspots); end loop;end producer;

Remind you of a printer queue problem?

Statements

Parallel Fortran

Statement-level Parallel Process

PARALLEL LOOP 20 K=1,20 PRIVATE (T) DO 20 I = 1, 500 T=0.0; DO 30 J = 1, 150030 T = T + ( B(J,K)*A(I,J))20 C(I,K)=T