operating systems - queuing systems

UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

Operating SystemsCMPSCI 377

Queuing SystemsEmery Berger

University of Massachusetts Amherst


Queuing Systems & Servers

Queuing systems

High-level model of concurrent applications

Flux

Language for building servers

2


Queuing Networks

Model of tasks or services

Node includes queue (line) & server

3


Queuing Networks



4


Queuing Networks



5


Queuing Networks



6


Queuing Networks



7

arrival rate()


Queuing Networks



8

waiting time


Queuing Networks



9

service time


Queuing Networks



10


Stable Systems

Stable queuing system:arrival rate = departure rate

11


Stable Systems


12


Stable Systems


13


Stable Systems


What happens if > departure rate?

14


Stable Systems



15


Stable Systems



16


Stable Systems



17


Networks of Queues

Can build system from connected servers

Latency = time for one thing to get through

Throughput = service rate

Throughput?

18

5/sec 5/sec 5/sec


Networks of Queues

Can build system with numerous connected servers

Latency = time for one thing to get through

Throughput = service rate

Throughput?

Lowest throughput = bottleneck19

5/sec 1/sec 5/sec


Little’s Law

Little’s Law – applies to any “blackbox” server

Queue length (N) =arrival rate () * average waiting time (T)

N = T

20

T

N


Applications of Little’s Law

Compute waiting time to get into restaurant, bar, etc.

If N = 20 people in front of you, = departure rate = 1 / 5 min.,how long will you wait in line?

21

T

N N = T



Required service time?

Arrival rate = one job @ 500 ms

Average queue length = 10

T = ?

What’s the average latency?

22

T

N N = T



Required service time?

Arrival rate = one job @ 500 ms

Average queue length = 5

T = ?

What’s the average latency?

23

T

N N = T


Motivating Example: Image Server

Client

Requests image @ desired quality, size

Server

Images: RAW

Compresses to JPG

Caches requests

Sends to client

http://server/Easter-bunny/200x100/75

not found

client

imageserver

http://www.flickr.com/photos/andysum/127382782/

http://www.flickr.com/photos/lawrence_evil/127052519/

http://www.flickr.com/photos/95466938@N00/126802300/

http://www.flickr.com/photos/r47z/126159584/

http://www.flickr.com/photos/jollyjoy/126604796/

http://www.flickr.com/photos/hudlu/127974417/

http://www.flickr.com/photos/trashd/128818423/

http://www.flickr.com/photos/ekarjala/126075837/

http://www.flickr.com/photos/dotlyc/128794472/

http://www.flickr.com/photos/ilumb/125871761/

http://www.flickr.com/photos/forbidden_emotions/128092082/

http://www.flickr.com/photos/robby/126371817/

http://www.flickr.com/photos/doc18/128640138/



Problem: Concurrency

Could write sequential code but…

More clients (latency)

Bigger server Multicores, multiprocessors

One approach: threads

Limit reuse,risk deadlock,burden programmer

Complicate debugging

Mixes program logic & concurrency control clients

imageserver

http://www.flickr.com/photos/andysum/127382782/

http://www.flickr.com/photos/lawrence_evil/127052519/

http://www.flickr.com/photos/95466938@N00/126802300/

http://www.flickr.com/photos/r47z/126159584/

http://www.flickr.com/photos/jollyjoy/126604796/

http://www.flickr.com/photos/hudlu/127974417/

http://www.flickr.com/photos/trashd/128818423/

http://www.flickr.com/photos/ekarjala/126075837/

http://www.flickr.com/photos/dotlyc/128794472/

http://www.flickr.com/photos/ilumb/125871761/

http://www.flickr.com/photos/forbidden_emotions/128092082/

http://www.flickr.com/photos/robby/126371817/



The Flux Programming Language

Flux = Components + Flow + Atomicity

Components unmodified C, C++ (or Java)

Flow implicitly || path thru components

Atomicity high-level mutual exclusion

Compiler generates:

Deadlock-free, runtime-independent server

Threads, thread pools, events, …

Path profiling

Discrete event simulator

High-performance & deadlock-freeconcurrent programming w/ sequential components


Flux Outline

Intro to Flux: building a server Components

Flows

Atomicity

Performance results Server performance

Performance prediction (QNMs)


Flux Server “Main”

Source nodes originate flows

Conceptually in separate thread

Executes inside implicit infinite loop

Here: initiates flow for each image request

ReadRequest WriteCompress CompleteListen

ReadRequest WriteCompress CompleteReadRequest WriteCompress Complete

source Listen Image;

image server


Flux Image Server

ReadRequest WriteCompress Complete

libjpeg sockethttp http

Basic image server requires: HTTP parsing (http) Socket handling (socket) JPEG compression (libjpeg)All UNIX-style C libraries

Abstract node = flow across nodes Concrete or abstract

Image = ReadRequest Compress Write Complete;

image server


Listen

Control Flow

Direct flow via user-supplied predicate types

Type test applied to output Note: no variables – dispatch on output “type”

Here: cache frequently requested images

ReadRequest

ReadInFromDisk

WriteCheckCache

Compress StoreInCache

Completehit

typedef hit TestInCache;Handler:[_,_,hit] = ;Handler:[_,_,_] = ReadFromDisk Compress StoreInCache;

handler

handler


Listen

ReadRequest

ReadInFromDisk

WriteCheckCache


CompletehitReadRequest

ReadInFromDisk

WriteCheckCache


Completehit

Supporting Concurrency

Many clients = concurrent flows

Must keep cache consistent

Atomicity constraints

Same name = mutual exclusion

Apply to nodes or whole flow (abstract node)

ReadRequest

ReadInFromDisk

WriteCheckCache


Completehit

atomic CheckCache {};atomic Complete {, };atomic StoreInCache {};

ReadRequest

ReadInFromDisk

WriteCheckCache


Complete

hit

handler

handler


More Atomicity

Reader / writer constraints

Multiple readers or single writer (default)

atomic ReadList: {listAccess?};atomic AddToList: {listAccess!};

Per-session constraints

User-supplied function ≈ hash on source

Added to flow ≈ chooses from array of locks

atomic AddHasChunk: {chunks(session)};

UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 33

Preventing Deadlock

Naïve execution can deadlock

Establish canonical lock order

Partial order

Alphabetic by name

atomic A: {z,y};atomic B: {y,z};

atomic A: {y,z};atomic B: {y,z};


Preventing Deadlock, II

A = B;C = D;atomic A{z};atomic B{y};atomic C{y,z};

B

AC

B

A:{z}C

B

A{z}C{y}

A = B;C = D;atomic A{y,z};atomic B{y};atomic C{y,z};

Harder with abstract nodes

Solution: Elevate constraints; fixed point

B

A{y,z}C


Listen

Handling Errors

What if image requested doesn’t exist?

Error = negative return value from component

Remember – nodes oblivious to Flux

Solution: error handlers

Go to alternate paths on error

Possible extension – can match on error paths

FourOhFour

handle error ReadInFromDisk FourOhFour;

ReadRequest

ReadInFromDisk

WriteCheckCache


Complete

hit

handler

handler


Almost Complete Flux Image Server

Listen

imageserver

Concise, readable expression of server logic No threads, etc.: simplifies programming, debugging

source Listen Image;Image = ReadRequest CheckCache Handler Write Complete;

Handler[_,_,hit] = ;

Handler[_,_,_] = ReadFromDisk Compress StoreInCache;

atomic CheckCache: {cacheLock};atomic StoreInCache: {cacheLock};atomic Complete: {cacheLock};

handle error ReadInFromDisk FourOhFour;

ReadRequest

ReadInFromDisk

WriteCheckCache


Complete

hit

handler

handler


Flux Outline

Intro to Flux: building a server

Components, flow

Atomicity, deadlock avoidance

Performance results

Server performance

Performance prediction

Future work


Flux Results

Four servers: Image server (+ libjpeg) [23 lines of Flux]

Multi-player online game [54]

BitTorrent (2 undergrads: 1 week!) [84]

Web server (+ PHP) [36]

Evaluation Benchmark: variant of SPECweb99

Three different runtimes here Thread: one per connection

Thread pool: fixed max # threads

Event-driven: helper threads for blocking calls

Compared to Capriccio [SOSP03], SEDA [SOSP01]


Web Server


Performance Prediction

observedparameters


The End

operating systems - queuing systems

Technology