cs4411 – operating systems practicum september 30, 2011 zhiyuan teo supplementary lecture 2

cs4411 – Operating Systems Practicum

September 30, 2011Zhiyuan Teo

Supplementary lecture 2

•Administrative Information

•Common mistakes on Project 1

Today’s lecture

•Project 2 FAQ

•Technical supplement 1

•Discussion

•Project 2 deadline has been extended to 11.59pm 5th October.

•Project 1 has been graded – please review comments and make amendments.

Administrative Information

- Regrade requests: through CMS or in person.

•Don’t need provide as much abstraction as in data structure courses.

•We prize correctness, but also elegance.

Project 1: general

•Comments and README.

•zip files and file content.- don’t create an extra folder to hold the content.- submit only changed source files, in addition to README.- don’t tar and rezip.- don’t encrypt!

Project 1: general

•Not checking for errors.- malloc()- queue_new(), queue_delete(), etc.- asserts are not checks!- should not exit unless there is no way for even the user to recover.

•Excessive debug output.

•Indentation and code readability.

•queue_delete/dequeue

•queue_append/prepend

Common mistakes: queues

- forgetting to set next or prev pointers.

- bugs when deleting from head/tail: pointers not updated or illegal access.- not returning -1 when item to delete isn’t found in queue.- not freeing up queue node after updating list.- forgetting to update head/tail pointers.

•queue_iterate- not checking the return value from function pointer.- short-circuit evaluation.

•semaphore_V

•semaphore_create vs semaphore_initialize

Common mistakes: semaphores

- creating the blocked queue in semaphore_initialize.

- relying on a blind dequeue or queue length to unblock threads.

•semaphore_destroy- forgetting to free up the blocked queue.

•TAS locks- yielding instead of spinning.- homemade TAS code.- non-atomic clears.

•minithread_yield

•minithread_fork/create

Common mistakes: minithreads

- reimplementing create in fork (by cutting and pasting code).

- self-switching.

•minithread_stop- beware of the semantics.- comments for minithread_stop indicate that it is used for blocking threads.- not to be used in place of minithread_yield, even if scheduling results are correct.

•cleanup thread

•cleanup handler

Common mistakes: minithreads

- directly starting the cleanup thread through minithread_start.- forcing the cleanup thread to run immediately (via context switch).- forcing the next yield to run the cleanup thread.

- blindly calling dequeue before semaphore_P.- freeing multiple items per semaphore_V called on it.

•global wait queue- replicating functionality already achieved in semaphores.- inefficient to delete.

•units of time

•malloc and library calls

Project 2 FAQ

- all library calls are safe: interrupts are automatically disabled upon calling. interrupts will be restored to its original state (enabled/disabled) after the call.

- PERIOD is defined as 50 ms, which is 50000 as a constant.- alarm and wakeup delays are specified in milliseconds.- you have to convert units; don’t blindly subtract PERIOD.

•irregular/random clock interrupts- this is normal.- be careful of introducing heisenbugs because of your debug statements.

•Typically manipulations on shared data structures.

•When you need to do something that must be done atomically.

When to disable interrupts

- data structures that can be accessed by multiple threads ‘simultaneously’.- eg. modifying the cleanup queue, ready queue, alarm list.

•Trivial way of achieving correctness: disable interrupts for everything.

-Why is this a bad idea?

•typical sem_P code:

Semaphores revisited

while (TAS(&lock) == 1);

sem->counter--;if (sem->counter < 0){ append thread to blocked queue atomically unlock and stop}else{ atomic_clear(&lock);}

•typical sem_V code:

Semaphores revisited


sem->counter++;if (sem->counter <= 0){ take one thread from blocked queue start the thread}

atomic_clear(&lock);

•Interrupts are not disabled in sem_P and sem_V.

Semaphore design considerations

•…but semantics must still be correct even if used in an interrupt-enabled environment.

•Semaphores should also be safely usable from places where interrupts are disabled.

•Why not just disable interrupts inside sem_P/V?

•Interrupts can arrive at any time.

Semaphore use in user threads

•If interrupts arrive while a TAS lock is held:- another thread that tries to acquire the TAS lock will spin until its time quanta is exhausted.- thread holding the TAS lock will eventually regain control and make progress.- progress ensures other threads can eventually get the TAS lock.


sem->counter--;if (sem->counter < 0){ append thread to blocked queue atomically unlock and stop}else atomic_clear(&lock);




semaphore_P semaphore_V

•Typically used to block some thread and wake it up on some condition.

Semaphore use in the kernel

- minithread_sleep_with_timeout()- wake up the thread after the elapsed time.

•Sleep with timeout requires calling sem_P on the TCB’s sleep semaphore. Done in user space.

•Waking up requires calling sem_V on that sleep semaphore. Done in kernel space with interrupts disabled.

struct tcb { int tid; … semaphore_t sleep_sem;};

•What if user calls sleep_with_timeout(0) ?

An unfortunate interleaving

- sem_P is called, and thread blocks itself.

•What if sem_P was interrupted just after placing thread on blocked queue but before clearing TAS lock?


sem->counter--;if (sem->counter < 0){ append thread to blocked queue

atomically unlock and stop}else atomic_clear(&lock);




user calls sleep_with_timeout(0)… …clock handler tries to wake that thread up

clock interrupt!

but interrupts are disabled in the clock handler!

•Disable interrupts for sem_P and sem_V.

Solution

- atomicity: sem_P will be done with everything before an interrupt can possibly arrive.- if interrupt arrives, acquisition of TAS lock is guaranteed in kernel space.

“You should disable interrupts first before making calls to semaphore_P from within your kernel code.”

•What about sem_V?- sem_V is called from interrupt handler.- interrupts are already disabled in the handler.

-project 2 slides

•If semaphore will be used in portions of your kernel where interrupts are disabled.

When is this applicable?

- right now: only the sleep semaphore.

•What about cleanup semaphore?- Cleanup semaphore is not signaled from any place where interrupts are disabled.

•Cleanup code should only disable interrupts while accessing the cleanup queue, not for semaphore signaling.

•Seems logical to yield if TAS lock cannot be acquired.

Why spin on TAS? Why not yield?







- behavior would still be correct in user threads.

while (TAS(&lock) == 1) yield();





•But interrupts are enabled after the yield.

Why spin on TAS? Why not yield?







- not a problem in user threads, since they should run with interrupts enabled.- presents problems in the kernel where you want interrupts to stay disabled when using semaphores.

No longer safe from interrupts;interrupt handler can be interrupted.

•Suppose interrupts are not disabled before calling sem_P/V in the kernel.

How a problem may arise

1. Suppose clock handler was invoked to service a clock interrupt. It runs with interrupts disabled.

void clock_handler() {

disable_interrupts();

while(sleeping_threads > 0) { if (time_to_wake_thread) sem_V(&tcb->sleep_semaphore); }

restore_interrupt_level();}

clock handler


2. Clock handler is unable to acquire TAS lock, so it yields.




semaphore_V()


3. Control returns after yield. Interrupts are now enabled. Suppose TAS lock succeeds (previous thread gave it up).




semaphore_V()


4. A network interrupt arrives and calls sem_V because it wants to wake up the thread.




semaphore_V()

network interrupt!

void network_handler() {

disable_interrupts();

sem_V(&net_tcb->sleep_semaphore); restore_interrupt_level();}

network handler


5. Network handler yields because it cannot acquire TAS lock.

No amount of yielding will allow progress in this scenario. Why?

Nested interrupts

Thread A

Clock interrupt handler

Network interrupt handler

•Network interrupt handler is waiting for clock handler to finish, but this is impossible.

•Yielding does not unwind the call stack.

•Only way to do this is for network interrupt handler to return.

•An interrupt is like an unscheduled function call.

0x7fffffff

0x00000000

Program text, heap, etc.

Q & A for Project 2

Technical supplement roadmap

•How pointer casts work.

•How the stack works; function calls and context switches; anatomy of buffer overflow & stack smashing exploits.

•x86 history and a (very short) introduction to x86 assembly.

•None of these are examinable nor strictly needed for 4410/4411 but are good to know as OS designers.

Pointer casting

•A pointer has two components:- starting address of the data being pointed to.- size of data type associated with this memory location.

0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0

int a = 5;int *x = &a;

0x00001000

0x00001004

x = 0x00001004*x = 0x00000005

pointer x: data starts from 0x00001004 and spans 4 bytes

Pointer casting

•Casting the pointer changes only the way the memory is interpreted.

int a = 5;int *x = &a;

x = 0x00001004*x = 0x00000005y = 0x00001004*y = 0x00

•It does not change the memory contents nor the address stored in the pointer.

0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0

0x00001000

0x00001004

pointer x: data starts from 0x00001004 and spans 4 bytes

char *y;y = (char*) x;

0 0 0 50

pointer y: data starts from 0x00001004 and spans 1 byte

Pointer casting

•We can change the values from the casted pointer if we like.

int a = 5;int *x = &a;char *y = (char*) x;

x = 0x00001004*x = 0x01000305y = 0x00001004*y = 0x01

•No array bounds check in C!

0 0 0 0 0 0 0 0 0 0 0 0

0x00001000

0x00001004

0 0 50

*y = 1;

1

y[2] = 3;

3

Structures

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0x00001000

struct queueable { struct queueable* next; struct queueable* prev;};

struct tcb { struct queueable reserved; int tid; semaphore_t sleep_sem;};

base struct ‘derived’ struct

struct queueable* queue_node;struct tcb* idle_tcb;

pointer queue_node: data starts from 0x00001000 and spans 8 bytes (struct is aligned)

0 0 0 0 0 0 0 0

next prev

Structures

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0x00001000

struct queueable { struct queueable* next; struct queueable* prev;};

struct tcb { struct queueable reserved; int tid; semaphore_t sleep_sem;};

base struct ‘derived’ struct

struct queueable* queue_node;struct tcb* idle_tcb;

pointer idle_tcb: data starts from 0x00001000 and spans 16 bytes (struct is aligned)

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

tid sleep_semreserved

Common questions on casting

•Casting from a derived struct pointer to the base struct pointer is safe.

- any operation that works on data members of the base struct will not write outside of the space allocated for the derived struct.

0 0 0 0 0 0 0 0

pointer idle_tcb: data starts from 0x00001000 and spans 16 bytes (struct is aligned)

c a f e b a b e b a a d f 0 0 d

pointer queue_node: data starts from 0x00001000 and spans 8 bytes.

d e a d b e e f

manipulation of next and prev fields in queue_node will only affect the first 8 bytesof the tcb structure.

other fields in the tcb struct (tid and sleep_semaphore) are not affected.

struct queue_node* node = (struct queue_node*) idle_tcb;node->prev = 0xdead; node->next = 0xbeef;

Common questions on casting

•What about casting from base struct to derived struct?- unless you are very sure you will not touch data fields other than those in the base struct.- but if that is the case, why did you perform the cast?

struct tcb* some_tcb = (struct tcb*) queue_node;

some_tcb->reserved.next = 0xdead;some_tcb->reserved.prev = 0xbeef;

some_tcb->tid = 1;

•Writing outside of known bounds may cause unpredictable behavior.

- may overwrite another variable.- may overwrite stack (and thus modify return address; a common exploit). - usually just crashes.

void* pointers

•Review: a pointer has two components.- starting address of the data being pointed to.- size of data type associated with this memory location.

•void* pointers are not associated with any data type.- has only a starting address.- no known size, therefore not possible to tell where data ends in memory.- malloc() is a good example.

•casting to void* from other pointer types.- automatic, no need to prepend (void*).- other pointers can easily cast to void*; just ignore the size of the data type.

•casting to other pointer types from void*.- explicit cast required.- destination data type supplies the size of the data type.

cs4411 – operating systems practicum september 30, 2011 zhiyuan teo supplementary lecture 2

Documents

malloc queue

blocked queue

deletedequeue queue

queue length

queue node

discussion slide

v semaphore

yield minithread