Advanced Operating Systems

COMP9242Introduction

2COMP9242 04S2

Staff

• Lecturer in Charge– Gernot Heiser

• Lecturer – Kevin Elphinstone

• Various Support Staff– TBA

3COMP9242 04S2

Why are you here?

• You’ve done comp3231– Did well (minimum credit)

– You would like to delve deeper into OS issues– You’d like to get your hands really dirty

• Curious about where field is heading.• Thinking about doing a thesis in OS• Thinking about a job in embedded

systems industry

4COMP9242 04S2

What can you expect?

• Challenging Project

• Lectures in general:– Background required for project– Exposure to local research projects– An in-depth look at OS issues

• Building upon the background in COMP3231

– Exposure to recent and seminal research papers– Guest lectures by active researchers (PhD students

and local researchers)

5COMP9242 04S2

Project Goal

Provide students with a deeper understanding of Operating Systems

through practical experience.

• Approach: Participate in the design and implementation of a simple operating system (SOS).

6COMP9242 04S2

Project Aims• Provide experience in OS design and

development, including:– Microkernel based systems (L4::Pistachio).

• User-level OS servers.• User-level page fault handlers.

– Device drivers– Performance evaluation– Implications of cache architectures– Exposure to alternative OS Designs

• Demonstrate the importance of design• Provide experience of being a team member in a

large software project.

7COMP9242 04S2

Project Aims

• Expose students to a mostly realistic OS development environment.– Similar to professional OS and or embedded systems

developer.

• Give an understanding of what’s involved in constructing an entire OS on bare hardware.

• Give an understanding of the interaction between low-level software and hardware.

• Encourage you to undertake a thesis, or do research within OS Group / NICTA.

8COMP9242 04S2

Prerequisites

• Students are expected to be very competent C programmers.

• Students are expected to be familiar with – basic computer architecture concepts.– Assembly language (read-only)

– Basic RISC processor characteristics (we’ll use a MIPS R4600 for the project)

9COMP9242 04S2

Lectures(subject to change)

• Introduction and Overview

• Introduction to the L4 Microkernel– L4 system calls and usage (to get you started on the

project)

• A close look at selected OS issues– Protection, capabilities – Caching, and its implications for OS – Page tables for wide address spaces – SMP issues: locking, cache coherence, scheduling – File systems

10COMP9242 04S2

Lectures(subject to change)

• Microkernels and User-level Servers– History and motivation for microkernel systems,

Hydra, Mach, discussion, experiences; second-generation microkernel systems, L4, Exokernel, Spin; design and implementation of microkernel-based systems, including user-level page fault handling and device drivers

• Microkernel Implementation– Detailed look at a real microkernel (L4Ka::Pistachio).

• Persistent and single-address-space systems• OS Projects at UNSW/NICTA

– Gelato@UNSW, NICTA Embedded OS, Mungi

11COMP9242 04S2

Project/Lab Work• Build a simple operating system (SOS) from the

ground up• Major component of the course• Use L4Ka::Pistachio

– ported to MIPS here by Carl van Schaik

• Develop and test on U4600 computers– R4600 based machines design and built by Kevin

Elphinstone and Dave Johnson– “Clean” machine to get your hands dirty

• Can also use CPU simulator Sulima– Developed locally– Demos must be on real hardware

12COMP9242 04S2

Project

• Some warm-up experiments

• Students will work in groups of two• End goal:

– To produce a small efficient operating system

• Project will have a series of due milestones– Demo to pass the milestone and be awarded marks– Help you manage your time– Avoid major problems

13COMP9242 04S2

Milestones

• Details on class web site• Late milestones:

– Max 1 week late: 25% of milestone mark lost– 1-2 weeks late: 50% of milestone mark lost– More than two weeks: all marks lost

• Bonus task– Much pain for little gain ;-)– Don’t go overboard– Full mark doesn’t require bonus!

14COMP9242 04S2

Alternative Projects

• Special arrangements might be made for particular student to do alternative projects– Must be at least as challenging as the original

project

– Must have some relevance to you or us.– Must convince us that you can actually do it.

– Unable to get bonus marks

15COMP9242 04S2

Assessment

• 65% for project work• 35% for final exam

– A minimum of 14 (40%) required in final exam to pass

• Final Exam– 24hr take-home exam– Read and analyse two recent research papers

and submit a critical report

16COMP9242 04S2

Textbook

• No particular textbook for course– See course web page for useful reference

books

• Selected research papers referred to in the course

• Manuals provided in hardcopy and via class web site

L4 and Microkernels

Background

18COMP9242 04S2HW

Bit Byte Word Register Instructions

Application

documents

windows

symbols

stacks & heaps

arrays & structures

variables

threads

coroutines

modules

procedures

statements

����

�����

���� ����������

������������������ ���������

����

�����������

����������

������

���� ���

���������������

19COMP9242 04S2

Monolithic Kernels - Advantages

• Kernel has access to everything, potentially:– All optimisations are possible

– All techniques/mechanisms/concepts are implementable

• Can be extended by simply adding more code to the kernel

20COMP9242 04S2

Linux Kernel Evolution

Linux Kernel Size (.tar.gz)

0

5

10

15

20

25

30

35

31-Jan-93 28-Oct-95 24-Jul-98 19-Apr-01 14-Jan-04

Date

Siz

e (M

b)

������ ������!

������"#$#%&�'�"#(���������������� ��� �

21COMP9242 04S2

Approaches to tackling complexity

• Monolithic approaches– Layered Kernels

– Modular Kernels– Object Oriented Kernels

• Alternatives– Extensible Kernels

– Microkernels

22COMP9242 04S2

HW

Bit Byte Word Register Instructions

µ-kernel

Address Space Thread

Application

documents

windows

symbols

stacks & heaps

arrays & structures

variables

threads

coroutines

modules

procedures

statements

ServerFile

23COMP9242 04S2

History

• monolithic kernels

24COMP9242 04S2

History• monolithic kernels

• 1st- generation µ-kernels– Mach – Chorus – Amoeba – (L3)

External PagerExternal Pager

UserUser--Level DriverLevel Driver

CMU, OSF

Inria, Chorus

Vrije Universiteit

GMD

25COMP9242 04S2

Brief History

• monolithic kernels

• 1st- generation µ-kernels– Mach – Chorus – Amoeba – (L3)

• 2nd- generation µ-kernels– (Spin) – Exokernel– L4

CMU, OSF

Inria, Chorus

Vrije Universiteit

GMD

U Washington

MIT

GMD / IBM / UKa

UserUser--Level Address SpaceLevel Address Space

External PagerExternal Pager

UserUser--Level DriverLevel Driver

26COMP9242 04S2

HW

L4

classicOS

Security

RTMM

classic +

HW

L4

nativeJava

em-bedded

app

thin

HW

L4

highly-specializedcomponent

specialized

27COMP9242 04S2

HW

L4

SOS

Security

sosh

Project

sosh

28COMP9242 04S2

The Great Promise

• coexistence of different

• APIs

• file systems

• OS personalities

• flexibility

• extensibility

• simplicity

• maintainability

• security

• safety

29COMP9242 04S2

The Great PromiseThe Big Disaster

� ����������������

� ��������������������������������

� �� ���� ���� ���� ��

• coexistence of different

• APIs

• file systems

• OS personalities

• flexibility

• extensibility

• simplicity

• maintainability

• security

• safety

Macro µµµµ

Performance Flexibilty

Simplicity

Performance Flexibility

Simplicity

Macro µµµµ

The 100The 100--µµs s DisasterDisaster

25 MHz 386 50 MHz 486 90 MHz Pentium 133 MHz Alpha

The 100The 100--µµs s DisasterDisaster

34COMP9242 04S2

0

100

200

300

400

0 2000 4000 6000

msg len

Mach

LN + cache flush

LN

raw move

[µs]

µs / ipcIPC Costs (486, 50 MHz)

35COMP9242 04S2

0

50

100

0 50 100 150

msg len

[µs]

Mach

LN0

100

200

300

400

0 2000 4000 6000

msg len

Mach

LN + cache flush

LN

raw move

[µs]

IPC Costs (486, 50 MHz)

36COMP9242 04S2

L4 IPC

0.73 µs (Pentium 166 MHz)

0.91 µs (R4600 100 MHz)

0.10 µs (21164 433 MHz)

[cycles]

236

180

200

50

100

150

200

250

Pent III P3 Sysops P3 Lipc ?

0.47 µs (P III 500 MHz)

0.36 µs (P III 500 MHz)

0.04 µs (P III 500 MHz)

(hopefully)

82

16

23

36

55

7

38

0

20

40

60

80

100

120

140

Pentium R4600 Alpha

37COMP9242 04S2

Minimality

Elegance

Architectural Integration

Efficiency

Flexibility

• A µ-Kernel does the Job• if Properly Designed

• if Carefully Implemented

Thesis:

38COMP9242 04S2

When analyzing IPC performance,

Cycles are not the only the to consider!!

39COMP9242 04S2

Processor-DRAM Gap (latency)

µProc60%/yr.

DRAM7%/yr.

1

10

100

100019

8019

81

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

DRAM

CPU

1982

Processor-MemoryPerformance Gap:(grows 50% / year)

Per

form

ance

Time

“Moore’s Law”

��� ������������� ����)����������*������+&

40COMP9242 04S2

Today’s Situation: Microprocessor

• Microprocessor-DRAM performance gap– time of a full cache miss in instructions executed1st Alpha (7000): 340 ns/5.0 ns = 68 clks x 2 or 1362nd Alpha (8400): 266 ns/3.3 ns = 80 clks x 4 or 3203rd Alpha (t.b.d.): 180 ns/1.7 ns =108 clks x 6 or 648

– 1/2X latency x 3X clock rate x 3X Instr/clock � �5X

��� ������������� ����)����������*������ +&

41COMP9242 04S2

99.85%

0.15%

L2 cache

• 8192 cache lines (256K)

• 12 lines used for IPC

98.83%

1.17%

L1 cache

• 1024 cache lines (16K + 16K)

• 12 lines used for IPC

Cache Working Sets

42COMP9242 04S2

Other Complications

• P4 trace cache– A cache of recently translated µ-ops

– Flushed on every page-table switch

• Virtual Caches– Need to be flushed on address space switch

43COMP9242 04S2

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

100 200 400 800 1600 3200 6400 12800 25600 51200 102400 204800

Mach 486

Mach Alpha

Chorus 486

Spin Alpha

L4 Pentium

average cycles between successive IPCs

ove

rhea

d d

ue

to IP

C

44COMP9242 04S2

Α µΑ µΑ µΑ µ-kernel does no real work.µµµµ-Kernel services are only required to overcome µµµµ-kernel constraints.

Therefore, µµµµ-kernels have to be infinitely fast!

Minimality is the key!

•• ThreadsThreads

•• Address SpacesAddress Spaces

IPCIPC

MappingMapping

45COMP9242 04S2

Threads, CommunicationAddressSpaces

Threads

46COMP9242 04S2

Drivers at User Level

User

DriverDevice

INTR

= ipc• IO ports: part of the user address space

• interrupts: messages from hardware

47COMP9242 04S2

Address Spaces

48COMP9242 04S2

Address Spaces

• map

• unmap

• grant

49COMP9242 04S2

Address Spaces

• map

• unmap

• grant

50COMP9242 04S2

Address Spaces

• map

• unmap

• grant

51COMP9242 04S2

Page Fault Handling

Pager

"PF" msg

map msg

Application

52COMP9242 04S2

Page Fault Handling

Pager

"PF" msg

map msg

Application

PF IPC

res IPC

53COMP9242 04S2

Address Spaces

Physical Memory

54COMP9242 04S2

Address Spaces

Physical Memory

Initial AS

55COMP9242 04S2

Address Spaces

Physical Memory

Initial AS

Pager 1 Pager 2

56COMP9242 04S2

Address Spaces

Physical Memory

Initial AS

Pager 1 Pager 2

Pager 3

Pager 4

57COMP9242 04S2

Address Spaces

Physical Memory

Initial AS

Pager 1 Pager 2

Pager 3

Pager 4

Application

Application Application

Application

58COMP9242 04S2

Address Spaces

Physical Memory

Initial AS

Pager 1 Pager 2

Pager 3

Pager 4

Application

Application Application

Application

Driver

Driver

59COMP9242 04S2

Mach Virtual MemoryIn comparison

Physical Memory

Paging Policy����

Application Application

Application

External Pager�� ����,��

60COMP9242 04S2

Size Comparison������-������� ����.!

"#(��������������

�����$��&/!

+0*000������

�$1�!!�������2��3"

%0*000������

61COMP9242 04S2

HW

LN

classicOS

Security

RTMM

classic +

HW

LN

nativeJava

em-bedded

app

thin

HW

LN

highly-specializedcomponent

specialized