linear regions are all you need

Linear Regions Are All You Need

Matthew Fluet

Cornell University

Greg Morrisett & Amal Ahmed

Harvard University

2

Memory Management

Dynamic allocation pervasive in computation

3

Memory Management

Dynamic allocation pervasive in computation

Region-based Memory Management– Memory is divided into regions– Objects are individually allocated in a region

constant-time operation

– All objects in a region are deallocated together constant-time operation

4

Application: Cyclone

Cyclone Safe-C Project– type-safety– with the “virtues” of C

low-level interface with manifest cost model

– range of memory management options regions are an organizing principle

5

Cyclone: Regions

Region varietyAllocation

(objects)

DeallocationAliasing

(objects)(what) (when)

Stack static

whole region

exit of lexical scope

unrestricted

Lexical

dynamic

Dynamicmanual

Dynamic seq.

Heap (`H)

single objects

automatic(BDW GC)

Unique (`U)

manual restrictedRef-counted (`RC)

6

Application: Cyclone

MediaNET– TCP benchmark (packet forwarding)– Cyclone v.0.1 (lexical regions & garbage collector)

High water mark: 840 KB 130 collections Basic throughput: 50 MB/s

– Cyclone v.0.5 (unique pointers & dynamic regions) High water mark: 8 KB 0 collections Basic throughput: 74MB/s

7

Application: Cyclone

MediaNET– TCP benchmark (packet forwarding)– Cyclone v.0.1 (lexical regions & garbage collector)

High water mark: 840 KB 130 collections Basic throughput: 50 MB/s

– Cyclone v.0.5 (unique pointers & dynamic regions) High water mark: 8 KB 0 collections Basic throughput: 74MB/s

8

Cyclone: Regions

Region varietyAllocation

(objects)

DeallocationAliasing

(objects)(what) (when)

Stack static

whole region

exit of lexical scope

unrestricted

Lexical

dynamic

Dynamicmanual

Dynamic seq.

Heap (`H)

single objects

automatic(BDW GC)

Unique (`U)

manual restrictedRef-counted (`RC)

Proving type safety of Cyclone is a nightmare!!

9

Cyclone: Regions

Region varietyAllocation

(objects)

DeallocationAliasing

(objects)(what) (when)

Stack static

whole region

exit of lexical scope

unrestricted

Lexical

dynamic

Dynamicmanual

Dynamic seq.

Heap (`H)

single objects

automatic(BDW GC)

Unique (`U)

manual restrictedRef-counted (`RC)

Goal: simple model where we can easily encode the key features of Cyclone in a target language with

a simpler type system.

10

Cyclone: Regions

Region varietyAllocation

(objects)

DeallocationAliasing

(objects)(what) (when)

Stack static

whole region

exit of lexical scope

unrestricted

Lexical

dynamic

Dynamicmanual

Dynamic seq.

Heap (`H)

single objects

automatic(BDW GC)

Unique (`U)

manual restrictedRef-counted (`RC)

Linear RegionsAre All You Need

11

Outline

Introduction

Monadic Type System (FRGN) [ICFP’04]

Substructural Type System (rgnUL)– Translation Sketch

Conclusion

12

Monadic Type System for Regions [ICFP’04]

Extend the runST “trick” to nested regions [L-PJ ’94]

– Polymorphic type system ensures safety

Key insights (FRGN):– Effects map to an indexed monadic type– Region subtyping witnessed by types– Sufficient for encoding Tofte-Talpin region calculus

and “core” Cyclone region features

13

RGN monad: Types

Monadic type

RGN computations in stack of regions returning values of type ;a “stack” transformer

14

RGN monad: Operations

Monadic unit and bind

returnRGN ::

8,. ! RGN

thenRGN ::

8,,. RGN ! ( ! RGN ) ! RGN

15

RGN monad: Operations

Monadic unit and bind

returnRGN ::

8,. ! RGN

thenRGN ::

8,,. RGN ! ( ! RGN ) ! RGN

16

RGN monad: Operations

Monadic unit and bind

returnRGN ::

8,. ! RGN

thenRGN ::

8,,. RGN ! ( ! RGN ) ! RGN

17

RGN monad: Types

Reference type

Ref values of type allocated in region

at the top of the stack of regions

18

RGN monad: Operations

Create and read region allocated values

new ::

8,. ! RGN (Ref )

read ::

8,. Ref ! RGN

19

RGN monad: Operations

Create and read region allocated values

new ::

8,. ! RGN (Ref )

read ::

8,. Ref ! RGN

20

RGN monad: Encapsulation

Encapsulate and run a monadic computation

runRGN ::

8. (8. RGN ) !

21

RGN monad: Encapsulation

Encapsulate and run a monadic computation

runRGN ::

8. (8. RGN ) !

22

RGN monad: Encapsulation

Encapsulate and run a monadic computation

runRGN ::

8. (8. RGN ) !

“for all stacks” ) no assumptions about

stack of regions

23

RGN monad: Encapsulation

Encapsulate and run a monadic computation

runRGN ::

8. (8. RGN ) !

“for all stacks” ) no assumptions about

stack of regions

24

RGN monad: Encapsulation

Encapsulate and run a monadic computation

runRGN ::

8. (8. RGN ) !

result is independent of stack ) 62 frv() )

region values don’t escape

“for all stacks” ) no assumptions about

stack of regions

25

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

1

26

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

1 a : 1

inputallocated in first region

27

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

1

2

a : 1

input allocated in first region

28

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

1

2

a : 1

b : 7

temporary allocated in second region

inputallocated in first region

29

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

1

2

a : 1

c : 8

b : 7

temporary allocated in second region

input and outputallocated in first region

30

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

1 a : 1

c : 8

temporary allocated in second region

input and outputallocated in first region

31

RGN monad: Example

runRGN ( 1.

do a à new [1] 1

c à runRGN ( 2.

do b à new [2] 7

… z = …

new [1] z )

… c … )

allocating in older region

requires RGN 1 type

allocating in younger region

requires RGN 2 type

32

RGN monad: Witnesses

Witness type

Pf(1 · 2) –

type-level proof that the stack of regions 1

is a substack of the stack of regions 2

33

RGN monad: Witnesses

Witness operations

coerceRGN ::

81,2,. Pf(1 · 2) ! RGN 1 ! RGN 2

transSub ::

81,2,3. Pf(1 · 2) ! Pf(2 · 3) ! Pf(1 · 3)

34

RGN monad: Regions

Regions are created and destroyedwith a lexically scoped construct

letRGN ::

81,. (82. Pf(1 · 2) ! RGN 2 ) ! RGN 1

35

RGN monad: Regions

Regions are created and destroyedwith a lexically scoped construct

letRGN ::

81,. (82. Pf(1 · 2) ! RGN 2 ) ! RGN 1

36

RGN monad: Example

letRGN ( 1. pf1.

do a à new [1] 1

c à letRGN ( 2. pf2.

do b à new [2] 7

… z = …

coerceRgn pf (new [1] z ))

… c … )

1

2

a : 1

c : 8

b : 7

37

Limitations of LIFO Regions

Lexical scope is ill-suited for– iterative computations

Conway’s Game of Life; copying GC

– CPS-based computations– event-based computations

38

Limitations of LIFO Regions

Lexical scope is ill-suited for– iterative computations

Conway’s Game of Life; copying GC

– CPS-based computations– event-based computations

But, lexical scope was ensuring that the stack of regions was used in a single-threaded manner

39

Substructural Type Systems

Provide core mechanisms to restrict the number and order of uses of data and operations– generalization of linear type systems

40

Substructural Type System: UL

Qualifiers

q ::= U j L

PreTypes

::= 1 j 1 £ 2 j 1 ! 2 j 8. j 9.

Types

::= q

41

Substructural Type System: UL

Qualifiers

q ::= U j L

PreTypes

::= 1 j 1 £ 2 j 1 ! 2 j 8. j 9.

Types

::= q

How maythe value be used?

42

Substructural Type System: UL

Qualifiers

q ::= U j L

PreTypes

::= 1 j 1 £ 2 j 1 ! 2 j 8. j 9.

Types

::= q

How maythe value be used?

How often maythe value be used?

43

Substructural Qualifiers

UnrestrictedDrop Copy

Linear

must be “used” exactly once

may be “used” an arbitrary # of times

44

Substructural Type System for Regions

Provide core mechanisms to restrict the number and order of uses of data and operations– generalization of linear type systems

Key insights (rgnUL):– Separate region names from region liveness– Region liveness witnessed by types– Sufficient for encoding FRGN calculus

and “advanced” Cyclone region features

45

rgnUL = UL + Regions

PreTypes

::= … j cap j ref j 8. j 9.

“capability” for region ;mediates all access to a region

for allocating, reading, and writing

46

rgnUL: Region Primitives

Regions are created and destroyedwith separate operations

newrgn ::U1 ! (9. Lcap )

freergn ::

8. (Lcap ! U1)

47

rgnUL: Region Primitives

Regions are created and destroyedwith separate operations

newrgn ::U1 ! (9. Lcap )

freergn ::

8. (Lcap ! U1)

Produces a capability.

Consumes a capability.

48

rgnUL: Region Primitives

Regions are created and destroyedwith separate operations

newrgn ::U1 ! (9. Lcap )

freergn ::

8. (Lcap ! U1)

49

rgnUL: Region Primitives

new ::

8,. ((Lcap £ U) !(Lcap £ Uref U))

read ::

8,. ((Lcap £ Uref U) !(Lcap £ U))

50

rgnUL: Region Primitives

new ::

8,. ((Lcap £ U) !(Lcap £ Uref U))

read ::

8,. ((Lcap £ Uref U) !(Lcap £ U))

Returns a capability.

Requires a capability.

51

rgnUL: Region Primitives

new ::

8,. ((Lcap £ U) !(Lcap £ Uref U)

read ::

8,. ((Lcap £ Uref U) !(Lcap £ U)

52

Translation: FRGN to rgnUL, Types

« RGN ¬ = U( ! L( £ «¬))

53

Translation: FRGN to rgnUL, Types

« RGN ¬ = ! ( £ «¬)

– operational behavior of monad is store/stack-passing

54

Translation: FRGN to rgnUL, Types

« RGN ¬ = ! ( £ «¬)

– operational behavior of monad is store/stack-passing

55

Translation: FRGN to rgnUL, Types

« RGN ¬ = ! ( £ «¬)

– operational behavior of monad is store/stack-passing– represent “stack of regions”

as a sequence of linear capabilities,formed out of nested linear tuples

56

Translation: FRGN to rgnUL, Types

« RGN ¬ = ! ( £ «¬)

– operational behavior of monad is store/stack-passing– represent “stack of regions”

as a sequence of linear capabilities,formed out of nested linear tuples

57

Translation: FRGN to rgnUL, Ops

« returnRGN [] [] e ¬ =let res : «¬ = «e¬ inUstk:. Lhstk,resi

« thenRGN [] [a] [b] e1 e2 ¬ =let f : «RGN a¬= «e1¬ inlet g : «a ! RGN b¬ = «e2¬ inUstk:. let hstk,resi = f stk in g res stk

58

Translation: FRGN to rgnUL, Ops

« returnRGN [] [] e ¬ =let res : «¬ = «e¬ inUstk:. Lhstk,resi

« thenRGN [] [a] [b] e1 e2 ¬ =let f : «RGN a¬= «e1¬ inlet g : «a ! RGN b¬ = «e2¬ inUstk:. let hstk,resi = f stk in g res stk

Store-passing

encoding

59

Translation: FRGN to rgnUL, Types

« Pf(1 · 2) ¬ = U(9’. Iso(2, L(1 £ ’)))

60

Translation: FRGN to rgnUL, Types

« Pf(1 · 2) ¬ = U(9’. Iso(2, L(1 £ ’)))

– Isomorphism between 2 and L(1 £ ’), for some “slack” ’

61

Translation: FRGN to rgnUL, Types

« Pf(1 · 2) ¬ = U(9’. Iso(2, L(1 £ ’)))

– Isomorphism between 2 and L(1 £ ’), for some “slack” ’

– Proof that 1 is a substack of 2 is persistent

– Liveness of 1 and 2 is ephemeral

62

Translation: FRGN to rgnUL, Types

« Ref ¬ = U(9. U(U(9’. Iso(, L(’ £ Lcap )))« Ref ¬ = U(9. U(£ Uref «¬))

63

Translation: FRGN to rgnUL, Types

« Ref ¬ = 9. (9’. Iso(, L(’ £ Lcap ))« Ref ¬ = 9. (£ Uref «¬)

Existential fixes region

64

Translation: FRGN to rgnUL, Types

« Ref ¬ = 9. (9’. Iso(, L(’ £ Lcap ))« Ref ¬ = 9. (£ Uref «¬)

Existential fixes region Isomorphism witnesses membership of in

65

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! Hnd 2 ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,hcap,hndi) = newrgn Lhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let phnd = Upack(,UhUpack(1,Uhid,idi),hndi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit phnd stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] Lhcap,hndi inUstk1:1.Lhstk1,resi

66

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,cap) = newrgn Uhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] cap inUstk1:1.Lhstk1,resi

Stack-passing encoding

67

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,cap) = newrgn Uhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] cap inUstk1:1.Lhstk1,resi

Create & destroy a new region

68

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,cap) = newrgn Uhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] cap inUstk1:1.Lhstk1,resi

Construct rep.of new stack

69

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,cap) = newrgn Uhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] cap inUstk1:1.Lhstk1,resi

Run comp. and recover old stack

and new cap

70

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,cap) = newrgn Uhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] cap inUstk1:1.Lhstk1,resi

Construct isomorphism

71

Translation: FRGN to rgnUL, Ops

« letRGN [1] [] e ¬ = let f : «82. Pf(1·2) ! RGN 2 ¬ = «e¬ inUstk1:1.let pack(,cap) = newrgn Uhi inUstk1:1.let stk2 = Lhstk1,capi inUstk1:1.let id = Ustk: L(1 ­ Lcap ).stk inUstk1:1.let pwit = Upack(Lcap ,Uhid,idi) inUstk1:1.let hstk2,resi = f [L(1 ­ Lcap )] pwit stk2 inUstk1:1.let hstk1,capi = stk2 inUstk1:1.let hi = freergn [] cap inUstk1:1.Lhstk1,resi

72

Encoding Cyclone Features

Many of Cyclone’s features fit into this framework– Lexical Regions– Dynamic Regions– Heap– Reaps– Unique Pointers

See paper for more details.

73

Future Work

In practice, capabilities shouldn’t be values

Encode results of region analyses– Aiken et.al. [PLDI’95], Henglein et.al. [PPDP’01]

Combine convenience of monadic encapsulation with power of substructural threading

74

Conclusion

Substructural type systems applicable to encoding region-based memory management– target-level language exposes commonalities

in source-level language features

Scope vs. Lifetime– Lexical scope of region name– Un-scoped lifetime of region capability

Unrestricted witnesses vs. Linear capabilities

75