multithreading design patterns

@gfraiteur

Surmounting complexity by raising abstraction

Multithreading Design Patterns

Gaël Fraiteur

PostSharp TechnologiesFounder & Principal Engineer

@gfraiteur

Hello!

My name is GAEL.

No, I don’t thinkmy accent is funny.

my twitter

@gfraiteur

My commitment to you:to seed a vision, not to sell a tool.

@gfraiteur

The vision:Code at the right level of abstraction,with compiler-supported design patterns

@gfraiteur

back in

1951

@gfraiteur

hardware was fun but quite simple to use

@gfraiteur

Univac Assembly Language

@gfraiteur

FORTRAN (1955)• 1955 - FORTRAN I

• Global Variables

• Arrays

• 1958 - FORTRAN II

• Procedural Programming(no recursion)

• Modules

@gfraiteur

COBOL (1959)• Data Structures

@gfraiteur

LISP (1959)• Variable Scopes (Stack)

• Garbage Collection

@gfraiteur

“The new invention caught quickly, no wonder, programs computing nuclear power reactor parameters took now

HOURS INSTEAD OF WEEKS

to write, and required much

LESS PROGRAMMING SKILL

@gfraiteur

1. The Memory Management Revolution

2. Models and Patterns3. Defining Threading Models

4. Designing with Threading Models

5. A Few Threading Models

6. Q&A

7. Summary

Section

@gfraiteur

How do programming languagesincrease productivity?

@gfraiteur

What you may think the compiler does for you.

Code Validation

Instruction Generation

@gfraiteur

Compilers translate codeFROM HIGH TO LOW ABSTRACTION language

@gfraiteur

Languages let us express ourselves against a model

Thing Concept Word

Model Language

∞ 1

World

abstracted into expressed with

∞ ∞

1 1

∞

1

1 1

abstracted into expressed with

@gfraiteur

Good Models are Easy• Allow for succinct expression of intent (semantics), with

less focus on implementation details

• less work

• fewer things to think about

• Allow for extensive validation of source code against the model

• early detection of errors

• Allow for better locality and separation of concerns

• “everything is related to everything” no more

• Are deterministic

• no random error

@gfraiteur

Good Models are Friendly• to human mind structure

• cope with limited human cognitive abilities

• to social organization

• cope with skillset differences, division of labor, time dimension, changes in requirements

@gfraiteur

The Classic Programming ModelQuality Realization

Succinct semantics • Concept of subroutine• Unified concept of variable

(global, local, parameters, fields)

Validation • Syntactic (spelling)• Semantic (type analysis) • Advanced (data flow analysis)

Locality • Information hiding (member visibility)• Modularity

Determinism • Total (uninterrupted single-threaded program)

@gfraiteur

1. The Memory Management Revolution

2. Models and Patterns

3. Defining Threading Models

4. Designing with Threading Models

5. A Few Threading Models

6. Q&A

7. Summary

Section

@gfraiteur

Why we need threading models• Multithreading is way too hard – today

• Too many things to think about

• Too many side effects

• Random data races

• Your colleagues won’t get smarter

• Increased demand – end of free lunch

@gfraiteur

@gfraiteur

The Root of All Evil

Access to Shared Mutable State• Memory ordering

• Atomicity / transactions

• Typical damage: data races

• Corruption of data structures

• Invalid states

@gfraiteur

Locks alone are not the solution

@gfraiteur

@gfraiteur

Locks alone are not the solution• Easy to forget

• Difficult to test

• Deadlocks

@gfraiteur

"Problems cannot be solved by the same level of thinking that created them."

(and you’re not a man if you haven’t cited Albert Einstein)

@gfraiteur

Threading Models

Reader-Writer

Sync’ed

PessimisticTransactions

Lock-Based

Lock-Free

OptimisticTransactio

nsTransactional

Avoid Mutable State

Thread Exclusive

Immutable

Actor

Avoid Shared Mutable State

Avoid Shared State

@gfraiteur

• Named solution

• Best practice

• Documented

• Abstraction

• Automation

• Validation

• Determinism

Threading Models are…

Models Design Patterns

@gfraiteur

1. The Memory Management Revolution

2. Models and Patterns

3. Defining Threading Models

4. Designing with Threading Models

5. A Few Threading Models

6. Q&A

7. Summary

Section

@gfraiteur

Golden rule: all shared state must be thread-safe.

@gfraiteur

Assign threading models to types

1. Every class must be assigned a threading model.

2. Define aggregates with appropriate granularity.

1.

@gfraiteur

Product

Invoice

Party

Store

Invoice InvoiceLine

Product

-lines

1

-invoice

*

-product *

1

Party

-invoices1

-customer*

ProductPart-parts

1

-product

*

Address

-addresses

1*

StockItem

Store

-items 1

-store *

*

-part1

reader-writer-synchronized

reader-writer-synchronizedreader-writer-

synchronized

actor

@gfraiteur

Ensure only thread-safe types are shared

1. Arguments of cross-thread methods

2. All static fields must be read-only and of thread-safe type.

2.

@gfraiteur

1. The Memory Management Revolution

2. Models and Patterns

3. Defining Threading Models

4. Designing with Threading Models

5. A Few Threading Models6. Q&A

7. Summary

Section

@gfraiteur

Immutable PatternNever changed after creation.Make a copy if you want to modify it.

Issue: multi-step object initialization (e.g. deserialization)

@gfraiteur

Freezable Pattern1. Implement interface IFreezable

Freeze() must propagate tochildren objects.

2. Before any non-const method, throw exception if object is frozen.

3. Call the Freeze method after any initialization.

@gfraiteur

Thread ExclusivePromises never to be involvedwith multithreading. Ever.

@gfraiteur

Thread Exclusivity Strategies• Exclusivity Policies:

• Thread Affinity (e.g. GUI objects)

• Instance-Level Exclusivity (e.g. most .NET objects)

• Type-Level Exclusivity

• Concurrency Behavior:• Fail with exception

• Wait with a lock

@gfraiteur

Implementing Thread Affine Objects

• Remember current thread in constructor

• Verify current thread in any public method

• Prevent field access from outside current instance

• Only private/protected fields

• Only accessed in “this.field”

@gfraiteur

Implementing Thread Excluse Objects

• Acquire lock before any public method

• Wait/throw if it cannot be acquired

• Prevent field access from outside current instance

• Only private/protected fields

• Only accessed in “this.field”

• Note: “Throw” behavior is non-deterministic

@gfraiteur

Stricter coding constraints increase code verifiability.

@gfraiteur

Lab: Checking Thread Exclusivity

@gfraiteur

Actor ModelBound to a single thread(at a time)

@gfraiteur

MessageQueue

PrivateMutable

State

SingleWorker

Actors:No Mutable Shared State

@gfraiteur

Composing Actors

@gfraiteur

Actor Sequence Diagram

Object1 Object2 Object3

Request1 Request2

Response1

Response2

Wait in message queue

@gfraiteur

• with compiler support:

• Erlang,

• F# Mailbox

• PostSharp

• without compiler support:

• NAct

• ActorFX

Actor implementations

@gfraiteur

Verifying the Actor Model• Public methods will be made async

• Parameters and return values of public methods must be thread-safe

• Methods with return value must be async

• Prevent field access from outside current instance

• Only private/protected fields

• Only accessed in “this.field”

@gfraiteur

Lab: Implementing Actors

@gfraiteur

• Performance• Theoretical Latency: min 20 ns

(L3 double-trip)

• Practical throughput (ring buffer): max 6 MT/s per core

• Difficult to write without proper compiler support

• Reentrance issues with waiting points (await)

• Race free (no shared mutable state)

• Lock free (no waiting, no deadlock)

• Scales across processes, machines

Actors

Benefits Limitations

@gfraiteur

Reader-Writer SynchronizedEverybody can read unless someone writes.

@gfraiteur

Reader-Writer Synchronized• 1 lock per [group of] objects

• e.g. invoice and invoice lines share the same lock

• Most public methods require locks:

Method Required Lock Level

Reads 1 piece of state None

Reads 2 pieces of state Read

Writes 1 piece of state Write

Reads large amount of state, then writes state

Upgradeable Read, then Write

@gfraiteur

Lab: Implementing RWS

@gfraiteur

Transactions• Isolate shared state into thread-specific storage

• Commit/Rollback semantics

• Platform ensures ACID properties:

• Atomicity, Consistency, Isolation, (Durability)

• Type of concurrency policies

• Pessimistic (lock-based: several “isolation levels” available)

• Optimistic (lock-free, retry-based)

As far as I'm concerned, I prefer silent vice to ostentatious virtue. Albert Einstein “

@gfraiteur

Q&AGael Fraiteurgael@postsharp.net@gfraiteur

@gfraiteur

Summary• Repeat the memory

management success with the multicore issue.

• Models decrease complexity to a cognitively bearable level.

• We need compilers that allow us to use our own modelsand patterns.

BETTER SOFTWARE THROUGH SIMPLER CODE