1

NC: A Verifiable, Executable Coordination Paradigm forReal-Time Communication

Sebastian Fischmeistersfischme@seas.upenn.edu

Department of Computer and Information ScienceUniversity of Pennsylvania

2

Real-Time Systems

Userinterface

Computer simulation

Internetvideo, audio

Cruisecontrol

Telecommunication

Flightcontrol

Electronic engine

Soft RT Hard RTNo RT

• Right value at the wrong time can cause an error!

How to guarantee timeliness in the system?

3

Distributed RT Systems

need for a coordination model for controlling access.

• Multiple nodes together realize one real-time application.

How to bound communication delays in the system?

Collisions, retransmissions, delays!

Brake

Sharedmedium

Brake pedal

Fuel tank

Tires

Boardcomputer

4

• TDMA is a method for realizing collision-free access to a shared medium.

• Partition access into time slices (=slots) and rounds.• Assigns slots to nodes for exclusive access using some algorithm.

• Example: Round robin with four nodes

Slot# Sender Receiver1 N1 …2 N2 …3 N3 …4 N4 …

N2N1 N3 N4

t

Time Division Multiple Access

Round

Slot

5

• Great for safety-critical systems, but too inflexible.

• Still:– Decisions are made at the beginning of the round/phase.– Phase order stays the same.– Typically independent of the software.

One static phase:statically allocated

static dynamicStatic + dynamic:

static dynamic freeStatic, dynamic, free:

Flavors of TDMA

dynamicOnly dynamic:

6

Goals of Network Code

• Goals:– Allow close interaction between the application and the

medium access control.– Allow developers to program their own access control.– Allow decisions during the communication round.– Should be apt for safety-critical hard real-time systems.

7

Network Code[ISCC05]

• Executable programmable coordination model for real-time communication with a domain specific language.– Split application into: program logic + communication logic– Communication logic is programmed separately (=Network Code

program)– An interpreter (=Network Code machine) executes the program at

run time

Medium

NC program Value space

NCM

Application

Node1 Node2

Value space

NCM

ApplicationProgram logic

set X

get X

tx M/ch

set X

get X

8

Research Results/Status

• Theory:– Underlying concept and idea of Network Code [ISCC05]

– Verification of Network Code programs [RTAS06]

• Given a set of programs, are they safe to run concurrently?– Metrics framework for measuring programs [EmSoft06]

• Overhead, throughput, avg. cycle length, avg. waiting time– Composition algorithms for programs [submitted]

• Given several applications, how can they run together?

• Simulation:– In round decisions increase throughput and performance. [EmSoft06]

– Gains are kept despite overhead introduced during composition. [submitted]

• Implementation:– Implementation RTLinuxPro[RTAS06,TC07]+Ethernet, PIC18F+CAN , FPGA

9

How does it work?

10

Sender

Receiver

t10 200

L0:create(M, X)send(ch, M, _)wait(20)

La:wait(10)

Example: Transmit X in M

• Slot length is 10t.• Node 1 transmits variable X every 20t in message M on channel ch.• Node 2 receives this message and stores it Y.

receive(ch, Y)wait(10)

goto(L0)

goto(La)

11

Complex Example

• Transmit a value with temporal replication tolerating 2 failures– Sensor values A to E. (2n+1)– Decide at the end of the round what the sensor value is.– Take the majority vote (e.g., ‘4,4,3,2,4’ is 4)

B

t

A C D EStandard:

B

t

A C D E

Vote decisive? Skip ahead!

Improved version:

12

Complex Example Code

B tA C D E

Node A:La: create(MA, A)

send(ch, MA)wait(20)receive(ch, B)wait(10)receive(ch, C)if(voteDecisive, La)wait(10)receive(ch, D)if(voteDecisive, La)wait(10)receive(ch, E)goto(La)

13

System Overview

RT application

Communication medium

Applicationlayer

Access control

Variable space

Hard queueSoft queue

Message transceiver

Netw

ork co

de

mach

ine

Queue

set/get

Coordination

set/get

Queue

Value det.layer

DynamicTDMAlayer

Network codeprogram

14

Data Flow (Hard Values)

RT application

Communication medium

Applicationlayer

Access control

Variable space

Hard queueSoft queue

Message transceiver

Netw

ork co

de

mach

ine

Queue

set/get

Coordination

set/get

Queue

Value det.layer

DynamicTDMAlayer

Network codeprogram

15

Data Flow (Soft Values)

RT application

Communication medium

Applicationlayer

Access control

Variable space

Hard queueSoft queue

Message transceiver

Netw

ork co

de

mach

ine

Queue

set/get

Coordination

set/get

Queue

Value det.layer

DynamicTDMAlayer

Network codeprogram

16

Verification & Checking[RTAS06]

• What if I have a bug in my Network Code program?

a verification framework with model checking

• Use operational semantics of instructions to model system state.• Convert program instructions to transformations of the system

state.• Check properties are reachability questions using model

checking.

Prgm 1:La: wait(20) send(_,_,_)

if(guardX, La)wait(10)send(_,_,_)goto(La)

Prgm 2:Lb:wait(10)

send(_,_,_)wait(30)goto(Lb)

17

Example Properties

• Integrity checks– All guards present– Valid jump addresses– Successor address– Consequential code

• Behavioral checks– Non-empty send– Local message lifecycle

• Distributed checks– Collision-free communication– Schedulability– Non-empty receive– Message reception within timeout

18

Metrics Framework[EmSoft06]

• What is the overhead? What is the throughput? What is the cycle?– Convert the Network Code Program into a timed tree

communication schedule.– Compute metrics on the tree schedule using DFS & model

checking.

• Timed tree communication schedule:– Nodes encode messages releases and delays– Transitions encode mutually excl. decisions

(exhaustive, deterministic)– Leaves cause a reset

(X,3)

(Y,4) (Z,2)

g1!g1

19

B

tA C D E

g1 g2

A

B

C

D

E

-

-

g2

g1 !g1

!g2

Example Metrics

• Longest cycle length = longest branch• Guard overhead• End-to-end constraints for messages

• Probabilistic tree schedules:– Average cycle length = average branch

length– Average waiting times

20

Is it better?

21

Inverted Pendulum

• Pendulum sitting on a cart with one degree of freedom.• Measure the angle w.r.t. to straight up.• The cart moves horizontally to balance the pendulum.

• Temporal replication to tolerate two faults:– 5 measurement units (A to E)– Report measurements to the controller.– Controller updates the kart at the round.

22

Simulation Results

Network Code is better*

B

t

A C D E

Standard schedule:

B

t

A C D E

Vote decisive? Skip ahead!

Network code:

23

Simulation Results

With an increase in the mean time to failure, standard TDMA stays the same.Network Code utilize this increase in reliability.

Mean time to failure

24

• Given the shortest program, what is the fastest execution rate?– Output queue must not overflow.– Input queue must not under-/overflow.– Slot structure must not be violated.

• RTLinuxPro + direct link (100 MBit/s)– 10% latency increase due to computation time– 2.96MByte/s with a 99% guarantee of a successful transmission– 2.65MByte/s with 99.999% guarantee

• Interrupts preempt instructions, cause jitter, cause delays.– Send() instruction: 372ns (39.47%), 733ns (99%), 19090ns (99.999%)

=> FPGA! (10us)

Implementation Measurements

25

Conclusions

• Communication behavior differs among applications; custom protocols/schedules can:– Significantly improve the application’s performance– Incorporate/guarantee new properties at the communication layer

• Network Code allows programming custom TT communication behavior.– Programs are checkable, guarantee properties in the

distributed system.– Programs are measurable with tree schedules.– Can run real-time and non-real-time traffic on the same

network.

• Future work: multi-resource tree schedules and resource DSLs, program generation, development methodology

26

Other/Past Research

• Logical execution model has deficits when considering communication.– Task chains[DASC04]

– Case study[RTAS05]

• How to deterministically reconfigure RT software?– Use virtualization and separate behavior & functionality[ECRTS05]

• Are mobile agents systems secure? (year 2000)

– No![MA01]

• How to build efficient proactive location-based systems?– Decentralized architecture[LNAI]

– Use notification service[PERCOM03]

27

Outlook

28

DSLs & Interfaces

FunctionalityCons

umpt

ion

malloc()

fun()spawn()

x++send()

receive()

X()

y+=5%x

write()

read()

open()close()

Current Approach

Software Component

Medium

Resources

DSL1

DSL2

DSL3

Network Code

• What are good abstractions for resources, what are good DSLs?• What makes the interface of a software component? Is the DSL the interface?• How can we generate code for a component?• How do we provide hardware support for DSLs? (e.g., programmable memory)