Self-* Programming: Run-Time Parallel Control Search

for Reflection Box

Olga Brukman and Shlomi DolevBen-Gurion University of the Negev

Israel

Problem

• Ideally– Systems should anticipate every possible scenario

• Reality– Engineers fail to create such systems despite the

effort– The number of possibilities of different

interactions with dynamic environment is enormous

– Relying on accumulated knowledge of human operator to deal with unexpected situations

2

Example 1

• Airplane flying into ash cloud– Engines stopped– Pilots managed to fly the plane out of the cloud,

waited till the engines cooled down, and were able to restart them

3

Example 2• Airplane crossing the

speed of sound – Airplane control

handles behave opposite to the expected behavior

– Pilots increase the plane speed so it becomes higher than speed of sound, plane control is back to normal

4

Our Contribution: Automatic Control Search Engine for Dynamic Environment

• No assumptions on possible environment changes• Experimentation on replicas• Parallelization of experiments• Polynomial search time – Parallelization– Exposing system state

• Observing system state (e.g., with Java reflection)• Setting system state to a certain state

5

System Settings: Environment

EnvironmentNon-deterministic infinite automaton

t

Environment

DA/PA(F1)Environment

DA/PA(F2)Environment

DA/PA(F3)

t

• Environment is large, sophisticated, dynamic– Non deterministic infinite automaton– At every given time slot environment is

– deterministic automaton – probabilistic automaton with a transition function Fi

6

System Settings: Environment vs. Plant

• Our program (control) interacts with some machinery in environment – plant

SystemEnvironment

Controlin1,.., ink

out1, …,outl

PlantPlant

Environment

in’1…...., in’m

out’1,……,out’n

7

System Settings: Environment and Plant

• Environment is– Reentrant : no mutual

replicas interference– History oblivious:

deterministic repetition of behavior for the plants in the same initial state and with the same control

SystemEnvironment

Control1

Plant

in’1….., in’m

out’1,……,out’n

Control2

Plant

in’’1...., in’’m

out’’1,……,out’’n

Control1

Plant

in’1...., in’m

out’1,……,out’n

8

SystemControl Plant

Environment

in’1…...., in’m

out’1,……,out’n

in1,.., ink

out1, …,outl

Control Search Engine

Observer

plant-environmentinteraction

Control Generator

Goals=Behaviors

bhv={io1,…,ioj}peout

peinjio

peoutj

peinj outin ,

pciniin

|AP|=N ≤Nmax

Recording plant-environment

interaction

9

Settings: Periodic Control

Plant automaton

+

=

in1

in4

in3

in2

io1

io2

io3

io4

P=3

peout

pein

10

bhv={io2, io3 , io4}, io2, io3 , io4

C={in1(in2, in3, in4)*}

Control Search AlgorithmsAlgorithm Plant State

ReflectionPlant State Set

Deterministic Environment

I: black box x x

II: set box x v

III: reflection box v x

IV: reflection-set box

v v

Probabilistic Environment

V: reflection-set box

v v

VI: reflection box v x

11

Control Search Algorithms for

Deterministic Environment

12

Algorithm I: black-box

• Complexity– Total number steps in experiments:– Longest experiment: O(PNmax)

s

s

s

s

…

C1

C2

CM

)||( 1max

max PNpcinPNO

scurr

x P∙

scurr sP_1 sP_2 sP_N_max+1…

0<i≤Nmax P

P(Nmax +1)

sP_3

P

max

1

||N

i

Pipcin

)1(

max

max

)(

0,)(

Nsuffixprefixj

Ppcinsuffix

ipcinprefix

CCC

C

NiC

13

sstart

Algorithm IV: reflection-set box

|| pcin

s1sN….

σ1 σ2

σΣ... σ1 σ2

σΣ...

|| pcin Ap

• Off line search of the constructed plant automaton• Try all controls of form from every state

• Complexity• Total number of steps in experiments:• Longest experiment: O(1)

1])[( NPpcin

|)|( pcinNO

14

Control Search Algorithms for Probabilistic Environment

Probabilistic Environment

• Plant is unaware of the entire state of the environment– Environment can be considered to be probabilistic

automaton – Plant transition function is probabilistic

• Control search algorithm executes all the time– Due to probabilistic transition function of plant automaton

• Monitoring– Recognize changes in the plant probabilistic transition

function

16

Probabilistic Plant Automaton

• 0≤pr(s, s’, σ, io) ≤ 1• prmin – minimal probability

sisj

)(, peout

peinj

pcini io

17

σ1, io1,1, pr1,1

σ1, io1,2 ,pr1,2

…σm, iom,1 ,prm,1

σm, iom,2 ,prm,2

Computing Probabilistic Plant Automata Graph (PPAG)

si

SF times

σ

:,, peout

pein

pcinPji ioSss σ, io s1 s2 … si-1 si+1 … sN

s1 0.02 0.08 … 0.06 0.05 … 0.25

s2 0.04 0.07 … 0.6 0.02 … 0.2

…

sN 0.5 0.01 … 0.02 0.1 … 0.4

PPAG[si,sj,σ,io]=pr

SF=1/prmin: number of experiments required to discover the edges with the smallest probability

sj?io?

18

Preprocessing: Behavior Suffix Probability (BSP) Table

• For every state s and j=1,…,|bhv|:– BSP[si ,j] = [prmax, σ]

• prmax is the maximal probability to obtain suffix (bhv,j) starting from plant in state si ;

σ is the first entry in this control

• BSP computed from PPAG

siσ

j

prmax

pr2

pr1

pr3

19

Preprocessing: Behavior Suffix Probability (BSP) Table Cont.• Base step: for every si compute BSP[si,1]

– For every si find input σ that produces bhv[k] with the highest probability : max{PPAG[si, *, σ,bhv[k]]}

pr1σ1 (io1)

si

1

prmax

pr2

pr3

σ (bhv[k])

σ2 (io2)

σ3(io3)

Preprocessing: Behavior Suffix Probability (BSP) Table Cont.• Assume we computed all entries BSP[si ,j] for every si and j≤m

• BSP[si , (m+1)]?

– Let σ be the one that yields the largest value of • Σs_j (PPAG[si,sj, σ,bhv[k-m-1]] BSP[s,m].pr)∙

– BSP[si, m+1]=[Σs_j (PPAG[si,s, σ,bhv[k-m-1]] BSP[s,m].pr), ∙ σ]

σ1 (bhv[k-m-1])

si

1

pr3= Σs_j (PPAG[si,sj,σ3,bhv[k-m-1]].pr BSP[s∙ j,m].pr)

σ (bhv[k-m-1])

σ2 (bhv[k-m-1])

σ3 (bhv[k-m-1])s,s’,s’’,…

s,s’,s’’,..

s,s’,s’’,..

s,s’,s’’,…

prmax=Σs_j (PPAG[si,sj, σ,bhv[k-m-1]] BSP[s∙ j ,m].pr)

pr2= Σs_j (PPAG[si,sj,σ2,bhv[k-m-1]].pr BSP[s∙ j,m].pr)

pr1= Σs_j (PPAG[si,sj,σ1,bhv[k-m-1]].pr BSP[s∙ j,m].pr)

Algorithm V: reflection-set box

sstart

snext

snext

sstart=max{BSP[s,|bhv|].pr}

BSP[snext,k-1]>BSP[sstart,k]

BSP[sstart,k].σ

pr=0.4

pr=0.6

BSP[snext,k-1]<BSP[sstart,k]

Algorithm VI: reflection box

sstart

BSP[snext,k-1]>BSP[sstart,k]

BSP[sstart,k].σ

s’best=max{BSP[s,k].pr}

sbest=max{BSP[s,k].pr}

sbest

snext

snext

23

pr=0.7

pr=0.3

BSP[snext,k-1]<BSP[sstart,k]

Conclusions

• Framework for automatic control search • Control, plant, environment – Deterministic plant– Probabilistic plant

• Polynomial time– Parallelization– Exposing plant state