sensor task manager (stm) v.s. subrahmanian university of maryland joint work with: f. ozcan, ibm...

Sensor Task Manager (STM)

V.S. SubrahmanianV.S. Subrahmanian

University of MarylandUniversity of Maryland

Joint work with:Joint work with:F. Ozcan, IBM AlmadenF. Ozcan, IBM Almaden

T.J. Rogers, University of MarylandT.J. Rogers, University of Maryland

Jan. 02 Darpa SenseIT PI Meeting 2

Scaling task handling Users specify tasks of interest:Users specify tasks of interest:

Where to monitorWhere to monitor When to monitorWhen to monitor Monitoring conditions to check forMonitoring conditions to check for What to do when monitoring conditions arise.What to do when monitoring conditions arise.

Data on the ground changes continuously.Data on the ground changes continuously. Monitoring conditions need to be evaluated Monitoring conditions need to be evaluated

continuously.continuously. LOTS of conditions, LOTS of sensed data. Scalability LOTS of conditions, LOTS of sensed data. Scalability

is key.is key.


How to Handle lots of tasks

Three pronged strategy:Three pronged strategy: Merge: Merge: Merge tasks to eliminate any redundancy using Merge tasks to eliminate any redundancy using

a cost model. Such merging only works well for a cost model. Such merging only works well for relatively small sets of tasks (or conditions to evaluate).relatively small sets of tasks (or conditions to evaluate).

Task Assignment:Task Assignment: Select sensors (and/or data Select sensors (and/or data sources) to handle merged tasks so as to optimize sources) to handle merged tasks so as to optimize performance criteria.performance criteria.

Partition: Partition: Given a large set of tasks (or conditions) to Given a large set of tasks (or conditions) to process, determine ways of partitioning into smaller process, determine ways of partitioning into smaller sets of manageable size.sets of manageable size.

For time reasons, only the last is discussed today.For time reasons, only the last is discussed today.


Task Partitioning Goal:Goal: Partition large number of tasks into disjoint Partition large number of tasks into disjoint

sets and minimize the total cost of executing the sets and minimize the total cost of executing the taskstasks

Cost estimation function (cost): Cost estimation function (cost): approximatesapproximates the the cost of executing a set of tasks together. Any cost of executing a set of tasks together. Any function satisfying the axioms:function satisfying the axioms:

TTi i T Tjj cost cost(T(Tii) ) costcost((TTjj)) cost(Ø) = 0cost(Ø) = 0


Partitions

Partition:Partition: A partition P of a set T of tasks is a set A partition P of a set T of tasks is a set

{ P{ P11,…,P,…,Pnn}, where each P}, where each Pi i is non-empty, iis non-empty, ij j PPi i

PPjj = = and and

Each PEach Pii is called a is called a componentcomponent of P. of P.

P is a sub-partition of Q ifP is a sub-partition of Q if


Task Partitioning Problem (TP)

Formal Problem Definition:Formal Problem Definition: Given as input a set T Given as input a set T of tasks, and a cost estimation function of tasks, and a cost estimation function costcost ,, find a find a partition P = { Ppartition P = { P11,…,P,…,Pnn} such that } such that

Need to balance execution time of tasks vs. Need to balance execution time of tasks vs. optimization time of tasks.optimization time of tasks.

)(cost1

n

i

iP is

minimized


TP Algorithms

Theorem:Theorem: The task partitioning problem is NP-The task partitioning problem is NP-complete. complete.

Proposed multiple types of algorithms to solve TPProposed multiple types of algorithms to solve TP A*-basedA*-based : Finds optimal solution : Finds optimal solution Branch-and-BoundBranch-and-Bound (BAB): Finds optimal solution (BAB): Finds optimal solution GreedyGreedy: Is not guaranteed to find optimal solution, has : Is not guaranteed to find optimal solution, has

polynomial running time – several variants proposed.polynomial running time – several variants proposed.


Adaptation of the A* Algorithm

State: A State: A sub-partitionsub-partition of T; P’ = of T; P’ = {P{P11,…,P,…,Pmm}} Start state: Empty partitionStart state: Empty partition Goal state: A Goal state: A partitionpartition of Tof T

Ex:Ex: T = {t T = {t11, t, t22, t, t33, t, t44, t, t55}} Example state s = {{tExample state s = {{t11, t, t33}, {t}, {t22, t, t55}}}} Goal State = {{tGoal State = {{t11, t, t3 3 , t, t44}, {t}, {t22, t, t55}}}}

g(s) = g(s) = PiPiP’ P’ cost(Pcost(Pii)) Ex:Ex: g(s) = cost({t g(s) = cost({t11, t, t33}) + cost ({t}) + cost ({t22, t, t55})})



Expansion functionExpansion function

Pick a task t and insert it into each component PPick a task t and insert it into each component P ii of of

P’P’

Create a new component PCreate a new component Pm+1 m+1 containing only tcontaining only t

Ex :Ex : {{t {{t11}, {t}, {t22}} and we pick t}} and we pick t44, then , then

{{t{{t11,t,t44}, {t}, {t22}}, }},

{{t{{t11}, {t}, {t22, qt, qt44}} }}

{{t{{t11}, {t}, {t22}, {t}, {t44}}, }},



h(s) = min{incr(t,s) | t h(s) = min{incr(t,s) | t P} P} incr(t,s) = min{cost(t), min{cost(t incr(t,s) = min{cost(t), min{cost(t P Pii) - cost(P) - cost(Pii) | P) | PiiP’}}P’}}

Ex:Ex: s = {{ts = {{t11}, {t}, {t22, t, t55}}}}

h(s) = min{incr(th(s) = min{incr(t33,s), incr(t,s), incr(t44, s)}, s)}

incr (tincr (t44, s) = min{cost(t, s) = min{cost(t44), (cost({t), (cost({t11, t, t44}) - cost(t}) - cost(t11)), )),

(cost ({t (cost ({t22, t, t5 5 , t, t44}) - cost ({t}) - cost ({t22, t, t55}))}}))}

Theorem:Theorem: The function The function h h is admissible and satisfies the monotone is admissible and satisfies the monotone restriction.restriction.

Theorem:Theorem: hence, A* finds an optimal partition. hence, A* finds an optimal partition.


Cluster Graphs

Canonical Cluster GraphCanonical Cluster Graph (T): Undirected weighted (T): Undirected weighted

graph wheregraph where

V = { {tV = { {tii} | t} | tii T } T }

E = { ({tE = { ({tii},{t},{tjj})| t})| tii, t, tj j T and w({t T and w({tii},{t},{tjj}) }) > > 0 0

w({tw({tii},{t},{tjj}) = cost(t}) = cost(tii) + cost(t) + cost(tjj) - cost ({t) - cost ({tii,t,tjj})})


Cluster Graph Example

T = {tT = {t11, t, t22, t, t33, t, t44, t, t55}}

cost(tcost(tii) =5, ) =5, cost({tcost({t11, t, t22}) = 8, cost({t}) = 8, cost({t33, t, t44}) = }) =

7 and cost({t7 and cost({t33, t, t55}) = 6}) = 6

t1 t2

2 t3

t4 t5

3 4


Greedy Partitioning Algorithm

Builds the partition iteratively using a cluster graph Builds the partition iteratively using a cluster graph representationrepresentation

In each iteration, finds the edge (tIn each iteration, finds the edge (t ii,t,tjj) with the ) with the

maximum weight and removes from the graphmaximum weight and removes from the graph Terminates when all edges are processedTerminates when all edges are processed Running time : O(|V|.|E|)Running time : O(|V|.|E|)


Greedy Partitioning Algorithm

At each step, four possible casesAt each step, four possible cases Case 1:Case 1: Both t Both tii and and ttj j are in the same component; are in the same component;

do nothingdo nothing Case 2:Case 2: One of t One of tii or or ttj j is in a component; insert the is in a component; insert the

other one into the same componentother one into the same component Case 3:Case 3: Neither is in any of the components; Neither is in any of the components;

create a new component with tcreate a new component with tii and and ttj j

Case 4:Case 4: t tii and and ttj j are in different components; are in different components; move one of them into the other component, or move one of them into the other component, or leave as it isleave as it is


t1 t2

2

t3

t4 t5

3

t1 t2

2 t3

t4 t5

3 4

Running ExampleT = {t1, t2, t3, t4, t5}

P = {{t3, t5}}

P={}


t1 t2

2

t3

t4 t5

Running Example, cont.P = {{t3, t4, t5}}

P = {{t3, t4, t5}, {t1, t2}}

t1 t2

t3

t4 t5


Variants of the Greedy Algorithm

Several variants of the basic greedy algorithm (5 in all we Several variants of the basic greedy algorithm (5 in all we worked with, 2 examples below) worked with, 2 examples below) Greedy with weight update (Greedy with weight update (Greedy w/ WUGreedy w/ WU))

After inserting tasks into components, it updates the After inserting tasks into components, it updates the weights of adjacent edges weights of adjacent edges

Greedy with no move around (Greedy with no move around (Greedy w/ NMAGreedy w/ NMA))Once a task is inserted into a component, it stays there.Once a task is inserted into a component, it stays there.


Running TimesNo of tasks

A*-based

BAB Greedy Basic

Greedy w/NMA

Greedy w/WU

5 68.3 81.1 2.2 0.3 1

6 328.5 417.1 2.75 0.35 1.55

7 1420.7 1570.5 2.8 0.4 1.7

8 9941.2 6752 3.3 0.45 3

9 18281.9 5243.4 4.7 0.5 4.8

10 44782.6 10109 6 0.5 6.6

Execution times (millisecs) (Cost-limit = 100, Overlap-degree=0.6, Overlap-prob = 0.4,0.6)

Only 10 tasks above as A* runs out of space. BAB cando 11 or 12. Greedy methods can handle thousands (see next slides).


Scalability of The Greedy Algorithms Cost-limit=100

Overlap- degree=0.2

Overlap- prob=0.4,0.6

0

2

4

6

8

10

12

14

16

18

20

0 300 600 900 1200 1500 1800

no of tasks

op

tim

izat

ion

tim

e (s

ecs)

GreedyBasic

Greedyw/NMA

Gredy w/WU

e


Cost Reduction

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

5 6 7 8 9 10

no of tasks

cost

red

uct

ion

per

cen

tag

e

greedy

A*-based

Cost-limit=100

Overlap-

degree=0.6

Overlap-

prob=0.4,0.6


Cost Reduction of Greedy AlgorithmsCost limit =

100

Overlap degree =

0.2

Overlap prob=0.4,

0.6

11

11.5

12

12.5

13

13.5

14

14.5

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

1600

1700

1800

no of tasks s

co

st

red

uc

tio

n p

erc

en

tag

e

GreedyBasic

Greedyw/NMA

Greedyw/WU


Bottom Line

Both A*-based and the BAB algorithm finds optimal Both A*-based and the BAB algorithm finds optimal solution, but do not scalesolution, but do not scale

Greedy algorithms find “good” solutions and scale up wellGreedy algorithms find “good” solutions and scale up well Greedy w/NMA scales very well, but achieves smaller Greedy w/NMA scales very well, but achieves smaller

cost reduction percentagescost reduction percentages Greedy w/WU achieves very large cost savings; it Greedy w/WU achieves very large cost savings; it

becomes the clear winner as the overlap degree becomes the clear winner as the overlap degree increasesincreases

Partitioning algorithms, in conjunction with merging Partitioning algorithms, in conjunction with merging algorithms promise substantial scalability algorithms promise substantial scalability improvements.improvements.


Other key contributions

Solved task assignment problem efficiently Solved task assignment problem efficiently despite NP-completeness. (Golubchik,Ozcan, despite NP-completeness. (Golubchik,Ozcan, Subrahmanian).Subrahmanian).

Temporal probabilistic relational DBs on top of Temporal probabilistic relational DBs on top of ODBC (TODS 2001)ODBC (TODS 2001)

Solved problem of scaling temporal probabilistic Solved problem of scaling temporal probabilistic databases –Built cost models and query optimizer. databases –Built cost models and query optimizer. (Dekhtyar, Ross, Ozcan, Subrahmanian)(Dekhtyar, Ross, Ozcan, Subrahmanian)

Probabilistic object base models (TODS 2001)Probabilistic object base models (TODS 2001) Temporal probabilistic object base models (sub)Temporal probabilistic object base models (sub)


SenseIT group demos

Developed gateway framework for Developed gateway framework for communicating with on-node cache maintained by communicating with on-node cache maintained by Fantastic data. Fantastic data.

STM provides data conduit behind Va Tech GUI.STM provides data conduit behind Va Tech GUI. Participated in Nov. 2002 SITEX experiments at Participated in Nov. 2002 SITEX experiments at

29 Palms. UMD gateway and conduit used there.29 Palms. UMD gateway and conduit used there. UMD Gateway and STM also to be used in joint UMD Gateway and STM also to be used in joint

demo with BBN, Va Tech, and other team demo with BBN, Va Tech, and other team members tomorrow.members tomorrow.


Contact Info V.S. SubrahmanianV.S. Subrahmanian Dept. of Computer ScienceDept. of Computer Science

AV Williams BuildingAV Williams BuildingUniversity of MarylandUniversity of MarylandCollege Park,MD 20742.College Park,MD 20742.

Tel: (301) 405-2711Tel: (301) 405-2711 Fax: (301) 405-8488Fax: (301) 405-8488 Email: [email protected]: [email protected] URL: www.cs.umd.edu/users/vs/index.htmlURL: www.cs.umd.edu/users/vs/index.html

sensor task manager (stm) v.s. subrahmanian university of maryland joint work with: f. ozcan, ibm...

Documents