Modeling Teamwork and Command and Control in Multi-Agent Systems

Thomas R. IoergerDepartment of Computer Science

Texas A&M University

TaskableAgents Architecture• similar to process networks, or HTN’s• has a customized knowledge representation

language (TRL) for encoding knowledge about tasks and methods (doctrine, mission)

• agents run as independent processes• each may have multiple parallel activities• agents represent staff positions (S2, S3...)• communicate with each other for teamwork• interact with humans (via forms: info/cmds)• interact with OTB for scenario simulation

Simulation

DIS

Agents

OneSAFTestbed

KBPDUs Translated To Facts (speed, location, unit type, etc.)

Cache

Periodic Updates From Simulation

High-Level Architecture of DBST

text, forms, map

actionsinform, request, direct, approve,

respond

RFS,CFF

mouse

PDUsOTB Agents

PucksterInterfac

e

BDE Interfac

e

PDUs

TaskableAgents Architecture

• Written in Java• TRL Knowledge Representation Language

- For Capturing Procedural Knowledge (Tasks & Methods)

• APTE Method Selection-Algorithm- responsible for building, maintaining, and

repairing task-decomposition trees• Inference Engine JARE

- Java Automated Reasoning Engine- Knowledge Base with Facts and Horn Clauses- back-chaining (like Prolog)- Updating World With Facts

TaskableAgents

TaskableAgents

sensingmessage

s

JARE KB: facts &Horn-

clauses

OTB(simulation)

operators

results

assert, query,retract

messages

APTEAlgorith

m

TRL TaskDecomposition

Hierarchy

ProcessNets

Other Agent

s

TRL KB:tasks &methods

Task Representation Language (TRL)

• Provides descriptors for: goals, tasks, methods, and operators• Tasks: “what to do”

– Can associate alternative methods, with priorities or preference conditions

– Can have termination conditions• Methods: “how to do it”

– Can define preference conditions for alternatives– Process Net

- Procedural language for specifying how to do things- While loops, if conditionals, sequential, parallel constructs- Can invoke sub-tasks or operators- Semantics based on Dynamic Logic

• Operators: lowest-level actions that can be directly executed in the simulation environment, e.g. move unit, send message, fire on enemy– Each descriptor is a schema with arguments and variables– Conditions are evaluated as queries to JARE

Example TRL Knowledge

(:Task Monitor (?unit) (:Term-cond (destroyed ?unit)) (:Method (Track-with-UAV ?unit) (:Pref-cond (not (weather cloudy)))) (:Method (Follow-with-scouts ?unit) (:Pref-cond (ground-cover dense)))) (:Method Track-with-UAV (?unit) (:Pre-cond (have-assets UAV)) (:Process (:seq (:if(:cond(not(launched UAV)))(launch UAV)) (:let((x y)(loc ?unit ?x ?y))(fly UAV ?x ?y)) (circle UAV ?x ?y))))

• useful for describing multiple ways of accomplishing tasks

• may encode preference conditions• APTE algorithm will automatically try another if one

method fails• examples:

– use of UAV vs. ATK helicopters vs. scouts for recon– suppression of direct fire with Arty/CAS– use of FASCAM to slow or re-direct advancing enemy– maintaining security:

• flank guard, patrols• neighboring units• use of terrain features• electronic surveillance

Alternative Methods

Task-Decomposition Hierarchy

T1 M1

T2 T5

T3 T4

M7 M12 M92 M60

T15 T18 T40 T45 T40 C T45

T2

level 1 level 2 level 3 level 4 level 5

Tx =Task Mx = MethodC = Condition

TOC Staff - Agent Decomposition

CDR

FSO

S3

S2

CompaniesScouts

Control indirect fire,

Artillery, Close Air,

ATK Helicopter Maintain enemy situation,Detect/evaluate threats,

Evaluate PIRs

Maintain friendly situation,Maneuver sub-units

Maneuver,React to enemy/orders,

Move along assigned route

Move to OP,Track enemy

Move/hold, Make commands/decisions,

RFI to Brigade

S2 Agent and Interactions

DP approva

l

Move to OP

Threat level,PIRs

RFI/RFSSALT/ INTSUM

intel intel

Enemy info

S2

BDE/DIV Sensors/ Recon

BDES2

Scout

CDR

CCIR S3

spot reports

Vignette 2 – Decision Point 1 [Shift Main Effort]

Variation A:Enemy major threat is on main route (AA) as in route (planned AA).

CompanySize forces

3-66=1-22 3-66=1-22 4ID X X

1CD

4ID X X

1CD CoCTmB CoC

TmATmB

TmA

AA3 AA4 AA5a

PL

IBIB

PL

PL PL

AA3 AA4

AA5a

AA5

AA6c

AA5

AA5c

3 234 2341 2351 1-2353-2341-234

CompanySize forces

Main Effort(ME)

Main Effort(ME)

Shift ME from Tm B to Co C?

DP1

• 2 Companies of 3-234 heading along AA3• 3-234 lead Bn of 234 Regt.• Situation unclear on AA5

INTEL

N Y

Variation B:Enemy major threat changes to secondary approach

• 2 Companies of 234 heading along AA3 & AA4• 3-234 lead Bn of 234 Regt intent is unclear.• Lead Bn (1-235) of 235 Regt on AA5a

INTEL

Shift ME from Tm B to Co C?

DP1

N Y

CompanySize forces

ME Switch

Vignette 3 – Decision Point 2 [Commit TF Reserve]

Variation A:Heavy enemy threat across across entire sector.

CompanySize forces

3-66=1-22 3-66=1-22 4ID X X

1CD

4ID X X

1CD CoCTmB CoC TmB

TmA

AA3 AA4 AA5a

PL

IBIB

PL

PL PLY

AA3 AA4

AA5a

AA5

AA6c

AA5

AA5c

3 234 23412351 1-2353-234

1-234

CompanySize forces

Main Effort(ME)

Main Effort(ME)

Commit the TF Reserve platoon?

DP2

• Company units of 3 different Bns on all 3 AAs • Estimate enemy will reach PL Y at same time• 238 Regt lead units not committed

INTEL

N Y

Variation B:Major enemy movement on one avenue (AA).

• 2 Companies of 234 heading down AA3• Uniform pressure on AA’s 4 & 5ar.• Calculations indicate Tm A unit can move to PL Y prior to lead of enemy unit.

INTEL

DP2

Y

Res…

Go to DP 3

Res…

Commit the TF Reserve platoon?

TmA(-)

Blocking PositionsTAI

CompanySize forces

(+)

N

Modeling Teamwork• TOC is more than just collection of staff members

• helping/backup behavior, information fusion, resource sharing, and joint decision-making

• how to model collaborative behavior?

• CAST (extension to TaskableAgents)

• adds features to TRL language for encoding team structure and process (MALLET)

• adds algorithms for coordination and communication within teams

• semantics based on joint intention theory and mutual awareness (beliefs)

Key Concept: Shared Mental Models

• Various components– static: structure of the team, communication policies...

– goals and plans

– dynamic: current situation, others’ workloads/status

• Team knowledge needed by agent team members:– roles, responsibilities, capabilities, team plans

– need to know who should act and when

– need to reason about each other

– need to know when to communicate for synchronization, coordination, disambiguation, infomation sharing, etc.

The CAST Agent Architecture• MALLET - team knowledge repres. language

– team structure (roles, capabilities, responsibilities)– team process (plans, policies)

• CAST kernel (interpreter)– convert to Petri nets (track progress, select actions)– use back-chaining theorem-prover for inference– dynamic role selection - make choices in context

• DIARG - information exchange algorithm– proactive: offer new info to those who need it

• primary references: (Yen et al., IJCAI, 2001), (Yin et al., Autonomous Agents Conf., 2000)

MALLET Multi-Agent Logical Language for Encoding Teamwork

• extension of TRL• basic definitions

– (team search-and-rescue (bill ted))– (role pilot) (role spotter)– (plays-role bill pilot)– (capable spotter use-IR-binoculars)

• conditions: (<predicate>*) with variables prefixed by ‘?’ – e.g. ((forward-scout ?unit) (location ?unit ?x ?y))

• team operators:(team-oper lift-heavy-object (?obj)

(pre-cond (at ?obj) (num-agents >= 2))(share-type AND)))

• share types: AND=together, OR=any, XOR=only 1 (excl.)

• team plans: can select certain agents or roles to do steps (like SharedPlans of Kraus and Grosz)(team-plan indirect-fire (?target) (select-role (scout ?s) (in-visibility-range ?s ?target)) (process (do S3 (verify-no-friendly-units-in-area ?target)) (while (not (destroyed ?target)) (do FSO (enter-CFF ?target)) (do ?s (perform-BDA ?target)) (if (not (hit ?target)) (do ?s (report-accuracy-of-aim FSO)) (do FSO (adjust-coordinates ?target))))))

• “compile” these into TRL using methods of Biggers and Ioerger (2001)

• other scouts can take over as backup in case of failure of ?s• responsibilities (such as monitoring, reporting); semantics

similar to joint intentions (Johnson and Ioerger, 2001)

CAST Kernel• compile team plans into Petri nets (expand sub-tasks)• cycle: sense/decide/act loop

1. update beliefs about environment in self’s KB2. check for any incoming messages from other agents3. find active steps in plan (transitions with tokens in all input places)4. if self is uniquely resp., consider executing oper.5. if oper is XOR and resp. is ambiguous, offer6. if oper is AND, broadcast READY and wait for others7. randomly choose among remaining actions and execute8. inform others of completed steps

• Dynamic Role Selection (DRS)– check role definitions, must satisfy any constraints, capable?– communicate when ambiguity exists– sync. for AND operators; select for XOR operators– could also allow individuals to vote/negotiate

DIARGDynamic Inter-Agent Rule Generator

• Info. sharing is a key to flexible teamwork

• Want to capture information flow in team, including proactive distribution of information

• Want to restrict to only the most relevant cases

• Ideal criterion:(Bel A I) ^ (Bel A (Bel B I)) ^ (Bel A (Goal B G)

^ [(Bel B I) � (Done B G)]

^ [(Bel B I) � (Done B G)]

(Goal A (Inform B I))

where is the temporal operator for ‘always’�

DIARG, continued• Explanation - A should send message I to B iff:

– A believes I is true– A believes B does not believe I (or believes it is false)– I is relevant to one of B’s goals

• i.e. pre-cond of current action that B is resp. for in team plan, • and that action would not succeed without knowing the info.

• Reasoning about observability– agents can sometimes infer that other team members

already believe certain information– e.g. based on common observability in environment– use this to filter out superfluous messages– recent work: (Rozich and Ioerger, submitted)

Command and Control• Need for tactical decision-making

– more flexibility in unplanned situation– commander agent

• How to represent of “tactics”?– battlefield geometry, relative force strength,

combined arms theory– terrain, effects on mobility– discovery of enemy intent

Naturalistic Decision-Making• NDM (Klein) - a cognitive model of human

decision-making in complex environments

• based heavily on situation assessment (SA)– 3 stages (Endsley):

• acquisition of factual information

• comprehension (abstraction, relevance, goal impact)

• projection (prediction of consequences)

• NDM is “satisficing”– take first adequate match; don’t extensively

evaluate and compare alternatives; respond

Evidence for NDM in TOCs• verbal protocol analyses

– characterize types of utterances and interactions

• representative studies– Serfaty, Entin, et al. (NDM, 1997)

• expertise as independent variable

– Pascual and Henderson (NDM, 1997)• reliance on recall from experience

– Schmitt and Klein (CCRTS, 1999)• recognitional processes in MDMP/COA

– Endsley (ARL report)• information flow in infantry platoon/urban combat

• lots of other similar C2 environments...– CIC/AAW, AWACS, fire fighting, ATC

Recognition-Primed Decision Making (RPD)

• a model of NDM• characterized by “feature-matching”• look for enough cues to trigger recognition• features could be weighted• for each situation, there is a typical response

(doctrinal, or learned from experience)• role of “mental simulation”?

RPD Flow Chart• Len Adelman and Denny Leedom

– integrate RPD within battalion TOC staff

situationclear?

status quoacceptable?

generatenew options

reduceuncertainty

currentmission plan

monitorprogress

modify

No

No Yes

Yes

generateresponse,mental sim,compat. test,modify plan...

Methods to Deal With Uncertainty

• often features are unknown, can’t evaluate

• options include: 1. suppress uncertainty 2. make default assumptions 3. confirmation bias (expectations) 4. take “probing” actions 5. forestalling until situation is more clear

Implementation of RPDin TaskableAgents

• task DetectSituation runs in parallel with other routines...

• loop until match enough features for one situation

• while some features are unknown, try various methods to findout (e.g. UAV, scouts, radar, JSTARS, Bde Int, feint...)

• drive information collection to discriminate situations

• trigger plans for response maneuvers once situation is determined (takes priority)

• go back to original mission once threat is handled

• all of this can be implemented as tasks and methods in TRL

Characterization of Situations• Situations must have lists of typical features

• Characterize by:– location of nearby enemy, combat strength– in regions relative to local axis (frame of ref.)– distance, speed, direction– intent? (e.g. attacking, bypassing, objective)– cover, uncertainty– effect of terrain on mobility/reachability

• roads, mountains, streams, bridges, marshes, forests

• also minefields, targeted areas of interest...

Types of Situations• Defensive

– getting: flanked, ambushed, enveloped, etc.• response: shift, request for support, withdraw...

– over-whelming enemy maneuver/main effort• response: impede, divert, CAS, inform Bde...

– forked maneuvers (intent?); bypass attempt

• Offensive (opportunities worth recognizing)– exploit gaps, isolate enemy units, envelopment,

use fixing force + flank attack, canalize enemy, bypass

Practical Issues• Managing priorities

– when to abandon mission plan & react tactically?– return to mission plan once done?

• Dependence of response on goals– ROE, aggression/initiative vs. defense/security– should SA involve impact on goals? (= threat?)

• Need to have “critics” to revise responses– avoid enemy TAI’s, minefields, contact...– stay near adjacent friendly units, air defense...

Conclusion

• TaskableAgents architecture

• CAST extensions for teamwork

• TOC staff agent model

• modeling command and control (on-going)

• HLA interoperability (on-going)