How much is enough?Requisite modelling for socio-technical problems

Graham Mathieson, Dstl Fellow

Kathryn HyndUK Defence Science and Technology Laboratory

Presentation to 21 ISMOR, Sept. 2004

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Structure of presentation

• Background and aims

• Things we know about socio-technical systems

• NCO Domains & current modelling

• Case Study 1 – Combat ID

• Case Study 2 – Network Effects C2

• Building practical requisite models

• Conclusions

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Background and aims

• Increasing awareness of socio-technical issues

• Military OR challenged to deal with wider range of factors

• Current modelling (both conceptual and practical) has relatively narrow coverage, largely limited to physical and informational domains (with a little of the cognitive)

• Presentation addresses the validity of modelling to support OR and seeks to describe a ‘requisite’ model

• Main focus has been to identify how much of the cognitive and social domains need to be added

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Some definitions and declarations

• Models are abstract representations

– descriptive/explanatory

– conceptual/practical

• Requisite (adj) made necessary by particular circumstances (Concise Oxford)

• Requisite model is minimum that is fit for purpose

• Requisite model:

– Contains all critical factors which may determine the study conclusions (e.g. factors significantly affecting option rank ordering)

– Can be defined in relation to an isolatable sub-problem (i.e. one in which a sub-set of factors are not too dependent on others)

• Non-requisite, by implication, means not fit for purpose

– Using non-requisite models carries risks. When is the risk too high?

Requisite

Non-requisite

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Things we know about socio-technical systems

Resources

Technology

Structure

Culture

Processes

People

People process information according to their culture, experience, expectation, emotional state

Humans naturally use very little of the information available to them

Organisation size correlates with formality of interactions

Technical system performance can critically depend upon human social phenomena

Degraded comms system performance can lead to improved information service

Even in safety-critical organisations, people do not consistently follow formal processes

Structures emerge in organisations despite the

intentions of the people involved

It takes years to change an organisational culture

without wholesale re-staffing

People create and use informal structures, which can be more influential than the

formal ones

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

NCO Domains & current modelling

Social domain

Cognitive domain

Information domain

Physical domain

Current coverage of OR models

Maturity of source

models in each dom

ain

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Case Study 1 – Combat ID• ‘The process of combining situational awareness, target

identification, specific tactics, techniques and procedures to increase operational effectiveness of weapon systems and reduce the incidence of casualties caused by friendly fire’

CID

TTPsTID

SA

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Case Study 1a - Combat ID

• Operation Provide Comfort, 14th April 1994

– 2 US army Black Hawk helicopters take off to pick up 16 members of UN humanitarian relief operation

– 2 USAF F-15s enforcing No Fly Zone report 2 low flying, slow moving radar contacts to USAF AWACS, which reports no known contacts in area

– F15s fail to obtain IFF responses from contacts. They conduct a visual ID pass, and identify the helicopters as Iraqi Hinds. They shoot down both Black Hawks, killing all passengers and crew

• Example of a breakdown in Combat ID. Provides a method for identifying key factors in the criteria for the requisite model?

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Case Study 1a – Factors

Social domain

Cognitive domain

Information domain

Physical domain

First time AWACS crew had flown together

Helicopter, AWACS and fighter communities socially isolated

Roles and responsibilities in AWACS crew confused

At time of VID, F15 pilots expected contacts to be Iraqi

Black Hawks not on Air Tasking Order

Black Hawks on different RF frequency to other air missions in area

Feature emphasised in recognition training similar for Black Hawks/Hinds

F15s briefed only on potential threats, not on friendly helicopters

Black Hawk visually identified as a Hind

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Combat ID - the requisite model

• This incident shows us social and cognitive factors were key (in fact dominant) in this breakdown of combat ID

• Implies that a modelling approach that doesn’t incorporate these factors is not requisite

• Borne out by recent operational experience that showed the worth of ‘social’ interventions - liaison officers

• Also supported by literature on human error, and accident analysis in complex civilian systems.

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Case Study 2 – Network Effects C2

• C2 concepts for Network Effects

• Based on work in progress by NATO SAS-050

• NATO Force under UN mandate undertaking an amphibious landing to restore sovereignty of Keswonian territory in the face of a Tetlovian invasion

• Conflict includes mixture of conventional joint and combined military assault, surveillance and border security, and restoring/maintaining security for humanitarian relief operations.

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

An intelligence failure has led to a ‘surprise’ incursion by Tetlovian forces on the eastern border, threatening to over-run Keswonian positions. First indication to Joint Force comes from a Liaison Officer with Keswonian Regt.

Help!

JFHQ

Re-planningMission

negotiation

DivisionCAOC BrigadeWOC TaskingTasking

Tasking

Tasking

Keswonians

Liaison Officer

Tim

e

Recce

Strike Manoeuvre

RecceEffectsgenerated

‘Traditional’ C2 Concept

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

An intelligence failure has led to a ‘surprise’ incursion by Tetlovian forces on the eastern border, threatening to over-run Keswonian positions. First indication to Joint Force comes from a Liaison Officer with Keswonian Regt.

JFHQ

DivisionCAOC BrigadeWOC

Tasking

Tasking

Keswonians

Liaison Officer

Tim

e

Recce

Strike Manoeuvre

RecceEffectsgenerated

‘Functional’ C2 Concept

ReportingReporting

Rep

ortin

g

Reporting

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Case Study 2 – Factors (sample)

Social domain

Cognitive domain

Information domain

Physical domain

Level/type of co-location

Persistence of teaming

Richness and reach of information networks

Availability of own force capability information

Task knowledge and competence

Motivational level

Cultural homogeneity

Behavioural predictability

Interpersonal and categorical trust

Teamwork experience

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Case Study 2 – Requisite model

• The Network Effects case study could not be effectively analysed using only physical and informational variables

• Socialisation factors affecting willingness of team member to work together creatively were at least as important as information sharing and collaboration tools

• The variables critical for assessing C2 options were roughly balanced across the four domains (PICS)

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Building practical requisite models

• UK is seeking to develop a demonstration of requisite modelling of C2, which includes social, cultural and organisational variables.

• Synthesis of a wide range of scientific theory is needed, covering social (including organisational) and cognitive theories and constructs, to complement conventional informational and physically-based modelling.

• Proposed architecture balances breadth and depth, as well as being sensitive to danger of too much complexity.

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Developing a modelling architecture

Social domain

Cognitive domain

Information domain

Physical domain

NCO Domains (PICS)

Expanded domain construct (PICOS)

Modelling architecture (active/interactive)

Organisation domain

Cognitive domain

Information domain

Physical domain

Social domain

Processesing

Organising

Teamworking

Socialising

Informing

Acting

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Taskwork (the ‘production’ process)

I/P O/PProcess as Task sequences

Changing Characteristics

Changing Scenario

Task evolution

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

‘Synchronisation’

Teamworking

Agent

AgentAgent

Physical interactions-Sharing environment-Communications-Sharing resource

Frame Frame

Frame

Abstract representation of processes involved in

achieving ‘synchronised’ or ‘shared’ understanding (using idea of overlapping

‘Frames’)

The ‘Frame’ represents the Agent’s set of knowledge,

beliefs, expectations, goals, understandings, etc

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Emerging meta-model

HQ Process as Task transition network

Task ‘team’‘Working together’

Agent Agent

Agent

AgentAgent

Physical

Frame Frame

Frame

Synchronisation

Gross Structure processes influencing agent relationships and links to/between resources

‘Being together’

Gross Social processes influencing knowledge in agent and team Frames

Teamwork/Taskwork ‘balance’ affects task performance/efficiency

Team knowledge determines whether team is forming, storming, norming, or performing

Teamworking processes influencing knowledge in agent and team Frames

© Dstl 2004, Dstl/CP12214September 3, 2004 Dstl is part of the

Ministry of Defence

Conclusions

• ‘Requisite’ models represent all critical factors (i.e. those which could significantly shape study conclusions).

• Modelling sub-sets of the problem is only requisite if the factors modelled are relatively isolated from others.

• The risks of not using requisite models are high.

• The default model (for C2 problems at least) should assume a balance of variables from all NCO domains as a start point.

• The practical construction of requisite models is possible, both in terms of underlying domain knowledge and modelling capability.

• The proposed modelling architecture is capable of hosting variables from all domains – thus making a good basis for requisite modelling.