B I 8 D m 0

UNCLASSIFIED

I ' THIS DOCUMENT IS THE PROPERTY OF HER BRITANNIC MAJESTY'S GOVERNMENT, and is issued for the information of such persons only as need to know its contents in the course of their official duties. Any person finding this document should hand it to a British Forces unit or to a police station for safe return to the Chief Security Officer, DEFENCE SCIENCE AND TECHNOLOGY LABORATORY, Farnborough, Hampshire GU14 OW, with particulars of how and where found. THE UNAUTHORISED RETENTION OR DESTRUCTION OF THE DOCUMENT IS AN OFFENCE

when released to persons outside Government service, this document is issued on a personal basis and the recipient to whom it is entrusted in confidence within the provisions of the Official Secrets Acts 191 1-1989, is personally responsible for its safe custody and for seeing that its contents are disclosed only to authorised persons.)

UNDER THE OFFICIAL SECRETS ACTS OF 1911-1989.

This document is subject to the release conditions printed on the reverse of this page

Rendering Account - The Holistic Evaluation of Operational Capability

I M Mitchell

August 200 1

CopyNo of

Dstl/TNO1436

Cover + vi + 21 pages

Dstl is part of the Ministry of Defence

U NCLASSI Fl ED

UNCLASSIFIED

This document contains commercially valuable information controlled by Dstl. Intellectual Property Department must must be consulted before it is released outside Dstl.

U NC LASS I FI ED

I

This document is released to ISMOR 2001 for limited purposes only, and is not to be used for other purposes of disclosed to any third party without Dstl consent.

0 Crown copyright 2001 Defence Science and Technology Laboratory UK

UNCLASSIFIED

Authorisation

Prepared by

Title

Signa tu re

Date

Location

Authorised by

Title

Signature

Date

Ian M Mitchell Principal Analyst

Room 337 Northumberland House, Northumberland Avenue London WC2N 5BP

Dstl/CBS

@ 6-08-2001

... I l l

U NC LASS I FI ED

UNCLASSIFIED

Issue

Record of changes

Date Detail of Changes

I Sep 2001 I Presentation paper for 18 ISMOR I I I

iv

U N CLASS I F I ED

DSTLlTN01436

m m m U 8 8

U N C LA S S I F I ED

m B m (.

Executive summary

In 2001 many studies are of capabilities composing systems on systems. This contrasts with previous approaches focussed on individual equipment projects. These act as lower level components of systems. The paper explores implications of recent developments. It argues that the general move towards commercial accounting concepts requires that “currencies” for the benefits derived from defence capabilities are found to allow meaningful representation in analysis and so evidence in support of decision making. The process is increasingly a socio-technical one, with modelling an integral part of the discussions leading to the rational allocation of resources.

DSTLlTN01436

UNCLASSIFIED

V 1

UNCLASSIFIED m ' I

m m m

List of contents

Authorisation

Record of changes

Executive summary

List of contents

1

1.1

1.2

2

2.1

2.2

2.3

3 3.1

3.2

3.3

3.4

4

4.1

4.2

5

4

Introduction

Context

Structure of paper

Accounts

Accountancy as modelling

Processing within the accounts model

C u rre n cy

Currencies For Operations

As good as money can buy

Without price

Four Metrics

An Exchange Rate For Effectiveness

Levels of OA

Structure

The computer says whatever it's been programmed to

The End

References

Report documentation page

vi

U NC LASS1 FI ED

iii

iv

V

vi

9

9

9

10

11

11

11

12

14

16

16

18

19

21

23

DSTLlTNOl436

UN CLASS IF I ED

1 In trod uct ion

1.1 Context

1.1.1 Operational Research (OR) studies cover the working of systems and capabilities composing systems of systems. In the defence field, Operational Analysis (OA or Military OR), informs balance of capability and investment decisions by providing evidence to illuminate clashes of interest and opinion.

1 .I .2 A focus on the acquisition of capability contrasts with approaches focussed on the acquisition of individual equipment projects. These equipments act as lower level components of the systems of systems, which generate capability.

1 .I .3 This paper explores implications of recent developments. It argues that there is a general move towards commercial accounting concepts, as evidenced by the introduction of accrual based accounts. It proposes that “currencies” for the benefits derived from defence capabilities are needed to allow meaningful representation in analysis. This analysis provides evidence to both inform and account for the decisions made on the acquisition of defence capabilities.

1 .I .4 Within OA the metrics are less tangible than those of business. The process of defining currencies and using them is necessarily a socio-technical one, with judgement an essential part of currency definition. Modelling becomes an integral part of the discussions leading to the rational allocation of resources as well as a generator of evidence.

1.2 Structure of paper

1.2.1 The paper considers commercial accounts as a type of model. It discusses their core processing and outputs before moving to those sought from military operations.

1.2.2 Four types of metric are described with the role of judgement seen to be essential but not sufficient to the formation of a holistic view of capability.

DSTLTTN01436

UNCLASSIFIED

Page 1

2

2.1

2.1.1

2.1.2

2.1.3

2.1.4

2.1.5

2.1.6

2.2

2.2.1

2.2.2

Page 2

UNCLASSIFIED

Accounts

Accountancy as modelling

Accounts describe the ebbs and flows of a commercial system in a numeric form. They allow an appreciation to be made of the situation of a business.

The accounts function at interlinked levels of resolution between an overview of the results achieved by a business and the individual transactions which generated the res u I t s.

The phrase "the bottom line" is derived from commerce and is especially relevant to income and expense accounts. It shows in a single figure what the outcome was in terms of a profit or loss. As with OR models accounts abstract reality. The full complexity of a business is not captured in its accounts. They provide views of the business, based on abstractions and assumptions.

The outcome of profit or loss depends on assumptions of the period covered with relevant revenues received set against relevant costs. Conventions deem how the revenues and costs are viewed, whether calculating gross profit of the business or the abstract charge for amortisation of a portion of the costs of maintenance of the office cat.

The Standard Statements of Accounting Practice (SSAPs) in the UK have prescribed how some transactions may be defined. This common view allows comparison between the accounts of a number of businesses.

The value of the commercial entity can be assessed by this output metric. Other factors are often applied but the profitability of an enterprise even if it is a charity (non-profit) body is essential to its continued functioning.

Processing within the accounts model

Within the representation of business models two fundamental formulae can be seen.

0

Revenue -Cost = Profit. Money (revenue or cost) = Price of units used x Number of units used

structures, whether cash flow or accrual based, can be seen to be types of model. They allow a view of the value of an operation to be formed based on a set of stated assumptions.

DSTLfTN01436

UN CLASS1 FI ED

I a 8 II

m 0 m II 8

2.2.3

2.2.4

2.2.5

2.2.6

2.3

2.3.1

2.3.2

2.3.3

UNCLASSIFIED

There are many debates on what should be recognised as a cost or revenue, particularly by other interested parties such as tax gatherers. Other metrics exist, such as ratios for price earnings or returns or investment. They draw on the core processes of business as defined by these two formulae.

Defining the boundaries of a business, where one identifies inputs and outputs, is critical. This includes what is deemed to be a part of a business and the timescales used in measuring its operation.

Accounting structures, whether cash flow or accrual based, can be seen to be types of model. They allow a view of the value of an operation to be formed based on a set of stated assumptions.

Accountancy is often retrospective. When OA models are run retrospectively it is usually for the purpose of verification and validation. The OA models’ purpose in use is to assist in the selection of better ways of conducting an operation in the future.

Currency

Commerce is blessed by the existence of a single metric for many of its works. This metric is money. Money is a means of symbolising the value of other goods and services despite its own intrinsically valueless nature. Accounts are useful because the metric for the system is well established and accepted by many different societies.

As with OR models underpinning assumptions for the accounting standards are fluid areas for debate. Robert Bruce explored this theme in his article for The Times on 12 July 2001. His closing words included:

... the importance of consistentjnancial reporting around the globe is at last seen as more important than the number of exhausted and angry standard-setters arguing about accounting perfection. ’’ (emphasis added)

His emphasis is on the importance of understanding of the assumptions underpinning numbers prepared on a common set of assumptions.

DSTWN01436 Page 3

U NC LASS1 FI ED

UNCLASSIFIED

3 Currencies For Operations

3.1 As good as money can buy

3.1 .I The costs of conducting military operations can be measured in terms of money or casualties. The value of the outcome obtained from the operations is far less clear and may have no direct equivalent in money terms.\l\lithout some measure of the benefit obtained no estimate can be made of the return on investment.

3.2

3.2.

Without price

The outputs of Peace Support Operations (PSOs) are particularly challenging. One view is that description of the output of a PSO may be assessed on the basis of movements within a Maslovian hierarchy as shown below. Intervention moves the state from one level to another. The approach is further explored in the proceedings of CornwallisVI (Reference 1) to be published later in 2001.

3.2.2 Whilst the evils in the abyssal level have provoked intervention how far up the pyramid an intervener should attempt to move the intervenee is not clear. The X level, at the equivalent of Maslow's self-actualisation, in particular may lie beyond the gift of an intervener.

Page 4

V W Y Governance

Society

Security Justice Economy

Relief needed

1 The Abyss

Figure I: Maslow derivative view of a health of a state

DSTL/TNO 1436

UNCLASSIFIED

UNCLASSIFIED

3.2.3 The value of an intervention in moving a failed or failing state from one tier to another will be determined on the basis of judgement of decision makers.

3.2.4 In the modelling of capability mechanisms to capture and communicate the values of the decision-makers are important but these can not give an answer alone.

3.2.5 The challenge of linking the individual equipments with overall capability has provoked many attempts as described in a 1994 presentation at Young OR I 1 by Paul Beardsmore on strategic planning of Naval Forces (Reference 2). There is no single analytical solution.

3.3 Four Metrics

3.3.1 The four metrics are summarised here and their place in scenarios is illustrated below. The diagram illustrates eight inputs for the generation of a scenario, in the sense of a sequence of events. This was originally used to describe the Close Action Environment (CAEN) model but has broader applications.

0 Measure of Performance (MOP) at the component or technology level- what can an item or person do? Examples include the maximum detection range of a sensor or maximum speed of a vehicle. These are the measures of inputs for the forces and technology elements from the scenario diagram.

0 Measure of Effectiveness (MoE) looks at the outputs often in terms of damage sustained to the force in the course of a scenario.

0 Measure of Outcome; (MOO) establish whether a scenario has been successful or not. These may be thresholds for example on casualties or simple yesho questions eg was the hill taken.

0 Measure of Capability (MoC) a set of missions, in terms of a number of outcomes.

DSTLlTNOl436

UN C LASS IF1 ED

Page 5

UNC LASS1 FI ED

Figure 2 Scenario elements and metrics

3.3.4 The four tiers have broad applicability. As David Oxenham has described for air warfare (Reference 3) and Alistair Dunlop observed on infantry operations (Reference 4) performance does not drive effectiveness as a linear function. Formulating relationships between MOPS and MoEs is often challenging.

3.3.5 The links between the higher metrics, of Outcome and Capability differ in their nature. A chain structure is proposed below.

Page 6 DSTLmN01436

UNCLASSIFIED

UNCLASSIFIED

I. 3.3.6

3.3.7

3.3.8

3.4

3.4.1

3.4.2

3.4.3

Potential

QKI lnterpre ation

Capability

Opinion

Figure 3 Chain structure of holistic system of military capability

As progress is made from left to right so the importance of judgement increases. A Measure of Performance can be established by experiment. Observation of existing forces, such as the Static scoring described by Matt Jolly (Reference 5) can also be brought to bear. MoPs are measures of potential.

The circumstances of the setting and the interaction with opponents or other participants generate opportunities for the potential to be realised. MoEs represent the results of these interactions.

MoEs can be found from simulations, experiments, trials or historical sources. In all of these interaction between the MoPs occurs and their consequences may be observed. Metrics such as casualties sustained and inflicted or ground retained or lost are the results.

An Exchange Rate For Effectiveness

The definition of what makes a successful outcome can only be based on judgement. This judgement is driven by the values of the judge. Capability is an aggregation of outcomes. Whilst this step may be simple addition it implicitly incorporates the judgements of the outcome level.

The number of casualties acceptable from an operation is a key issue to many societies. This number is not universal by time or location.

Wellington, the “Iron Duke”, observed that “Nothing except a battle lost can be held so melancholy as a battle won.” in his despatch of 1815 (Reference 6), cited in his tomb. There was no doubt that Waterloo had been a victory from the British perspective, despite the numbers of casualties sustained. The headmaster at Waterloo’s book on the battle (Reference 7) cites the Allies as losing 22,000 compared with the French losses of 27,000 dead and 8,000 prisoners.

DSTWNO1436

U NC LASS I F I ED

Page 7

UNCLASSIFIED

3.4.4 To make the statement “Waterloo was a victory” depends on the way that the facts of the battle, as can best be discerned, and the outcome’s effect on the campaign overall are interpreted. For modelling capability and outcome the identification and representation of the values of observers is inevitable.

3.4.5 Quantification of the values provides the “exchange rates” with which an individual views the world. These values drive preferences on the basis of which decisions are made on aim selection and resource allocation to activities to achieve that aim. Currency as a means of language allows the development of common understanding of a situation, although consensus may not be feasible between all observers.

3.4.6 Measures of Outcome are based on the interpretation of these results from a scenario. As such they are driven by the qualitative views of those making them.

3.4.7 Measures of Capability look at a sum of the outcomes. Again qualitative inputs set the standard of “enough”.

3.4.8 A shift from heavily qualitative towards heavily quantitative is apparent in the progression from Performance to Capability.

3.4.9 The next section considers the structure of studies to provide content for the met rics.

Page 8 DSTLTTN01436

U NCLASS I FI ED

UNCLASSIFIED

4 Levels of OA

4.1 Structure

4.1.1 Andrew Trethewey used a triangle diagram (Reference 8) in the Nuclear Boilogical and Chemical (NBC) OA Strategy Paper to describe four levels of studies. These have evolved and are shown below in context with the measures of Capability, Outcome, Effectiveness and Performance (C, 0, E and P).

Figure 4 Studies levels and metrics

4.1.2 Level 1 covers the study of capabilities as the outputs estimated to be achievable by the systems of systems. Level 2 studies, linking system effectiveness with campaign outcome, comprise a relatively lightly explored area.

4.1.3 Level 3 covers the operation of individual equipments. These act as lower level components of systems. In the triangle of studies much “hard” OA has been conducted at Level 3 within the context of a higher level scenario or campaign.

4.1.4 Picking a winner from amongst a number of candidates requires a focus on level 3 to allow discrimination for the different sets of Performance characteristics. Some consequences can be demonstrated in the forms of Measures of Effectiveness.

DSTLITNO1436 Page 9

UNCLASSIFIED

4.1.5 A major gap in capability can sometimes be seen at Level 3 without the intervening

4.1.6 The following diagram illustrates the linking up of the levels of studies.

tier, if it is sufficiently dire, to prevent the achievement of the campaign mission.

Page 10

eg How much of what kind of successful operation can be done?

combinations

Acceptable Casualty threshold

I I

MOO

Best Case eg no casualties

Worst Case eg annihilation I MoE

MOP 4 Range of minimum acceptable performance

Range of minimum acceptable effect

component eg Casualties sustained in presence of a level of performance

Figure 5 Conversions with ranges

DSTUTN01436

UNCLASSIFIED

UNCLASSIFIED

4.1.7

4.1.8

4.2

4.2.1

4.2.2

4.2.3

4.2.4

4.2.5

4.2.6

The lower level relationships may offer greater scope for quantitative analysis but they are also more complex. There are greater numbers of entities interacting.

The MOP to MoE link is not one to one. A single MoE is a function of the MOPS not only between the members of one side but also with those of the other participants, all using their resources to attain objectives within a fluctuating situation.

The computer says whatever it’s been programmed to

The process is a socio-technical one, with modelling an integral part of the discussions leading to the rational allocation of resources. This reflects a mix of quantitative and qualitative inputs. As Daniel Goleman describes in his works on “Emotional Intelligence” (Reference 9) has described the use of preferences is essential to the process of making any decision. Data alone is not enough.

The preferences of an individual drive the solution selected by the individual from a number of options. Preferences can be captured in the form of numerical weights in Multi-Criteria Decision Analysis.

This process is described in Valerie Belton’s paper from 1985 (reference 10). The approach which it describes has been developed and has been implemented in the Equity model produced by Enterprise LSE.

There is often tension perceived between judgement and the modelling of physical processes. This is often described as the “Soft” versus “Hard” debate. In fact, the two rely on each other to allow a holistic view to be taken. Judgement without evidence is likely to mislead. Evidence without judgement can provide no guidance.

As Level 1 is approached the essential importance of qualitative input increases. Multi- criteria decision analysis offers a structured way to consider issues at the capability level.

Its criteria are the currencies with which an account can be rendered allowing decisions to be made and subsequently audited.

DSTUTN01436

UNCLASSIFIED

Page 1 1

UNCLASSIFIED

5 The End

5.1 This paper has looked at the process of rendering accounts to holistic evaluate of operational capability.

In 2001 many studies are of capabilities composing systems of systems. This contrasts with previous approaches focussed on the merits of individual equipment projects These act as lower level components of systems.

5.2

/ Performan

Figure 6 A holistic view of measures

5.3 The paper explored implications of developments from 2000 to 2001. It has argued that the general move towards commercial accounting concepts requires that “currencies” for the benefits derived from defence capabilities be found to allow meaningful representation in analysis and so evidence in support of decision making.

5.4 The process is a socio-technical one, with modelling an integral part of the discussions leading to the rational allocation of resources. Judgement can never be excluded from the system. Its presence is increasingly important at the higher levels.

Page 12 DSTVTN01436

UNCLASSIFIED

UNCLASSIFIED

5.5 Means of capturing the judgements and putting them into a rational framework are as valid OA as that conducted within the lower level models which inform the judges.

5.6 Defence capability is never cheap. Current circumstances have risks rather than a defined threat, so the ability to describe capability holistically has value. It allows an account to be rendered for the resources committed. Such explanation may be essential to its own defence against other uses for the resources.

DSTLmN01436

U N C LASS I FI ED

Page 13

4

1. 2.

3.

4.

5.

6. 7. 8. 9. I O .

Page 14

U NC LASS1 FI E D

References

Proceedings of Cornwallis VI 2001-08-14 Paul Beardsmore ‘The Strategic Planning of Naval Forces Using The Concept Of Sea Power’ presented to Young OR 11, 1994 D Oxenham ‘Determining the effectiveness of air-launched weapons.’ Journal of Defence Science Vol. 3 No4 pages 490-496 October 1998 UK Restricted. I Mitchell ‘Notes from Soldier Modelling Workshop 26-31 January 1997’ CDA/LS/WP2/97 M Jolly ‘The Shepherd’s Balls: The Roles and Limitations of Static Scoring’ The Net Assessments Unit 1994 h tt p :www . st pa u Is. Co. u k/a bout/t hecry pt/wel I ing ton. h tm . FR. Libert Waterloo Crast Printers Nivelles Belgium Andrew Trethewey ‘NBC OA Strategy Paper’ 1998 Restricted D Goleman ‘Emotional Intelligence’ pages 27-28 Bloomsbury London 1996 V Belton ‘The use of a simple Multiple-Criteria Model to Assist in Selection from a Shortlist’ 2000 reprint of Journal of the Operational Research Society Volume 36 No 4 pp 265-274 1985

DSTLTTN01436 I D II’

m A UNCLASSIFIED

UNCLASSIFIED

5. Report Classification and Caveats in use:

UNCLASSIFIED

Report documentation page

6. Date written: Pagination: References:

July 2001 vii + 20 10

1. Originator’s report number: Dstl/CBS/TN000554

2. Originator’s Name and Location: Ian M Mitchell Room 337 Northumberland House Northumberland Avenue Main Building Whitehall London WC2N 5BP

3. MOD Contract number and period covered:

4. MOD Sponsor‘s Name and Location:

NIA

NIA

7a. Report Title: RENDERING ACCOUNT - THE HOLISTIC EVALUATION OF OPERATIONAL CAPABILITY

7b. Translation / Conference details (if translation give foreign title / if part of conference then give conference particulars):

ISMOR 2001: The 18th International Symposium on Military Operational Research at Eynsham Hall North Leigh Witney Oxon OX8 6PW.

7c. Title classification: UNCLASSI Fl ED

8. Authors: I M Mitchell

9. Descriptors / Key words: Measures of Effectiveness, Performance, Outcome, Ca pa bi I i ty , Accountancy. Equity , M C DA

~~~ ~~ ~~ ~

loa. Abstract. (An abstract should aim to give an informative and concise summary of the report in up to 300 words).

In 2001 many studies are of capabilities composing systems of systems. This contrasts with previous approaches focussed on individual equipment projects. These act as lower level components of systems. The paper explores implications of recent developments. It argues that the general move towards commercial accounting concepts requires that “currencies” for the benefits derived from defence capabilities are found to allow meaningful representation in analysis and so evidence in support of decision making. The process is increasingly a socio-technical one, with modelling an integral part of the discussions leading to the rational allocation of resources.

1 Ob. Abstract classification: UNCLASSIFIED I FORM MEETS DRlC 1000 ISSUE 5

DSTWN01436 2

U NC LASS I FI ED

U NC LASS1 FI ED

This page is intentionally blank

U N C LASS I FI ED