formal modelling and analysis of socio-technical...

Formal Modelling and Analysis of Socio-technical Systems

Christian W ProbstTechnical University of Denmark

inria, Rennes, June 10, 2016

2

Virtual(malware)

Physical(damage to

furnace)

Social(spear phishing)

3

Motivation

• Attacks on systems and organisations exploit non-technical aspects.

• How do we formalize these human elements? – Social Engineering – Honest mistake – “Experimentation”

4

The TRESPASS Approach to Risk Assessment

• Information security threats to organisations have changed completely over the last decade.

• New attacks cleverly exploit multiple organisational vulnerabilities, involving physical security and human behaviour.

• Defenders need to make rapid decisions regarding which attacks to block, as both infrastructure and attacker knowledge change rapidly.

5

Key project goals

Predict complex attack scenarios spanning digital, physical and social engineering steps

Prioritise these scenarios via a planning tool that tells defenders where to expect the most serious issues

Prevent attacks by calculating and comparing cost-effectiveness of countermeasures

6

From organisational models to attacks

Attack Attack Attack Attack Attack Attack Attack Attack Attack

• Attack trees• Descriptive Method• Success based on experience and imagination of the

consultant/defende

• System Model• Analytic approach• Success based on experience and imagination of the

modeller

7

ModelOrganisation

Visualise

AnalysisModel

Choose most beneficial

countermeasures

Identify attacks and their impact

The TRESPASS Process

8

home

McScrooge card pin

pin

city Money BinMB

bank

$$$

$$$computer

C

settop boxSB

MB,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e

REM: e(Ptransfer)TECH: e(Pfirmware)

SB: e

Beagle Boys

C: i

door

trust M: m

get cash

get cashat ATM

goto ATM get card[(pin,X)], (pin,X)& input cash at ATM

get Charlie’s credentialsand perform action

input cashat ATM

in cash at ATM

get Alice’s credentialsand perform action

get credentials

get card

get pin

goto Home

goto Door &get trust

SE Alice move Door move Door

move Home

perform inat Alice

in cardat Alice

SE Alicein Card

goto Home



move Home

perform inat Alice

SE Alicein Pin

get card

perform inat Alice

in cardat Alice

SE Alicein Card

in pinat card

input cashat ATM

in cash at ATM

9

Organisation = Socio-technical Model

Organisation

• InfrastructureIT infrastructure

• Policies • Employees• Visitors

Socio-Technical System

• Interaction between people and technology in workplaces.

• Interaction between society’s complex infrastructures and human behaviour.

10

Our Contribution

• Explore the formal modelling and analysis of systems containing informal components

• For risk assessment in socio-technical systems.

14

The Attack Navigator

• Identifies and ranks attacks on an organisation.– Supports prediction, prioritisation, and prevention of

complex attack scenarios.

• Analytic risk assessment based on system model of organisation – infrastructure, policies, and employees.

• Identifies all possible attacks in the model!– Based on attacker profiles.

15 Satellite View

16 Satellite View

17 High Level View

19

Attack Navigator Map

• Models real-world systems – physical, virtual, social layer.– Relevant properties/actions – Maps to an analysable formalism – Apply static analyses and model checking

• Directed graph. • Models all locations that can be accessed/store data. • Models all entities that can move in the system.

21 Ehancing model through serious play

22

Socio-Technical Modelling Language

• Elements – Mobile components actors, programs – Data knowledge, artefacts – 3 basic actions • Input read, find something • Output print, loose something • Movement well, move around

23

Socio-Technical Modelling Language

Based on the Klaim process calculus.

24

home

McScrooge card pin

pin

city Money BinMB

bank

$$$

$$$computer

C

settop boxSB

MB,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Beagle Boys

C: i

door

trust M: m

25

Information Processing

26

Evaluating processes, moving around.

27

What are “valuable” assets?

• Depends on the modelled organisation:– Virtual or physical assets,– Operational goals, or– Global policies.

• Assets can be accessed by actors or processes, and• Can move/be moved around the system!

Ø This blurred location results in additional challenges!

28

Data and Data Handling Data Handling

Data

Policies

29

home

McScrooge card pin

pin

city Money BinMB

bank

$$$

$$$computer

C

settop boxSB

MB,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Beagle Boys

C: i

door

trust M: m

get cash

get cashat ATM



input cashat ATM

in cash at ATM


get credentials

get card

get pin

goto Home



move Home

perform inat Alice

in cardat Alice

SE Alicein Card

goto Home



move Home

perform inat Alice

SE Alicein Pin

get card

perform inat Alice

in cardat Alice

SE Alicein Card

in pinat card

input cashat ATM

in cash at ATM

30

Systematic generation of attacks from TRESPASSmodels

• Defender specifies undesirable states of the organisation – the destinations of an attacker!

• Attack navigator identifies all possible ways of reaching that states – the routes of an attacker!

Goal: use M’s card to get $$$;

home

Margrethe card pin

pin

city ATMA1

bank

$$$

$$$computer

C

settop boxSB

A1,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Fred

C: i

door

trust M: m

31

How do I get the money?

home

Margrethe card pin

pin

city ATMA1

bank

$$$

$$$computer

C

settop boxSB

A1,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Fred

C: i

door

trust M: m

32

Ask the TRESPASS Attack Navigator!

home

Margrethe card pin

pin

city ATMA1

bank

$$$

$$$computer

C

settop boxSB

A1,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Fred

C: i

door

trust M: m

33

Outside Home Living Room

Remoteserver

Settopbox

Bank Account

M

Dongle

Malware

The Attack Navigator

34

• Goal: Actor Fred, Be at A1, perform action with card and pin

• Policy: have card and pin• Goal: Obtain card and pin

– Location: Magrethe– Location: Home

• Action: Go to home• Policy: be trusted by M

– Action: Impersonate trusted person or

– Action: Break in• Action: Steal• Action: Social engineer• Action: Go to ATM, Obtain money

home

Margrethe card pin

pin

city ATMA1

bank

$$$

$$$computer

C

settop boxSB

A1,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Fred

C: i

door

trust M: m

Goal: use M’s card to get $$$;

$$$

card

pin

• This is a navigator map and the identified routes!

• Specify what to protect – the attack navigator shows you how that will fail (in your model).

35

Attack Generation

• Identify Attackers based on policy to invalidate. • Identify target locations in system.• Generate attacks for reaching target location.

– This will identify and obtain required assets to perform any of these actions, and obtain all assets required to reach the target location.

• Move to Target Location and Perform Final Attack.

36

Policies in TREsPASS

• The TREsPASS policy-specification language is a combination of policies and processes.

• Captures local access-control policies and global organisational policies.

• The resulting policy language is designed to be easily extended.

37

The TREsPASSPolicy Specification Language

Provide triggers for processes

Local policies

38

Powerful Policy Resolution

• Policies and model are translated to Datalog– inference rules, specified as Horn clauses

• Datalog programs deduce required credentials against knowledgebase (model!)

• Basis for powerful extensions, eg, PEAL

39

Cloud Case Study

40 TREsPASS Model of the Cloud Case Study

RoomInternal

DoorDataCenter

RoomDataCenter

server1

vm1

fileX,42

Grey

Finnpin,4

pwd,3

cardpin, 4

owner, Finn

server2

dataStore

DoorInternal

Hallway

DoorHallway

Outside

laptop switch1

WindowInternal

WindowDataCenter

Ethanpin,2

pwd,1

cardpin, 2

owner, Ethan

Bigpin,8

pwd,9

cardpin, 8

owner, Big

Sydneypin,6

pwd,7

cardpin, 6

owner, Sydney

Cleopin,5

cardpin, 5

owner, Cleo

Terrypin,10

pwd,11

cardpin, 10

owner, Terry

41

Generated Representation of Actors, Assets, and Policies

isActor(’Terry’).isItem(’id001’).hasName(’id001’,’card’).contains(’Terry’,’id001’).isData(’id002’).hasName(’id002’,’pin’).contains(’Terry’,’id002’).contains(’id001’,’id002’).

Terry• Terry is an actor, • who has a card (id001) • and a pin (id002) • that is also stored on the

card.

42

Generated Representation of Actors, Assets, and Policies

isPolicy(’id003’). hasName(’cred001’,’card’). requires(’id003’,’cred001’). enables(’id003’, ’move’). contains(’DoorInternal’,’id003’). isPolicy(’id004’). requires(’id004’,’cred001’). hasName(’cred002’,’pin’). requires(’id004’,’cred002’). enables(’id004’, ’move’). contains(’DoorDataCenter’,’id004’).

Policies• A policy (id003), • Requires a card (cred001) • And enables "move"• At DoorInternal.

• A policy (id004),• Requires a card (cred001)• And a pin (cred002)• And enables "move"• At DoorDataCenter.

43

Start Rule

44

Moving in

45

Models vs Attack Trees

• Models and policies guide the creation of attacks– ”white box testing” of system models– Cluttered models result in cluttered attack trees

• Level of detail is important• The model contains only relevant artefacts

– Technical details should be ”hidden” from the model– Instead they are added in a library approach

46

Libraries

• Libraries support individualization of models, attack trees, attackers, defenders, etc– By providing patterns for elements.– Establish interfaces to central repositories, community

portals.

47

Attack Patterns

• Explain how attacks can be implemented.– Dependent of the type of the actual elements of the attack.– Credit card with a chip vs magnetic strip.

• Separate the what (attack) from the how (concrete attack steps) in attack trees.

IN A item:I actor:C

A steals I from CA social

engineers C to give I

MAKE A B IN B item:I actor:C

A threatens B to execute IN B I C A blackmails B

A collects intel about B A blackmails B to execute IN B I C

A bribes B to execute IN B I C A social engineers B

A impersonates authority

A orders B to execute IN B I C

48

Applying Patterns

• Patterns are applied to the generated attack trees.• Leaf nodes are matched against available patterns.

– Attack pattern library performs property lookups, and – Decorates the leaf nodes, e.g., with necessary resources, skill level,

or risk of detection.

label match { case IN attacker item container:

// get type attacker from attacker profile // get type item from knowledge base // get type container from knowledge base // insert APL attacks that extract item from container

case MAKE attacker actor action: // get type attacker from attacker profile // get type actor from attacker profile // insert APL attacks based on types and action //...

}

49

get key

get key from McScrooge

get key from shelf

IN BeBo key McScrooge

MAKE BeBo Donald IN Donald key McScrooge

get key


get key from shelf



BeBo steals key from

McScrooge

BeBo social engineer McScrooge to give

them key

IN A item:I actor:C









IN A item:I actor:C









get key


get key from shelf


BeBo steals key from

McScrooge

BeBo social engineer McScrooge to give

them key

BeBo threaten Donald to execute

IN Donald key McScrooge

BeBo blackmails Donald

BeBo collects intel about Donald

BeBo blackmails Donald to execute


BeBo bribes Donald to execute


BeBo social engineers Donald

BeBo impersonates authority

BeBo orders Donald to execute



50

home

McScrooge card pin

pin

city Money BinMB

bank

$$$

$$$computer

C

settop boxSB

MB,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Beagle Boys

C: i

door

trust M: m

get cash

get cashat ATM



input cashat ATM

in cash at ATM


get credentials

get card

get pin

goto Home



move Home

perform inat Alice

in cardat Alice

SE Alicein Card

goto Home



move Home

perform inat Alice

SE Alicein Pin

get card

perform inat Alice

in cardat Alice

SE Alicein Card

in pinat card

input cashat ATM

in cash at ATM

51 Use of Colour and Form

52

Dealing with Uncertain Data

• Data available in practice often based on expert opinion – Can produce a wide range of estimates – Analysis methods must be able to cope

• Challenges – Data must be presented in an unambiguous form – Link data to the appropriate Basic Attack Steps – The uncertainty itself must be represented

• Can be addressed by sensitivity analysis or fuzzy reasoning

53

Analyses

• Valuate attack trees based on different factors– Impact on organisation, resources of the attacker,

probability of success– Attacker profiles guide resources and probability of

success• Resources are important

• Compute different quantitative properties

– Success/budget– Impact– Cost (time, resources, etc)

54

Data

Key metrics:probability, cost, time

Low-level model:CTMC, Interactive Markov Chain, Games

Quantitative & qualitative data

Quantitative results

Model

Stochastic analysis/

Model Checking

Model extraction

Dynamics Task 3.5

Attack model

Attack generation

Attacks,Countermeasure generation

ADVERTISEMENT SPACEHere could be your data

ADVERTISEMENT SPACEHere could be your analysis

55

home

McScrooge card pin

pin

city Money BinMB

bank

$$$

$$$computer

C

settop boxSB

MB,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Beagle Boys

C: i

door

trust M: m

get cash

get cashat ATM



input cashat ATM

in cash at ATM


get credentials

get card

get pin

goto Home



move Home

perform inat Alice

in cardat Alice

SE Alicein Card

goto Home



move Home

perform inat Alice

SE Alicein Pin

get card

perform inat Alice

in cardat Alice

SE Alicein Card

in pinat card

input cashat ATM

in cash at ATM

56 Attack Navigator Map

57

A Visual Vocabulary

58

Where is the Behaviour?

59

What is missing?

• Analyses over-approximate – All performable actions are performed – Needed to guarantee correctness of the result

• Whenever an actor can loose something... – ...he will do so

60

Adding Behaviour

• “naïve” approach: whenever needed, assume behaviour

• Add likelihoods for actions to be performed – Annotate items with a probability of a certain action

being performed on them– This may depend on time of day, mood, item, location, ... – Compute probabilities for actions to be performed

• Add a behaviour component – Analysis “asks” each analysed process for next action – Based on system state and actor knowledge and state

61

quantitative data

InfrastructureBehaviour

Actors

Risk of detection

Assets

Policies

Cost

Time

Risk appetite

Goal

Social Engineering

Data

Locations

Locations

62

Security Dashboard forSocio-Technical Systems

• Based on identified risks.• Generate surveillance mechanism

based on possible attacks.• Tracks observable movements of

actors and data (logged manually or automatically).

JAN

HallLJanJANFRENT m:FRENT

Out-side

m:CLSRV

m:CLUSR

m:CLUSR

m:CLSRVm:CLUSRCLUSRUSRHALLLJanJAN

PC1 PC2e:PCe:PC1 e:PC2

m:CLSRV

m:FREXIT

CLSRVSRVHALLLJanJAN

home

Margrethe card pin

pin

city ATMA1

bank

$$$

$$$computer

C

settop boxSB

A1,card,pin: i

IPTV pwdip

IPTV remoteREM

LAN

card, pin, ip: e


SB: e

Fred

C: i

door

trust M: m

63

Conclusion

• Socio-technical security models provide tools to assess the risk faced by modern organisations.– Spanning the physical, virtual, and social domain.

• The TREsPASS project has developed the attack navigator to identify all possible attacks in socio-technical models.

• Analyses of attacks provide insight in– Risk level of the organisation and contribution of parts of

the organisation to attacks.– But we need a better understanding of actor motivation

and behaviour.

formal modelling and analysis of socio-technical...

Documents