security metrics

© 2006 Spire Security. All rights reserved.

Pete Lindstrom, CISSPResearch Director

Spire Security, LLCwww.spiresecurity.com

[email protected]

Security Metrics


Risk and Control Metrics


3

Agenda

1. System & Information Assets

2. Estimating Loss

3. Controls - Four Disciplines

4. Risk & Control Metrics

5. “Business” Metrics

6. Measure Security Spending

7. Activity-based Costing

8. ROI & ROSI


4

You Can’t Quantify Risk

Yes, you can quantify risk.


5

The Precedent is Set

“Furthermore, the affidavit of Plaintiffs' expert conclusorily posits that Plaintiffs' risk of identity fraud is significantly increased without quantifying this risk. Defining "significant" for the purpose of awarding credit monitoring is a matter of law for the Court, however, and mere allegations that an increase is significant do not constitute evidence.

Similarly, although Plaintiff's expert opines that credit monitoring will "substantially" reduce the risk of identity fraud, he fails to quantify the reduction of risk in objective terms. Because the Court finds that there is no evidence in the record before it that Plaintiffs‘ personal information itself endured significant exposure, that Plaintiffs' risk of identity fraud is significantly increased, or that credit monitoring will reduce the risk of identity fraud to the necessary degree, Defendant's motion for summary judgment must be granted as to this case even if credit monitoring were available in other circumstances.”

Stollenwerk v. TriWest Health Care Alliance, No. 03-0185 PHX SRB (D. Ariz.)


6

Risk is NOT What You Think it isYou may have some preconceived notions (let’s dispel them

now):

Risk is NOT static; it is dynamic and fluctuates constantly with potentially high degrees of variation – like any financial index.

Risk is NOT about the possibility that something bad could happen; it is about the probability that it might happen.

Risk is NOT some pie-in-the-sky academic exercise; you have all of the necessary information available to you today.

Risk is NOT a vague, ambiguous concept; it is a continuum along which you can plot many levels of tolerance and aversion.


7

What is Risk, then?

Risk is the likelihood that something unwanted will happen.

You define what is unwanted (I will give you some ideas).

The goal of any information security program is to minimize risk within the constraints set forth by the organization, technical environment, and available resources.


8

Risk = likelihood of negative outcome, where the impact of the negative outcome is understood (and sometimes quantified).

Three faces of risk:o Manifest Risk – The risk of attack or

compromise associated with end-user system events. (Activity)

o Inherent Risk – the risk associated with the “possibility” of attack due to the availability or exposure of targets. (Asset)

o Contributory Risk – the risk related to control process failure and/or incompletene. (Admin)

Three Faces of Risk


9

1. Define Unwanted Outcomes

Step 1 to Quantifying Risk

Take C, I, A objectives to the next level:o Content is read by inappropriate people

(confidentiality breach).o Content is inappropriately modified (Integrity and

authenticity breach) o Content access is interrupted (availability breach).

The Internet has made resources as important as data/content:o Program access is interrupted (productivity breach).o Programs are abused (liability breach).

UO’s are compromises of these objectives, precipitated by attacks.


10

The Ginsu Approach to UOs

Confidentiality

Integrity

Availability

Productivity

Liability

Dat

a/In

form

atio

nD

ata/

Info

rmat

ion

Res

ou

rce

sInbound

(In-Transit)Stored

(At-Rest)Outbound(In-Transit)

Sniff Copy (“steal”) Leak

Spoof, Replay, Insert Modify Redirect

Overload Delete Overload

Overload Distract Consume

Relay/Bounce Abuse (illegal) Propagate

Attacks Compromises


11

2. Define the Event Set “Universe”

Once we have defined unwanted outcomes, we can “back up” to identify the event set of all possible outcomes (that include both wanted and unwanted outcomes):o 1) similar events that can create the outcome

(attack/exploit vectors); oro 2) all objects that could be affected by the outcome.

(more later).

The most common events in data processing are:o Network Flowso [Host- or application-based] Sessionso Program Operationso [Content-oriented] Transactions / Messages


12

Network Layer: Flowso Source IP, Dest IP, Dest Porto Inbound and/or Outbound

Host Layer: Sessionso Sessions under managemento Number of logins

Application Layer: Program Operationso System callso Application calls

Data Layer: Transactionso Messageso Business Activities (financial trades, purchase orders,

published articles, etc.)o Queries – Record Retrieval

Recall IT Activities (Events)


13

Good Events Bad Events

Total Events

Step 1: Define unwanted outcomes

Step 2: Define event set

Step 3: Calculate risk: count and divide step 1 by 2:

Risk = Bad Events

Total (Good + Bad) Events

Risk = Bad Emails

Total (Good + Bad) Emails

3. Calculate Risk


14

This is “Manifest” Risk

Manifest Risk is that risk associated with “real-time” or ongoing data processing activities in your computing environment.

(I repeat) The most common events in data processing are:o Network Flowso [Host- or application-based] Sessionso Program Operationso [Content-oriented] Transactions / Messages

The philosophy is “you can’t have a compromise without an attack”; if your systems and data never get touched, they can’t result in a compromise.

But it’s really not that simple, is it? Enter the INFORMATION SECURITY PROFESSIONAL!


15

Manifest Risk

Events occurring within the computing environment. (Actual)

Philosophy: A compromise can’t occur without online activity.

Count discrete activities.o Actual Flows (network)o Actual Sessions (system)o Actual Program Commands (application)o Actual Transactions (data)

Count number of “bad” activities.


16

4. Calculate Control Coverage Recall that risk is dynamic.

The key factor in minimizing risk is the set of controls we place over the risk.

We have existing controls and apply new ones throughout our computing environment constantly.

Control Coverage is the percent of events that any or all of our controls evaluate.


17


ControlledControlled UncontrolledUncontrolled

Total Events

Coverage = Controlled Events

Total Events

Note: some set of good events and some set of bad events are uncontrolled.

Calculate Control Coverage


18


ControlledControlled UncontrolledUncontrolled

Allowed Denied Allowed Denied

Success SuccessFailureFailureLuckyFailure

Total Events

(false positive) (omission) (false negative)

5. Control Success/Failure


19

6. Calculate “Residual” Manifest Risk

Total EventsGood Events(“wanted”)

Bad Events(“unwanted”)

Controlled Events

Uncontrolled Events

allowed

denied

“Residual” manifest risk = False Negatives + Omissions

Total Events


20

Legitimate Email Spam

Controlled Uncontrolled

Allowed Filtered


Email Messages

(false pos) (omission) (false neg)

Uncontrolled Controlled

Allowed Filtered

Risk =Spam

Email Msgs

1

2

1

2

43

Coverage =ControlledEmail Msgs 2

3 4+Effectiveness =SuccessEmail Msgs 2

5 6+

65 7 8

“Resid” Risk =IncidentsEmail Msgs 2

7 8+

Email Risk


21

Legitimate Calls Overflows

Controlled Uncontrolled

Allowed Blocked


System Calls

(false pos) (omission) (false neg)

Uncontrolled Controlled

Allowed Blocked

Risk =OverflowsSys Calls

1

2

1

2

43

Coverage =ControlledSys Calls 2

3 4+Effectiveness =SuccessSys Calls 2

5 6+

65 7 8

“Resid” Risk =IncidentsSys Calls 2

7 8+

Buffer Overflow Risk


Pete [email protected]

www.spiresecurity.com

Agree? Disagree?

security metrics

Documents