measuring and maturing an appsec program · 2015-04-21 · presenter . bruce c jenkins cissp,...

Measuring and Maturing an AppSec Program

Presenter Bruce C Jenkins CISSP, CSSLP, CISM Fortify Security Lead & AppSec Program Strategist Hewlett-Packard Company Contact: bcj at hp dot com

2 (ISC)2 e-Symposium

Agenda

• Why Measure

• Preparing to Measure

• What to Measure


Why Measure


Why Measure Humans have a natural tendency to want to measure


Why Measure We have been measuring (and comparing) since ancient times


Source: wikipedia.org/wiki/Cubit

Source: www.theguardian.com

Why Measure Numerous guides, standards, and frameworks speak to measurement


2005 2006 2008* 2010 2010 2013

*OpenSAMM update scheduled for CY2015

Why Measure Bottom line: Decision Support


Bruce

Source: HP Fortify on Demand

Why Measure Views about the priority of security in custom software development


Source: Osterman Research White Paper, Jan 2015

Preparing to Measure


Preparing to Measure First some basic definitions




goal long-term aims that you want to accomplish

Preparing to Measure Goals often are broad or lofty and long term; Example Personal Goal: Be taller




objective concrete attainments achieved by following a certain number of steps; time-constrained, measurable goals


Preparing to Measure Objectives are concrete, measurable and time-constrained achievements on the path to reaching a particular goal


Obtain medieval-certified rack by 2015-05-31

Complete medieval rack Train-the-Trainer program by 2015-09-18

Train and certify four rack operators by 2015-12-31

Complete Phase I Stretching Program by 2016-04-01



objective

metric

concrete attainments achieved by following a certain number of steps; time-constrained, measurable goals

a quantitative measure




objective

metric



KPI Key Performance Indicator (KPI) is used to evaluate the success of an organization or of a particular activity


Preparing to Measure Sidebar: Top challenges in achieving software security goals*


Source: Gatepoint Research Pulse Report, Oct 2014 n = 300 executives

*Read as: software security assurance (SSA) program goals

Preparing to Measure

“It is necessary that people work together in unison toward common objectives and avoid working at cross purposes at all levels if the ultimate in efficiency and achievement is to be obtained.”


Dave Packard Co-founder, Hewlett-Packard

Preparing to Measure Sidebar: Sound software security assurance (SSA) programs are based on business needs


Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training

Preparing to Measure Use security goals to establish SSA program direction, achieve stakeholder unity


Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training

• Establish security-related goals that are directly tied to the firm’s mission

http://www8.hp.com/us/en/hp-information/about-hp/corporate-objectives.html

Preparing to Measure Use security goals to establish SSA program direction, achieve stakeholder unity Example: Hewlett-Packard Co.


Profit

Customer Loyalty

Growth

Market Leadership

Commitment to Employees

Leadership Capability

Global Citizenship

Hewlett-Packard

See HP’s Corporate Objectives at http://www8.hp.com/us/en/hp-information/about-hp/corporate-objectives.html



Profit

Customer Loyalty

Growth

Market Leadership



Global Citizenship

Hewlett-Packard

...

Goal 1

... ...

Goal n

HP Software ...

Goal 1 ...

Goal n

Fortify

Goal 1

... ...

...

Ent. Security

...

Goal n

…

Security Goal 1

…

…

…

Security Group

Security Goal n




Profit

Customer Loyalty

Growth

Market Leadership



Global Citizenship

Hewlett-Packard

...

Goal 1

... ...

Goal n

HP Software

Goal 1

... ...

...

Ent. Security ...

Goal 1 ...

Goal n

Fortify

…

Security Goal 1

…

…

…

Security Group

...

Goal n Security Goal n




Profit

Customer Loyalty

Growth

Market Leadership



Global Citizenship

Hewlett-Packard

...

Goal 1

... ...

Goal n

HP Software

Goal 1

... ...

...

Ent. Security ...

Goal 1 ...

Goal n

Fortify

…

Security Goal 1

…

…

…

Security Group

...



Preparing to Measure Example goal for anchoring security program (real-world Financial)


Corp Mission Statement

Goal 1

Goal 2

Goal 3

Protect our customers’ data

Goal n




Goal 1

Goal 2

Goal 3


Goal n




Goal 1

Goal 2

Goal 3


Goal n

Corp Security Group

Security Goal 1

Security Goal 2

Security Goal 3

Security Goal n




Goal 1

Goal 2

Goal 3


Goal n

Corp Security Group

Security Goal 1

Security Goal 2

Security Goal 3

Security Goal n

Proactively identify security risk in

Business Critical applications



Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training





Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training


• Develop a security strategy that is designed to support achievement of the security goal(s)




Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training



• Based upon business priorities and portfolio risk*, design time-constrained, measurable objectives

*portfolio is known, classified and risk-ranked




Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training




• Only choose metrics and construct KPI’s that show progress toward meeting the objectives; nothing else



What to Measure


What to Measure Revisited: Example goal for anchoring security program Focus: Security Objectives



Goal 1

Goal 2

Goal 3


Goal n

Corp Security Group

Security Goal 1

Security Goal 2

Security Goal 3

Security Goal n



What to Measure

(ISC)2 e-Symposium 41

Security Goals Security Objectives & Tasks* Metrics [measurement frequency]

SG-3 Proactively identify security risk in Business Critical applications

SO-3.1 Reveal baseline risk exposure of all Mission Critical applications no later than FY15Q2 ST-3.1.1 Conduct baseline static scans of all Mission Critical apps no later than FY15Q2 ST-3.1.2 Conduct baseline dynamic scans of all Mission Critical apps no later than FY15Q2

SM-3.1.1 Number of static scans remaining [w] SM-3.1.2 Number of dynamic scans remaining [w]

*implemented in accordance with the SSA program strategy

What to Measure




SO-3.1 Reveal baseline risk exposure of all Business Critical applications no later than FY15Q2 ST-3.1.1 Conduct baseline static scans of all Mission Critical apps no later than FY15Q2 ST-3.1.2 Conduct baseline dynamic scans of all Mission Critical apps no later than FY15Q2



What to Measure







# of Scans Remaining by Week Ref. SM-3.1.1, SM-3.1.2


What to Measure




SO-3.1 Reveal baseline risk exposure of all Business Critical applications no later than FY15Q2 SO-3.2 Identify security skills amongst application Architects, Developers and Testers no later than FY15Q2 SO-3.3 Qualify gap between application risk exposure and security skills no later than FY15Q3 SO-3.4 Implement training program to address security skills gap no later than FY15Q3 SO-3.5 Evaluate risk exposure of all Mission Critical applications on a quarterly basis

SM-3.5.1 # of Critical findings [q] SM-3.5.2 # of High findings [q]


What to Measure




SO-3.1 Reveal baseline risk exposure of all Business Critical applications no later than FY15Q2 SO-3.2 Identify security skills among application Architects, Developers and Testers no later than FY15Q2 SO-3.3 Qualify gap between application risk exposure and security skills no later than FY15Q3 SO-3.4 Implement training program to address security skills gap no later than FY15Q3 SO-3.5 Evaluate risk exposure of all Mission Critical applications on a quarterly basis



What to Measure







What to Measure




SO-3.1 Reveal baseline risk exposure of all Business Critical applications no later than FY15Q2 SO-3.2 Identify security skills among application Architects, Developers and Testers no later than FY15Q2 SO-3.3 Qualify gap between application risk exposure and security skills no later than FY15Q3 SO-3.4 Implement training program to address security skills gap no later than FY15Q4 SO-3.5 Evaluate risk exposure of all Business Critical applications on a quarterly basis



What to Measure







What to Measure Consolidated dashboards are effective at providing the status of key metrics (Key Performance Indicators)


Bruce

Source: HP Fortify on Demand

What to Measure Use security goals to establish SSA program direction, achieve stakeholder unity


Mission

Goals

Objectives

Strategy

m m m KPI

Policy

Standards

Training




• Only choose metrics and construct KPI’s that show progress toward meeting the objectives; nothing else



Summary • Anchor your security program to the business


Summary • Anchor your security program to the business • Develop security goals that directly support the business goals


Summary • Anchor your security program to the business • Develop security goals that directly support the business goals • Strategies are used to achieve goals; develop a strategy


Summary • Anchor your security program to the business • Develop security goals that directly support the business goals • Strategies are used to achieve goals; develop a strategy • Goals must have supporting objectives; some may be long-lived


Summary • Anchor your security program to the business • Develop security goals that directly support the business goals • Strategies are used to achieve goals; develop a strategy • Goals must have supporting objectives; some may be long-lived • Never use a metric that does not support an objective; it will be

interesting at best, but will not add value to your program



interesting at best, but will not add value to your program • Report your progress to maintain program justification & budget



interesting at best, but will not add value to your program • Report your progress to maintain program justification & budget • Adjust the strategy as business goals, threats and risks change


hp.com/go/fortifyssa

Q&A

measuring and maturing an appsec program · 2015-04-21 · presenter . bruce c jenkins cissp,...

Documents