visualize this! meaningful metrics for - rsa conference · pdf filevisualize this! meaningful...
TRANSCRIPT
![Page 1: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/1.jpg)
SESSION ID:
Moderator:
Panelists:
Visualize This! Meaningful Metrics for Managing Risk
GRC-F02
John Johnson Global Security Strategist John Deere
Alex Hutton Director of Operations Risk & Governance A Financial Organization
David Mortman Chief Security Architect and Distinguished Engineer Dell Enstratius
Jack Jones President CXOWARE Inc.
Caroline Wong Security Initiatives Director Cigital
![Page 2: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/2.jpg)
#RSAC #RSAC
Agenda
What do you mean by meaningful metrics?
Build your own metric – audience participation!
Metrics Categorization
Leveraging Frameworks and Models
What decisions do your metrics focus on supporting? Examples.
Every organization has loss events. What loss metrics do you capture and how do you leverage them?
Painting a picture with meaningful metrics
2
![Page 3: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/3.jpg)
#RSAC #RSAC
Vocabulary • Measurement vs. Metric – what’s the difference?
o I had 2 eggs for breakfast this morning o It’s 46 degrees in Sterling, VA o This workshop is 105 minutes long
• A measurement is the value of a specific characteristic of a given entity
• A metric is the aggregation of one or more measurements to create a piece of business intelligence.
o What is the question the metric answers? o What is the decision the metric supports? o What is the environmental context?
![Page 4: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/4.jpg)
#RSAC #RSAC
Real Life Metrics
• What metrics are you using to answer questions and make decisions about software security? o What question, what decision
o Who’s asking, who’s answering
o What’s the goal
o What environmental context
![Page 5: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/5.jpg)
#RSAC #RSAC
Build Your Own Metric
![Page 6: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/6.jpg)
#RSAC #RSAC
Example
6
Risk Landscape Visibility –helps us understand how well informed (or not) our risk decisions
are. The values represent data and estimates regarding four elements (asset population, threat
conditions, value/liability at risk, and control conditions). This helps us to focus on specific areas of
poor visibility, thus improving our ability to make well-informed risk decisions.
![Page 7: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/7.jpg)
#RSAC #RSAC
Example
7
Root Cause Analysis — which helps us understand why undesirable conditions exist (e.g., non-compliance with policy). This enables us to focus on our efforts to systemically improve.
![Page 8: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/8.jpg)
#RSAC #RSAC
Example
8
► Loss Exposure Comparison – allows us to compare a current level of loss exposure against a proposed future level assuming the application of new controls. Forms the benefit component of a cost-benefit analysis
![Page 9: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/9.jpg)
#RSAC #RSAC
Example
9
► Risk Scenario Prioritization – allows us to compare the level of loss exposure from multiple scenarios, which improves our ability to prioritize effectively
![Page 10: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/10.jpg)
#RSAC #RSAC
Example
10
► Large Scale Scenario Prioritization – allows us to compare the level of loss exposure from many scenarios, which improves our ability to prioritize effectively
![Page 11: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/11.jpg)
#RSAC #RSAC
The probability of a loss event occurring and the probable magnitude of loss that results
Definition of “Risk”?
![Page 12: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/12.jpg)
#RSAC #RSAC
Take Aways
Developing metrics and applying models that are meaningful in the context of your organization
Breaking down metrics by category
Choosing frameworks and models
Delivering the right metrics for your audience, so they can make informed decisions about business risk management
Applying useful examples to help you quantify risk at your organization and present it concisely to your management Good metrics and practices Good Governance Risk Reduction
12
![Page 13: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/13.jpg)
#RSAC #RSAC
Appendix
![Page 14: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/14.jpg)
#RSAC #RSAC
My Fitness Pal
• I ask questions and make decisions about my health every day
What should I eat for breakfast?
How much? How often?
What kind of exercise should I do?
For what length of time? How often?
• I can change my behavior by setting goals and measuring progress
SMART goals
Specific, measurable, actionable, reasonable, time-based
![Page 15: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/15.jpg)
#RSAC #RSAC
Software Security Metrics
![Page 16: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/16.jpg)
#RSAC #RSAC
A Software Security Framework
Four domains
Twelve practices
See informIT article on BSIMM website http://bsimm.com
![Page 17: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/17.jpg)
#RSAC #RSAC
BSIMM Scorecard
![Page 18: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/18.jpg)
#RSAC #RSAC
Earth (67)
![Page 19: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/19.jpg)
#RSAC #RSAC
BSIMM as a Measuring Stick
• Compare a firm with peers using the high water mark view
• Compare business units
• Chart an SSI over time
![Page 20: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/20.jpg)
#RSAC #RSAC
BSIMM Scorecard with FAKE Firm Data Top 12 activities
purple = good?
red = bad?
“Blue shift”
practices to emphasize
![Page 21: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/21.jpg)
#RSAC #RSAC
Each of us is a Special Snowflake (NOT)
ISV (25) results are similar to financial services (26)
![Page 22: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/22.jpg)
#RSAC #RSAC
BSIMM Longitudinal: Improvement over Time
21 firms measured twice (an average of 24 months apart)
Show how firms improve
An average of 16% activity increase
![Page 23: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/23.jpg)
#RSAC #RSAC
BSIMM by the Numbers
![Page 24: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/24.jpg)
#RSAC #RSAC
The Predictive Security Model
![Page 25: Visualize This! Meaningful Metrics for - RSA Conference · PDF fileVisualize This! Meaningful Metrics for ... Real Life Metrics • What metrics are you using to answer ... which improves](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5aa45d4e7f8b9afa758bd158/html5/thumbnails/25.jpg)
#RSAC #RSAC
Top Ten Risks