Is finding security holes a good idea?

Presented By: Jeff WheelerCSC 682

Outline

• Introduction• Vulnerability Lifecycle• Cost of Disclosure• Finding rate to pr

• Rate of Vulnerability Discovery• Sources of Error

Introduction

• Assertions1. It is better for vulnerabilities to be found by

good guys than bad guys.2. Vulnerability finding increases total software

quality

The life cycle of a vulnerability

• Introduction – the vulnerability is first released as part of the software.

• Discovery – the vulnerability is found.• Private Exploitation – the vulnerability is

exploited by the discoverer or a small group known to him or her.

• Disclosure – a description of the vulnerability is published.

The life cycle of a vulnerability

• Public Exploitation – the vulnerability is exploited by the general community of black hats.

• Fix Release – a patch or upgrade is released

The life cycle of a vulnerability

• These events do not occur strictly in this order.– Ex: software manufacture releases disclosure

and fix

White Hat Discovery

• Discovery, Fix, and Disclosure: Best Case– The vulnerability is discovered by a

researcher with no interest in exploiting it.– The researcher notifies the vendor– The vendor releases an advisory and a fix– Public exploitation begins at time of disclosure

White Hat Discovery

Black Hat Discovery

• Discovery, Fix, and Disclosure: Worst Case– The vulnerability is first discovered by

someone with an interest in exploiting it.– Black hat community exploitation– Knowledgeable person identifies exploit being

used against a system and notifies vendor – The vendor releases an advisory and a fix– Public exploitation begins at time of disclosure

Black Hat Discovery

WHD versus BHD

• WHD eliminates period of Private Exploitation

• CBHD – CWHD = Cpriv

• Are administrators more likely to patch if they know a vulnerability is being actively exploited?– Total number of vulnerable systems will

decline more quickly, minimizing peak exploitation rate

Cost-Benefit Analysis of Disclosure

• Best Case– White hat discovery, never rediscovered or

exploited• Worst Case

– Black hat discovery• Cpriv + Cpub

Cost-Benefit Analysis of Disclosure

From finding rate to pr

• Assumption: Vulnerability discovery is a stochastic process.– Overall rate of vulnerability discovery in a

particular application is a good estimate for pr

– Pr upper bound current percent discovery

Determining the Vulnerability Discovery Rate

• Assumption: Software undergoes multiple releases– If we assume patches/releases do not

introduce new bugs, only fixes, we can assume overall software quality increases with time

• How does one determine this rate?

Determining the Vulnerability Discovery Rate

• ICAT vulnerability metabase– A searchable index of computer

vulnerabilities.– Entire database available for public download

and analysis• Relevant Information

– Rate of discovery over time, Program and version effected

• Data Cleansing

Sources of Error• Unknown Versions• Bad Version Assignment• Announcement Lag• Severity of Vulnerabilities• Operating System Effects

– Packages included with OS, use OS release date instead of package release date

• Effort Variability• Different Vulnerability Classes• Data Errors

Is it worth disclosing vulnerabilities?

• If there is no depletion of vulnerabilities, then disclosing vulnerabilities is always harmful. This implies there is an infinite number of vulnerabilities and pr approaches zero.

• If we assume the pool of vulnerabilities is depleting, and all vulnerabilities will eventually be discovered, pr=1, and disclosing vulnerabilities makes sense.

Conclusions

• This research does not provide sufficient evidence that vulnerability finding and disclosure provides in increase in software security sufficient to offset the effort being invested.

• This research does not provide sufficient evidence that vulnerability finding and disclosure is a bad idea.

Conclusions

• Prefer continuous white hat discovery with no disclosure until exploitation by black hat?

• How do we estimate the number of vulnerabilities in an application, both discovered and undiscovered?