Bug HuntingBug HuntingSimon PolkowskeSimon Polkowske

Senior SeminarSenior Seminar

2/26/072/26/07

Security BugsSecurity Bugs

No one likes themNo one likes them

Their cost is calculated in the damage that Their cost is calculated in the damage that is doneis done

Its big news, not only exploiting bugs but Its big news, not only exploiting bugs but just telling people they exist.just telling people they exist.

Has spawned “Responsible Disclosure”Has spawned “Responsible Disclosure”

Responsible Disclosure Responsible Disclosure

Companies have asked bug hunters to Companies have asked bug hunters to wait a reasonable amount of time before wait a reasonable amount of time before publishing bugspublishing bugs

Releasing bugs before a patch is out can Releasing bugs before a patch is out can put users at riskput users at risk

OR gives companies more time to waste OR gives companies more time to waste while problem is not fixed says bug while problem is not fixed says bug huntershunters

Open VS ProprietaryOpen VS ProprietaryFIGHT! FIGHT!

Often debated over which leads to better Often debated over which leads to better quality of code.quality of code.

Open source thought of as better.Open source thought of as better.

Study done by Coverity showedStudy done by Coverity showed On average Open had less bugs per LOCOn average Open had less bugs per LOC Top of the line Proprietary was more than 5 Top of the line Proprietary was more than 5

times better than best Open sourcetimes better than best Open source

Proprietary CheatsProprietary Cheats

Open source enthusiast say this was not a Open source enthusiast say this was not a fair test.fair test.

They argue that comparing mission critical They argue that comparing mission critical software to software that is not critical is software to software that is not critical is bad comparisonbad comparison

Comparing like software is better Comparing like software is better comparisoncomparison Ex: MS Internet Explorer vs Mozilla FirefoxEx: MS Internet Explorer vs Mozilla Firefox

Bug Hunter UnrealityBug Hunter Unreality

““Only very bright, knowledgeable people can find Only very bright, knowledgeable people can find security bugs.”security bugs.”Helps to have experience, time is more Helps to have experience, time is more important.important.““There is always a shady motivation behind the There is always a shady motivation behind the search for security bugs.”search for security bugs.”As many reasons as there are peopleAs many reasons as there are people““Person that finds a security bug knows Person that finds a security bug knows everything there is to know about it”everything there is to know about it”The full scale of security bugs is often missed.The full scale of security bugs is often missed.

Getting Your BugGetting Your Bug

Some things to keep in mindSome things to keep in mind How many people you haveHow many people you have What their experience and technical skills areWhat their experience and technical skills are How much time you haveHow much time you have What kind of tools at your disposalWhat kind of tools at your disposal

ApproachesApproaches

Depending on the resources there are a Depending on the resources there are a few different bug finding approachesfew different bug finding approaches Lone Ranger ModeLone Ranger Mode Time-Constrained Peer AuditTime-Constrained Peer Audit Assembly-Line TeamworkAssembly-Line Teamwork TournamentTournament Rotating TeamsRotating Teams

Lone Ranger ModeLone Ranger Mode

One person or moreOne person or more

Their sole responsibility is to find bugsTheir sole responsibility is to find bugs

Best for long term searchingBest for long term searching

Time-Constrained Peer AuditTime-Constrained Peer Audit

Two to three peopleTwo to three people

Looking where bugs are suspectedLooking where bugs are suspected

Similar to Extreme Programming Similar to Extreme Programming techniquestechniques

Good for short time periods and when the Good for short time periods and when the group has similar skills and experiencegroup has similar skills and experience

Assembly-Line TeamworkAssembly-Line Teamwork

Two or more peopleTwo or more people

Good for groups with varying skills and Good for groups with varying skills and time is shorttime is short

Puts people on one specific area that they Puts people on one specific area that they are good atare good at

TournamentTournament

Multiple GroupsMultiple GroupsGive the same thing to check to each Give the same thing to check to each group, first to find bugs (or the most bugs) group, first to find bugs (or the most bugs) winswinsGood way to find numerous bugs in a Good way to find numerous bugs in a short periodshort periodCan also rules in tournament that help Can also rules in tournament that help establish a processestablish a processFun way to keep moral upFun way to keep moral up

Rotating TeamsRotating Teams

Several TeamsSeveral Teams

Cycling the teams into the bug hunting hatCycling the teams into the bug hunting hat

Ongoing processOngoing process

Hard for bugs to hide from all those eyesHard for bugs to hide from all those eyes

Doesn’t let people get bored doing just Doesn’t let people get bored doing just one thingone thing

Criteria of Bug HuntingCriteria of Bug Hunting

A clear definition of the target technologyA clear definition of the target technology Know exactly what the software will be onKnow exactly what the software will be on

Example: a default installation of Windows XP ProExample: a default installation of Windows XP Pro

Process DocumentationProcess Documentation Everyone must understand what is being Everyone must understand what is being

donedone Important if there are people at different Important if there are people at different

amounts of expertise or even different amounts of expertise or even different locationslocations

Criteria of Bug Hunting Criteria of Bug Hunting (cont.)(cont.)

Results DocumentationResults Documentation Helps people coming in after youHelps people coming in after you

DiversificationDiversification Use a mixture of techniques and a team of Use a mixture of techniques and a team of

people to hunt bugspeople to hunt bugs

TechniquesTechniques

Source Code Audit Source Code Audit Reverse Engineering: Debug & Reverse Engineering: Debug & DisassemblyDisassemblyReverse Engineering: Network TrafficReverse Engineering: Network TrafficBlack Box TestingBlack Box TestingBrute ForceBrute ForceTop-Down AnalysisTop-Down AnalysisInformation GatheringInformation Gathering

Source Code AuditSource Code Audit

RTFS – “Read the fine source”RTFS – “Read the fine source”

Reading the source code looking for bugs Reading the source code looking for bugs and poor programming and poor programming

Requires little knowledge of the systemRequires little knowledge of the system

Good to have experience in bug huntingGood to have experience in bug hunting

Also good to have the development team Also good to have the development team on handon hand

Reverse Engineering: Debug & Reverse Engineering: Debug & DisassemblyDisassembly

DebuggingDebugging Team actively monitors the execution and Team actively monitors the execution and

inputs for bugsinputs for bugs Source code not necessarySource code not necessary

DisassemblyDisassembly Looks at the source code in assemblyLooks at the source code in assembly The tool must know the underling technologyThe tool must know the underling technology

Only good on small and simple softwareOnly good on small and simple software

Reverse Engineering: Network Reverse Engineering: Network TrafficTraffic

Needs to have network componentsNeeds to have network components

Good when the project is large and Good when the project is large and complexcomplex

Requires a tool to capture packetsRequires a tool to capture packets

Try to spot possible flaws Try to spot possible flaws

Usually requires a lot of experienceUsually requires a lot of experience

Black Box TestingBlack Box Testing

Manipulating the environment and inputs Manipulating the environment and inputs to produce bugsto produce bugs

Looking for usual bugs in type of softwareLooking for usual bugs in type of software

Requires more experience than technical Requires more experience than technical skillsskills

Doesn’t have knowledge of how it works or Doesn’t have knowledge of how it works or access to source codeaccess to source code

Brute ForceBrute Force

Black Box approach using tools.Black Box approach using tools.

Tries to test every possible inputTries to test every possible input

Requires little to no expertiseRequires little to no expertise

Needs good toolsNeeds good tools

Top-Down AnalysisTop-Down Analysis

A review of the high-level area and will A review of the high-level area and will look deeper if something is suspectlook deeper if something is suspect

Quick to yield resultsQuick to yield results

Can lead to many dead endsCan lead to many dead ends

Information GatheringInformation Gathering

““Quick and Lazy”Quick and Lazy”

Searching for information on bugsSearching for information on bugs

Helps to find bugs to verifyHelps to find bugs to verify

It WorksIt Works

October 2000, MySQL authentication bugOctober 2000, MySQL authentication bug Used a time-constrained source code audit Used a time-constrained source code audit

and assembly-line teamworkand assembly-line teamwork Problem was in their cryptographyProblem was in their cryptography

June 1998, SSH CRC-32 insertion attackJune 1998, SSH CRC-32 insertion attack Top-down overview and peer auditTop-down overview and peer audit Weak integrity checks allow for multiple attack Weak integrity checks allow for multiple attack

attemptsattempts

ToolsTools

Help fix programming styles and identify where bugs could occur Help fix programming styles and identify where bugs could occur for C: for C:

Lint (Sun Microsystems)Lint (Sun Microsystems) LCLintLCLint/Splint (University of Virginia)/Splint (University of Virginia)

for C++: for C++: PC-Lint (PC-Lint (GimpelGimpel Software) Software) CodeWizard/C++TestCodeWizard/C++Test ( (ParasoftParasoft))

for C#: for C#: FxCopFxCop (Microsoft) (Microsoft) ClockSharpClockSharp (TIOBE) (TIOBE)

for Java: for Java: CheckStyleCheckStyle ( (SourceForgeSourceForge project) project) JCSC (JCSC (SourceForgeSourceForge project) project) JTest/CodeWizardJTest/CodeWizard ( (ParasoftParasoft))

LintLint

A library that you can include in programA library that you can include in program

It will doIt will do Analysis structure and flow of the source Analysis structure and flow of the source

program program Analysis of control flow and data flow Analysis of control flow and data flow Analysis of data types usage Analysis of data types usage Constant propagations and constant Constant propagations and constant

expression evaluations expression evaluations

Squashed BugSquashed Bug

No one bug hunting approach or technique No one bug hunting approach or technique is bestis best

Mixing different approaches, techniques, Mixing different approaches, techniques, and tools will better the chances of finding and tools will better the chances of finding more bugsmore bugs

Work CitedWork CitedArce, Ivan. Arce, Ivan. Bug Hunting: the Seven Ways of the Security SamuraiBug Hunting: the Seven Ways of the Security Samurai. Core Security . Core Security Technologies, 2002. 1-5.Technologies, 2002. 1-5.

Chelf, Ben. "Insecurity in Open Source." Chelf, Ben. "Insecurity in Open Source." ViewpointViewpoint (2006). 25 Feb. 2007 (2006). 25 Feb. 2007 <http://www.businessweek.com/technology/content/oct2006/tc20061006_394140.htm<http://www.businessweek.com/technology/content/oct2006/tc20061006_394140.htm?campaign_id=bier_tco.g3a.rss1007>. ?campaign_id=bier_tco.g3a.rss1007>.

Cmot. "October 2006 Archives." Cmot. "October 2006 Archives." Raw MatterRaw Matter. 25 Feb. 2007 . 25 Feb. 2007 <http://fortytwo.ch/blog/archives/2006-10.html#e2006-10-06T09_04_13.txt>.<http://fortytwo.ch/blog/archives/2006-10.html#e2006-10-06T09_04_13.txt>.

Evers, Joris, and Marguerite Reardon. "Bug Hunters, Software Firms in Uneasy Evers, Joris, and Marguerite Reardon. "Bug Hunters, Software Firms in Uneasy Alliance." Alliance." CNET News.ComCNET News.Com (2005). 25 Feb. 2007 (2005). 25 Feb. 2007 <http://news.com.com/Bug+hunters,+software+firms+in+uneasy+alliance/2100-<http://news.com.com/Bug+hunters,+software+firms+in+uneasy+alliance/2100-1002_3-5846019.html>. 1002_3-5846019.html>.

Lemos, Robert. "Flaw Finders Go Their Own Way." Lemos, Robert. "Flaw Finders Go Their Own Way." CNET News.ComCNET News.Com (2005): 1-2. 25 (2005): 1-2. 25 Feb. 2007 <http://news.com.com/Flaw+finders+go+their+own+way/2100-1002_3-Feb. 2007 <http://news.com.com/Flaw+finders+go+their+own+way/2100-1002_3-5550430.html>. 5550430.html>.

Work CitedWork Cited

"Ling Source Code Checker." Sun Microsystems. 25 Feb. 2007 "Ling Source Code Checker." Sun Microsystems. 25 Feb. 2007 <http://docs.sun.com/source/806-3567/lint.html>. <http://docs.sun.com/source/806-3567/lint.html>.

"Proof-of-Concept Code Increases Risk to Computer Users." "Proof-of-Concept Code Increases Risk to Computer Users." Microsoft.ComMicrosoft.Com. Microsoft. 25 Feb. 2007 . Microsoft. 25 Feb. 2007 <http://www.microsoft.com/security/incident/im_info.mspx>. <http://www.microsoft.com/security/incident/im_info.mspx>.

Rasch, Mark. "'Responsible Disclosure' Draft Could Have Legal Rasch, Mark. "'Responsible Disclosure' Draft Could Have Legal Muscle." Muscle." SecurityFocusSecurityFocus (2002). 25 Feb. 2007 (2002). 25 Feb. 2007 <http://www.securityfocus.com/columnists/66>. <http://www.securityfocus.com/columnists/66>.

Robers, Paul. "Do Bug-Hunting Security Firms Put Users At Risk?" Robers, Paul. "Do Bug-Hunting Security Firms Put Users At Risk?" IDG News ServiceIDG News Service (2002). 25 Feb. 2007 (2002). 25 Feb. 2007 <http://www.pcworld.com/article/id,106517-page,1/article.html>.<http://www.pcworld.com/article/id,106517-page,1/article.html>.