© 2007 mcafee, inc. vulnerability in the real world lessons from both sides of the fence ryan...
TRANSCRIPT
© 2007 McAfee, Inc.
Vulnerability in the Real WorldLessons from both sides of the fence
Ryan PermehManager of Product SecurityMcAfee Security Architecture [email protected]
Who I amWhy should I listen to this guy?
• Several years on the Research side
• Years as a reverse engineer and exploit developer
• Front lines experience as a vendor and developer
• Currently manage a large security vendor’s product security.
• 29 shipping projects with millions of lines of code
Bugs, Bugs, and more BugsAn accelerating trend
• Rates of found vulnerabilities increasing
• Severity of bugs increasing• Why?
— Research isn’t a black art— Attacker tools improving— More software— Do reasons matter that much?
Graph from IBM/ISS X-force Threat analysis 2007
Examining the PlayersWhat motivates Researchers?
• Common Good (For the Internet)• Fame• Money• Demonstrate Technical Excellence• Responsibility• Very Bad Things
Examining the PlayersWhat motivates Vendors?
• Common Good (for our customers)• PR and Brand Image • Money• Demonstrate Technical Excellence• Responsibility
Today’s RealityIt’s not your Daddy’s Vulnerability Market
• An emerging economic market for bugs• Fame is less likely• Vendors that make things hard for researchers may find
that researchers prefer the market for their bugs• Even efficient vendors may not get reports first• Vendors need to “fill the gap”
Lack of reports != Lack of bugs
Researcher RelationsSeek the common ground
• Money from vendors is not likely— Extortion?
• Fame from vendors is more likely— Credit in advisories— Acknowledgment in any pr items
• Hire researchers with good track records— Many large software companies hire researchers
• Avoid antagonism on either side— Disclosure policies and communication
Vendors Thinking Like Researchers
Design Coding Internal
Testing
Beta
Testing
Post
release
Growing a Secure OrganizationWhy secure software is good business
• Cost of post release vulnerability is oppressive
• Implementation in the SDLC is paid for itself by only a few serious bugs
• Damage to brand, to customers, and lawsuits
An ounce of prevention is worth
a pound of cure.
Benjamin Franklin
~35x more expensive to fix a bug
post release than in design
5x
15x
25x
35x
Selling Security in Your OrganizationDemonstrate a need, then fill the Need
Building Security into your process is a long process• Starts Slow
— Keep scope realistic for your resources
• Demonstrate Risk— Initial reports— Focus on growth, not fear
• Increase Scope• [Daily,Weekly,Monthly,Quarterly] wins add up• Keep it Consistent and meet commitments
Calculating ROI for Security There ‘Aint No Such Thing as a Free Lunch
• Every activity has a cost— Human Capital— Technical Capital— Cost of Opportunity
• Focus on high ROI activities• Plan for the short term and the long term
— Some activities have short term ROI, others take longer
• To justify increases, you must have the data• Clear improvements justify costs and allow for additional
resources
Tracking Security MetricsWhat to measure and why
• Automated• Repeatable• Track over Time
• Security Report Card
— Number of external security bugs— Number of internal security bugs— Bugs per KLOC— Fuzzing Coverage %— Automated tools coverage %— Input Validation Coverage %— Cost per Sev 1 (internal and
external)— Cost per Developer Trained— Cost per Penetration test— Cost per …
How do researchers find bugs?Code Audits
Hand Auditing
• Relatively low initial cost• Relatively high ongoing costs• Poor scaling• Variable output levels• Deep analysis possible• Relatively poor ROI• Dependant on quality of
auditors
Static Code Analysis
• Relatively high initial cost• Relatively low ongoing costs• Good scaling• Consistent output levels• Deep analysis usually not
possible• Reasonable ROI• Dependant on quality of
tool
How do researchers find bugs?Reverse Engineering
• Useful bug finding skill• Usually not necessary if you have code• Good for analysis of 3rd party libraries (check the license)• Can find very deep bugs• Specialized, expensive skill to keep on staff• Audits can take longer• Tools are getting better, but still require skills• Pretty low ROI
How do researchers find bugs?Fuzzing
• Large scale fault injection• Use a public framework or write your own• Great for covering
— File formats— Network servers— Web input testing
• Find bugs while you sleep• Not very precise• Variable initial cost• Low ongoing cost• Good ROI
How do researchers find bugs?Threat Models
• Structural /Architectural Analysis with threat enumeration• Pioneered by Microsoft• Discovers architectural flaws and missing pieces• Can be very formal or less so• Scope can be focused or “big picture”• Utilize your architects, developers and QA• Very low initital and ongoing costs• Very good ROI with quick realization
— Some items found may require large effort to fix
How Vendors Fix and Reduce BugsPenetration Testing / Blackbox
• Use standard security testing practice• Can be easily integrated into QA cycle• Requires some specialized knowledge• Relatively mature tools• Dependant on quality of testers• Good candidate for outsourcing• Variable initial costs• Relatively low ongoing costs (unless outsourced)• Fair ROI with good short term gains• Pairs very nicely with a threat model
How Vendors Fix and Reduce BugsTraining
• Training developers in Secure coding techniques• Training QA engineers in security testing techniques• Requires time to train• Requires repetition to lock in skills• Can be sped up with intensives
— Microsoft’s “stop and train”
• Ultimately it does good in reducing bugs, but its slow• ROI is pretty poor, it takes a long time to recoup
— Requires trainers and training materials— Employee turnover affects costs— Screening new employees for security experience helps
How Vendors Fix and Reduce BugsIntegration into the SDLC
• Strong integration of security at all points of the SDLC• Security Requirements• Threat modeling and Secure Design Principles• Secure Coding Guidelines and Automated analysis• Security Test plans, fuzzing, penetration testing• Vulnerability Management• The end goal, an accumulation of all else• ROI dependant on implementation, gains are cumulative
and long term
Helping Researchers to Understand Vendors
Disclosure PoliciesFull Disclosure
• Instant information• Helps attackers and bleeding edge defenders• Theoretically trades instant pain for a potentially quicker
remediation cycle• Can work with vendors or can be 0day• Gives all involved the chance to understand the attack on
a deeper level, allowing them to better understand their risks
Disclosure PoliciesNon Disclosure
• No public information, ever• May be held by vendor or researcher (or bad guy)• Silent fixes (if any), no notification• Subject to patch diffing
• Anti-Sec• Legal battles• Heavy regulation
Disclosure PoliciesThe Middle Ground
“Responsible” Disclosure• Covers all ground between full and non disclosure• Not incompatible with full or non disclosure• Dependant on defined policy• Researcher waits until patch
— May release full technical details post patch— May not release anything
• Middle ground that people can work with
Disclosure PoliciesSome Examples
OIS – Organization for Internet Safety— Coalition of Security and Software Companies— Complete end to end disclosure policy— Fully documented— McAfee and many other vendors follow this model
RFPolicy— Full disclosure policy
IETF— Steve Christey and Chris Wysopal— Draft, not RFC
NIAC – DHS— Large scale study
Disclosure ResponseWhat to do when a bug is found
• Have a coordinated security policy in place• Make it easy for researchers to contact you
— [email protected]— Consider using PGP or GPG for communications— Have a public web site with contact details and PGP public key— Consider posting your policy
• Paypal
• Respond as soon as possible— Acknowledge receipt of issue immediately— Determine Risk— Treat Security bugs as important
Communication with Researchers
• Communication with researchers is important— Recommend at least weekly updates— Updates should include any status or movement in the fix
• Keep honesty and integrity as priorities• Use this Coordinate release schedule
Always Remember, researchers are helping you, even if it’s for their own reasons. Communication with researchers keeps them happier and less likely to engage in hostilities.
Releasing DetailsCoordination and Release Process
• Credit helpful researchers• Try hard not to slip on schedules• Coordinate release times
— Realize time zones may cause difficulties
• Be prepared for questions— Customers— Internal resources— Media
Baby StepsNext steps and beyond
Building BridgesEngaging the Research Community
• Keep building bridges• Quality external researchers are good for the software
development process• Continue to learn from the research community
— Security Conferences— Trainings— Microsoft’s Bluehat
• Stay vigilant of trends and changes
