BRINGING THE HACKER

MINDSET INTO

REQUIREMENTS AND

TESTING

Jason Petry

Eapen Thomas

Nationwide Insurance

AGENDA

How bad is our application security? Why are we so bad at securing our applications?

An example application: money transferring application

2 Views of Quality – Producer vs. Consumer

Why are security requirements so hard?

Tools to aid requirements and test analysts

Threat Models

Attack trees

Securing our applications is getting more challenging

Call to action

2

YOUR PERSONAL INFO IS PUBLIC

3Source: Symantec Internet Security Threat Report

A New Zero-Day Vulnerability Was Discovered on Average Each Week in 2015

Advanced attack groups continue to profit from previously undiscovered flaws in browsers and website plugins

Over Half a Billion Personal Records Were Stolen or Lost in 2015

Major Security Vulnerabilities in Three Quarters of Popular Websites Put Us All at Risk

Web administrators still struggle to stay current on patches

MOBILE VULNERABILITIES ARE OFF THE CHART

4Source: Symantec Internet Security Threat Report

Android users willingly downloaded over two billion malicious mobile applications last year

Source: http://www.itproportal.com/2016/02/26/smartphone-users-still-taking-cavalier-approach-mobile-security/

Nearly 25 percent of mobile apps contain at least one high-risk security flaw

An average mobile device connects to 160 different IP addresses daily

35 percent of mobile device communications are unencrypted

There is a 50 percent greater chance that games include a high-risk vulnerability than the average app.

SOME SCARY INDUSTRY NUMBERS

84% of cyber attacks are happening at the application layer

Source: Forbes / SAP (March 2015)

98% of applications scanned by Trustwave harbored one or more security vulnerabilities. Meanwhile, the median number of vulnerabilities was 20 - up from six the year prior.

Source: 2015 Trustwave Global Security Report

Exploiting many of these application vulnerabilities is “VERY EASY”

5

SECURITY VS FUNCTIONAL RQRMTS & TESTING

Security requirements and testing is different from functional requirements & testing.

In security testing the goal is to find out if the system can stand up to abusers. Negative tests are critical.

Security test scenarios may not be realistic from a common user standpoint . Especially when considering web applications, attackers may interact with the application in critically different ways compared to regular users.

Anticipating and planning for these scenarios is vital for security testing

Security requirements and testing requires an adversarial mindset, a "what if" mindset, i.e., the same one hackers use to break systems.

6

EXAMPLE REQUIREMENT STORY

Cyclone Transfers – A Pay-pal like service.

“As a logged in customer, I can transfer money, so that I move money from one of my accounts to another customer’s account.”

Acceptance Criteria:

The amount of money I transfer must be less than the amount of money available.

All amounts are in US Dollars.

Transfers may be for fractions of a penny.

7

DEMO

8

WHAT IS QUALITY?

What are the two views of Quality?

The producer view and the customer view.

The producer view of quality: a product is a quality product if it meets or conforms to the product requirements. This statement is usually shortened to: quality means meets requirements.

The customer view of quality: fit for use; the product or service meets the customer’s needs regardless of the requirements

9

THE PRODUCER VIEW OF QUALITY

Ok, so, what is the problem?

We don’t have much/any application security requirements (this is an industry wide problem)

To create good security requirements, the analyst should review (Organizational requirements, Privacy requirements, statutory requirements and industry requirements)

Organizational security policies and standards

Org privacy policies

Regulatory requirements (Sarbanes-Oxley, HIPAA etc.)

Other standards such as PCI DSS, ANSI-X9 for banks etc.

What is the solution? The requirements analysts have to be Creative

10

CUSTOMER VIEW OF QUALITY

Let’s now switch to the customer view of quality

Oops! We have a bigger problem!

To the customer, a product is a quality product if it meets the customer’s needs, regardless of whether the requirements were met.

We have to go beyond requirements (even if we have some security requirements).

For this, we definitely have to be creative, requires an adversarial mindset

We talk about some resources from OWASP and other organizations that can help

11

TESTERS HAVE TO BE CREATIVE

Test analysts should be creative in the absence of good security requirements:

Automated web application security testing tools can help

Explore & Discover – Exploratory testing comes handy

Use Common Sense & Experience - common knowledge that comes from experience

Discussions, Emails and Meeting Notes

Create and review the high level test scenarios with the business

12

OWASP TOP 10

When we talk about web application vulnerabilities, we have to talk about Open Web Application Security Project (OWASP) Top 10 vulnerabilities.

Please note: this is “a” list, not the exhaustive list of all possible web application vulnerabilities (there are hundreds of them)

13

CURE FOR MOST PROBLEMS: INPUT VALIDATION

Many vulnerabilities are exploited by injecting malicious commands/code through input forms

TYPE

Always check the data type of the input and make sure it matches the expected data type. For example, if there is an input box which accepts numeric data and the letter ‘O’ is typed instead of the number zero, it should not be accepted.

LENGTH

Always check that the data lies within the acceptable range of lengths for the values expected. For example, a zip code field will be either 5 or 10 (dashes included) digits in length. If nothing is entered, or if 11 or more digits are entered, it should not be accepted.

FORMAT

Always check that data is in a specified format. For example, dates should be in a specific format (such as MM/DD/YYYY). If it is not in the correct format, it should not be accepted.

RANGE

Always check that data lies within a specified range of values. For example, the month of a person’s date of birth should lie between 1 and 12. If it does not fall within that range, it should not be accepted.

14

THREAT MODELLING

“Threat modeling is about using models to find security problems.” – Adam Shostack, Threat Modeling: Designing for Security

“Coming up with a set of possible attacks you plan to protect against” – Electronic Frontier Foundation (https://ssd.eff.org/en/glossary/threat-model)

15

THREAT MODELLING EXAMPLE

May be many Data Flow

Diagrams (DFD’s) for one

application/process, at

varying levels of detail.

THREAT MODELLING

Can be done with varying levels of formality, and with different focuses; method adopted should be tailored to specific application need.

One Common Methodology is Microsoft’s STRIDE model.

Model: decompose the application as a data flow diagram (DFD) to drive the overall risk analysis process.

Identify: In the next step, threats to the modeled system are identified and enumerated

Mitigate: After threats have been identified, mitigations to those threats are selected

Validate: Implement tests to validate threat is mitigated.

Spoofing: Impersonating something or someone elseTampering: Modifying data or codeRepudiation: Claiming to have not performed an actionInformation Disclosure: Exposing information to someone not authorized to see itDenial of Service: Deny or degrade service to usersElevation of Privilege: Gain capabilities without proper authorization

Steal someone’s account

Trick someone into giving me

money

Enter a negative

number for a transfer I make

ATTACK TREES

Get someone else to give me

money

Fool the system into giving me

money

Get Funds Transferred to

me with no work.

18

NEGATIVE TEST EXAMPLES

#1 Test: The Single Quote: '

Helpful to make sure SQL injection attacks have been properly mitigated against

Also useful for ensuring anti-SQL injection protections properly deal with single quote characters in user data.

19

EXAMPLE SINGLE QUOTE TEST

Cyclone Transfers: Test Procedure

Login

Go to all users.

Use Search function to find abcdef

Use Search function to find O’Brian

Expected result: No users found same results for both

Actual Result: Error Message in second case.

20

NEGATIVE TEST CASE: HTML CONTENT

Another important test case: Allowing entry of HTML input, and properly displaying the result. (i.e., as text)

Cross-Site Scripting is the single most commonly encountered security issue in web applications.

21

EXAMPLE HTML CONTENT TEST

Cyclone Transfers: Test Procedure

Click on Sign In, then Sign Up.

Create new account, in Profile Statement section include following content: <script>alert(123)</script>

Log in a different user, go to all users and search for newly created user.

Expected Result: Profile Statement is displayed in search results as typed above.

Actual Result: Alert Box Created.

22

NEGATIVE TEST CASE: DIRECT OBJECT ACCESS

If URL’s to content are static, ensure that user’s cannot access other users’ content.

Unlike previous two cases, very difficult for multi-purpose scanner tools to detect and respond; requires knowledge of the application and data access rules.

23

EXAMPLE DIRECT OBJECT ACCESS

Cyclone Transfers: Test Procedures

Create New User, as in last Test

In New User, add account, and upload test PDF as “Bank Statement”.

Click on link to uploaded PDF to validate. Create Bookmark to PDF.

Log in a different account, use bookmark to go to PDF

Expected Result: Access should be denied.

Actual Result: PDF Displayed

24

THE RUGGED MANIFESTOHTTPS://WWW.RUGGEDSOFTWARE.ORG/

I am rugged and, more importantly, my code is rugged.I recognize that software has become a foundation of our modern world.

I recognize the awesome responsibility that comes with this foundational role.I recognize that my code will be used in ways I cannot anticipate, in ways it was

not designed, and for longer than it was ever intended.I recognize that my code will be attacked by talented and persistent adversaries

who threaten our physical, economic and national security.

I recognize these things – and I choose to be rugged.

I am rugged because I refuse to be a source of vulnerability or weakness.I am rugged because I assure my code will support its mission.

I am rugged because my code can face these challenges and persist in spite of them.

I am rugged, not because it is easy, but because it is necessary and I am up for the challenge.

25

THINGS ARE NOT GETTING ANY EASIER

Things are not getting any easier, they are getting really complicated, very fast Human beings are still the weakest link

Ballooning attack surface

The number of Mobile devices are growing, mobile apps are getting very functional/complicated

Internet of things is making application security more difficult with the amount of interconnected devices

26

CALL TO ACTION

Don’t be left behind, security requirements elicitation & testing skills are essential, not optional

A skill you must have to be competitive/marketable/just to survive

Get trained, get competent in software security requirements elicitation techniques & security testing techniques

Resources are lacking

Conferences that cater to analysts, have no tracks or have very few tracks on application security requirement/testing

27

RESOURCES

OWASPhttps://www.owasp.org

OWASP Testing Guide (200+ page PDF document is free to download)https://www.owasp.org/index.php/OWASP_Testing_Project

OWASP Application Security Verification Standard 3.0https://www.owasp.org/images/6/67/OWASPApplicationSecurityVerificationStandard3.0.pdf

OWASP Broken Web Application Projecthttps://www.owasp.org/index.php/OWASP_Broken_Web_Applications_Project

SANS SWAT checklisthttps://software-security.sans.org/resources/swat

Microsoft SDL Threat Modeling Toolhttps://www.microsoft.com/en-us/download/confirmation.aspx?id=49168

28

THANK YOU!If you would like to contact us:

Jason Petry (petryj2@nationwide.com)

Eapen Thomas (eapen@nationwide.com)

29