How secure is your software? A whitepaper by the Software Improvement Group

October 20, 2013

© Software Improvement Group

2

Software Improvement Group whitepaper

Abstract This whitepaper describes the growing IT security problem and how the improvement of software quality is essential in solving it. Security incidents are increasingly frequent, increasingly costly and increasingly difficult to prevent. Typically, security is addressed by approaching software as a black box. However, security risks often find their origin in the internal structure of software systems. As a result, it is essential to look at the inside of applications: design, specifications, source code and configuration. A model and a method to find vulnerabilities in software and rate security is presented.

1 The software security problem

IT security incidents are increasingly frequent, increasingly costly and increasingly difficult to prevent. The probability of security issues is growing because IT is increasingly complex, open and interconnected in terms of components and services. Growing IT complexity makes it hard to structurally locate and address security concerns. Organizational IT is directly connected to the Internet and often provides many points of access. The same principle of interconnection holds true for the internal application landscape. Breaching a single weak system therefore often allows for access to many others. In addition, attacks receive broad coverage by the media, motivating perpetrators and embarrassing organizations. Compounding the problem of prevention, attacks are carried out increasingly professionally as cyber crime is getting better organized. Additionally, the impact of security issues is increasing. Our growing reliance on IT means that IT becomes more valuable. Moreover, every day we store more sensitive, private data. A recent study by Forrester [1] found that half of the 240 participating organizations in Europe and North America experienced at least one web application security incident. A fifth reported that such breaches cost their organizations at least $500,000. In addition, the reported impact of software security breaches include damage to professional reputation, revenue loss, loss of customer confidence, damage to brand image and customer data loss. Software applications currently create the biggest digital security risk. Gartner and CERT report that 75% of security incidents are caused by software mistakes as opposed to insufficient technical infrastructure. As a result, security cannot be assessed well without looking at the internal structure of software applications: design, specifications, source code and configuration. Furthermore it is important to see what the organization has done to prevent incidents or respond to them correctly.

2 Existing approaches to IT security Rapid increases of attacks and reports of security incidents leave organizations worried about how secure their systems are. Security control is hard to attain both in terms of processes and products. Figure 1 shows an overview of the variety of existing approaches to security.

Figure 1 - Synthesis of cyber security assessment and risk mitigation approaches

product

operationsdevelopment

process

processmodels

managementsystems

modeling &measurement

testing &monitoring

intrusiondetection

CVSS

ethicalhacking

ISO 27001

BSIMMOpenSAMM

MS-SDL

CommonCriteria

ISO 25010

OWASPASVS

CWE/SANSTop-25

penetrationtesting

SAS 70

NEN 7510

© Software Improvement Group

3

Software Improvement Group whitepaper

The dimensions used in the circle in this image are the software life cycle (from development to operations) and a distinction between the software product and the processes governing its life cycle. The quadrants are complementary in the sense that organizations should address IT security by using approaches covering all sections. Security assessments typically encompass penetration tests, vulnerability scans, and organization audits. Evaluation of security risks in source code and software architecture (code/design review), a decidedly more technical approach, is less common. Well-known models are the Common Criteria [2] and the Open Web Application Security Project (OWASP) Application Security Verification Standard [3]. Application of the former model typically involves a major effort over a protracted time period, while the latter is limited in scope to web applications.

2.1 How does code/design review differ from penetration testing? A common way of assessing security is black box penetration testing: an organized attack on an IT system. The goal of such an attack is to discover vulnerabilities by using or abusing the system – not by looking at how it is constructed. While this is useful for finding vulnerabilities, it has three key limitations that can be complemented by performing a code/design review:

1. A penetration test does trial and error on a selection of known attacks on a selection of system elements. All known attacks cannot be tried because the test is bound by time. In contrast, by looking at the inside of the system through code/design review, vulnerabilities can become logically apparent without trial and error.

2. A penetration test is limited by what can be seen from the outside; the perimeter. By testing a system one cannot see how well and how generic all internal parts of the system are built to protect against (future) threats and data leaks. Is data stored securely in the database? Are backups treated with care? Is sensitive data accidentally logged on the server? What has been done to maximize resilience, to detect, reconstruct, and recover from attacks? In contrast, these aspects can be analysed through code/design review.

3. A penetration test can only be applied to a system or component that is finished and therefore its

benefit is limited during development. Adding or fixing security when software is already finished is expensive. Some studies report it to be 100 times more expensive to add security after deployment compared to addressing it in the design stage [7]. In contrast, a code/design review is possible from the design stage of the software development life cycle.

3 Addressing security at the source

The ISO/IEC 25010 standard for software product quality [4] offers a powerful framework for analyzing software quality aspects including security, as depicted in Figure 2.

Figure 2 - ISO/IEC 25010 quality characteristics for software product quality. Specifically, security is defined as: “The degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization.”

Software Quality

ISO 25010

Functional Suitability

Performance Efficiency

Compatibility Usability Reliability

Security

Maintainability

Portability

© Software Improvement Group

4

Software Improvement Group whitepaper

3.1 The SIG security model ISO has not yet provided concrete models and measurements to evaluate software product security. The Software Improvement Group (SIG) has therefore developed a security model [5] based on ISO 25010 [6] to make it possible to rate security based on how a system is constructed: design, specifications, source code and configuration – to complement existing approaches that address aspects like network infrastructure and penetration tests. An overview of the SIG security model is depicted in Figure 3. Figure 3 - Mapping of 11 system properties to ISO/IEC 25010 security sub-characteristics. The five security sub-characteristics are:

• Confidentiality: Ensures that data are accessible only to those authorized; • Integrity: Prevents unauthorized access or modifications; • Non-repudiation: Actions or events can be proven to have taken place; • Accountability: Actions of an entity can be traced uniquely to the entity; • Authenticity: The identity of a subject or resource can be proven to be the one claimed.

The SIG security model is used to perform a code/design review to measure how well best practices have been applied in 11 system properties:

• Data transport; • Data storage; • Authorized data access; • Authorization; • Input and output verification; • Strength of proof; • Logging completeness; • Unique identification; • Access management strength; • Session management strength; • User management.

The model provides tooling and a method to dissect a system into elements that are relevant for security and each element is analyzed to assess the relevant system properties. This analysis is performed by a SIG specialist using dedicated tooling to look at specifications, design, source code and configuration to see how well best practices have been implemented and weaknesses avoided, no matter the technology. These are the best practices and weaknesses as they are collected and documented by organizations such as OWASP and CWE/SANS. By rating the quality of several aspects of the system properties, each property is scored on a scale between one and five stars, which then is mapped to a score for each sub-characteristic and then into a final system score.

3.2 From the inside out – seeing everything A code/design review can be compared with testing the security of a bank by studying the construction blueprints and assessing the building from the inside: How solid is the construction? Are there special passages? Is every type of intrusion detected? The advantage of this way of looking is that one can see how well or how poorly something has been built and pinpoint the causes of weaknesses. This makes it much easier to spot vulnerabilities, as opposed to trying many potential vulnerabilities, like a penetration test

Secure data transport

Secure data storage

Authorised data access

Secure authorisation

In/output verification

Strenght of proof

Logging completeness

Unique identification

Access management strength

Session management strength

Secure user management

- Confidentiality - Integrity X X X X X - Confidentiality - Integrity X X X X X

- Non-repudiation - Accountability X X X

X - Authenticity X X

© Software Improvement Group

5

Software Improvement Group whitepaper

does. In that sense the outside approach can be compared to attempting to open windows and doors to get inside. Both approaches are complementary. A penetration test assesses whether a system is able to withstand a set amount of known attacks but cannot establish if this is the case for unknown, future threats. A view inside a bank building can reveal that the vault door is unlocked or that a passage exists to an adjacent building. Not being able to enter a bank building today does not imply that the money is protected from future attacks. In similar fashion, a code review has predictive abilities by eliciting vulnerabilities that form a threat now and/or in the future.

3.3 Related tooling

A code review according to the SIG security model can make use of static code analysis tools, but needs more than what those tools can offer to determine how the software has been implemented. Therefore, SIG recommends the use of static code analysis tools to help developers find vulnerabilities, being aware that the scope of these tools is limited and does not provide understanding of specific security mechanisms, build quality, system configuration and security aspects on the architecture level.

4 Software security in context As discussed in this whitepaper, in order to determine security risks, code/design reviews and penetration testing are important, complementary methods. For a more complete approach, however, it is necessary to look further and incorporate the build quality of the system, as well as the context: organization and processes involved.

4.1 Importance of build quality A good code review not only locates inadequate protection against attacks but also situations that encourage software defects. Defects can cause security vulnerabilities or data leaks, for example an user that sees the personal data of another user because of a bug in the way sessions are handled. An important aspect of security therefore is the extent in which software is maintainable: How easy is it to make alterations to software without introducing new issues? The proven SIG ISO 20510 software maintainability model [8] is a useful method to analyze this risk. The available processes and tools for functional testing of the system are an important factor to further mitigate maintainability risks.

4.2 Importance of process and organization As discussed in section 2, process and organization play an important role in IT security, although good processes are no guarantee for secure software. Are the people involved aware of security risks? Have developers been trained at building security in and are they aware of the organization risks? Are system logs analyzed to detect attacks?

4.3 SIG Security Risk Assessment SIG offers a Security Risk Assessment (SecRA) service that is targeted at technology, rooted in science and based on international standards. The SecRA looks for vulnerabilities and security risks by investigating from the inside: How systems are constructed from the perspective of technology, architecture and source code and how the involved organization works. The objective of a SecRA is to identify the most salient security risks and to give advice on effective approaches to mitigate these. Typically, one out of ten SIG findings could have been found by a penetration test. Likewise, one out of ten findings is directly based on static code assessment. The rest of the findings originate from implementation details, architecture, build quality and configuration. The SecRA approach encompasses the SIG security model, the SIG maintainability model and a method to assess the role of processes and organization, based on what is available in ISO27001, BSIMM and MS-SDL. See Figure 3 for a diagram of the SecRA process, involving several sessions in order to get a clear picture of strategy, organization, process and technology.

© Software Improvement Group

6

Software Improvement Group whitepaper

Figure 3 - Software Improvement Group “Security Risk Analysis” process outline. SIG has a proven track record in software quality and assessment of code in which technical findings are linked to processes. SIG’s distinguishing capacity lies in the unique combination of technical analysis, understanding of the software life cycle and advisory capabilities at the C-level. In addition, SIG offers a more thorough analysis than is provided by a simple scan based on the OWASP top-10 but a more pragmatic and lightweight analysis than a Common Criteria assessment, which is done only in rare cases. The advisory services and the underlying rating model enable a client to invest in security in an efficient manner, in order to:

• Avoid financial loss; • Prevent reputational damage; • Stop information leakages; • Minimize downtime; • Minimize, and trace back, the impact of security issues when they do happen; • Limit security development cost.

In addition, the SecRA expresses the vulnerability level with a rating between one and five stars. This objective and repeatable star rating scores the security relative to an industry benchmark. The result of a SecRA is security control: A clear understanding of the security risks and a roadmap with recommendations to mitigate them. SIG is independent and does not implement security solutions itself.

5 Conclusion Since the growing number of security incidents are for a large part caused by software problems it is essential to 1) test the software through penetration testing and 2) analyze software quality to determine how well security aspects have been implemented in the relevant parts of an application. Apart from these security aspects, the general build quality of the software also needs to be controlled, as that determines the risk of new vulnerabilities or data leaks to appear. By looking at the design, architecture, source code and configuration of software systems, the causes for current and future weaknesses can be spotted directly. In this way security is visible and controllable, from the early stages of development, taking into account not only known attacks, but also new threats, and taking into account the many security aspects that cannot be seen from the outside. SIG offers a security risk assessment service to measure software quality and to look at relevant processes in order to find out what security risks exist and recommend how to improve security by addressing the causes of those risks.

Analysis in SIG laboratoryReview of processes and architecture

FinalpresentationKick off

Strategy session

Technicalsession

Technicalvalidation

session

Riskvalidation

sessionFinal

report

© Software Improvement Group

7

Software Improvement Group whitepaper

About the Software Improvement Group (SIG) SIG is a management consultancy that focuses on software-related challenges. We provide management with fact-based insight into their current IT situation, along with razor-sharp, pragmatic and highly actionable recommendations on how to improve on that situation. We know how to govern software projects effectively, when to invest in quality improvement, how to rationalize, and how to control cost. By consolidating low-level technical analysis and high-level financial analysis we make sure our recommendations are the best fit for your specific business needs. References [1] Forrester. The software security risk report: The road to application security begins in development.

Technical report, Forrester Consulting (Commissioned By Coverity), 2012. [2] Nine governmental organisations. Common methodology for information technology security

evaluation (evaluation methodology) version 3.1 revision 4 (CCMB-2012-09-004). Technical report, Nine governmental organisations, September 2012.

[3] The OWASP Foundation. Open web application security project verification standard. Technical report, The OWASP Foundation, 2009.

[4] ISO. ISO/IEC25010:2011 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models. Technical report, International Organization for Standardization, 2011.

[5] H. Xu, J. Heijmans, and J. Visser. A practical model for ratings software security. In Proceedings of the 7th International Conference on Software Security and Reliability, 2013.

[6] ISO. ISO/IEC27010:2012 Information technology — Security techniques — Information security management for inter-sector and inter-organisational communications. Technical report, International Organization for Standardization, April 2012.

[7] B. Boehm and V. R. Basili. Software defect reduction top 10 list. IEEE Computer, 34:135–137, 2001. [8] I. Heitlager, T. Kuipers, and J. Visser, A Practical Model for Measuring Maintainability, In proceedings of

the 6th International Conference on the Quality of Information and Communications Technology (QUATIC 2007), pages 30-39, IEEE Computer Society Press, 2007.

Contact Rob van der Veer Security Practice Leader Email: r.vanderveer@sig.eu Phone: +31 6 20 43 71 87

www.sig.eu