security guidelines for the usage of open source software

INOM EXAMENSARBETE TEKNIK,GRUNDNIVÅ, 15 HP

, STOCKHOLM SVERIGE 2020

Security Guidelines for the Usage of Open Source Software

SEBASTIAN DOMAR BOLMSTAM

SIAVASH HANIFI

KTHSKOLAN FÖR ELEKTROTEKNIK OCH DATAVETENSKAP

Abstract

Open-source software is in average used in more than 65% of the applicationswithin the domains of enterprise software, retail and e-commerce, cyberse-curity and internet of things (Synopsys, 2019). With the frequent use ofopen-source software, security issues arise which need to be handled. Theseinclude among other issues; non-patched vulnerabilities and malicious code(Schryen, 2011). Security guidelines for open-source software usage have beendefined by numerous security organizations as an effort to increase effectivesecurity handling of open source software within organizations. These guide-lines often cover directives on many layers of an organization and are oftenlacking information necessary for them to be understandable, reliable, anduseful to the person using them.

The purpose of this study is to contribute to increased software security re-lated to open-source software usage, by exploring and providing informationon the topic, and by defining a set of improved security guidelines that coverboth what measures to take to minimize security risks, and how to imple-ment it, based on the published state-of-the-art security guidelines for usingopen-source software.

The subject was investigated through a research process focused on answeringwhether the current state-of-the-art security guidelines could be improved,using a qualitative research type based on a document analysis data collec-tion method. The research was exploratory in its design and the main focuswas to explore the subject by trying to answer the posed research question.

By investigating the state of contemporary security guidelines found in lit-erature, and evaluating them against a set of desirable attributes for highquality guidelines, it became evident that the contemporary guidelines could

be improved. An effort was therefore made to build on the found guidelinesand improve them by trying to resolve the issues found through the evalua-tion.

The effort of trying to improve existing guidelines resulted in a new setof guidelines including added information and reformulations, however, thechanges made could not be said to be conclusive or objective improvements.Instead they present suggestions for how and in what aspects the contempo-rary guidelines could be improved.

Keywords:Open Source (OS), Open Source Software (OSS), Security Guideline (SG),Open Source Software Security

Sammanfattning

Mjukvara med oppen kallkod (open-source software) anvands i genomsnitti mer an 65% av applikationerna for mjukvara till foretag, detalj- och e-handel, cybersakerhet och sakernas internet (Synopsys, 2019). Den frekventaanvandningen av oppen kallkod ger upphov till sakerhetsrisker som behovermotverkas. Dessa risker inkluderar bland annat; skadlig kod och sakerhets-brister som inte atgardas (Schryen, 2011). Sakerhetsriktlinjer har blivitdefinierade av ett flertal organisationer med malet att effektivisera saker-hetshanteringen av oppen kallkod och pa sa satt minska risken for attacker.Dessa riktlinjer tacker ofta manga olika delar av en organisations hierarkioch saknar ofta information som ar nodvandig for att gora dem begripliga,palitliga och anvandbara for de personer som anvander dem.

Syftet med denna studie ar att bidra till en okad sakerhet vid anvandan-det av oppen kallkod genom att dels oka kunskapen om amnet, och genomatt definiera en samling forbattrade sakerhetsriktlinjer som bade beskrivervad som ska goras for att minska sakerhetsrisker och hur det ska goras.De forbattrade riktlinjerna ska baseras pa befintliga sakerhetsriktlinjer foranvandning av oppen kallkod.

Amnet studerades genom en forskningsprocess med fokus pa att besvarafragan om huruvida befintliga sakerhetsriktlinjer kan forbattras, dar infor-mation samlades in genom en kvalitativ forskningstyp baserad pa dokumen-tanalys. Forskningsdesignen var av utforskande karaktar, dar huvudfokusetlag i att utforska amnet genom att forsoka besvara forskningsfragan.

Genom att undersoka kvaliten av befintliga sakerhetsriktlinjer som hittatsi litteraturen och utvardera dessa med stod av en stor samling onskvardaegenskaper hos riktlinjer av hog kvalitet, blev det uppenbart att befintliga

riktlinjer kan forbattras. Dafor genomfordes ett forsok att vidareutvecklabefintliga riktlinjer for att forbattra dem genom att losa de problem somhittats genom utvarderingen.

Forsoket att forbattra existerande riktlinjer resulterade i en ny uppsattningriktlinjer med tillagd information och omformuleringar. Dessa forandringarkan dock inte sagas representera konklusiva eller objectiva forbattringar.Istallet representerar de forbattrade riktlinjerna ett forslag pa hur riktlin-jer skulle kunna forbattras.

Keywords:Oppen Kallkod, Mjukvara med Oppen Kallkod, Sakerhetsriktlinje, Sakerhetfor Oppen Kallkod

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Research Method . . . . . . . . . . . . . . . . . . . . . . . . . 31.6 Commissioned Work . . . . . . . . . . . . . . . . . . . . . . . 31.7 Target Audience . . . . . . . . . . . . . . . . . . . . . . . . . . 41.8 Scope and Limitations . . . . . . . . . . . . . . . . . . . . . . 41.9 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.10 Benefits, Ethics and Sustainability . . . . . . . . . . . . . . . 5

1.10.1 Social Benefits . . . . . . . . . . . . . . . . . . . . . . 61.10.2 Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.10.3 Sustainability . . . . . . . . . . . . . . . . . . . . . . . 6

1.11 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Extended Background 92.1 Software Security and Risks . . . . . . . . . . . . . . . . . . . 92.2 Open Source Software . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1 Definition of Open Source Software . . . . . . . . . . . 102.2.2 Open Source Licenses . . . . . . . . . . . . . . . . . . . 122.2.3 Platforms and Providers . . . . . . . . . . . . . . . . . 132.2.4 By Who and How Is It Used? . . . . . . . . . . . . . . 142.2.5 Open Source Software and Security Risks . . . . . . . . 14

2.3 Closed Source Software . . . . . . . . . . . . . . . . . . . . . . 152.3.1 Where Is It Found and Who Provides It? . . . . . . . . 152.3.2 How Is It Used? . . . . . . . . . . . . . . . . . . . . . . 162.3.3 Closed Source Software and Security Risks . . . . . . . 16

2.4 Benefits and Drawbacks of Open Source Software . . . . . . . 162.5 Security Guidelines for OSS Usage . . . . . . . . . . . . . . . 18

2.5.1 Providers and Authors . . . . . . . . . . . . . . . . . . 192.5.2 Covered Security Areas . . . . . . . . . . . . . . . . . . 19

2.6 Guideline Recommendations . . . . . . . . . . . . . . . . . . . 202.6.1 Traits of a Good set of Guidelines . . . . . . . . . . . . 21

3 Research Methodology 233.1 Choice of Methodology . . . . . . . . . . . . . . . . . . . . . . 24

3.1.1 Research Question . . . . . . . . . . . . . . . . . . . . 253.1.2 Required Data . . . . . . . . . . . . . . . . . . . . . . 253.1.3 Choice of Research Type . . . . . . . . . . . . . . . . . 253.1.4 Research Tools . . . . . . . . . . . . . . . . . . . . . . 273.1.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 Research Process and Phases . . . . . . . . . . . . . . . . . . . 283.2.1 Information Gathering and Analysis Phase . . . . . . . 293.2.2 Evaluation Criteria Definition Phase . . . . . . . . . . 303.2.3 Problem Identification Phase . . . . . . . . . . . . . . . 313.2.4 Guideline Improvement Phase . . . . . . . . . . . . . . 313.2.5 Evaluation Phase . . . . . . . . . . . . . . . . . . . . . 32

3.3 Suitability of Research Approach and Research Validity . . . . 333.3.1 Suitability of Research Approach . . . . . . . . . . . . 333.3.2 Research Validity . . . . . . . . . . . . . . . . . . . . . 33

4 Results 354.1 Gathered Information . . . . . . . . . . . . . . . . . . . . . . . 35

4.1.1 Security Guidelines for OSS Usage . . . . . . . . . . . 354.1.2 Guideline Recommendations . . . . . . . . . . . . . . . 37

4.2 Evaluation Criteria . . . . . . . . . . . . . . . . . . . . . . . . 394.3 Issues Identified in Existing Guidelines . . . . . . . . . . . . . 42

4.3.1 Results of Synopsys Guideline Evaluation . . . . . . . . 434.3.2 Results of Iron Bastion Guideline Evaluation . . . . . . 444.3.3 Results of Michael Cobb’s Guideline Evaluation . . . . 464.3.4 Results of CSPS Guideline Evaluation . . . . . . . . . 48

4.4 Guideline Improvements . . . . . . . . . . . . . . . . . . . . . 504.5 Result of Final Evaluation . . . . . . . . . . . . . . . . . . . . 52

5 Analysis and Discussion 55

5.1 Result Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 555.1.1 Analysis of Information Gathering . . . . . . . . . . . . 555.1.2 Analysis of Evaluation Criteria . . . . . . . . . . . . . 565.1.3 Problem Analysis . . . . . . . . . . . . . . . . . . . . . 575.1.4 Analysis of Improvements . . . . . . . . . . . . . . . . 575.1.5 Analysis of Final Evaluation . . . . . . . . . . . . . . . 57

5.2 Analysis of the Chosen Research Method and Research Out-comes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.2.1 Management of Validity Threats . . . . . . . . . . . . . 59

6 Conclusion and Future Work 616.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

A Security Guidelines for OSS Usage 67A.1 Synopsys Security Guidelines . . . . . . . . . . . . . . . . . . 67A.2 Iron Bastion Security Guidelines . . . . . . . . . . . . . . . . . 69A.3 Michael Cobb Security Guidelines . . . . . . . . . . . . . . . . 70A.4 CSPS Security Guidelines developed for the Ministry of Trans-

portation and Communications Qatar . . . . . . . . . . . . . . 71

B Record of Evaluation Criterion Selection Process 75

C Detailed Evaluation Protocols 93C.1 Protocol for Synopsys Guideline Evaluation . . . . . . . . . . 93C.2 Protocol for Iron Bastion Guideline Evaluation . . . . . . . . . 96C.3 Protocol for Michael Cobb’s Guideline Evaluation . . . . . . . 99C.4 Protocol for Q-CERT Guideline Evaluation . . . . . . . . . . . 102C.5 Protocol for Improved Guideline Evaluation . . . . . . . . . . 105

D Improved Security Guidelines 109D.1 Reason for Creating this Document . . . . . . . . . . . . . . . 109D.2 Founding and Support . . . . . . . . . . . . . . . . . . . . . . 109D.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109D.4 Sources of Information . . . . . . . . . . . . . . . . . . . . . . 110D.5 Method used to formulate recommendations . . . . . . . . . . 110D.6 Limitation and Scope . . . . . . . . . . . . . . . . . . . . . . . 110D.7 Pilot-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110D.8 The Security Guidelines for OSS usage - Overview . . . . . . . 111

D.8.1 Organizational Governance . . . . . . . . . . . . . . . . 111D.8.2 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . 111D.8.3 Software Security . . . . . . . . . . . . . . . . . . . . . 112D.8.4 OSS Health . . . . . . . . . . . . . . . . . . . . . . . . 117

E Pilot-test: Project X 119

Chapter 1

Introduction

In a society with freedom of speech, anyone can write a book and makestatements on how to solve problems. If we follow these statements withoutquestioning their credibility and ideas we are bound to be subjects of ma-nipulation or misinformation. Open source software (OSS) can be similar toa book in these aspects (Schryen, 2011). One affects computer software andthe other affects people. A book might misinform a person, an open sourcelibrary might infect a program with malicious code or expose it to attackers.

With the significant amount of OSS used in applications today, the need forguidelines for assessing the security of using them has increased (Synopsys,2019). This is of the utmost importance in organizations and stately au-thorities that handle sensitive data. A single line of code in the midst ofthousands of lines can be malicious and seek to manipulate or leak data.Even if the authors of the OSS have no intentions of harming their users,if they do not patch technologies whose vulnerabilities become known, theyindirectly expose the users for attacks.

There are published security guidelines for the usage of OSS provided bycompanies, organizations and experts within the field of software security.However, many of the published security guidelines fail to provide necessaryinformation to be fully understandable or to establish reliability and usabil-ity for the user, which could lead to inefficiency and many times confusion.In extension, this could potentially result in dire consequences when entiresoftware systems within organizations could be compromised due to a mis-understanding or an overseen security threat.

1

Furthermore, low quality security guidelines could potentially contribute toadding fuel to the flames regarding poor security within OSS, and therebyintimidate organizations from using OSS, thus missing the opportunity toutilize powerful, easily accessible, and oftentimes well supported software.

1.1 Background

The characteristics of the open source development process makes it prone tosecurity risks. Since the development process is defined by granting permis-sion to any volunteer to modify and redistribute the software (opensource.org,2007), there can be no guarantees that all contributors have good intentions(or that they are skilled enough to implement secure code).

The security risks that come with the usage of OSS could pose a great threatwhen used within organizations handling sensitive data. Devastating conse-quences have been the result for numerous organizations when sensitive datahas been leaked. By identifying the potential risks and providing relevantactors within organizations with guidelines for how to handle them, the ben-efits of using open source software could be utilized more efficiently and therisks be reduced. The question is; could the contemporary security guidelinesfor open source software usage be improved?

1.2 Problem

The problem in focus on this study is that many contemporary security guide-lines for open source software usage do not provide necessary and completeinformation on how they were developed, what evidence certain recommen-dations are based on, explicit descriptions of whom the intended users are,what the users should do, and how to do it.

2

1.3 Purpose

The purpose of this study, based on the definition of effect goals (effektmal)in Eklund (2011), is to contribute to increased software security related tothe usage of open source software, by highlighting flaws in the current bestpractice recommendations for how to handle security risks and by providinga proposal of improved security guidelines.

1.4 Goal

The goal of this study, based on the definition of performance goals (resul-tatmal) in Eklund (2011), is to identify strengths and weaknesses in contem-porary security guidelines and propose improvements based on the findingsin the form of an improved set of guidelines for open source software usage.

1.5 Research Method

With consideration to the limitation of this work and the nature of the re-search question, the research method for this study was based on a qualitativemethodology using an exploratory research design. This could be described asan approach where the understanding of the subject in focus is successivelyexpanded throughout the study with the end-goal to explore the researchtopic. This is done through a process guided by the effort of trying to an-swer the research question; Could the contemporary security guidelines foropen source software usage be improved?.

1.6 Commissioned Work

The Swedish Transport Administration (STA), is a public authority respon-sible for long-term planning of infrastructure as well as building, operatingand maintaining public roads and railways. The IT-department at STA im-plements a micro-service architecture that over time has grown more andmore complex through continuous development and added micro-services,and a need of a visualization tool has been identified. Therefore, STA hascommissioned us to develop a visualization tool referred to as Project X (to

3

not disclose the actual name) that should provide an overview of their archi-tecture, show dependencies and usage in the system, and increase the generalknowledge of the system amongst all employees. The visualization tool wasdependent two third party open source libraries supplying an API for man-aging network graph, these are referred to as OSS X and OSS Y.

Since STA is a public authority that handles sensitive data and infrastructureoperations, the libraries used need to guarantee a high level of security beforegetting introduced to their system. Because of this an interest for securityguidelines for open source software usage has emerged within STA.

The commissioned work and the study described in this thesis are not verystrongly connected, thus no technical details of Project X will be provided.However, the improved guidelines produced from this study were pilot-testedon Project X so that it’s security-state could be evaluated. The results fromthis pilot-test are described in Appendix E with certain sensitive informationcensored as requested by our supervisor at STA.

1.7 Target Audience

The target audience for this study includes any person that might use opensource software or seeks to learn more about the domain. Mainly this in-cludes students, researchers and actors within organizations that produce ormaintain software.

1.8 Scope and Limitations

This study focuses on the textual contents of contemporary security guide-lines, that is, the quality and coverage of information, which represents afirst barrier to using the guidelines. The thesis does not explore the impactor effectiveness of guidelines, since this would require extensive research andcomparison over a longer period of time which is beyond the capabilities andresources available. Instead, the claims of expertise authors of the contem-porary guidelines is at this point regarded to be reliable in terms of impact.

4

It would be reasonable to assume that different companies and organisationsmight have their own in-house security guidelines for OSS usage, guidelinesthat are customized for their needs. However, extracting this information fora representative set of companies is deemed to be outside the scope of thisresearch. Therefore, this research will focus on general security guidelinesextracted from public sources, that still could represent a basis which couldbe further customized for specific contexts.

Although aspects of open source software other than security might be dis-cussed for context, the main focus of the research is the security aspects andthe study will disregard other potential flaws in software.

1.9 Terminology

The keywords and what we mean when we use them throughout this thesisare listed below:

• Open Source Software - OSS - Applications, frameworks and librarieswhose source code is publicly available.

• Closed Source Software - CSS - Applications, frameworks and librarieswhose source code is publicly unavailable.

• Security Guideline - Information encouraging or discouraging certainaction or behaviour with the purpose to increase security. In this thesisthe focus is on security related to OSS usage, thus what we mean withsecurity guidelines is really Security Guidelines for OSS usage.

• Guideline Recommendations - Information describing what attributesguidelines ought to have in order to be ”good”. The word recommen-dations might be used for short in some cases.

1.10 Benefits, Ethics and Sustainability

Social benefit is the total benefit to society from producing or consuming agood/service. Social benefit includes all the private benefits plus any exter-nal benefits of production/consumption. If a good has significant externalbenefits, then the social benefit will be greater than the private benefit.

5

The beneficial, ethical and sustainability-related effects of this research aredescribed in this section. Section 1.10.1 - Social Benefits covers potentialpositive and negative social effects of the research. Section 1.10.2 - Ethics,covers the ethical considerations related to the research and the content ofthis thesis, and Section 1.10.3 - Sustainability presents the potential sustain-ability effects this work might have.

1.10.1 Social Benefits

The potential social benefits of this work were identified to be related to com-munication and effectiveness. By relying on the set of guidelines collectedand improved through this study, a developer could argue for and againstthe use of a certain OSS. This could imply utilization of powerful function-ality without having to ”reinvent the wheel” and prevent the introduction ofvulnerabilities.

1.10.2 Ethics

Since this study is based on a qualitative research methodology which largelyrelies on previous studies and research work, it is important to credit thesources from which the information is gathered. This is done throughout thethesis by referencing the sources when used in text.

Regarding the commissioned work conducted in parallel to the research, theremight be some ethical issues related to confidentiality. Therefore, the con-tents of this thesis have been revised by STA employees to ensure that noconfidential material has been disclosed.

1.10.3 Sustainability

The results gathered in this study might contribute to a more sustainableusage of technology and resources, reducing the need for reinventing the wheeland increasing lifespan and quality of open source technologies.

6

1.11 Thesis Outline

The contents of this thesis in short are:

• Chapter 2 - Extended Background - This chapter provides an overviewof existing work in the field. The information presented here is reliedupon in the research process described in Chapter 3. The gatheredresults are also largely based on the information covered in this chapter.

• Chapter 3 - Research Methodology - This chapter presents the researchmethodology, methods used for collecting and analyzing data and toolsused when we conducted our study.

• Chapter 4 - Results - This chapter presents the findings obtained fromconducting our study.

• Chapter 5 - Analysis and Discussion - This chapter analyses and dis-cusses the findings presented in the result chapter.

• Chapter 6 - Conclusion and Future Work - This chapter summarizesthe study and describes how future work might be conducted to increaseour understanding of the domain.

7

Chapter 2

Extended Background

To understand the problem in focus for this study, it is necessary to under-stand the domain and the current state of research. This chapter aims tolay the ground on which this study stands. Section 2.1 - Software Securityand Risks, describes what software security is and what it’s risks and theircountermeasures are. Section 2.2 - Open Source Software, explains the back-ground of OSS and relates it to software security risks. Similarly Section2.3 - Closed Source Software, explains the background of CSS and how itrelates to software security risks. Section 2.4 - Benefits and Drawbacks ofOpen Source Software, explains the strengths and weaknesses of OSS in com-parison to CSS. Section 2.5 Security Guidelines for OSS Usage, provides anoverview of the contemporary published security guidelines for OSS usage bydescribing what they look like, which areas they cover and who their authorsare. Section 2.6 - Guideline Recommendations, explains the contemporaryguideline recommendations.

2.1 Software Security and Risks

The topic of software security is broad, complex and ever evolving, to a pointwhere it might be overwhelming to account for all aspects of it. One generaldefinition of software security is that it is the idea of protecting softwarefrom different types of malicious attacks aimed to compromise integrity, au-thentication and availability, to ensure continuous, proper and intended func-tionality (Oliinyk, 2019b). Gollmann (2011) stresses the difference betweensoftware security and software reliability, where reliability could be degraded

9

by mistakes on a developer’s behalf, resulting in unwanted behaviour. Secu-rity, on the other hand, is an effort to prevent attackers from intentionallycompromising the system in some way. The main difference in how reliabilitydegradation are tackled compared to security threats, is that mistakes madeby the developers could easily be tested with traditional testing methods,while testing for security threats needs to cover more atypical usage pat-terns, which makes it harder to accomplish.

While there are many ways of detecting a security vulnerability such as codereview, analysis tools, and in the worst case scenario, by actually being at-tacked, it is imperative for developers to be aware of the risks and how toprotect against them. It is, however, a daunting if not impossible objective toachieve this awareness of all security threats, their causes and consequences.

Examples of the most prevalent security threats, their causes, consequencesand remedies are well described in a paper published by van der Stock, Glas,Smithline, and Gigler (2017). See references for a link to the paper.

2.2 Open Source Software

Throughout this section, the concept of OSS is presented through a definitionof what it is in Section 2.2.1 - Definition of Open Source Software, an overviewof OSS licenses in Section 2.2.2, information on who provides it and who usesit in Section 2.2.3 - Platforms and Providers and Section 2.2.4 - The securityaspects of OSS is described in Section 2.2.5.

2.2.1 Definition of Open Source Software

OSS is a type of software that is distinguished by the way that it is developed,maintained and distributed, where all source code is kept publicly visible foranyone to inspect, modify and enhance (opensource.com, 2020).

The idea evolved in the 1970’s as a opposition towards companies sellingcompiled programs that were designed and tailored for specific purposes,which did not fully satisfy the different needs of all customers (Buxmann,Diefenbach, & Hess, 2012). By opening up the source code for the public,users of the software could actively contribute to the software evolution by

10

implementing the specific use cases they them self sought, and at the sametime contributing to fixing bugs and enhancing the software in general (ibid).In this way, different OSS projects attracted communities of developers thatvoluntarily and collaboratively produced flexible often high quality software.

The term ”Open Source” was coined in the 1990s, when the Open Source Ini-tiative (OSI) was founded. This non-profit corporation established a defininglist of criteria that a project needs to fulfill to classified as open source:

1. Free Distribution: The license shall not restrict any party from sell-ing or giving away the software as a component of an aggregate softwaredistribution containing programs from several different sources. The li-cense shall not require a royalty or other fee for such sale.

2. Source Code: The program must include source code, and must al-low distribution in source code as well as compiled form. Where someform of a product is not distributed with source code, there must bea well-publicized means of obtaining the source code for no more thana reasonable reproduction cost, preferably downloading via the Inter-net without charge. The source code must be the preferred form inwhich a programmer would modify the program. Deliberately obfus-cated source code is not allowed. Intermediate forms such as the outputof a preprocessor or translator are not allowed.

3. Derived Work: The license must allow modifications and derivedworks, and must allow them to be distributed under the same terms asthe license of the original software.

4. Integrity of The Author’s Source Code: The license may restrictsource-code from being distributed in modified form only if the licenseallows the distribution of ”patch files” with the source code for thepurpose of modifying the program at build time. The license mustexplicitly permit distribution of software built from modified sourcecode. The license may require derived works to carry a different nameor version number from the original software.

5. No Discrimination Against Persons or Groups: The license mustnot discriminate against any person or group of persons.

11

6. No Discrimination Against Fields of Endeavor: The license mustnot restrict anyone from making use of the program in a specific fieldof endeavor. For example, it may not restrict the program from beingused in a business, or from being used for genetic research.

7. Distribution of License: The rights attached to the program mustapply to all to whom the program is redistributed without the need forexecution of an additional license by those parties.

8. License Must Not Be Specific to a Product: The rights attachedto the program must not depend on the program’s being part of aparticular software distribution. If the program is extracted from thatdistribution and used or distributed within the terms of the program’slicense, all parties to whom the program is redistributed should have thesame rights as those that are granted in conjunction with the originalsoftware distribution.

9. License Must Not Restrict Other Software: The license mustnot place restrictions on other software that is distributed along withthe licensed software. For example, the license must not insist that allother programs distributed on the same medium must be open-sourcesoftware.

10. License Must Be Technology-Neutral: No provision of the licensemay be predicated on any individual technology or style of interface(opensource.org, 2007).

2.2.2 Open Source Licenses

All software that claim to be open source should have a license attachedto them, proclaiming permission for anyone to use, modify and share thesoftware. For a license to be valid under the OSI regulations, the OSI needsto review and approve the license (opensource.org, 2007). There are manylicenses that could be used that include different conditions and restrictions.The users of OSS need to be aware of and abide to these restriction to lawfullymake use of the software. The most popular used licenses that are recurringin many project are:

12

• Apache License 2.0

• BSD 3-clause ”new” or ”revised” License

• BSD 2-clause ”simplified” or ”freeBSD” License

• GNU General Public License (GPL)

• GNU Library or ”Lesser” General Public License (LGPL)

• MIT License (x11 License)

• Mozilla Public License 2.0

• Common Development and Distribution License

• Eclipse Public License version 2.0 (ibid)

2.2.3 Platforms and Providers

OSS are typically provided through public repositories (Hasan, 2017). Theserepositories usually hold the source code of the software and relevant files thatdescribe the project and provides licensing information. The repositories canbe hosted by any repository provider for example GitHub or SourceForge(ibid).

Since OSS are often developed and evolved through a collaboration of in-dividual developers that share an interest in the project, it might be hardto pinpoint the provider to one single organization, group or person. How-ever, there are cases where organizations or groups of developers initiate aproject and restrict external developers from contributing to the code (Ray-mond, 2001). This form of approach within an open source project is calledCathedral style, whereas the alternative of Bazaar style allows anyone to alsocontribute to the code (ibid).

In the same way that cathedral style projects have distinct providers, manybazaar style projects could be seen to have distinct providers. Even thoughbazaar style allows for anyone to contribute, many projects keeps a structurewith project leaders, core members and peripheral members (Crowston etal., 2016), where the project leaders and core members could be viewed asthe providers.

13

2.2.4 By Who and How Is It Used?

OSS is used by a broad spectrum of developers (Synopsys, 2019). Rangingfrom enterprise employees to hobby programmers, OSS is used to interlacetechnologies so that reinvention of the wheel can be avoided (Dikbas, Ergen,& Giritli, 2009).

OSS-based applications pervade our whole society. In an audit made byBlack Duck Audit Service and reported by Synopsys (2019), where over 1200commercial code-bases from different fields of business were scanned for opensource software, over 96% of the code-bases were found to include some opensource software.

The interlacing of technologies is usually made by either importing OSS inthe form of libraries or adding finished applications to systems. Softwaredevelopers that see the need of certain functionality for their projects canthus try to find OSS to provide them with this functionality.

2.2.5 Open Source Software and Security Risks

Since OSS differ from other types of software only in how it is developed,and how it is distributed, it is subject to the same technical security vulner-abilities as any other software. Regardless of this, software security withinOSS has been a controversial topic (Schryen, 2011).

Since the openness and transparency of OSS do not discriminate against anyperson, actors that are looking for ways to exploit software vulnerabilitiesfor malicious conduct does also have full access to the source code. Thisimplies that if an attacker finds a vulnerability that has not been patched,the vulnerability could be exploited to launch attacks against all users ofthe software as long as the vulnerability remains unresolved (Levy, 2000).Furthermore, even if a vulnerability would be found and reported by a per-son with good intentions, the reporting effectively disclose the vulnerabilityfor any attacker to exploit which means that even reporting an issue couldincrease the risks of an attack. With this in mind, the OSS community needsto be quick to act when a vulnerability is reported, and make sure that thesecurity risk is terminated before causing harm (Schryen, 2011).

14

Even though OSS licenses are typically permissive to allow users the freedomto alter the software and redistribute it as a part of a commercial product,it does represents a legal risk when using OSS within an organization. If thelicenses applied to an open source project is not thoroughly investigated bythe user, the organization might be at risk of overstepping restrictions andthereby legal repercussions.

The perceived security threats related to OSS (both real and exaggerated),often tends to make organizations opt for more secure alternatives. How-ever, viewing OSS as more prone to security vulnerabilities in comparison toproprietary code might be a mistake. In a report posted by Synopsis Cyber-security Research Center (Synopsys, 2019), a claim is made that proprietarycode is no more secure than OSS, and that the main cause of security risksis the lacking drive within organizations to apply patches to resolve vulner-abilities. Furthermore, the report claims that the main difference betweenOSS and commercial software is the distribution of responsibilities. Whilecommercial software providers could detect vulnerabilities, patch them, andthen push the patched code to a customer as an update, OSS users needs tomanage the updates by actively pulling the patched software when an updateis made.

2.3 Closed Source Software

The opposite of OSS is called closed source software, or proprietary software.Whereas the OSS source code is published for anyone to use, modify anddistribute, CSS source code is known only to the original authors and keptsecret and protected. The actors developing the CSS are in full control of thesource code, the software development and maintenance, and the software isonly provided to paying customers as a compiled program (Oliinyk, 2019a).

2.3.1 Where Is It Found and Who Provides It?

CSS is provided by companies with commercial incentives to develop anddistribute software functionality to authorized and paying users. If a needof certain functionality emerges, the user reaches out to a provider witha suitable solutions and an agreement is made to use their product underrestrictive licensing rules (Saltis, 2020).

15

2.3.2 How Is It Used?

The user purchases the product as is, and does not have access to the sourcecode. Therefore, the user can not implement changes to the product to betteraccommodate their needs. Instead, all requirements and issues of the softwareneed to be reported to the provider support service for further processing.

2.3.3 Closed Source Software and Security Risks

While OSS security relies on the project community to find and fix securityvulnerabilities, vulnerabilities within CSS could only be fixed by the vendor.If a user of the program suspect that an attack has been launched on theirsystem, using vulnerabilities in the CSS, the user needs to report this tothe vendor and wait for the problem to be solved (Oliinyk, 2019b). Thestrengths related to CSS and security is that potential vulnerabilities withinthe software will be harder for an attacker to find and exploit, since the sourcecode is kept secret. The case might also be made, that the personnel hiredto develop and maintain the CSS has greater knowledge of the software thana random contributor to OSS might have, and that expertise might indicatea higher code quality.

2.4 Benefits and Drawbacks of Open Source

Software

OSS oftentimes seems to have a dual nature, where many of the aspects thatmakes it a really powerful and flexible approach towards software develop-ment, also presents equal but opposite effects. There are several benefitsto opting for OSS instead of proprietary software, however, there are also anumber of disadvantages that needs to be considered. In this section, thebenefits and drawbacks of OSS with regard to security, cost, quality and us-ability is presented under corresponding headline.

Security: Within OSS communities, volunteering developers continually re-view the code to find improvements, bugs and vulnerabilities. If a vulnera-bility is found it is typically reported along with proposals of improvementsor a completed patch to resolve the issue. In this way, numerous developerscould contribute to finding and resolving security vulnerabilities and thereby

16

improving the general security in the software. Proponents of OSS develop-ment claim that the larger the community and public access to source codeis, the higher rate of vulnerability detection and patching will be. This phe-nomenon is called ”Linus’s law” by Raymond (2001) as he puts it: ”Givenenough eyeballs, all bugs are shallow.”.

OSS does have a strength in numbers if the project community is big enough,where a potentially huge number of developers could contribute to code re-view and debugging. Compared to CSS having relatively limited resourcesin regard to manpower, OSS does have a better chance of finding and elimi-nating security risks at a faster rate. The transparency of OSS also providesa chance for a potential user to review the code and make decisions based onfound and reported security issues before deciding to use the software, whichmight not be the case with CSS (Bromhead, 2017).

The major issue of OSS with regard to security is the fact that potential at-tackers have access to the source code, which enables them to both find andexploit existing vulnerabilities, and potentially introduce new vulnerabilitiesby building it into new functionality and hope that it is not detected in thereview process(Schryen, 2011).

Cost: One great advantage of OSS is the low, or non-existing cost of using it.This is first and foremost related to the initial acquirement of the software,where the software is usually completely free, whereas CSS software couldcost thousands of dollars to acquire. The same functionality gained from theOSS could of course be implemented in-house, however, this would generatebig time and personnel costs (Bromhead, 2017).

It would, however, be wrong to expect OSS to be completely free in the longrun. Acquiring an OSS could come with some extra efforts, such as tailoringit to fit the specific context, continuously monitor changes and adapt there-after, and possibly hire expertise staff for managing certain parts of the OSS.This also generates costs that needs to be considered (dcomisso, 2016).

Quality: The case could be made, that the crowd attracted to OSS com-munities to a great extent is highly knowledgeable within their fields, whichwould be an indication of high quality code being produced. Furthermore,the rapid pace at which many OSS evolve through contributions of sometimes

17

thousands of developers is a good environment to quickly improve the codequality if proper and rigorous review is being applied (Bromhead, 2017).

The downside of OSS in regard to quality is the often lacking structure ofthe projects and the work being done. There are often no explicit processesor methods in place to guarantee high quality and proper documentation(blogs.worldbank.org, n.d.).

Usability: Perhaps the greatest benefit of using open source software isthe flexibility it grants. A user with specific requirements could easily cus-tomize the open source software to better suit their needs, compared to closedsource software, where a program that fits perfectly to the specific require-ments could prove hard to find. Urging a vendor to change the software toaccommodate the specific needs could also be an expensive and time con-suming process (Regoli, 2015).

OSS communities and forums often provides information on how to use thesoftware and how to contribute, which for an experienced user makes it veryeasy to get started. However, the information given is often aimed towardsthe experienced users, and it might not be enough for novices (ibid).

2.5 Security Guidelines for OSS Usage

By providing developers with security guidelines, the risks of OSS usage ei-ther getting totally ignored or some important aspects getting overlookedcan be reduced (Broughton & Rathbone, 2001).

Even though security related to open source software use is a controversialtopic, few empirical studies have been conducted on the subject (Schryen,2011). The same seems to hold for security guidelines for OSS usage. Thesecurity guidelines that can be found throughout different internet forums,blogs and reports, do not seem to be based on scientific research, but insteadon a set of commonly agreed upon best practices.

Security guidelines for OSS usage can be found in different reports and ar-ticles, these come in varying formats and cover security areas differently.Security areas can be seen as divisions of security based on specific domains

18

or aspects, such as license risks or software vulnerabilities. Section 2.5.1- Providers and Authors describes the backgrounds of the found securityguidelines. Section 2.5.2 - Covered Security Areas explains which securityareas the security guidelines cover.

2.5.1 Providers and Authors

Guidelines and best practices related to OSS security can be found in vari-ous reports provided by organizations specialized in software security and inarticles written by experts.

Synopsys Cybersecurity Research Center (CyRC) provides six guidelines intheir report on OSS security (Synopsys, 2019). They are a part of the Synop-sys enterprise and publish research supporting strong cybersecurity practices(Cybersecurity Research Center (CyRC) | Synopsys , n.d.).

The Qatar Computer Emergency Response Team (Q-CERT) is a national,government sponsored organization setup under the auspices of Qatar’s Min-istry of Transport and Communications (qcert.org, n.d.). They have devel-oped the security guidelines for OSS usage presented in CSPS (2018).

Another set of security guidelines for OSS usage is provided by Michael Cobb,a renowned security author with over 20 years of experience in the IT indus-try (Cobb, 2012). An Australian company, Iron Bastion, provides informa-tion on how to approach OSS usage securely (blog.ironbastion.com.au, 2019).The company is specialized in cyber security and provides services protectingagainst different digital threats.

Links to the reports or articles are included in the references.

2.5.2 Covered Security Areas

The guidelines briefly presented in Section 2.5.1 - Providers and Authors allcontains directives that could be assigned a specific area of concern withinOSS security. The authors of this thesis have identified three distinct mainareas which are described below:

19

Software Security: Recommendations that are directly related to usingdifferent tools and techniques for finding and eliminating security vulnera-bilities. This security area also covers methods for monitoring the softwareevolution to detect vulnerabilities when they surface to maintain securityfrom a long-term perspective.

Licensing risks: This is the least technical security area since it is relatedto legal risks rather than the risk of being attacked. It is concerned with therisks of legal repercussions when overstepping potential restrictive limits puton the software via its license.

OSS health: This security area is concerned with investigating the generalstatus of an OSS project to get information on what to expect from it interms of quality and support, and thereby assess how secure the softwaremight be.

2.6 Guideline Recommendations

A broad definition of what a guideline entails is given by dictionary.cambridge.org(n.d.):

”Information intended to advise people on how something should bedone or what something should be”

”a piece of information that suggests how something should be done”

The only requirement for something to be considered to be a guideline underthis definition, is that it should provide information on how to do something.

There are, however, more precise suggestions on what constitutes ”good”guidelines within specific practical and scientific fields, where the quality ofguidelines could be controlled by identifying certain desired traits and at-tributes of guidelines.

Broughton and Rathbone (2001) proposes a checklist consisting of numerousrequirements that guidelines should meet for them to be considered ”good”within the field of healthcare. Another example is given by Aboulsoud,Huckson, Wyer, and Lang (2012) where desired traits for making healthcare-

20

guidelines more useful is identified through a survey involving 206 respon-dents where the majority were physicians.

Effort has been made to find similar study’s within the field of software se-curity specifically, but to no avail. When inspecting the desired traits listedin the proposed checklist, and in the results of the survey, it is evident thatmany of them could be tailored to be used to evaluate any type of guidelineto ensure its usefulness and validity.

2.6.1 Traits of a Good set of Guidelines

The traits of a good set of guideline according to Broughton and Rathbone(2001) are;

• Valid – leading to the results expected of them.

• Reproducible – if using the same evidence, other guideline groupswould come to the same results

• Cost-effective – reducing the inappropriate use of resources.

• Representative/multidisciplinary – by involving key groups andtheir interests

• Clinically applicable – patient populations affected should be unam-biguously defined

• Flexible – by identifying the expectations relating to recommendationsas well as patient preferences

• Clear – unambiguous language, which is readily understood by clini-cians and patients, should be used

• Reviewable – the date and process of review should be stated

• Amenable to clinical audit – the guidelines should be capable oftranslation into explicit audit criteria

21

Chapter 3

Research Methodology

In this chapter, the research methodology for conducting this study is de-scribed together with the steps taken to produce results, the research pro-cess. Firstly the choice of methodology is described in Section 3.1 - Choiceof Methodology. In this section the motivations behind the posed researchquestion is presented along with tools and data required to answer the ques-tion. The chosen methodology a research design is presented, and a methodfor evaluating the findings of the study is presented. Section 3.2 - ResearchProcess and Phases presents the general research process including detailedinformation on how the study was conducted to answer the research question.The suitability of the chosen approach is discussed in Section 3.3 - Suitabilityof Research Approach and Research Validity, to further establish the motiva-tions behind decisions made, and to present identified validity threats relatedto the chosen approach.

An overview of the chosen methodologies, research instruments and processesis depicted in Figure 3.1.

23

Figure 3.1: Research strategy overview

3.1 Choice of Methodology

The choice of methodical approach was based on the research question formu-lated for directing this study. The research question is presented in Section3.1.1 - Research Question along with a brief summary of why this questionwas relevant, and a general outline of how to answer the question. With thisresearch question as a basis, required data to answer the question was iden-tified and presented in Section 3.1.2 - Required Data. The type of researchmethodology chosen for this study is presented in Section 3.1.3 - Choice ofResearch Type, and tools used for the research and evaluation is presented insection 3.1.4 - Research Tools and Section 3.1.5 - Evaluation.

24

3.1.1 Research Question

The question we asked ourselves before conducting this study was:

Could the contemporary security guidelines for open source software usage beimproved?

Since the application developed in our commissioned work uses OSS and thework provider (STA) asked us to take considerations to the security risks itmight imply, the topic became relevant for us. We first looked for securityguidelines for OSS usage to follow up and found a few samples. These sam-ples although informative, came in different forms and were not very straightforward to follow up. Thus, we became interested in the idea of possiblyimproving them.

The next step was to see if there were any recommendations or conventions forhow well defined,good or ideal security guidelines ought to be. Unfortunatelythere were not many recommendations for security guidelines within the do-main of OSS usage, however a number of recommendations were found forguidelines within the domain of healthcare. Although different domains, thecontent provided in these recommendations for healthcare guidelines seemedlike they in great parts could be applied for security guidelines for OSS usage.

3.1.2 Required Data

To answer the research question certain data was required. In order to un-derstand if the contemporary security guideline for OSS usage could be im-proved, two groups of data were gathered; the contemporary security guide-lines for OSS usage and recommendations or standards for how guidelinesshould be formed.

3.1.3 Choice of Research Type

The chosen research type used for this research was a qualitative researchtype combined with a document analysis data collection-method. The qual-itative research types are characterized by a rigorous process where under-standing and defining of the studied phenomena and the question at issue issuccessively expanded throughout the study (Olsson, 2011). The document

25

analysis data collection method puts emphasis on the validity of the writtenword by not only taking what is written into account, but by whom and inwhat context.

A quantitative research type was deemed unsuitable because it builds onboth establishing causality of the results and generalizing the results to dif-ferent contexts (Bryman, 2018). Furthermore, the quantitative methodologyseeks to provide results that describe a static representation of reality, withemphasis on variable relations (ibid). This is an approach that would not bereasonable for this study. Since the research question posed in Section 3.1.1is interpretative in its nature, and would be hard to quantify and measuredue to the limited time frame, and various unknown factors such as indi-vidual preference and application context. The research conducted in thisthesis does not have the structural organization or statistical basis to use aquantitative method.

When deciding upon which research design that was most suitable for thisstudy, two different designs were considered; conclusive research design andexploratory research design.

The purpose of conclusive research is to provide a final answer or solution toa research question or problem (Dudovskiy, n.d.). This approach was deemedunsuitable due to the limitations of the study. The reasons for this is thatthe quality of the ”improved” guidelines would be a matter of interpretationand that the measurements of their effect would require rigorous testing andcomparison against current guidelines.

Instead, the chosen research design was Exploratory where the research ques-tion is put into focus to guide the exploration of the research topic. Eventhough the main focus of an exploratory research design is to highlight thetopic and thereby form a foundation on which to build future research, andnot to present a conclusive solution to a problem(ibid), this study accom-plishes both of objectives in part. In a pursuit to answer the research ques-tion, the goal of producing an improved set of guidelines was also accom-plished, however, not conclusively.

26

An exploratory research design is also strongly related to a qualitative re-search type which further suggests that the choice was suitable for the con-text.

3.1.4 Research Tools

The research tools used includes:

• Literature

• Example Software Project (Commissioned work)

Literature: The required data described in Section 3.1.2 - Required Data wasacquired by searching for information using the keywords; Security Guide-lines, Open Source Software, Guideline Recommendations, Good Guidelines,Software Security and Open Source Software Security Guidelines. The por-tals used to find this information includes DiVA, Google Scholar, refseek andMicrosoft’s Academic Research.

Example Software Project: One of our evaluation criteria demanded thatthe produced set of guidelines ought to be pilot-tested. Thus, we found thatthe software development-project Project X conducted in parallel with thisstudy would be appropriate to test the guidelines against.

3.1.5 Evaluation

To evaluate the contemporary sets of security guidelines and the produced setof improved security guidelines, a checklist of guideline recommendations wasused. This checklist was a subset of the guideline recommendations foundin Broughton and Rathbone (2001) and Lemieux-Charles, Palda, Brouwers,Gagliardi, and Grimshaw (2011). For each guideline recommendation therewere three assessment-checkboxes; Yes, No, Cannot tell,

In the process of evaluating the sets of security guidelines, they were analyzedand assessed so that the appropriate assessment-checkboxes could be checked.

27

3.2 Research Process and Phases

The research process model followed to conduct this study were created withthe main focus of answering the research question posed in Section 3.1.1 andin addition, reach the goal of producing a set of improved security guidelinesfor OSS usage. The research process consists of five research phases depictedin Figure 3.2, where each phase was designed to have a distinct part in an-swering the research question.

The Information Gathering and Analysis Phase, described in Section 3.2.1,was focused on gathering and analyzing the data needed to answer the re-search question through a literature study using the document analysis datacollection method. Consideration to the credibility of the authors were takenin the process of gathering information by making sure that their backgroundshowed some form of expertise within the domain. This implied the exclusionof certain articles.

The gathered and analyzed data was then used to define evaluation crite-ria in the Evaluation Criteria Definition Phase, described in Section 3.2.2.To be able to answer the research question, a well-defined baseline to checkthe guidelines for potential improvements needed to be established, whichis what was done when we established the evaluation criteria. In the Prob-lem Identification Phase described in Section 3.2.3, guidelines found throughthe literature study were individually evaluated using the evaluation criteriadefined in the previous phase. By conducting this evaluation, the researchquestion could be partly answered depending on the results of the evaluation.

Based on the issues of the guidelines identified through the Problem Iden-tification Phase, a draft of improved guidelines was formulated through theGuideline Improvement Phase described in Section 3.2.4, building on theguidelines found through the literature study. The improvement phase wasan effort to create a basis to be used to confirm that the found guidelinesnot only could be improved, but that it was actually possible to implementthe improvement.

The results from the Guideline Improvement Phase were then evaluatedthrough the Evaluation Phase using the same evaluation criteria as in theProblem Identification Phase. If the improved guidelines did not fulfill the

28

Figure 3.2: The Research Process

evaluation criteria, the Guideline Improvement Phase would be reiteratedto further improve the guidelines. At the point when the guidelines weredeemed satisfactory, that is, an improved version of the initial guidelines hadbeen produced, the research question would be answered in full. Descriptionsof the Evaluation Phase is found in Section 3.2.5.

3.2.1 Information Gathering and Analysis Phase

Guided by the research question, and the need for certain data to answerthe question as described in Section 3.1.1 and Section 3.1.2, the informationgathering and analysis phase was initiated by searching for existing securityguidelines for OSS usage using the keywords and portals presented in Section3.1.4 - Research Tools. The process was initially focused on finding scientificresearch papers and articles provided on well known portals. However, whenno relevant information could be found, the search was broadened to useGoogle to find interesting entry points.

29

In parallel with gathering the information, analysis through reading was con-ducted to determine if it was relevant data for our study. Also, analysis onthe credibility of the authors was conducted by making sure there was infor-mation proving their expertise within the domain.

The same process was used for finding information on what defines ”good”security guidelines. This information would be used when defining evaluationcriteria to be able to distinguish improvements in the guidelines throughoutthe process. Since no suitable information was found for the specific contextof OSS usage, trusted resources from other fields of research were deemed ac-ceptable if the information could be adapted to be applicable to the contextof interest, without loosing integrity and validity.

The result of the Information Gathering and Analysis Phase was collectionsof security guidelines and recommendations for how guidelines should formed.These artifacts became the basis for the next phase described in Section 3.2.2- Evaluation Criteria Definition Phase and in Section 3.2.3 - Problem Iden-tification Phase.

3.2.2 Evaluation Criteria Definition Phase

To be able to answer the research question of whether or not existing guide-lines could be improved, it was critical to establish a framework of evaluationcriteria on which the quality of guidelines could be evaluated, to first establisha baseline quality measure of the existing guidelines which could be comparedto the quality of the improved guidelines.

Through this research phase, the defined attributes of ”good” guidelines iden-tified in the information gathering and analysis phase were used to create theevaluation criteria to be used later in the research process. The gathered in-formation was first reviewed to find which attributes were applicable to thecontext of this study. The selection criteria applied in the review process tocollect a suitable set of attributes were:

• The attribute should be relevant to the context of OSS usage

• The attribute should be reasonable with respect to the scope of thisthesis.

30

After having selected a collection of attributes, the attributes were dividedinto groups of which evaluation criteria they were a part of fulfilling suchas clarity or maintainability. Also in the cases when certain criterion wereredundant the ones that were deemed most suitable were picked and their al-ternatives dismissed. The result of this phase was a framework for evaluation,consisting of a list of distinct evaluation criteria.

3.2.3 Problem Identification Phase

The primary objective of the research process is to answer the research ques-tion on whether existing research questions could be improved. One step ofthis process was to investigated whether the guidelines would need improve-ments at all, and in the case that they would, to identify how they could beimproved.

In the problem identification phase, guidelines found in literature were eval-uated using the evaluation criteria defined in the evaluation Criteria Phase.Various issues found through the evaluation process were registered and con-sidered the output of this phase.

3.2.4 Guideline Improvement Phase

With a collection of identified issues from the previous phase, the next andfinal step of answering the research question could be executed. This phasecould be said to answer the question; even if the guidelines need improve-ment, could they be improved?

In order to be able to improve the contemporary security guidelines, all thesets of guidelines were merged into one set. In the cases of redundant guide-lines the ones living up to the distinct evaluation criteria mostly, were chosenand their alternatives excluded.

Using the contemporary guidelines as a basis, an attempt to improve themagainst each recommendation was made. Since some issues of the contem-porary guidelines were outside the limitations of this study to be improved,they had to be left unresolved.

Throughout the improvement phase the guidelines were sought to be formu-

31

lated in accordance with the suggestions of writingcenter.ashford.edu (n.d.).

Since the issues of the contemporary security guidelines were identified latein the process, there was no way to know what this phase would entail interms of subject, effort or time. However, it is typical for an exploratoryresearch design for the researchers to be flexible and adapt to the findingsthroughout the process.

The existing guidelines were improved with complementary information andcompiled into one single set of improved guidelines which is also the outputof this phase.

3.2.5 Evaluation Phase

To investigate whether the new set of guidelines was in fact improved incomparison to the existing guidelines, and thereby answering the researchquestion, an evaluation was conducted on the new set using the same evalu-ation criteria as in the Problem Identification Phase.

Through the evaluation process, remaining issues were identified and theresult was compared to the results of the Problem Identification Phase toinvestigate whether improvements had been made or not.

A confirmatory answer at this point would have been sufficient to say thatthe research question had been answered, however, to make sure that no re-solvable issue was missed, the Guideline Improvement Phase was reiteratedto resolve any remaining issues that could be improved. The final result ofthis phase was called the complete set of improved guidelines.

32

3.3 Suitability of Research Approach and Re-

search Validity

Section 3.3.1 - Suitability of Research Approach, further explains the suit-ability of the chosen research approach in relation to this specific study, theresearch process, the type of data collected, and the nature of the goal thisstudy strives to reach. Validity threats related to the study and its resultsare addressed in Section 3.3.2 - Research Validity.

3.3.1 Suitability of Research Approach

Before conducting the study, there were a number of uncertainties amongstthe authors regarding the chosen topic of research, such as uncertaintiesabout the current state of security guidelines, the need of improvements,and possibility to improve the guidelines. These uncertainties were all takeninto account when formulating the research question Could the contemporarysecurity guidelines for open source software usage be improved?. Therefore,the topic needed to be explored through research guided by the researchquestion and the findings produced through the process. This type of researchis typical for a Exploratory research design (Dudovskiy, n.d.). Furthermore,The requirements posed on the contrary Conclusive research design is thatthe research should produce a conclusive solution to a problem. Even if thisstudy strives to produce guidelines of the highest quality possible, it couldnot be said that the improved guidelines is in any way conclusive or final.

The choice of a qualitative research type was in many ways an easy decision.Firstly, the Exploratory research design is closely tied to qualitative researchin that it is interpretive in its nature, and that it focuses on expandingthe question throughout the process. Secondly, no set of data or numericmeasurements were found to support a quantitative research type. Instead,the study builds on observations and analysis of written text which is typicalfor a qualitative research type.

3.3.2 Research Validity

The usage of validity and reliability criteria within research has a strongconnection to quantitative methods, whereas its application on qualitative

33

research has been criticized (Bryman, 2018). However, experts within thefield has proposed a set of sub-criteria to use when evaluating the validityand reliability of qualitative research methods (ibid). The qualitative re-search validity criteria includes: credibility, transferability, dependability andconfirmability.

The credibility-criterion focuses on whether the research and its results couldbe interpreted in another way than what is presented, or if the research couldbe accepted as a representation of reality. In qualitative research this validitycriterion is often ensured by letting experts in the studied field partake in theresults to evaluate if the result is valid or not, a method called respondentvalidation. It could also be validated through a method called triangulation,where several methods and sources of data is being used to ensure consis-tency (Bryman, 2018).

Transferability entails that results obtained through the study should alsobe valid even though the context is changed, or if a new study was to beconducted at another time.

Dependability is a method for ensuring the research reliability by allowingcolleagues or peers audit the work and report while it is conducted, or whenit is almost finished.

Confirmability entails an assurance that the researchers remain neutral andunbiased and do not let personal values influence the research or results.

34

Chapter 4

Results

This chapter presents all results gathered through the research process. Sec-tion 4.1 - Gathered Information presents results from the information gather-ing and analysis phase, Section 4.2 - Evaluation Criteria presents the resultsof the Evaluation Criteria Definition Phase, Section 4.3 - Issues Identified inExisting Guidelines presents the results of the problem identification phase,Section 4.4 - Guideline Improvements presents improvements made to theguidelines, and how they were implemented, and Section 4.5 - Result of Fi-nal Evaluation presents results from the final evaluation of the improvedguidelines.

4.1 Gathered Information

This section presents the sources of gathered information found through theInformation Gathering Phase, which represents the basis of this study. Sec-tion 4.1.1 - Security Guidelines for OSS Usage presents the found sources ofcontemporary security guidelines and Section 4.1.2 - Guideline Recommen-dations presents the sources of information regarding recommendations forhigh quality guidelines.

4.1.1 Security Guidelines for OSS Usage

The information gathering of security guidelines resulted in four sets of guide-lines from four different sources that were deemed to be reliable. The guide-lines differed in both content and style of formulation, however, a great part

35

of the contents were similar in terms of security areas covered. The sourcesand information about their authors and format is presented below:

Synopsis Report Security Guidelines

The 2019 Open Source Security and Risk Analysis report from Synopsys Cy-ber Security Research Center (Synopsys, 2019) provides a detailed overviewof the state of security within OSS including information on different securitythreats related to OSS usage. The report also includes a set of recommen-dations on what to consider before, and while using OSS. The main areasof concern are licensing risks and vulnerability handling. The full set ofguidelines can be viewed in Appendix A.1.

Iron Bastion Security Guidelines

Iron Bastion is an Australian company operating in the field of cyber security.The company has published information on OSS security and recommenda-tions of how to identify secure OSS and how to protect your data when usingOSS through a blog-post (blog.ironbastion.com.au, 2019). The informationfocuses on vulnerability handling, and controlling the status of the OSS com-munity. The full set of guidelines can be viewed in Appendix A.2

Michael Cobb’s Security Guidelines

In a Q&A section of computerweekly.com, security author Michael Cobbwith over 20 years of experience in the IT industry, addresses the questionof how to judge the security of OSS (Cobb, 2012). The recommendationsgiven are mainly focused on the status of the OSS community. The full setof guidelines can be viewed in Appendix A.3.

Q-CERT Security Guidelines

The most detailed and extensive set of guidelines were found in a reportwritten by Q-CERT, a government sponsored computer emergency responseteam operating under the Ministry of Transport & Communication (MOTC)of Qatar. The report proposes guidelines to be used by the MOTC to in-crease security when using OSS. The guidelines cover information on what toconsider before using OSS and how to securely implement and monitor the

36

software (CSPS, 2018). The full set of guidelines can be viewed in AppendixA.4.

4.1.2 Guideline Recommendations

The effort of finding data on how a set of guidelines should be constructedfor them to be considered guidelines of good quality, resulted in informationfrom two different sources targeting the domain of healthcare.

Broughton and Rathbone (2001) provides a checklist containing requirementsthat a set of guidelines should adhere to be considered high quality guidelines.The checklist is targeted at healthcare but deemed to also be useful in thecontext of this study. The full checklist is depicted in Figure 4.1 and Figure4.2.

Figure 4.1: Checklist for evaluating guidelines (source: (Broughton & Rath-bone, 2001))

37

Figure 4.2: Checklist for evaluating guidelines cont. (source: (Broughton &Rathbone, 2001))

Figure 4.3: Survey results showing desirable guideline attributes (source:(Aboulsoud et al., 2012))

38

Information on guideline quality criteria was also found in a report presentingthe findings of a survey conducted to establish what attributes a good set ofguidelines should have Aboulsoud et al. (2012). The subjects of the surveyidentified a number of attributes of which a number could also be found inthe checklist presented in Figure 4.1 and Figure 4.2. Parts of the surveyresults is depicted in Figure 4.3

4.2 Evaluation Criteria

The evaluation criteria defined were a subset of the guideline recommenda-tions, described in Section 4.1.2 - Guideline Recommendations. This subsetincluded the attributes we deemed reasonable within the limitations of ourstudy and that were not specific for the healthcare or clinical domain. Thefull process of selecting attributes to include in the evaluation criteria is pre-sented i Table B.1 found in Appendix B.

The evaluation criteria are grouped by clarity, implementability, measureabil-ity, validity and understandability. The full collection of evaluation criteriaare presented in Table 4.1 below:

Table 4.1: Evaluation criterion

Attribute Yes No Cannottell

ClarityThe overall objective(s) of the guideline is (are)specifically describedDo the guidelines describe the condition to bedetected,treated or prevented in unambiguous terms?Are the different possible options for management ofthe condition clearly stated in the guidelines?Can each major recommendation be found easily andare they clearly presented?

39



Are the reasons for developing the guidelines clearlystated?The recommendations are specific and unambiguousImplementabilityAre the proposals realistic and/or practical?The guideline describes facilitators and barriers to itsapplicationThe guideline provides advice and/or tools on how therecommendations can be put into practiceThe potential resource implications of applying therecommendations have been consideredMeasurabilityDoes the guideline identify clear standards or targets?Does the guideline document define measurableoutcomes that can be monitored?ValidityIs there a description of the individuals – forexample,professionals and interest groups – who wereinvolved in developing the guidelines?Was external funding and/or support received fordeveloping the guidelines?If they were received,is there evidence that potentialbiases were taken into account?Is there a description of the sources of informationused to collect (that is,identify and select) the evidenceon which the guidelines are based?Are the sources of information and search strategiesadequate and referenced?Is there a description of the methods used to interpretand assess the strength of the evidence?Are these methods satisfactory,in terms of weighting orrating the evidence?

40



Is there an explicit link between the majorrecommendations and the level of supporting evidence?Were the guidelines subjected to independent reviewby experts or outside panels (for example,a peer reviewjournal) prior to their publication or release?If so,is explicit information given about methods andhow comments were addressed?Were the guidelines piloted or pre-tested?Is information given about the pilot or pre-test processand findings?Is there an accurate summary in the document thatreflects the methods,contents and recommendations?Competing interests of guideline development groupmembers have been recorded and addressedUnderstanabilityIs there a description of the methods used to formulaterecommendations?If so,are these methods satisfactory?If formal expert or group judgement techniques wereused to reach consensus,are the techniques explicitlydescribed?The target users of the guideline are clearly definedSystematic methods were used to search for evidenceThe criteria for selecting the evidence are clearlydescribedThe strengths and limitations of the body of evidenceare clearly describedMaintainabilityIs there a mention of a date for reviewing or updatingthe guidelines?Is there an adequate description of how this will takeplace?

41



Is the body responsible for reviewing and updating theguidelines clearly identified?ComparabilityIs there a mention of other sets of guidelines that dealwith the same topic?If so,is there a discussion of possible conflicts amongexisting guidelines and the reasons for them?

4.3 Issues Identified in Existing Guidelines

The issues and strengths of the existing sets of guidelines were depicted inthe checklist-fields Yes, No and Cannot tell, that describe their possessing ofcertain attributes. To clarify, the listed and checked against attributes varyin being desirable or not. An attribute that is checked as a yes does notnecessarily mean it is desirable and similarly an attributed checked as no isnot necessarily undesirable. In the figures 4.4, 4.5, 4.6 and 4.7, the desirableattributes identified are described as approved, the undesirable attributes aredescribed as issues and the attributes whose possession could not be deter-mined were described as unknown.

Section 4.3.1 - Results of Synopsys Guideline Evaluation presents the resultsof assessing the guidelines from Synopsys (2019). The results for assessing theguidelines from Iron Bastion (blog.ironbastion.com.au, 2019) are presentedin Section 4.3.2. Section 4.3.3 presents the results from assessing the guide-lines in Cobb (2012). The results for assessing the guidelines from Q-CERT(CSPS, 2018) are shown in Section 4.3.4.

42

4.3.1 Results of Synopsys Guideline Evaluation

An overview of issues found in the guidelines in relation to approved at-tributes is depicted in Figure 4.4. Descriptions of the found issues are listedbelow.

• The recommendations are not specific and unambiguous

• The potential resource implications have not been considered

• The guidelines do not identify clear standards or target.

• There are no definitions of measurable outcomes

• There is no description of methods used to interpret and assess strengthof evidence

• There is no information given on if the guidelines were subject to in-dependent review or how comments on the guidelines were addressed

• There is no information on whether the guidelines have been pilot-tested or not

Figure 4.4: Overview of results from the Synopsys guideline evaluation

43

• Competing interests of guideline development group have not beenrecorded

• There is no description of the methods used to formulate recommen-dations

• There is no mention of expert or group judgment techniques for reach-ing consensus

• The criteria for selecting evidence is not clearly described.

• There is no adequate description of how guideline review or update willbe conducted.

• There is no mention of other sets of guidelines

• There is no discussion of possible conflicts among existing guidelines

4.3.2 Results of Iron Bastion Guideline Evaluation

An overview of issues found in the guidelines in relation to approved at-tributes is depicted in Figure 4.5. Descriptions of the found issues are listedbelow.


• The potential resource implications have not been considered

• The guideline do not identify clear standards or target.


• There is no description of the sources of information used to collect theevidence on which the guidelines are based

• The sources of information and search strategies are not adequate andreferenced


• There is no explicit link between major recommendations and the levelof supporting evidence

44

Figure 4.5: Overview of results from the Iron Bastion guideline evaluation

• There is no information given on if the guidelines were subjected toindependent review or how comments on the guidelines were addressed


• There is no accurate summary that reflects the methods, contents andrecommendations




• Target users of the guidelines are not clearly described

• The strengths and limitations of the body of evidence are not clearlydescribed

45


• There is no mention of a date for reviewing or updating the guidelines


• No responsible body for reviewing or updating the guidelines are iden-tified



4.3.3 Results of Michael Cobb’s Guideline Evaluation

An overview of issues found in the guidelines in relation to approved at-tributes is depicted in Figure 4.6, together with descriptions of found issues.

• The major recommendations are hard to find and not clearly presented


• The guidelines do not provide any advice and/or tools on how recom-mendations can be put into practice

• The guidelines do not describe any facilitators and barriers to its ap-plication

• The guidelines do not identify clear standards or target



46

Figure 4.6: Overview of results from the Michael Cobb guideline evaluation

• The sources of information and search strategies are not adequate andreferenced



• There is no information given on if the guidelines were subjected toindependent review or how comments on the guidelines were addressed


• Competing interests of the guideline development group have not beenrecorded



47








4.3.4 Results of CSPS Guideline Evaluation

Overview of issues found in the guidelines in relation to approved attributesis depicted in Figure 4.7, together with descriptions of found issues.







48

Figure 4.7: Overview of results from the CSPS guideline evaluation

• There is no accurate summary that reflects the methods, contents andrecommendations








49




4.4 Guideline Improvements

This section presents the issues found in the contemporary guidelines andhow they were improved. The improvements are presented in the group ofevaluation criteria they belong to, such as clarity or maintainability. Thefull compilation of contemporary security guidelines for OSS usage completewith improvements is included in Appendix D.

Comparability: The issue resolved regarding comparability was that allcontemporary guidelines was failing to provide any mentions of other sets ofguidelines that deal with the same topic. In this case, only a minor improve-ment was needed. Since the improved guidelines was based on the work offour different sources, all four sources were referenced in the guidelines.

Maintainability: Since no plan has been laid out for reviewing or updatingthe improved guidelines, the maintainability criterion remains unchanged.

Understandability: Many of the contemporary guidelines did not mentionwho the intended users were. Instead, the recommendations were targeted atwhoever was reading the guidelines, while the guidelines involved measureson both organizational- and developer levels. In the improved guidelines,information has been added on all recommendations informing the reader ofthe target users.

The providing of clear descriptions of criteria used for selecting evidence waspartly improved. By referencing this thesis and how information gatheringand analysis was conducted, some information was given on the develop-ment work. This would have to be provided in the guidelines and be moreclearly defined to be acceptable. The same strategy was used to describe thestrengths and limitations of the evidence.

50

Validity: Some contemporary guidelines did not provide specific informationon why and on what research basis they were created. Even though guidelineswritten by security experts could be deemed as being reliable, informationshould be provided to allow the reader to make their own assessment. In theimproved guidelines, information on the authors of the guidelines, and thereasons for developing the guidelines has been provided.

To clarify that no external interests were influencing the guideline develop-ment, information on the level of external monetary support was provided inthe guidelines.

To establish where information was gathered to formulate the guidelines, ref-erences to the contemporary guidelines were made throughout the improvedset of guidelines.

One aspect that all contemporary guidelines were lacking, was informationon how the guidelines had been tested. Improvements were therefore madeby firstly conducting a pilot-test of the improved set of guidelines to evaluatethe web application Project X. A brief description of Project X is describedin 1.6. Some details of the pilot-test are not disclosed as requested by theproviders of the commissioned work. The results of this pilot-test can befound in Appendix E

To provide an overview of the methods used, contents and recommendations,a table of contents was included in the guidelines, an aspect that the con-temporary guidelines did not consider (or did not need).

Measurability: Attempts were made to find information on how to evaluateOSS on a more empirical basis, where the findings of the evaluation couldbe compared to clear standards and targets, but to no avail. This remains agreat issue, the could prove helpful if resolved.

Implementability: Many of the guidelines recommend certain actions, butfails to provide examples of implementation. Therefor, the improved guide-lines includes suggestions of tools to use to follow up certain recommenda-tions.

51

Clarity: To clearly describe the objective of the guidelines, information re-garding this was provided before listing any recommendations.

In the improved guidelines, step by step recommendations are provided wheredifferent possible options are provided within the steps. This was done toclarify different options of management to the user.

To clarify the context of specific recommendations, all recommendations wereplaced in sections dedicated to that specific are of concern, highlighted byclear headlines.

The question of whether the recommendations of the contemporary guidelineswere specific and unambiguous could not been answered, however, improve-ments were made to make the recommendations less ambiguous and moreclear by using techniques found at (writingcenter.ashford.edu, n.d.).

4.5 Result of Final Evaluation

Overview of issues found in the guidelines in relation to approved attributesis depicted in Figure 4.8, together with descriptions of remaining issues.

• The guidelines do not describe facilitators and barriers to its application

• The potential resource implications of applying the recommendationshave not been considered

• The guidelines do not identify clear standards or targets

• The guidelines do not define measurable outcomes

• The guidelines could not present an explicit link between major recom-mendations and the level of supporting evidence.

• There is no documentation on how comments on the guidelines wereaddressed

• There are no record of future reviews or updates of the guidelines, whowill conduct them or how it will take place.

• There are no discussions of possible conflicts among existing guidelines.

52

Figure 4.8: Overview of results from the Improved guideline evaluation

53

Chapter 5

Analysis and Discussion

Through this chapter, analysis of the presented results, chosen research meth-ods and outcomes are analyzed and discussed. In Section 5.1 - Result Analy-sis, the outcomes of all research phases are analyzed separately. Section 5.2- Analysis of the Chosen Research Method and Research Outcomes focuseson analysis of the chosen methodology, how suitable to the study it provedto be, and how validity threats were managed.

5.1 Result Analysis

Analysis of the obtained results from the different research phases are pre-sented in corresponding subsection. Analysis of the information gatheringphase is presented in Section 5.1.1 - Analysis of Information Gathering, thedefined collection of evaluation criteria is analyzed in Section 5.1.2 - Analysisof Evaluation Criteria, identified problems are analysed in Section 5.1.3 -Problem Analysis, the guideline improvement phase is discussed in Section5.1.4 - Analysis of Improvements, and the final evaluation of the improvedguidelines is discussed in Section 5.1.5 - Analysis of Final Evaluation.

5.1.1 Analysis of Information Gathering

The results of the information gathering phase might seem sparse concerningthe number of sources that provides security guideline, however, finding in-formation on a topic that is relatively niche, and further limiting the resultsto contain only authors deemed reliable based on their expertise within the

55

specific subject proved really hard.

Much of the information found in the contemporary guidelines could also befound in different forums, however, not in a compiled format where multiplerecommendations come together to provide a completeness that could coverbroad aspects of security. This type of sources, along with sources failing toprove the expertise of the author, was therefore not selected to be furtherinvestigated.

Even though the selected sources were deemed reliable based on the authorsexpertise, very few of them seemed to base their recommendations on anyscientific evidence, or be willing to present the motivations behind the rec-ommendations. However, many of the recommendations were reoccurring indifferent sources and thus indicating some kind of commonly accepted bestpractice approach.

The definitions of ”good” guidelines used were targeted to healthcare andclinical procedures, however, many of the key-points in the definitions wereideal for evaluating guidelines outside the intended domain. Since no similarinformation targeted on security guidelines were found, the clinical guidelinerecommendations were a satisfactory substitute even though some contenthad to be tailored for the context of this study.

5.1.2 Analysis of Evaluation Criteria

Even though the checklist indeed presents a highly reasonable overview ofwhat to require from a set of guidelines, the individual attributes were of-ten found to be too vaguely defined. As for the guidelines themselves, theevaluation checklist should have had well-defined standards and targets thatcould be confirmed through measurements.

With that said, the evaluation checklist still was a helpful tool for measur-ing differences in guideline quality. This was apparent when comparing theoutcomes of different evaluations, and in the way attributes that clearly weremissing were highlighted.

56

5.1.3 Problem Analysis

The problem identification phase proved successful in identifying issues. Eventhough a number of the reported issues were heavily based on personal sub-jectivity, many of the issues were indeed factual flaws of the guidelines whichwould improve the guidelines if corrected.

One aspect that was not considered during the problem identification phase,was the possibility that the evaluated guidelines did not to cover the entirespectra of software security risks, or that the guidelines were in fact effectivein fulfilling their purposes. This would however cover a different scope ofresearch which would require more time and resources to investigate.

5.1.4 Analysis of Improvements

Once again, the results of this phase is very much a subject for debate sinceit largely builds on personal subjectivity and style of formulation. It does,however, provide an example of an alternative approach that presents sug-gestions on how guidelines could be slightly improved by presenting them ina different manner and adding complementary information.

Some found problems were deemed to represent a too time-consuming ef-fort to improve, and some improvements were hard to accomplish becauseof lacking standards and targets. The lack of well-defined standards andtargets were also identified to be one of the most obvious and problematicissues found in the security guidelines, since they many time suggest the in-vestigation of certain OSS aspects, but fail to describe what is desirable orconstitute a level at which the measurement could be acceptable.

Many of the identified problems might not have been a problem at all. Forexample, problems related to validity, where the evidence on which the guide-lines had been established were questioned, might not have been an issue ofvalidity but of lacking information on the underlying creation process.

5.1.5 Analysis of Final Evaluation

The main focus of discussion regarding the final evaluation of the improvedguidelines is the great risk of biased assessment. Since the authors of this

57

thesis both identified the problems to be resolved, implemented the improve-ments, and evaluated the final result, it must be stressed that the evaluationis strongly prone to biases. This is however not considered a major issue forthe study itself, since the main focus were to find an answer to the researchquestion, and not to provide a conclusive solution to the problem of ques-tionable quality of guidelines.

Some improvements were made to the contemporary guidelines, however, abig part of these improvements were related to the manner of presentationand the providing of information regarding their background. In many casesevaluation criteria could not be assessed due to a lack of information withinthe documents providing the contemporary guidelines.

Disregarding these issues, the proposed improved guidelines did manage toprovide complementary information to better adhere to the definitions ofhigh quality guidelines, even though there is still much work to be done.

5.2 Analysis of the Chosen Research Method

and Research Outcomes

Based on the nature of this study, and how it has evolved throughout theprocess, the exploratory research design proved suitable. Since a large partof the produced results was based on the authors interpretations of textualcontent with support from frameworks of evaluation, it would be safe to saythat the results obtained are in no way conclusive. The study did howeverexplore the subject of open source software, related risks and the quality ofcontemporary security guidelines for the usage of open source software, andsome conclusions could be made through that exploration.

The research question, Could the contemporary security guidelines for opensource software usage be improved?, is deemed to have been answered in part.Yes, the guidelines could be improved based on the results of the conductedguideline evaluations, however, the question of how they should be improvedremains.

The study has also highlighted a number of problems related to open sourcesoftware and contemporary guidelines, even though only a few of the found

58

problems could be conclusively resolved. However, it does make for a goodbasis of future work, where a qualitative research type could be applied toeither confirm the findings of this study through empirical evidence, or findbetter ways to deal with OSS security.

5.2.1 Management of Validity Threats

The results of this thesis, and the process of producing them have been sub-ject to the different validity threats listed in Section — (credibility, trans-ferability, dependability and confirmability). Different measures have beentaken to manage these threats throughout the work process and these mea-sures are presented here.

To ensure credibility, the produced guidelines were be based on findings fromliterature supported by experts of the studied field, and the guidelines weresubsequently tested and adjusted. The result was presented for chosen ex-perts within the field of software security for further validation.

The guidelines were subsequently evaluated using criteria based on estab-lished methods for assessing guideline quality which is seen to increase thecredibility of the presented results, however, many steps of the process wasprone to biases which was difficult to avoid. Therefore, claims of credibilityon a detailed level could not be made, while the more general ideas pre-sented could still be considered credible. transferability remains a validitythreat that is potentially minimized by detailed instructions of how to usethe guidelines. This might however not be enough to guarantee transferabil-ity since much of the results is based on personal subjectivity.

Dependability was ensured by allowing our company supervisor to auditingour work, while also making it available to security experts at STA. In ad-dition, the report has been continuously audited by our academic supervisorand other students participating in our seminar group.

To fulfill the confirmability criteria, we have been meticulous about beingtransparent and describing the process and all phases of the research in de-tail. Continuous auditing could also be viewed as an extra safeguard formaintaining confirm-ability.

59

Chapter 6

Conclusion and Future Work

OSS is widely used throughout commercial applications to enhance function-ality without ”re-inventing the wheel”. The usage of OSS does however comewith added software security risks if adequate measures for preventing themis not defined and implemented.

The problem identified through this study is the lacking quality of securityguidelines for OSS usage. If the guidelines proposed for preventing securitythreats related to OSS usage do not provide reliable recommendations withsufficient and clear information, the person implementing them might be atrisk of misinterpreting or completely missing important steps. The potentialconsequences of this problem is an increasesd risk of organizations becomingvulnerable to attackers.

The purpose of this study was therefor to increase software security by in-vestigating the quality of contemporary security guidelines, and propose animproved set of security guidelines based on the problems found.

To achieve this, an exploratory research design based on a qualitative re-search type were used. The method for collecting data was a documentanalysis method based on interpretations and analysis of textual content.

Contemporary security guidelines were found through a literature studywhere the sources of information were considered based on the expertiseof the authors. These guidelines were then evaluated using evaluation cri-teria that were developed based on established definitions of high quality

61

guidelines. An effort was made to produce and evaluate a set of improvedguidelines based on the problems found in contemporary guidelines, however,the quality of these guidelines remains questionable.

The research process used, and the results produced were focused on answer-ing the research question; Could the contemporary security guidelines foropen source software usage be improved?. The question was answered in partby providing indications on that the contemporary security guidelines indeedcould be improved, and that they are in need of improvement. However, aconclusive and universally applicable solution on how to achieve this has notbeen proven.

6.1 Future Work

The results of this study could be viewed as an initial proposal of what toconsider when evaluating OSS, however, there is much more work that couldbe done. To start with, the improved set of guidelines would have to bemore thoroughly evaluated in a controlled environment to be able to reliablymeasure the impact of the improvements. This evaluation would also haveto be conducted over a longer period of time to observe the impacts post-implementation, and include comparison to results of contemporary guide-lines.

There are also a number of remaining problems in the guidelines that werenot resolved, which needs to be addressed. For example, the importance ofthe OSS community is mentioned in the guidelines, while there is no explicitmention of what to look for, and what would be considered acceptable. Thisspecific issue was investigated, however, no information was found on whatwould be considered an acceptable in terms of community size, communityactivity, reliability of provider, and so on. Therefore, research could be tar-geted at trying to measure how different OSS attributes affects the prevalenceand longevity of security vulnerabilities within the software.

62

References

Aboulsoud, S., Huckson, S., Wyer, P., & Lang, E. (2012). Survey of preferredguideline attributes: what helps to make guidelines more useful foremergency health practitioners? International Journal of EmergencyMedicine, 5 (1), 4.

blog.ironbastion.com.au. (2019, October). A Guide to Open-SourceSoftware Security Risks. Retrieved 2020-05-18, from https://

blog.ironbastion.com.au/open-source-software-security

-risks-practices/ (Library Catalog: blog.ironbastion.com.au)blogs.worldbank.org. (n.d.). Quality of Open Source Software:

how many eyes are enough? Retrieved 2020-06-21, fromhttps://blogs.worldbank.org/opendata/quality-open-source

-software-how-many-eyes-are-enough (Library Catalog:blogs.worldbank.org)

Bromhead, B. (2017). 10 advantages of open source for the enterprise.Retrieved 2020-05-17, from https://opensource.com/article/17/

8/enterprise-open-source-advantages (Library Catalog: open-source.com)

Broughton, R., & Rathbone, B. (2001). What makes a good clinical guideline?(Vol. 1) (No. 11). Hayward Medical Communications, a division ofHayward Group plc. Retrieved from http://www.bandolier.org.uk/

painres/download/whatis/WhatareClinGuide.pdf

Bryman, A. (2018). Samhallsvetenskapliga metoder (3rd ed.).Buxmann, P., Diefenbach, H., & Hess, T. (2012). Open source software. The

Software Industry .Cobb, M. (2012). Open source software security issues: How

to review OSS for security. Retrieved 2020-05-12, fromhttps://www.computerweekly.com/answer/Open-source-software

-security-issues-How-to-review-OSS-for-security (Library

63

https://blog.ironbastion.com.au/open-source-software-security-risks-practices/



https://blogs.worldbank.org/opendata/quality-open-source-software-how-many-eyes-are-enough

https://blogs.worldbank.org/opendata/quality-open-source-software-how-many-eyes-are-enough

https://opensource.com/article/17/8/enterprise-open-source-advantages

https://opensource.com/article/17/8/enterprise-open-source-advantages

http://www.bandolier.org.uk/painres/download/whatis/WhatareClinGuide.pdf

http://www.bandolier.org.uk/painres/download/whatis/WhatareClinGuide.pdf

https://www.computerweekly.com/answer/Open-source-software-security-issues-How-to-review-OSS-for-security

https://www.computerweekly.com/answer/Open-source-software-security-issues-How-to-review-OSS-for-security

Catalog: www.computerweekly.com)Crowston, K., Hammouda, I., Lundell, B., Robles, G., Gamalielsson, J.,

& Lindman, J. (2016). Open Source Systems: Integrating Com-munities: 12th IFIP WG 2.13 International Conference, OSS 2016,Gothenburg, Sweden, May 30 - June 2, 2016, Proceedings. SpringerInternational Publishing. Retrieved from https://books.google.se/

books?id=QP40DAAAQBAJ

CSPS. (2018). Cyber security guidelines for using open-source software.Retrieved from https://www.qcert.org/sites/default/files/

public/documents/cs-csps guidelines open source software

eng v1.0.pdf

Cybersecurity Research Center (CyRC) | Synopsys. (n.d.). Retrieved 2020-05-19, from https://www.synopsys.com/software-integrity/

cybersecurity-research-center.html (Library Catalog:www.synopsys.com)

dcomisso. (2016, May). Disadvantages of open source software [Text].Retrieved 2020-05-17, from https://www.nibusinessinfo.co.uk/

content/disadvantages-open-source-software (Library Catalog:www.nibusinessinfo.co.uk)

dictionary.cambridge.org. (n.d.). GUIDELINE | meaning in the CambridgeEnglish Dictionary. Retrieved 2020-05-18, from https://dictionary

.cambridge.org/dictionary/english/guideline (Library Catalog:dictionary.cambridge.org)

Dikbas, A., Ergen, E., & Giritli, H. (2009). Managing IT in Construc-tion/Managing Construction for Tomorrow. CRC Press. (Google-Books-ID: B0NZDwAAQBAJ)

Dudovskiy, J. (n.d.). Exploratory Research. Retrieved 2020-05-20,from https://research-methodology.net/research-methodology/

research-design/exploratory-research/ (Library Catalog:research-methodology.net)

Eklund, S. (2011). Arbeta i projekt: individen, gruppen, ledaren. Lund:Studentlitteratur. (OCLC: 732309493)

Gaff, B. M., & Ploussios, G. J. (2012). Open source software. Computer ,45 (6), 9–11.

Gollmann, D. (2011). Computer security (3rd ed.).Hasan, M. (2017, October). 15 Best Websites for Downloading Open Source

Software. Retrieved 2020-05-17, from https://ubuntupit.com/best

-websites-downloading-open-source-software/ (Library Catalog:

64

https://books.google.se/books?id=QP40DAAAQBAJ

https://books.google.se/books?id=QP40DAAAQBAJ

https://www.qcert.org/sites/default/files/public/documents/cs-csps_guidelines_open_source_software_eng_v1.0.pdf



https://www.synopsys.com/software-integrity/cybersecurity-research-center.html

https://www.synopsys.com/software-integrity/cybersecurity-research-center.html

https://www.nibusinessinfo.co.uk/content/disadvantages-open-source-software

https://www.nibusinessinfo.co.uk/content/disadvantages-open-source-software

https://dictionary.cambridge.org/dictionary/english/guideline

https://dictionary.cambridge.org/dictionary/english/guideline

https://research-methodology.net/research-methodology/research-design/exploratory-research/

https://research-methodology.net/research-methodology/research-design/exploratory-research/

https://ubuntupit.com/best-websites-downloading-open-source-software/

https://ubuntupit.com/best-websites-downloading-open-source-software/

www.ubuntupit.com)Lemieux-Charles, L., Palda, V. A., Brouwers, M. C., Gagliardi, A. R.,

& Grimshaw, J. M. (2011). How can we improve guidelineuse? a conceptual framework of implementability. Implementa-tion Science, 6 (1), 26. Retrieved from https://doaj.org/article/

66a371f1bfe947d987530d9487bb2d45

Levy, E. (2000). Wide Open Source. Retrieved 2020-05-17, from https://

www.securityfocus.com/news/19

Oliinyk, K. (2019a, September). 5 Differences Between Open Sourceand Closed Source Software - API2Cart. Retrieved 2020-05-18, from https://api2cart.com/business/5-differences-between

-open-source-and-closed-source-software/ (Library Catalog:api2cart.com Section: Business)

Oliinyk, K. (2019b). What is Software Security? - Definition from Techo-pedia. Retrieved 2020-02-26, from https://www.techopedia.com/

definition/24866/software-security

Olsson, H. (2011). Forskningsprocessen : kvalitativa och kvantitativa per-spektiv (3rd ed.). Stockholm: Liber.

opensource.com. (2020). What is open source? Retrieved 2020-05-08, fromhttps://opensource.com/resources/what-open-source (LibraryCatalog: opensource.com)

opensource.org. (2007). The open source definition. Open Soure Initiative.Retrieved from https://opensource.org/docs/osd

qcert.org. (n.d.). About Q-CERT | Q-CERT. Retrieved 2020-05-26, fromhttps://www.qcert.org/about-q-cert

Raymond, E. S. (2001). The cathedral and the bazaar: Musings on linux andopen source by an accidental revolutionary. (1st ed.).

Regoli, N. (2015, June). 7 Main Advantages and Disadvantages ofOpen Source Software. Retrieved 2020-05-17, from https://

connectusfund.org/7-main-advantages-and-disadvantages-of

-open-source-software (Library Catalog: connectusfund.org)Saltis, S. (2020). Comparing Open Source vs Closed Source Soft-

ware. Retrieved 2020-05-18, from https://www.coredna.com/

blogs/comparing-open-closed-source-software (Library Catalog:www.coredna.com)

Schryen, G. (2011, May). Is open source security a myth? Communicationsof the ACM , 54 (5), 130. Retrieved from https://doi.org/10.1145/

1941487.1941516 doi: 10.1145/1941487.1941516

65

https://doaj.org/article/66a371f1bfe947d987530d9487bb2d45

https://doaj.org/article/66a371f1bfe947d987530d9487bb2d45

https://www.securityfocus.com/news/19

https://www.securityfocus.com/news/19

https://api2cart.com/business/5-differences-between-open-source-and-closed-source-software/

https://api2cart.com/business/5-differences-between-open-source-and-closed-source-software/

https://www.techopedia.com/definition/24866/software-security

https://www.techopedia.com/definition/24866/software-security

https://opensource.com/resources/what-open-source

https://opensource.org/docs/osd

https://www.qcert.org/about-q-cert

https://connectusfund.org/7-main-advantages-and-disadvantages-of-open-source-software



https://www.coredna.com/blogs/comparing-open-closed-source-software

https://www.coredna.com/blogs/comparing-open-closed-source-software

https://doi.org/10.1145/1941487.1941516

https://doi.org/10.1145/1941487.1941516

Synopsys. (2019, Apr). Open source security and risk analysis (ossra) report.Retrieved from https://www.synopsys.com/software-integrity/

resources/analyst-reports/2019-open-source-security-risk

-analysis.html

van der Stock, A., Glas, B., Smithline, N., & Gigler, T. (2017). Owasp top 10- 2017 the ten most critical web application security risks. The OWASPFoundation. Retrieved from https://github.com/OWASP/Top10/raw/

master/2017/OWASP%20Top%2010-2017%20(en).pdf

writingcenter.ashford.edu. (n.d.). Writing Clearly & Concisely | AshfordWriting Center. Retrieved 2020-05-26, from https://writingcenter

.ashford.edu/writing-clearly-concisely

66

https://www.synopsys.com/software-integrity/resources/analyst-reports/2019-open-source-security-risk-analysis.html



https://github.com/OWASP/Top10/raw/master/2017/OWASP%20Top%2010-2017%20(en).pdf

https://github.com/OWASP/Top10/raw/master/2017/OWASP%20Top%2010-2017%20(en).pdf

https://writingcenter.ashford.edu/writing-clearly-concisely

https://writingcenter.ashford.edu/writing-clearly-concisely

Appendix A

Security Guidelines for OSSUsage

A.1 Synopsys Security Guidelines

”CREATE AND ENFORCE OPEN SOURCE RISK POLI-CIES AND PROCESSES. Educate developers about the need formanaged use of open source. Put in place an automated process thattracks the open source components in a codebase and their known secu-rity vulnerabilities, as well as operational risks such as versioning andduplications, and prioritizes issues based on their severity.”

”PERFORM A FULL INVENTORY OF THEIR OPEN SO-URCE SOFTWARE. Organizations can’t defend against threats thatthey don’t know exist. It’s essential to obtain a full, accurate, andtimely inventory of the open source used in their codebases. The inven-tory should cover both source code and information on how open sourceis used in any commercial software or binary deployed in production orused as a library in an application.”

”MAP OPEN SOURCE TO KNOWN VULNERABILITIES.Public sources, such as the NVD, are a good first stop for informa-tion on publicly disclosed vulnerabilities in open source software. Keepin mind, however, that over 90 organizations contribute entries to theNVD. Not only does the NVD reflect their priorities, but there can be

67

significant lags in data reporting, scoring, and actionability of the datain a CVE entry.

Don’t rely solely on the NVD for vulnerability information. Instead,look to a secondary source that provides earlier notification of vulnera-bilities affecting your codebase and, ideally, delivers security insight,technical details, and upgrade and patch guidance.”

”CONTINUALLY MONITOR FOR NEW SECURITY TH-REATS. The job of tracking vulnerabilities doesn’t end when applica-tions leave development. Organizations need to continuously monitorfor new threats for as long as their applications remain in service. Im-portantly, any continuous monitoring strategy must take into accountthe composition of the software under attack, lest it become overwhelmedby the volume of vulnerability disclosures we experienced in 2018.”

”IDENTIFY LICENSING RISKS. Failure to comply with opensource licenses can put organizations at significant risk of litigationand compromise of IP. Educate developers on open source licenses andtheir obligations. Involve your legal advisors in the education processand, of course, in reviewing licenses and compliance with legal obliga-tions.”

”MAKE SURE OPEN SOURCE IS PART OF M&A DUEDILIGENCE. If you’re engaged in an acquisition, understand thatthe target company is using open source—and likely not managing itwell. Don’t hesitate to ask questions about their open source use andmanagement. If the software assets are a significant part of the valua-tion of the company, have a third party audit the code for open source.”

68

A.2 Iron Bastion Security Guidelines

”HOW TO IDENTIFY SECURE OPEN-SOURCE SOFT-WARE

To investigate the suitability of a software solution, you need to evalu-ate the security and reputation of each piece of software you’re inter-ested in using. An excellent place to start is to review its version his-tory and look at previous security issues for red flags.

Particularly for software providing cryptographic services, check if therehave been independent audits of the product’s design and implementa-tion. Security, and especially cryptography, is hard so having indepen-dent reviews by bona fide experts in the relevant fields is especially im-portant for security-sensitive code. Even word of mouth from a trustedcolleague can give you a good idea of what’s reliable. Additionally, findout what software your partners, competitors, and established organisa-tions in your field are using.

It may be that the best option for you is proprietary software or even amix of closed and open-source tools. The critical thing is that you makeyour decision based on thorough research.”

”HOW TO SECURE YOUR DATA WHEN DOWNLOAD-ING OPEN-SOURCE SOFTWARE

The key to keeping your data secure is to monitor for new threats con-tinuously. Are you using a current version of the open-source project?Is it the most secure version? Is the code actively maintained by anexpert, trustworthy community?

It’s vital to stay up to date - either by checking for insecurities foundand logged in online sources, or through automated security tools. Au-tomation is usually the best option for most companies, as manuallychecking your open-source use will require significant investments oftime, resources, and budget.”

69

A.3 Michael Cobb Security Guidelines

”Look at the development team.

When considering an OSS product, look for a company or developmentteam that has a proven track record of delivering quality products andissuing patches quickly once a vulnerability has been discovered. Manyof the larger, well-established products within the OSS community (suchas Apache) are well-managed with sound change-control processes thateasily stand up to or surpass those of some well-known commercial soft-ware vendors.

There are some open source projects that are not well supported andlack robust processes, similar in many ways to some smaller or lesssecurity-conscious companies making commercial software. Opting fora product from a lesser known or less experienced team or companymeans you are taking a security risk with the product.”

”Look at the user community.

Be sure to consider the user base when evaluating an OSS product forsecurity. Look for a mature community with a clear policy about howcontributions are evaluated and included, and a reliable regime in placeto handle errors or problems. Case studies of trouble-free real-worlddeployments are of more value than claims of how many times a prod-uct has been downloaded.

”Look at the documentation.

The quality of the product’s documentation is also an important con-sideration. How easy is it to find relevant information? Are there third-party specialists who can be hired to provide support? Don’t get caughtup with promises of lifetime support from a company or project thathas only been in existence for a few years; development and supportcan be discontinued at any time. You only have to look at consumerexperiences with white goods to know that vendors can come and go,disappearing overnight and leaving behind all guarantees, spare partsand promises of help.”

70

A.4 CSPS Security Guidelines developed for

the Ministry of Transportation and Com-

munications Qatar

1. Governance: Organizations should develop a comprehensive policygoverning the usage of OSS. Amongst other things, the policy shouldcover an acceptable usage of OSS and the acceptable risk appetite forOSS. Risk Assessment should on a minimum cover risks related to li-cense requirements, operations (support for the software and stabilityof the software) and security (vulnerabilities and known exploits).

2. Asset Inventory: Organizations should maintain a detailed inven-tory of OSS used within the organization detailing instances (number/ quantity) of use, version etc. Where OSS is used as a componentof your in-house application, a data call should be performed to de-termine what components are OSS and what versions are currently isuse. The inventory should include libraries, frameworks, middlewareand applications. Maintain a profile of each OSS to include the code’sorigin, where to get updates, and how often the community releases newversions

3. Documentation: Where OSS is used as a component of your in-house application, a comprehensive documentation of the componentsused should be maintained. This includes libraries, frameworks, mid-dleware and applications.

4. Source Code Repository: Maintain a repository of the source codeof OSS deployed within the organization.

5. Whitelisted OSS: OSS should be qualified after conducting relevantusability, stability and security tests. Only pre-approved and qualifiedOSS should be used and deployed within the organization.

6. Baseline Controls: The OSS should be treated as any other informa-tion system deployed within an enterprise and should comply with theentire relevant baseline and recommended controls prescribed within theNational Information Assurance Policy including its classification.

7. Risk Management: Organizations should conduct a Risk Assessmenton the OSS, based on the criticality of the services that it will deliver

71

and implement necessary controls to manage any risks that may arise.

8. OSS Installation: Any OSS installed / deployed should adhere tothe organization’s system installation procedure. This should ensurethat:

a. Only whitelisted OSS is deployed in the organization.

b. All deployments are vetted and approved through a formal syst-em deployment / change management process.

c. All deployments are inventoried in the asset register.

d. Only authorized individuals such as the system administratorsshould install / deploy OSS.

e. Use OSS from reliable and trusted sites. Wherever possi-ble prefer source code to binaries. It is always good to down-load the source code, verify against the MD5 checksums provided.Examples of trusted sites as recommended by Open Source Ini-tiative include freshmeat.net, sourceforge.net, osdir.com, devel-oper.berlios.de and bioinformatics.org

f. Ensure that the OSS is tested and updated with the latestpatches.

9. Security: Perform a security assessment to identify and patch anyknown vulnerabilities in the OSS. For critical applications, it is recom-mended to do a combination of an automated static analysis (sourcecode scanning) and dynamic analysis to find vulnerabilities in individ-ual applications. On identifying a vulnerability the organization should:

a. Check if a patch or an updated version is available to fix theidentified vulnerability.

b. With caution and responsible disclosure, ask the open sourcecommunity for help. Post the issues to the community and seeif a member can help with the fix.

c. Fix them yourself. Use your own or third-party developmentresources to resolve the issues.

10. Application Hardening: As with any other software, the OSS shouldbe configured in a secure manner.

72

a. Disable unwanted services.

b. In a development environment, unused and unwanted libraries,APIs and DLL should be removed / disabled.

c. User permissions should adhere to least privileges and a needto have model.

d. Use a Defense in Depth strategy to include all the possiblemeasures that need to be taken for all OSS, along with other prod-ucts in the network.

11. Patch Management: The organization’s Patch Management pro-cess should monitor and update patches released for the OSS.

a. Check the community associated with your open source code.These communities typically have bug-tracking systems and mail-ing lists that give information on known security issues and pro-vide the latest news and exploits.

b. When a new vulnerability is identified, the organization shouldexplore possible mitigation strategies that can be implemented un-til a patch is available.

c. Once a patch is released, test the patch for stability and ap-plicability within your test environment prior deploying on yourproduction systems.

d. Have a time bound approach to patch all vulnerable OSS inplace.

12. Compliance: From time to time, verify the inventory of OSS, toensure compliance with the OSS Usage Policy. Software compositionanalysis (SCA) products may be used to analyze application composi-tion to detect components known to have security and/or functional-ity vulnerabilities or that require proper licensing.

13. Business Continuity / Capacity Building: Organizations shouldensure that their employees have the relevant skills required to main-tain an OSS. Trainings should be regularly imparted and skills dis-tributed to avoid single point of failures. Knowledge bases of the OSSshould be developed and maintained within the organization.”

73

Appendix B

Record of Evaluation CriterionSelection Process

Table B.1: Table of chosen evaluation criterion

MeasureID

Attribute Source Accepted Motivation Adapted form

1

”The overallobjective(s) ofthe guideline is(are) specificallydescribed”

(Aboulsoudet al., 2012)

Yes

Theattributeisapplicable

Unchanged

2

”The healthquestion(s)covered by theguideline is(are) specificallydescribed”


No

Theattributeis specifictohealthcare,and iscovered byattribute 1

-

75


MeasureID


3

”The population(patients, public,etc.) to whomthe guideline ismeant to applyis specificallydescribed”


NoRedun-dancy

-

4

”The guidelinedevelopmentgroup includesindividuals fromall the relevantprofessionalgroups”


No

Toolooselydefined incontext

-

5

”The views andpreferences ofthe targetpopulation(patients, public,etc.) have beensought”


No

Attributeis outsidethe scopeof thestudy

-

6

”The targetusers of theguideline areclearly defined”


Yes


Unchanged

7

”Systematicmethods wereused to searchfor evidence”


Yes


Unchanged

76


MeasureID


8

”The criteriafor selecting theevidence areclearlydescribed”


Yes


Unchanged

9

”The strengthsand limitationsof the body ofevidence areclearlydescribed”


Yes


Unchanged

10

”The methodsfor formulatingthe recommen-dations areclearlydescribed”


NoRedun-dancy

-

11

”The healthbenefits, sideeffects and riskshave beenconsidered informulating therecommenda-tions”


No

Theattributeis specifictohealthcare

-

12

”There is anexplicit linkbetween the rec-ommendationsand thesupportingevidence”


NoRedun-dancy

-

77


MeasureID


13

”The guidelinehas beenexternallyreviewed byexperts prior toits publication”


NoRedun-dancy

-

14

”A procedure forupdating theguideline isprovided”


NoRedun-dancy

-

15

”The recommen-dations arespecific andunambiguous”


Yes


Unchanged

16

”The differentoptions formanagement ofthe condition orhealth issue areclearlypresented”


No


-

17

”Key recom-mendations areeasilyidentifiable”


NoRedun-dancy

-

18

”The guidelinedescribesfacilitators andbarriers to itsapplication”


Yes


Unchanged

78


MeasureID


19

”The guidelineprovides adviceand/or tools onhow the recom-mendations canbe put intopractice”


Yes


Unchanged

20

”The potentialresourceimplications ofapplying the rec-ommendationshave beenconsidered”


No


-

21

”The guidelinepresents(includes)monitoring and/or auditingcriteria”


No

Attributenotapplicableto context

-

22

”The views ofthe funding bodyhave notinfluenced thecontent of theguideline”


NoRedun-dancy

-

79


MeasureID


23

”Competinginterests ofguidelinedevelopmentgroup membershave beenrecorded andaddressed”


No


-

24

”Is the agencyresponsible forguidelinedevelopmentclearlyidentified?”

(Broughton& Rathbone,2001)

NoRedun-dancy

-

25

”Was externalfunding and/orsupport receivedfor developingthe guidelines?”


Yes


Unchanged

26

”If they werereceived,is thereevidence thatpotential biaseswere taken intoaccount?”


Yes


Unchanged

27

”Are the reasonsfor developingthe guidelinesclearly stated?”


Yes


Unchanged

80


MeasureID


28

”Are theobjectives of theguidelinesclearly defined?”


NoRedun-dancy

-

29

”Is there adescription ofthe individuals –for exam-ple,professionalsand interestgroups,includingpatients – whowere involved indeveloping theguidelines?”


Yes

Theattributeisapplicable,but has tobe adapted

”Is there adescription ofthe individuals –for exam-ple,professionalsand interestgroups – whowere involved indeveloping theguidelines?”

30

”If so,did thegroup representall the keygroups/stakehold-ers/disciplines?”


No

Attributeoutside thescope ofthe study

-

31

”Is there astatement ofhow potentialbiases orconflicts ofinterest of thepanel membersare taken intoaccount? Werethese dealt withadequately?”


No


-

81


MeasureID


32

”Is there adescription ofthe methodsused to seekviews ofinterestedparties not onthe panel?”


No


-

33

”Is there adescription ofthe sources ofinformationused to collect(that is, identifyand select) theevidence onwhich theguidelines arebased?”


NoRedun-dancy

-

34

”Wheredatabases areused,is there adescription ofthe searchstrategy used?”


No


-

35

”If so,are thesources andsearch strategiesadequate andreferenced?”


No

Attributeon whichit is builtis notaccepted

-

82


MeasureID


36

”Is there adescription ofthe methodsused to interpretand assess thestrength of theevidence?”


Yes


Unchanged

37

”Are thesemethodssatisfactory,interms ofweighting orrating theevidence?”


Yes


Unchanged

38

”If a synthesismethod was usedto summarisethe evidence (forexample,meta-analysis ordecisionanaly-sis) is themethod explicitlydescribed?”


No


-

39”Is this methodsatisfactory?”


No

Attributeon whichit is builtis notaccepted

-

83


MeasureID


40

”Is there anexplicit linkbetween themajor recom-mendations andthe level ofsupportingevidence?”


Yes


Unchanged

41

”Is there adescription ofthe methodsused toformulaterecommenda-tions?”


Yes


Unchanged

42”If so,are thesemethodssatisfactory?”


Yes


Unchanged

43

”If formalexpert or groupjudgementtechniques wereused to reachconsensus,arethe techniquesexplicitlydescribed?”


Yes


Unchanged

84


MeasureID


44

”Does thedocument giveexplicitinformationabout thestrength of theconsensus?”


NoRedun-dancy

-

45

”If the guidelinewas developedlocally,has itbeen adaptedfrom nationalguidelines?”


No

Outsidethe scopeof thestudy

-

46

”Does theguideline haveaccompanyingpatientinformationleaflets?”


NoAttributespecific tohealthcare

-

47

”Is there anadequatedescription ofthe healthbenefits that arelikely to begained from therecommendedmanagement?”


No


-

85


MeasureID


48

”Is there anadequatedescription ofthe potentialharms and risksthat may occuras a result of therecommendedmanagement?”


No


-

49

”Is there anadequatedescription ofthe costs andresources likelyto be incurred orreleased by therecommendedmanagement?”


No


-

50

”Were theguidelinessubjected toindependentreview byexperts oroutside panels(for example,apeer reviewjournal) prior totheir publicationor release?”


Yes


”Unchanged”

86


MeasureID


51

”If so,is explicitinformationgiven aboutmethods andhow commentswereaddressed?”


Yes


”Unchanged”

52

”Were theguidelinespiloted orpre-tested?”


Yes


”Unchanged”

53

”Is informationgiven about thepilot or pre-testprocess andfindings?”


Yes


”Unchanged”

54

”Is there amention of adate forreviewing orupdating theguidelines?”


Yes


”Unchanged”

55

”Is there anadequatedescription ofhow this willtake place?”


Yes


”Unchanged”

87


MeasureID


56

”Is the bodyresponsible forreviewing andupdating theguidelinesclearlyidentified?”


Yes


”Unchanged”

57

”Is there amention ofother sets ofguidelines thatdeal with thesame topic?”


Yes


”Unchanged”

58

”If so,is there adiscussion ofpossible conflictsamong existingguidelines andthe reasons forthem”


Yes


”Unchanged”

59

”Is there anaccuratesummary in thedocument thatreflects themeth-ods,contents andrecommenda-tions?”


Yes


”Unchanged”

88


MeasureID


60

”Overall,havethe potentialbiases ofguidelinedevelopmentbeen adequatelyaddressed?”


NoRedun-dancy

-

61

”Are the healthprofessionals forwhom theguidelines arein-tended,identified?”


No


-

62

”Is there asatisfactorydescription ofthe patients towhom theguidelines aremeant toapply?”


No


-

63

”Is there adescription ofthecircumstances(clinical ornon-clinical) inwhich exceptionsmight be madein using theguidelines?”


No


-

89


MeasureID


64

”Is there anexplicitstatement ofhow thepatient’spreferencesshould be takeninto account inapplying theguideline?”


No


-

65

”Do theguidelinesdescribe thecondition to bedetected,treatedor prevented inunambiguousterms?”


Yes


”Unchanged”

66

”Are thedifferent possibleoptions formanagement ofthe conditionclearly stated inthe guidelines?”


Yes


”Unchanged”

67

”Can eachmajorrecommendationbe found easilyand are theyclearlypresented?”


Yes


”Unchanged”

90


MeasureID


68

”Does theguidelinedocumentsuggest possiblemethods fordisseminationand implemen-tation?”


NoRedun-dancy

-

69

”Are theproposalsrealistic and/orpractical?”


Yes


”Unchanged”

70

”Is provisionmade for theadaptation ofthe guidelineinto a localguideline?”


No

Attributeoutside thescope ofthe

-

71

”If so,does theguidelinesuggest/specifythe methods forlocaldevelopment?”


No

Attributeoutside thescope ofthe

-

72

”Does theguidelinedocumentspecify criteriafor monitoringcompliance?”


No


-

91


MeasureID


73

”Does theguidelineidentify clearstandards ortargets?”


Yes


”Unchanged”

74

”Does theguidelinedocument definemeasurableoutcomes thatcan bemonitored?”


Yes


”Unchanged”

92

Appendix C

Detailed Evaluation Protocols

C.1 Protocol for Synopsys Guideline Evalua-

tion

Table C.1: Evaluation of Synopsys report


ClarityThe overall objective(s) of the guideline is (are)specifically described

3

Do the guidelines describe the condition to bedetected,treated or prevented in unambiguous terms?

3

Are the different possible options for management ofthe condition clearly stated in the guidelines?

3

Can each major recommendation be found easily andare they clearly presented?

3

Are the reasons for developing the guidelines clearlystated?

3

The recommendations are specific and unambiguous 7

ImplementabilityThe guideline provides advice and/or tools on how therecommendations can be put into practice

3

93



Are the proposals realistic and/or practical? ?The guideline describes facilitators and barriers to itsapplication

3

The potential resource implications of applying therecommendations have been considered

7

MeasurabilityDoes the guideline identify clear standards or targets? 7

Does the guideline document define measurableoutcomes that can be monitored?

7

ValidityWas external funding and/or support received fordeveloping the guidelines?

?

If they were received,is there evidence that potentialbiases were taken into account?

?

Is there a description of the individuals – forexample,professionals and interest groups – who wereinvolved in developing the guidelines?

3

Is there a description of the sources of informationused to collect (that is,identify and select) the evidenceon which the guidelines are based?

3

Are the sources of information and search strategiesadequate and referenced?

3

Is there a description of the methods used to interpretand assess the strength of the evidence?

7

Are these methods satisfactory,in terms of weighting orrating the evidence?

?

Is there an explicit link between the majorrecommendations and the level of supporting evidence?

3

Were the guidelines subjected to independent reviewby experts or outside panels (for example,a peer reviewjournal) prior to their publication or release?

?

94



If so,is explicit information given about methods andhow comments were addressed?

7

Were the guidelines piloted or pre-tested? ?Is information given about the pilot or pre-test processand findings?

7

Is there an accurate summary in the document thatreflects the methods,contents and recommendations?

3

Competing interests of guideline development groupmembers have been recorded and addressed

7

UnderstanabilityIs there a description of the methods used to formulaterecommendations?

7

If so,are these methods satisfactory? ?If formal expert or group judgement techniques wereused to reach consensus,are the techniques explicitlydescribed?

7

The target users of the guideline are clearly defined 3

Systematic methods were used to search for evidence ?The criteria for selecting the evidence are clearlydescribed

7

The strengths and limitations of the body of evidenceare clearly described

3

MaintainabilityIs there a mention of a date for reviewing or updatingthe guidelines?

3

Is there an adequate description of how this will takeplace?

7

Is the body responsible for reviewing and updating theguidelines clearly identified?

3

ComparabilityIs there a mention of other sets of guidelines that dealwith the same topic?

7

95



If so,is there a discussion of possible conflicts amongexisting guidelines and the reasons for them?

7

C.2 Protocol for Iron Bastion Guideline Eval-

uation

Table C.2: Evaluation of Iron Bastion guidelines



3


3


7


7


3

Are the objectives of the guidelines clearly defined? 3



3

Are the proposals realistic and/or practical? ?The guideline describes facilitators and barriers to itsapplication

3

96




7



7


?


?


3


7


7


7


?


7


?


7

Were the guidelines piloted or pre-tested? ?

97



Is information given about the pilot or pre-test processand findings?

7

Is there an accurate summary in the document thatreflects the methods, contents and recommendations?

7


7


7

If so,are these methods satisfactory? ?If formal expert or group judgement techniques wereused to reach consensus,are the techniques explicitlydescribed?

7



7


7


7


7


7


7


7

98

C.3 Protocol for Michael Cobb’s Guideline

Evaluation

Table C.3: Evaluation of Michael Cobb’s guidelines



3


3


3


7


3




7

Are the proposals realistic and/or practical? 3

The guideline describes facilitators and barriers to itsapplication

7


3



7


?

99




?


3


7


7


7


?


7


?


7


7


3


7


7

If so,are these methods satisfactory? ?

100



If formal expert or group judgement techniques wereused to reach consensus,are the techniques explicitlydescribed?

7



7


7


7


7


7


7


7

101

C.4 Protocol for Q-CERT Guideline Evalua-

tion

Table C.4: Evaluation of Q-CERT guidelines



3


3


3


3


3




3



3


3



7


?

102




?


3


7


?


7


?


7


?


7


7


3


7


7


103




7



7


7


7


7


7


7


7

104

C.5 Protocol for Improved Guideline Evalu-

ation

Table C.5: Evaluation of the Improved Guidelines



3

Do the guidelines describe the condition to bedetected, treated or prevented in unambiguous terms?

3


3


3


3



3



7


7



7


7

105




7


3


3


3


3

Are these methods satisfactory, in terms of weightingor rating the evidence?

?


7


3


7

Were the guidelines piloted or pre-tested? 3

Is information given about the pilot or pre-test processand findings?

3


3


7


3


106




7


Systematic methods were used to search for evidence 3

The criteria for selecting the evidence are clearlydescribed

3


3


7


7


7


3


7

107

Appendix D

Improved Security Guidelines

D.1 Reason for Creating this Document

The reason for creating this document was that an attempt of producing animproved set of security guidelines for OSS usage was needed for a bachelor’sthesis written at The Royal Institute of Technology in Stockholm, Sweden.The study was conducted by two students and was guided by the followingquestion:

Could the contemporary security guidelines for open source software usage beimproved?

D.2 Founding and Support

No founding or support was provided in the process of creating this docu-ment.

D.3 Objectives

The objective of the set of security guidelines for open source software (OSS)usage presented in this document is to provide security guidance to softwaredevelopers, development managers and organization management teams thatuse or consider using open source software. This documents seeks to raisethe reader’s awareness regarding risks and remedies by providing relevant

109

information in a structured and organised manner. Furthermore, this doc-ument aims to enable the utilization of potential benefits of using OSS byminimizing the related risks.

D.4 Sources of Information

The guidelines presented in this document are all based on the guidelinesdescribed in Synopsys (2019), Cobb (2012), blog.ironbastion.com.au (2019)and CSPS (2018). Links to these documents are presented under references.The process for finding and selecting these sources are described in the thesis-report in Section 3.2.1 - Information Gathering and Analysis Phase. Allcredits for the content of the guidelines go to the authors of the mentionedsources, the only thing we have contributed with is the organization andmerging of the guidelines.

D.5 Method used to formulate recommenda-

tions

The method used to formulate recommendations are described in the thesisreport under Section 3.2.4 - Guideline Improvement Phase.

D.6 Limitation and Scope

This document has been produced under the limitation described in thethesis-report in Section 1.9 - Scope and Limitations.

D.7 Pilot-Test

The guidelines were pilot tested against an Angular-web-application projectthat was developed in parallel with the study. The results for this pilot-testcan be found in the report Appendix E .

110

D.8 The Security Guidelines for OSS usage -

Overview

The security guidelines are divided into three different categories presentedin three sections these include; Section 7.1 - Organizational Governance, Sec-tion 7.2 - Licensing, Section 7.3 - Software Security and Section 7.4 - OSShealth.

Each guideline is presented in a green box with accompanying informationregarding the target users i.e the actors the guideline is meant to be fol-lowed up by. For some guidelines detailed information and/or follow upsuggestions are included. The follow up suggestions are more practical orimplementation-oriented directives.

D.8.1 Organizational Governance

This section describes security guidelines that focus on the way organizationmanagement teams can govern the way OSS is used within their organization.

Develop a comprehensive policy governing the usage of OSS

Target User: Organization Management Teams, Development Managers

Details:

• The policy should cover an acceptable usage of OSS and the acceptablerisk appetite for OSS.

D.8.2 Licensing

Security guidelines targeting license-oriented risks are covered in this section.

Educate Developers on OSS Licenses and their risks

Target User: Development Managers, Organization Management Teams

Follow Up Suggestions:

111

• Involve legal advisors in the process of educating them.

• Provide the developers with relevant literature such as:

– Open source software by Gaff and Ploussios (2012)

Evaluate the license of OSS

Target Users: Development Managers, Software Developers


1. Identify an OSS’s license type.

• If there is no license, contact the author(s) and ask them to licensetheir work.

2. Compare the license’s restrictions and permissions to your desired use-cases. Have your legal advisors confirm that they comply with thedesired use-cases before using them.

D.8.3 Software Security

Security Guidelines for OSS usage with focus on the software-aspect arepresented in this section.

Assess OSS Risks

Target Users: Developement Managers, Software Developers

Details:

• Conduct a Risk Assessment on the OSS, based on the criticality of theservices that it will deliver and implement necessary controls to manageany risks that may arise.

112

Educate developers about the need for managed use of OSS

Target Users: Development Managers, Organization Management Teams

Follow Up Suggestion:

• Provide the developers with relevant literature such as:

– Open source security and risk analysis (ossra) report by Synopsys(2019)

Keep an inventory of open source components


Details:

• The inventory should hold information about the OSSs’ known vulner-abilities and operational risks.

• The inventory should cover both source code and information on howopen source is used in any commercial software or binary deployed inproduction or used as a library in an application.

• The inventory should include libraries, frameworks, middleware andapplications.

• The process of keeping an inventory ought to automated.


• Use one (or more) of these tools:

– SonarQube

– OSSIndex (Java, JavaScript, Go, PHP, .NET, C/C++, Python)

– OWASP Dependency-Check

– Node Audit Analyser (JavaScript/NPM)

113

– RetireJS (JavaScript)

– Hakiri (Ruby and Rail-based github-projects)

– Snyk (JavaScript and NPM)

– Gemnasium(Ruby, NPM/JavaScript, PHP, Python, Bower/JavaScript)

– Veracode Application Analysis

Maintain a comprehensive documentation of the OSS compo-nents used

Target Users: Software Developers

Details:

• Make sure the documentation shows where the OSS component is used

• Make sure the documentation describes what the OSS component does

• Make sure the documentation describes how the OSS works


• Have a text-document with headlines for each OSS component andwhere and how it is used and what it does.

Look up vulnerabilities of OSS


Details

• For critical applications, it is recommended to do a combination of anautomated static analysis (source code scanning) and dynamic analysisto find vulnerabilities in individual applications.

Follow Up Suggestions

• As a first stop public sources such as NVD can be used to find infor-mation on publicly disclosed vulnerabilities in open source software.

114

• Look to a sources that provide early notification of vulnerabilities af-fecting your codebase and, ideally, delivers security insight,technicaldetails, and upgrade and patch guidance.

• Use automated security tools to find vulnerabilities. Automation isusually the best option for most companies, as manually checking youropen-source use will require significant investments of time, resources,and budget. Some of these tools include:

– SonarQube

– OSSIndex (Java, JavaScript, Go, PHP, .NET, C/C++, Python)

– OWASP Dependency-Check

– Node Audit Analyser (JavaScript/NPM)

– RetireJS (JavaScript)

– Hakiri (Ruby and Rail-based github-projects)

– Snyk (JavaScript and NPM)

– Gemnasium(Ruby, NPM/JavaScript, PHP, Python, Bower/JavaScript)

– Veracode Application Analysis

Patch vulnerabilities



1. Check if a patch or an updated version is available to fix the identifiedvulnerability.

2. With caution and responsible disclosure, ask the open source commu-nity for help. Post the issues to the community and see if a membercan help with the fix.

3. Fix them yourself. Use your own or third-party development resourcesto resolve the issues.

115

Configure applications appropriately



• Disable unwanted services.

• In a development environment, unused and unwanted libraries, APIsand DLL should be removed / disabled

• User permissions should adhere to least privileges and a need-to-havemodel

When taking over projects gather information regarding theirOSS usage and management

Target Users: Organization Management Teams, Development Managers,Software Developers

Continuously monitor for new threats

Target Users: Project Managers, Software Developers


• When a new vulnerability is identified, the organization should explorepossible mitigation strategies that can be implemented until a patch isavailable.

• Once a patch is released, test the patch for stability and applicabil-ity within your test environment prior deploying on your productionsystems.

• Check the community associated with your open source code. Thesecommunities typically have bug-tracking systems and mailing lists thatgive information on known security issues and provide the latest newsand exploits

116

Adhere to the organization’s system installation procedure wheninstalling or deploying OSS



• Only whitelisted OSS should be deployed in the organization.

• All deployments should be vetted and approved through a formal sys-tem deployment /change management process.

• All deployments should be inventoried in the asset register.

• Only authorized individuals such as the system administrators shouldinstall / deploy OSS.

• Prefer source code to binaries wherever possible. It is always good todownload the source code, verify against the MD5 checksums provided.Examples of trusted sites as recommended by Open Source Initiativeincludefreshmeat.net, sourceforge.net, osdir.com, developer.berlios.deand andbioinformatics.org

• Ensure that the OSS is tested and updated with the latest patches.

D.8.4 OSS Health

Security Guidelines regarding the maintainability and quality of OSS arepresented in this section.

Use OSS from reliable providers


Details:

• Look for companies or development teams that have a proven trackrecord of delivering quality products and issuing patches quickly oncea vulnerability has been discovered.

117

Follow Up Recommendations:

• Review its version history and look at previous security issues for redflags.

• Find out what software your partners, competitors, and establishedorganisations in your field are using.

• Case studies of trouble-free real-world deployments are of more valuethan claims of how many times a product has been downloaded.

• Look for a mature community with a clear policy about how contri-butions are evaluated and included, and a reliable regime in place tohandle errors or problems.

• Maintain a profile of each OSS to include the code’s origin, where toget updates, and how often the community releases new versions

• The quality of the product’s documentation is also an important con-sideration. How easy is it to find relevant information?

• Don’t get caught up with promises of lifetime support from a companyor project that has only been in existence for a few years; developmentand support can be discontinued at any time.

• Check if there are third-party specialists who can be hired to providesupport?

118

Appendix E

Pilot-test: Project X

Licensing

Evaluate the license of OSS

The license for OSS X usage reads as follows:

Copyright (c) 2016-2020,

Permission is hereby granted, free of charge, to any person

obtaining a copy of this software and associated documentation

files (the \Software"), to deal in the Software without restriction,

including without limitation the rights to use, copy, modify, merge,

publish, distribute, sublicense, and/or sell copies of the Software,

and to permit persons to whom the Software is furnished to do so,

subject to the following conditions:

The above copyright notice and this permission notice shall be

included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,

EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES

OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND

NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS

BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN

AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR

IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.

119

The license found is the MIT license which grants full permission to the userwhile also disclaiming any responsibilities. There is therefor no risks relatedto license in this case.

The license for OSS Y extension reads as follows:

Copyright (c) 2016-2018,

Permission is hereby granted, free of charge, to any person

obtaining a copy of this software and associated documentation

files (the \Software"), to deal in the Software without restriction,

including without limitation the rights to use, copy, modify, merge,

publish, distribute, sublicense, and/or sell copies of the Software,

and to permit persons to whom the Software is furnished to do so,

subject to the following conditions:

The above copyright notice and this permission notice shall be

included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,

EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES

OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND

NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS

BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN

AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR

IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.

The license grants full permission to the user while also disclaiming any re-sponsibilities. However, the license is issued 2016 and was only valid to 2018.This poses a risk that needs to be overseen. The issue was reported to theOSS Y community and permission was granted by one of their authors asshown in the snapshot below.

Software Security

Keep an inventory of open source components

• OSS X

120

Figure E.1: Author’s permission

• OSS Y

Look up vulnerabilities of OSS

OSS X

• NVD: No match

• Snyk: No match

• CVE: No match

• SonarQube: No match (except related to files outside of projectscope)

121

OSS Y

• NVD: No match

• Snyk: No match

• CVE: No match

• SonarQube: No match

Patch vulnerabilities:

None found and therefore nothing to be resolved.

Continuously monitor for new threats

Measure: Daily lookup of vulnerabilities for used OSS.

OSS Health

Use OSS from reliable providers:

OSS X health

Community:

• Size: OSS X is used by close to 3000 actors while 6600 has signaled aninterest of the software on GitHub. The project has forked 1200 times.

• Contributions: The project has 77 contributors over a 9 year pe-riod. At the day of investigation, there are 7 open issues reported ofwhich 2 concerns major changes that has been ongoing for over 2 years.Amongst the remaining 5 issues, the oldest reported issue has remainedunresolved for 29 days. The latest reported issue was reported 19 hoursago. There are currently 5 active pull-requests with lifetimes rangingfrom 60 days to 8 days.

• Maintenance response: The number of open issues and pull-requestsare stated above, and most of them has not been allowed to remainunattended for very long. Looking at the history, there are 2385 issues

122

that has been closed which could be an indicator of that the relativesmall number of issues currently opened also could be resolved in aacceptable timespan. In comparison to the 5 open pull-requests, thereare 279 closed requests.There are in total 5179 commits, where thelatest commit was performed 6 days ago.

• Code of conducte and information: The project rigorous informa-tion and documentation related to how to use the software and how tocontribute, partly through the project homepage ,and partly through README.md and CONTRIBUTING.md files. Thereis also licencing information in place. GitHub propose an explicit codeof conduct description, which is missing in the project however, this isdealt with through the README and CONTRIBUTING files. GitHubalso proposes an issue template for reporting issues. This is also miss-ing, however, there are templates for pull-requests.

Provider:

OSS X is provided by the , an independent non-profitorganization consisting of leadership and developers from the

. Funding for continued development and maintenance is founded byby the , and user support,education and new initiatives are supported by the

. There is no explicitly defined security policy in place,however, the nature of the provider and its supporters presents a trustworthyorganization. In addition to the OSS X software,are responsible for a number of application presented at their official website

.

Stability:

Generally, the reported issues now concerns minor changes related to styleand additional functionality that is not crucial for usability. As mentionedabove, there is ongoing work on the next version containing greater changes,however, it seems like this update is being conducted in a structured andwell thought through manner.

123

Code metrics:

SonarQube static code analysis was used to analyse the source code, whichgenerated the results shown in Figure E.2 and Figure E.3.

The found security vulnerability is related to a demo instance where smallchanges should be made to avoid phishing attacks. This is however notrelated to the software of interest. The security hotspots refers to sections ofcode that might be prone to vulnerabilities, however, no explicit vulnerabilitywas found and a developer review of these sections could be conducted to besure.

Figure E.2: Code metrics for OSS X

124

Figure E.3: Information on found vulnerability

Internal dependencies:

There are 743 different internal dependencies which is specified in the package-lock.json and package.json files of the project.

OSS Y

community:

• Size: OSS Y is used by 229 actors while 79 has signaled an interest inthe extension on GitHub. The repository has been forked 33 times.

• Contributions: The project has 11 contributors over a 5 year period.At the day of investigation, there are 16 open issues reported where theoldest remaining issue is five years old. There are few or no discussionsrelated to the raised issues and it seems like solutions to the issues mightnot be implemented soon, judging on the lack of active pull-requests.At the time of investigation, there are no active pull-requests.

• Maintenance response: The number of open issues and pull-requestsare stated above, and most of them does not seem to have raised a greatinterest from the community. Looking at the history, there are 40 issuesthat has been closed which could be an indicator of low communityactivity in general, and a relatively slow maintenance response. Thereare in total 257 commits, where the latest commit was performed 7

125

months ago, which is a clear indicator of slow maintenance responseand low community activity.

• Code of conduct and information: In comparison to OSS X thereare relatively sparse information on the project. There is a description,a README file and licence information, however, there are no codeof conduct, issue- or pull-request-templates or information on how tocontribute.

Provider:

The software provider is the research lab at. In addition to the OSS Y, the provider has created 38

different libraries of which a great part is extensions to OSS X.

Stability:

It is hard to assess the maturity of the software in this case. Since the workbeing conducted on the project seems to have stagnated, it might be thatthe final version has been established, and that the community has movedon to other projects.

126

Figure E.4: Code metrics for OSS Y

Code metrics:

SonarQube static code analysis was used to analyse the source code, whichgenerated the results shown in Figure E.4.

Internal dependencies:

There are 938 different internal dependencies which is specified in the package-lock.json and package.json files of the project.

127

TRITA-EECS-EX-2020:397

www.kth.se

security guidelines for the usage of open source software

Documents