managing information systems security: critical success factors and indicators to measure...

S.K. Katsikas et al. (Eds.): ISC 2006, LNCS 4176, pp. 530 – 545, 2006. © Springer-Verlag Berlin Heidelberg 2006

Managing Information Systems Security: Critical Success Factors and Indicators to

Measure Effectiveness

Jose M Torres, Jose M Sarriegi, Javier Santos, and Nicolás Serrano

Department of Industrial Management Engineering, TECNUN, University of Navarra, Paseo Manuel Lardizabal 13, 20018 San Sebastian, Spain

{jmtorres, jmsarriegi, jsantos, nserrano}@tecnun.es

Abstract. For how long can a business remain without its information systems? Current business goals and objectives highly depend on their availability. This highly dynamic and complex system must be properly secured and managed in order to ensure business survivability. However, the lack of a universally accepted information security critical factors’ taxonomy and indicators make security management of information systems (SMIS) a tough challenge. Effective information security management requires special focus on identifying the critical success factors (CSFs) when implementing and ensuring SMIS. The purpose of this paper is to share a group of 12 CSFs identified in the current information security literature as well as a set of 76 indicators which are easy to calculate and attempt to provide valuable information to organizations seeking information security level measurements.

Keywords: Information systems, security management, critical success factors and indicators.

1 Introduction

Organizations, regardless of their size, are adopting Information Systems (IS) at a fast tempo in order to be more competitive. They have realized all the advantages that IS interconnections bring to organizations. This new way of communicating and doing business has placed information as one of the most critical assets for the majority of today’s organizations.

The fast technology acquisition and IS “openness” increases systems’ complexity and dependency. These two factors plus the ever increasing interruptions of critical business systems, uninvestigated security incidents, gaps in user awareness and the sophistication of threats make current business reactive security strategies highly risky and irresponsible approaches [1].

Security management of information systems (SMIS) challenges cannot be addressed in isolation. However, the lack of a universally accepted information security (infosec) framework, theories or tendencies and the absence of ways to measure the effectiveness of implemented infosec controls restrain organizations from identifying the real mechanisms that control information security behaviors [2]. As a result, SMIS has been understood so far as the set of point solutions (patching and

Managing Information Systems Security 531

fixing breaches) instead of adopting more preventive and dynamic information security strategies.

Effective information security requires special focus on identifying the critical success factors (CSFs) for SMIS implementation and maintainability1. In addition, information security indicators2 should be implemented and analyzed in order to measure SMIS’ effectiveness and better allocate security resources. This way, organizations could align information security with business goals and improve future security strategies, investments, and policy’s enforcement.

Throughout this paper, the endlessly discussed but still vague information security definition is presented. However, the real purpose of this research is to point out on the one hand, which CSFs should be looked at in order to achieve robust and optimal SMIS’s design, implementation and maintainability tasks. On the other hand, share a set of 76 indicators, identified in the current infosec literature, which could provide relevant information for organizations seeking ways to measure SMIS effectiveness. Certainly, there could be more complex and accurate ways to measure information security. However, practitioners’ demand indicates the current need to find simple and fast ways to calculate organizational information security levels.

2 Information Security Definition

Information security is a concept that still lacks of unambiguous definitions. After many years of debates, information security has not found a worldwide definition. In fact, neither a widely accepted information security definition nor standardized critical success factors taxonomy exists [4].

Throughout the years, people and organizations responsible for critical business assets and business survivability have found different terms to make reference to the same activity: “protecting the organization’s critical assets”. In some cases, it is referred as information systems security, in others as digital security, or security itself. Information security regulation initiatives such as Basel II, Sarbanes-Oxley Act, and the Companies Act have defined “the need to protect information resources and prevent unauthorized access of the organization” as network security [5]. It can also be found as security management, business security, and the list goes on.

Information security, despite the variety of terms, has historically been defined as the process of ensuring information confidentiality, integrity and availability (CIA). This traditional way to understand information security helps us to handle this abstract, dynamic and complex phenomenon in a concrete way [6]. The recognized impact that human factors have on information security, along with new organizational structures which encourage employees’ self control and responsibility is only one aspect, out of many others, that must be considered when reformulating a complex definition such information security.

Information security experts have tried to extent the existing CIA definition. This extension includes four human-oriented security aspects which are responsibility, 1 The ease with which information systems can be modified to correct faults, improve per-

formance, or other attributes, or adapt to a changed environment [3]. 2 Taking measurements over time and comparing two or more measurements with predefined

baselines [6].

532 J.M. Torres et al.

integrity, trust and ethicality [7]. The combination of these four principles with CIA was one of the first indications that highlighted the need to find a wider and more accurate information security definition. Other authors have shared their viewpoint about information security. “The discipline responsible for protecting a company’s information assets against business risks” [8]. “The degree to which malicious harm is prevented, reduced and properly reacted to” [4]. “The contributor to strengthening the organization’s ability to adapt to new risk environments and accomplish its mission” [9]. Despite the correctness of these definitions, security professionals have to understand and agree on a worldwide definition in order to make information security to work [10].

After reviewing several definitions, one thing gets clear: information security is about technology, processes and people [11]. Therefore, we propose a definition which includes the following approaches: Information security is a well-informed sense of assurance that information risks and technical, formal and informal controls are in dynamic balance. Firstly, well-informed sense of assurance must be achieved because if there is not knowledge and practical assurance about the organization’s status, then information security gets very hard to accomplish [10]. Secondly, technical, formal and informal security controls (which are synonyms of technology, processes and people) must be implemented and managed since the absence of any of them also compromise information security [12,13,14]. Lastly, in dynamic balance makes reference to the fact that not only these controls must be implemented and managed, but also they must be equally and dynamically treated. In fact, it has been demonstrated how using the latest technology and having security policies and procedures worth nothing if they are not upgraded on a regular basis or if the human side of security is ignored [15]. The controls mentioned above are defined as follow:

• Technical controls: Hardware and software tools that restrict access to buildings, rooms, computer systems and programs in order to avoid unauthorized access or incorrect uses (antivirus, firewalls, IDS, backups, etc).

• Formal controls: Set of policies and procedures to establish and ensure effective use of technical controls. For example, identifying roles, responsibilities, implementing indicators and training employees.

• Informal controls: Interventions related to deploying digital information security through the workforce by enhancing users’ willpower and willingness.

The proposed information security definition should be seen as a first approach into motivating organizations, security experts and researchers to agree upon a unique definition. A worldwide definition could help us explore new perspectives and moving from technical approaches to more business, human and law oriented definitions. We have reached a time where engineers, economist, lawyers, and policymakers must try to forge common approaches in the name of information security management improvements [16].

3 SMIS Critical Success Factors

The following critical success factors (CSFs) for SMIS implementation and maintainability have been grouped based on already defined concepts:


• The CSFs have been grouped utilizing the technical, formal and informal approach discussed above. As it was stated in the definition, these three components work together as a whole and must be equally managed.

• As a result, they have been arranged based on the “Swiss cheese” model developed by James Reason where the holes on the cheese represent equipment failures, policy’s failures or human errors waiting to happen. According to his model, the cheese has several slices (defense in depth) and each slide on the cheese represents an obstacle or defense to protect the system. When the holes in the cheese line up “trajectory”, vulnerabilities are exposed and an incident occurs [17].

However, Reason’s approach was modified, from a multi-layer approach to a 3 dimensional cheese approach. Instead, of looking at the cheese slices as layers of security, each dimension of the cheese is seen as a security control. This 3D perspective highlights how by neglecting any of the three security controls, a failure could occur. Besides, the holes are constantly moving in all the three dimensions and if there is a trajectory in any of the three axes (neglecting a single control) an unwarranted event could happen [15].

��

��

��

��

��

��

��

��

!�"��

#��$��$��

��

��

��

��

Fig. 1. CSFs arrangement using a 3D version of Reason’s Swiss cheese model

The lack of information security data and empirical studies has forced us to identify these 12 CSFs based on revising published security experts’ perspectives. However, the 76 indicators have been designed by combining the information security literature review with the data from a current project that links university researchers, IS security experts and an engineering firm which aims to measure security effectiveness.

3.1 Technical Components

3.1.1 IS Security Architecture Defined as the way in which existent hardware and software business structures are introduced, arranged, protected and used.


Information security as well as IS reliability begins with an appropriate and robust IS security architecture design. The infrastructure that holds IS must be physically robust (indicators 1,2) and logically robust (indicators 3,4,5,6). Robust IS security architectures are also characterized by being highly dynamic (indicators 7,8) and organized (indicators 9, 10).

Table 1. IS Security Architecture

Indicator Formula Unit

1 % of securized areas (Σ of securized areas*100)/(Total areas defined

as secure) [%]

2 % of critical equipment with adequate physical protection

(Physically protected equipment*100)/(Critical business equipment)

[%]

3 % of secured configurations3

(Σ of successfully secured configurations*100)/(total configurations)

[%]

4 Σ of users with administrator passwords per workstation

Σ of users with administrator privileges to critical workstation(s)

[users/critical workstation (s)]

5 % of users with superuser

privileges (Σ of users with superuser privileges*100)/(Σ of

users) [%]

6 % of viruses and worms hits (Σ of hits*100)/( Σ of incoming viruses per year) [%] 7 Σ of architecture changes Σ of all per year [changes/year]

8 Σ of technical internal/external

audits Σ of all per year [audits/year]

9 % of software and hardware

classified4

(Σ of software&hardware classified*100)/(Σ of total software&hardware)

[%]

10 Σ of contracts with third parties

service suppliers (Σ of contracts*100)/(Total externalized services) [%]

3.1.2 Business Connections Defined as external and internal connections to the organization’s intranet or critical data.

Organizations have been approaching IS security as an external issue focusing on intrusions and connections from the outside. However, recent successful insider attacks experiences are changing the scenery making information security an internal as well as external business issue [18,19]. Security is not longer the perimeter [20] and therefore, it is vital to control remote accesses, business wireless equipment and their current security level (indicators 11,12).

The new way of doing business allows outsiders (users, clients, guests, suppliers, stakeholders, etc) access the organization’s intranet or critical data. These connections represent a great advantage for organizations production-wise. However, it only takes an organization’s guest connecting his/her laptop to the system, to lower your IS security level to a highly vulnerable state.

Unauthorized access testing and penetration assessments are countermeasures that should be implemented to ensure proper organizational information security from the technical point of view (indicator 13). If these connections are left unattended, then the business’ IS security level will be as good as the security level of the connected device of the outsider or insider [21].

3 Activating security features by default and ensuring robust security restrictions (i.e. robust

firewall configuration, Wi-Fi restrictions, etc). 4 To detect illegal downloaded software, verified licenses and control (if possible) external

hardware devices (pendrives).


Table 2. Business Connections

Indicator Formula Unit 11 Σ of remote accesses and

wireless devices Σ of accesses per month [accesses/month]

12 Average wireless devices upgrade date (laptops)

Σ (date of current upgrade (s)-date of last review (s))/(Σ of total wireless devices)

[days]

13 % of vulnerability and penetration assessments

conducted

(Σ of assessments conducted*100)/( Σ of assessment scheduled)

[%]

3.2 Formal Components

3.2.1 IS Security Strategy Defined as well-planned and structured SMIS improvement process. It includes having “clearly defined” SMIS plan of action’s goals, scope, resources, implementation team, their responsibilities and realistic completion times for goals set.

During the last decade, information security experts have provided organizations with several strategies to secure IS. Although they all try to achieve IS security, nearly all security experts’ strategies differ from each other. Some security experts recommend separating information security from information technology (IT) and integrate it with physical security [22]. Others highlight how security strategies based on deterrence, prevention, detection and response can be the difference between SMIS success and failure [23,24]. In addition, the information security ISO17799 recommends designing the security strategy aligned with business goals.

In order to achieve well-planned and structured security strategies, organizations need to compare their strategy with a universally accepted standard (indicator 14). Next, the strategy adopted must satisfy the organization’s security needs (indicators

Table 3. IS Security Strategy

Indicator Formula Unit 14 % of strategy robustness

5 (Σ of security actions achieved*100)/( Σ of total

actions recommend by an standard) [%]

15 % of outsourced infosec processes

(Σ of infosec processes outsourced*100)/( Σ of total business processes)

[%]

16 Σ of audits to the infosec outsourced firm(s)

Σ of audits per year [audits/year]

17 % of qualified IS staff6 (Qualified staff*100)/(Average IS staff during 1

year) [%]

18 Responsibility sharing7 (Σ of responsibilities assigned to a single staff

member*100)/(Σ of total security responsibilities) [%]

19 Project delays8 ((Completion hours-Estimated hours for phase

“n”)/project (s)) [hours behind

schedule/project (s)] 20 Evolution of infosec plan of

action (Σ of infosec activities from last year)-( Σ of

infosec activities from current year) [infosec activities]

5 Using a universally accepted standard such as ISO 17799 or CobiT (If certified = 100%). 6 Qualified person with one or more information security certificates. 7 In order to achieve responsibilities assignment balance (detecting or avoiding workloads). 8 The project should be divided in “n” phases in order to accomplish smaller goals and witness

progress.


15, 16). For example, depending on the business activity (financial vs. manufacturing) the needed strategy will vary (preventive, reactive or outsourcing). In fact, there are some information security studies that demonstrate how adopting a preventive security strategy is not the same as a corrective security strategy [25].

Proper allocation of information security human resources is the key to achieve robust information security strategies in the estimated completion time. Therefore, after evaluating the best strategy fit, organizations need to set and enforce realistic deadlines and have available qualified staff (indicators 17,18,19,20).

3.2.2 Dynamic Evaluation of Information Security Effectiveness Defined as continuous evaluation of the SMIS’ effectiveness: Understanding and managing the highly dynamic mechanisms that control information security behaviors.

Table 4. Dynamic Evaluation of Information Security Effectiveness

Indicator Formula Unit 21 Σ of internal and external systems

audits (Σ of audits per month) [audits/month]

22 % of daily monitorized processes (Σ of infosec processes monitorized daily*100)/(Σ of total business processes)

[%]

23 % of in house specialized staff dedicated to assessment of

infosec activities

(Σ of hours dedicated to evaluate monthly)/(Average available qualified staff

during 1 month)

[%]

24 Average time to respond to incidents

Σ (detection time (s)- response time (s) in days)/(Σ of total incidents)

[time/detected incident]

25 % of incidents stopped per month

(Σ of incidents stopped*100)/(Σ of total incidents detected per month)

[%]

26 Σ of monthly incident responses (Σ of incidents responses*100)/(Σ of total incidents detected per month)

[%]

27 % of monthly systems' performance and assurance

scheduled activities

(Σ of fixed IS anomalies*100)/(Σ of total detected anomalies per month)

[%]

28 % of nonconformity aspects fixed (found during audits activities)

(Σ of nonconformity aspects fixed*100)/(Σ of total nonconformity aspects detected)

[%]

29 % of maintenance processes executed

(Σ of maintenance process executed*100)/(Σ of total maintenance processes scheduled)

[%]

30 Σ of data recovery testing activities

(Σ of business data recovery testing activities) [activities]

Despite the powerful high-tech security countermeasures and the available process and procedure guidelines, organizations still fail to identify the real mechanisms that control information security behaviors. The need to analyze the dynamic aspects of information security is now in evidence and some studies have already shown results [18,19,26].

Any organization using a modern operating system has to actively manage this complex and dynamically changing environment (indicators 21,22). Information security has become a challenging system to manage and near impossible to predict since attacking-tools’ developers, hackers, crackers and insiders dictate the speed of IS’ insecurity. Dynamic evaluation of information security has become the only way to keep up with the pace at which threat sophistication is traveling (indicators 23,24,25,26,27,28,29,30).


3.2.3 Risk Assessment Process Defined as accurate identification, classification and prioritization of critical assets, vulnerabilities, threats, their impacts, and probability of happening.

Information security studies have shown that accurate risk assessment, data analysis and economic evaluations are hard to achieve because organizations find difficult to collect or keep track of information security indicators [27]. As a result, it becomes hard to correlate formal security structures and actual security behaviors.

Information security consultants and auditors have identified poor risk assessment processes as one of the most frequent reasons in SMIS implementation projects’ failures [28]. On the one hand, inaccurate risk assessment processes happen when organizations do not collect and analyze information security measurements (indicators 31,32,33,34). On the other hand, when the personnel involved in the process fail to identify critical assets, areas and threats (indicators 35,36,37,38,39) due to lack of knowledge about the organization’s security needs.

Table 5. Risk Assessment Process

Indicator Formula Unit 31 % of Risk Assessment (R.A)

automatization (Σ of automated R.A. tasks*100)/(Σ of total R.A.

tasks) [%]

32 Current level of risk by area9 Depends on the tools and methodology used [risk level]

33 % of countermeasures implemented

(Σ of implemented countermeasures*100)/(Σ of identified countermeasures)

[%]

34 Average risk assessment review time

(Time between consecutive reviews)/(Σ of reviews)

[days]

35 Σ of high-impact incidents on processes not contemplated in

previous R.A.

Σ of incidents not contemplated [incidents not contemplated]

36 Σ of critical assets Σ of critical assets [assets] 37 Σ of critical areas Σ of critical areas [areas] 38 Σ of identified potential threats Σ of potential threats [threats] 39 Σ of new threats identified (Σ of threats identified during previous revision)-

(Σ of threats identified during current revision) [threats]

3.2.4 Information Security Integration Defined as the connection between information security and the organizations’ core activities and processes with the purpose of aligning information security with business objectives.

The design of IS has been focused in enhancing organizations’ core competencies. Thus, it needs to be managed and secured like all the other critical business systems (indicator 40). SMIS relies on the interface between technology, policies-procedures and users. “If you think technology can solve your security problems, then you do not understand the problems and you do not understand the technology” [29]. Therefore, aligning end-users’ information security tasks with their professional goals is a reasonable solution for such problem (indicators 41,42).

Technology has allowed us to enjoy acceptable secure IT systems. However, if end-users ignore information security activities, the system can be left susceptible to breaches and failures (indicator 43). End-users are more likely to adopt guidelines and

9 Applications, operating systems, servers, etc.


procedures during the SMIS implementation process if information security activities contribute to fulfilling their daily duties more effectively. The suggested indicators allow organizations to evaluate the degree of alignment between protecting the organization’s core assets and processes and business objectives.

Table 6. Information Security Integration

Indicator Formula Unit 40 % of systems availability (Σ of hours available*100)/(hours expected to be

available) [%]

41 Σ of BSP incentives (Σ of best security practice incentives given/month)

[incentives/month]

42 Σ of protected files (Σ of files in the backup folder*100)/(Σ of critical files)

[%]

43 Σ of point solutions (patch, access controls, etc)

(Σ of infosec point solutions) [point solutions]

3.2.5 Project Accomplishment Defined as the degree on which starting information security strategic, operational and technical goals are met and enforced.

Several organizations have strictly followed proper SMIS implementation methodologies but have failed to protect their critical assets. The main cause of such failure is because the level of success, implementation-wise, at which the information security goals are reached, does not meet the high level of expectations set by management or consultants. In fact, it has been demonstrated how achieving fully accomplishment of previously stated SMIS implementation goals rarely happens [2].

A qualified, involved and motivated SMIS implementation team can be the difference between SMIS implementation’s failure and success (indicators 44,45,46,47). The leader of the implementation team should know the organization and possess legal knowledge (indicators 48,49). In addition, abilities such as leadership, accuracy when

Table 7. Project Accomplishment

Indicator Formula Unit 44 % of policies and procedures into

the design phase (not approved) (Σ of policies & procedures into design*100)/(Σ

of total p&p identified in a standard) [%]

45 % of policies and procedures documented and approved

(Σ of policies & procedures approved*100)/(Σ of total p&p identified in a standard)

[%]

46 (average) hours dedicated to policies and procedures design

(Hours dedicated to design)/(Average qualified staff during 1 year)

[hours/team member-year]

47 (average) hours dedicated to policies and procedures

implementation

(Hours dedicated to implement)/(Average qualified staff during 1 year)


48 (average) hours dedicated to policies and procedures reviews

and upgrading activities

(Hours dedicated to review and upgrade)/(Average qualified staff during 1 year)


49 Internal audits (Σ of internal audits perform*100)/(Σ of internal audits scheduled or planned)

[%]

50 Maturity level of current controls (Σ of incidents responses from current year)- (Σ of incidents responses from previous year)

[incidents responses]


estimating project costs, realistic evaluation of the “before and after” information security situation, downstream-upstream communication and time availability for information security activities, are key factors for successful SMIS implementation and maintainability (indicator 50).

3.2.6 Law Enforcement and Compliance Defined as the degree of enforcement and compliance of implemented information security controls. Externally done by regulatory institutions such as the Sarbanes Oxley Act and the Spanish LOPD10 and internally done through internal controls within the organization.

New regulations such as the Sarbanes-Oxley Act and the Spanish LOPD have had a visible impact on SMIS implementations. These regulations are slowly forcing organizations to increase their information security level. However, some organizations have taken these regulations as their security strategy by default. Adopting regulations as security strategy can lead organizations towards the compliance requirements of the moment leaving critical assets unattended. Regulations not only change security focus but also can cause costly and inefficient investments when organizations only consider assets and processes subjected to regulations [9].

At first, organizations should start by separating the operational part of information security from the compliance and enforcing part [30] in order to execute effective internal audits (indicator 51). Next, “no-compliance” severe sanctions should be applied to disobey departments within the organization (indicator 52). If so, they would not have a choice but to obey the information security practices implemented. Higher degrees of enforcement as well as no-compliance severe sanctions increase deterrence mechanisms that in the long run prevent incidents from happening [23].

Table 8. Law Enforcement and Compliance

Indicator Formula Unit 51 % of fulfilled regulations

11 (Σ of regulations fulfilled*100)/( Σ of regulations

enforced by authorities) [%]

52 % of penalties imposed to

users12

(Σ of penalties imposed*100)/( Σ of infosec bad practices detected)

[%]

3.2.7 Budget Defined as the percentage of IT economical resources dedicated to information security.

Organizations do not routinely require return of investment calculations on security investments since so far, information security activities have being treated as expenses [31]. Operational expenses (patch a breach) are usually easy to justify budget-wise but information security capital investments are not straightforward. As information security gets more expensive, infosec leaders are asked to show infosec budget allocations as well as cost-benefits analysis (indicators 53,54).

10 Ley Orgánica de Protección de Datos www.belt.es/legislacion/ 11 Confidential data only for organization internal purposes . 12 Confidential data only for organization internal purposes.


The lack of information security measurements and analysis makes the IT vs. security investments grow at disproportional rates. If the IT budget grows in isolation, then the budget needed to keep the system in a reliable state becomes highly expensive. Therefore, organizations need to track the evolution of both budgets (indicator 55). They should also perceive security expenses as an opportunity to improve upon IS availability, reputation and ability to accomplish business mission and ability to adapt to changing risk environments [9].

Table 9. Budget

Indicator Formula Unit 53 Security budget segregation

13 (Budget spent on the analyzed

area*100)/(Security budget) [%]

54 Σ of cost-benefit analysis (NPV,

IRR, ROSI, ALE, GLEIS)14

(Σ of cost-benefit analysis) [analysis/year]

55 Security budget evolution (Budget spent on IS security*100)/(IT budget every year)

[%]

3.3 Informal Components

3.3.1 Information Security Awareness Defined as the appreciation, at all levels within the organization, about the needs and benefits of information security.

Organizations have misplaced information security resources in the IT department and failed to identify people’s operational weaknesses as the root causes of the majority of information security breaches [9]. Information security must be much more than legislators, policies, and procedures [32]. It is a business problem and a people’s problem that requires active, involve, and aware users [8].

Information security awareness will receive more attention this year that any other because current insider threat are called to be the biggest information security threat during 2006. This upcoming year, attackers will not spend time looking for system’s vulnerabilities. Instead, they will focus on convincing employees to execute cyber attacks [33]. Therefore, developing information security awareness among the organization is no longer a choice but a necessity [34].

Security awareness is one of the few countermeasures capable to stop security incidents motivated by greed, economic and other personal problems [13]. By improving end-user behavior, organizations can minimize insider threat probability as well as approach security more as a business problem instead of a merely technical issue (indicators 56,57,58).

In the near future, role model organizations are going to be characterized by promoting ongoing users’ training and education programs in information security as well as effective communication of information security goals (indicator 59). They will also achieve robust SMIS (indicators 60,61) and fast incident responses (indicators 62,63).

13 Critical assets and critical areas first. 14 To compare the operational cost of security vs. the investment cost.


Table 10. Information Security Awareness

Indicator Formula Unit 56 Incidents reported per employee (Σ of incidents reported)/(Average employees

during 1 year) [incidents/user-year]

57 Average training hours received per year

(Hours dedicated to infosec training)/(Average staff during 1 year)

[hours/user-year]

58 Degree of awareness (by type of user: management, IT staff, end

users, etc)15

(Survey score*100)/(Optimal survey score) [%]

59 Σ of infosec-related reports, newsletters, memorandums sent

(Σ of memorandums sent per year) [memorandum/year]

60 % of incidents investigated (Σ of incidents investigated*100)/(Σ of detected incidents per year)

[%]

61 Certification status16

(Σ of hours needed to achieve certification) [hours]

62 Business critical data recovery

time17

Σ of hours needed to recover lost critical data or system functioning

[hours]

63 Average critical data recording

date18

(Time between consecutive recordings) [days or hours]

3.3.2 Information Security Awareness Defined as the degree of understanding and support from top management about the impact of information security on the business future and stakeholders.

Organizations’ dependency on IS, information security legal regulations and business competition are triggering top management commitment. Their support and understanding are critical because business decisions usually drive technical and operational decisions (indicators 64,65,66).

Table 11. Information Security Awareness

Indicator Formula Unit 64 % of SMIS policies approved (Σ of policies approved*100)/(Σ of policies

suggested per year) [%]

65 % of SMIS procedures approved (Σ of procedures approved*100)/(Σ of procedures suggested per year)

[%]

66 % of security budget spent on training

(Budget spent on training*100)/(Security budget) [%]

67 Downstream infosec communication

(Σ of meeting between infosec leaders&top management*100)/(Σ of meetings between

department leaders&top management per year)

[%]

68 Average training hours received per year

(Hours dedicated to training)/(Average staff during 1 year)

[hours/user-year]

69 % of satisfactory accomplishment per training activity

(Average of satisfaction*100)/(Σ of training activities per year)

[%]

70 % of infosec reports asked (Σ of infosec reports asked*100)/(Σ of business reports asked per year)

[%]

15 Done through an internal information security survey. 16 The accuracy of this indicator is no the final aim but to situate the business with respect to

others. Measured in hours or also in requirements needed to achieve certification. 17 The higher the IS dependency, the shorter the time. 18 The higher the IS dependency, the shorter the time.


Underestimating information security is not the only critical factor. The degree of information security top-management commitment varies across organizations’ size. Top management from large corporations has been practically forced to commit to information security due to companies’ fusions, external regulations and international business relations. However, in small and medium size enterprises (SMEs), management’s involvement tends to be much lower since they do not seen themselves as potential targets for cyber attackers [27].

In order to raise top-management commitment, regardless of enterprise size, information security must be directly related to business success. If the responsible for SMIS in the organization makes top-management understand that information security is the discipline which mitigates business risks, then effective security decisions will be made (indicators 67,68,69,70).

3.3.3 Administrators and End Users Competence Defined as IT knowledge and skills that can be used to properly utilize and secure IS but also used toward exploiting dishonest advantages through the use of IS.

Computing practices of system administrators and users continue to be one of the greatest information security challenges. People who administer IT systems sometimes practice insecure practices “shortcuts” claiming IS efficiency’s improvements or simply being helping certain end users. For example, a system administrator who changes a firewall rule, despite security rules and management approval, in order to help a remote user who has trouble sending email [35].

Between 80 and 90 % of organizational problems are due to human errors or bad practices [17]. Therefore, having honest, competent, smart and skillful systems administrators is a CSF for ensuring SMIS (indicators 71,72,73,74). It does not matter how IS are protected or the safety devices that have been settled, what is really important is who is using and defending the system [36].

End-users competence though, has the exact opposite effect on information security. Recent end user sophistication studies have demonstrated the strong

Table 12. Administrators and End Users Competence

Indicator Formula Unit 71 Staff responsible for infosec

training hours received per

year19

(Hours dedicated to training)/(Average infosec staff during 1 year)


72 % of qualified IS staff20

(Qualified staff*100)/(Average IS staff during 1 year)

[%]

73 % of satisfactory accomplishment per training activity

(Average of satisfaction*100)/(Σ of training activities per year)

[%]

74 Upstream infosec communication (Σ of meetings or reports with executive management per year)

[meetings/year]

75 % of reported incidents in users' PCs

(Σ of reported incidents in users’ PCs*100)/(Σ of total reported incidents per year)

[%]

76 Degree of organizational climate

satisfaction21

(Survey score*100)/(Optimal survey score) [%]

19 Due to threat sophistication and technology dynamism. 20 Person with one or more information security certificate. 21 Paying close attention to users’ behavior or attitude can be a powerful insider threat indicator.


relationship between end user sophistication and the potential of misusing IT systems [37]. These studies can be corroborated by the fact that, the majority of computer abuse comes from current employees who have managed to modify or swerve existing security controls [14]. Therefore, segregation of duties, enhancing incident reporting and monitoring users’ behaviors are effective countermeasures to protect business critical assets and areas (indicator 75,76).

4 Conclusions

The presented SMIS critical success factors and indicators are the result of a combination of current information security literature, security experts’ perspectives and an ongoing project which is trying to identify and simulate the security structure that generates current organizational security behaviors. The current standing information security situation indicates how the actual focus of information security research is not coinciding with the most critical factors and even worse there are not well-defined methods to measure information security.

Identifying information security CSFs and measuring the countermeasures’ effectiveness is a common goal for almost all organizations today. These two critical activities are, to a great extent, an internal activity for organizations. Therefore, as information systems become more complex and indispensables, getting feedback and measuring the level of information systems performance represents more than ever a business priority [6].

This work has allowed us to corroborate that current organizations struggle with the most basic but still unknown information security interrogates. How much security does my organization need? How much do we currently have? What to measure? And probably the most important question: How should we measure it? These questions must be properly answered in order to successfully implement, maintain and manage the security of information systems.

We predict that the future of information security will be based on the dynamic balance between technology (technical), processes (formal) and people (informal). The majority of the reviewed authors coincide in the fact that these 12 CSFs are going to be crucial to improve the overall organization’s critical assets protection. However, it is important to highlight that organizations should only choose the CSFs that better fit their security needs and implement the associated indicators. Although, these 12 CSFs are the most demanded by current information security practitioners, it is not strictly necessary to manage and implement all of them.

Achieving bulletproof information security is simply impossible or just too expensive. However, by implementing and analyzing these simple but helpful set of indicators, organizations will be able to easily witness their SMIS project’ evolution as well as measure its effectiveness. Only after that, security resources will be allocated and used more efficiently and accurate assets and economic evaluations will be achieved.


References

1. Ernst&Young.: Global Information Security Survey (2002) www.ey.com 2. Bjorck, F.: Institutional Theory: A New Perspective for Research into IS/IT Security in

Organizations. Proceedings of the 37th Hawaii International Conference on System Sciences. (2004)

3. Institute of Electrical and Electronics Engineers: IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York, NY (1990)

4. Firesmith, D.G.: Common Concepts Underlying Safety, Security and Survivability Engineering. December (2003) CMU/SEI-2003-TN-033.

5. Burling. M.: The key to compliance. www.net-security .org. 6. Kajava. J., Savola. R.: Towards Better Information Security Management by Under-

standing Security Metrics and Measuring Processes (2005) 7. Dhillon, G., Backhouse. J.: Information System Security Management in the New

Millennium. Communication of the ACM. July (2000) Vol. 43. No. 7 8. Von Sloms. S.H., Von Sloms. R.: From Information Security to…. Business Security?

Computer & Security (2005) Vol. 24 271-273 9. Caralli. R. A., Wilson. W. R.: The challenges of Security Management. Networked

Systems Survivability Program, SEI. 10. Anderson. James. M.: Why We Need a New Definition of Information Security. 11. Schneier. B.: Monthly Newsletter www.schneier.com 12. Dhillon. G.: Managing and Controlling Computer Misuse. Information Management &

Computer Security. 7/4 (1999) 171-175. 13. Dhillon. G.: Violating of Safeguards by Trusted Personal and Understanding Related

Information Security Concerns. Computer & Security Vol 20 No. 2 (2001) 165-172 14. Dhillon. G., Moores. S.: Computer crimes: Theorizing About the Enemy Within.

Computer & Security Vol 20 No. 8 (2001) 715-723 15. Torres. J. M. Sarriegui. J. M.: Dynamics Aspects of Security Management of Information

Systems. Proceedings of Systems Dynamic Society Conference, July (2003) Oxford, UK. 16. Anderson. R.: Why Information Security is Hard: An Economic Perspective (2001) 17. Reason. J.: Managing the Risk of Organizational Accidents. Hants, UK: Ashgate

Publishing Ltd, (1997) 18. Andersen. D., Cappelli. D., Gonzalez. J., Mojtahedzadeh. M., Moore. A., Rich. E.,

Sarriegui. J.M., Shimeall. T., Stanton. J., Weaver. E., Zagonel. A.: Preliminary System Dynamics Maps of the Insider Cyber-Threat Problem. Proceedings of System Dynamics Society Conference. Oxford, UK (2004)

19. Melara. C., Sarriegui. J.M., Gonzalez. J., Sawicka. A., Cooke. D.L.: A System Dynamics Model of an Insider Attack on an Information System. In: From Modeling to Managing Security: A System Dynamics Approach, Norwegian Academic Press Kristians, Norway 2003

20. Wilson. S.: The Future of Vulnerability Management: Information Security Bulletin, Vol 8 March (2003) 69.

21. Schneier. B.: Information Security Management. Conference in Bilbao, Spain (2005) 22. Berinato. S., Cosgrove. L.: Six Secrets of Highly Secure Organizations. CIO magazine,

Sep. 15 (2004) 23. Theoharidou. M., Karida. M., Kokolakis. S.: The Insider Threat to Information Systems

and the Effectiveness of ISO 17799. Computer&Security 24 (2005) 472-848 24. Parker. D.: Fighting Computer Crime. New York, NY. John Wiley & Sons (1998)


25. Torres. J. M., Sarriegui J. M., Santos. J.: Searching for Preventive-Corrective Security Balance. Proceedings of Systems Dynamic Society Conference, Boston U.S.A, July (2005)

26. Gonzalez. J., Rich. E.: Helping Prevent Information Security Risks in the Transition to Integrated Operations. Teletronikk 1, (2005)

27. Sarriegui. J. M., Eceiza. E., Torres. J. M. Santos. J.: Security Management of Information Systems Report (2005)

28. Bjorck. F.: Implementing Information Security Management System: Empirical Study of Critical Success Factors.

29. Schneier. B.: Applied Cryptography: Protocols, Algorithms and Source Code in C. New York: John Wiley& Sons, Inc., (1994)

30. Von Sloms. S.H.: Information Security Governance: compliance management vs. operational management. Computer&Security 24 (2005) 433-447

31. Gordon. L., Loeb. M.: Managing Cyber Security Resources. A cost-benefit analysis. New York, NY: McGraw-Hill (2006)

32. Schneier. B.: Beyond Fears. 1 edn. New York, NY: Copernicus Book (2003) 33. IBM Global Business Security index survey. Potential threats to information security

during 2006 (2005) 34. Mitnick. K.: The Art of Deception. Indianapolis, Indiana: John Wiley, Inc (2002) 35. Schultz. E.: The human Factor in Security. Computer&Security 24 (2005) 425-426 36. Schneier. B.: Managed Security Monitoring: Network Security for the 21st Century.

Computer and Security 20 (2001) 491-503 37. Magklaras. G.B., Furnell S.M.: A Preliminary Model of End User Sophistication for

Insider Threat Prediction in IT Systems. Computer&Security 24 (2005) 371-380

managing information systems security: critical success factors and indicators to measure...

Documents