service provider assessment framework - data security ... · dsci project advisory group ... hdfc...
TRANSCRIPT
Service Provider Assessment Framework A Platform for Building Synergies between Clients and Service Providers for Trusted Global Sourcing
A Study Report
Data Security Council of India in collaboration with Ernst & Young
December 2010
Under Cyber Security Awareness Program, Department of Information Technology, Government of India
Data Security Council of India (DSCI) is a section 25, not-for-profit company, setup by NASSCOM as an independent Self Regulatory Organization (SRO) to promote data protection, develop security and privacy codes & standards, and encourage the IT/BPO industry to implement the same.
For more information about DSCI or this report, please contact:
Data Security Council of IndiaNiryat Bhawan, 3rd Floor, Rao Tula Ram Marg, New Delhi – 110057, IndiaPhone: +91-11-26155070Fax: +91-11-26155072Email: [email protected]
© 2010 DSCI. All rights reserved.
About DSCI
DisclaimerThis document contains information that is Intellectual Property of DSCI. DSCI expressly disclaims to the maximum limit permissible by law, all warranties, express or implied, including, but not limiting to implied warranties of merchantability, fitness for a particular purpose and non-infringement. DSCI disclaims responsibility for any loss, injury, liability or damage of any kind resulting from and arising out of use of this material/information or part thereof. Views expressed herein are views of DSCI and/or its respective authors and should not be construed as legal advice or legal opinion. Further, the general availability of information or part thereof does not intend to constitute legal advice or to create a Lawyer/ Attorney-Client relationship, in any manner whatsoever.
Service Provider Assessment Framework
The IT (Amendment) Act, 2008 has established a strong data protection regime in the country, by requiring body corporates to implement ‘reasonable security practices’ to protect ‘sensitive personal information’. What is ‘reasonable security’ though? An organization is expected to have a comprehensive information security program, with appropriate controls that are commensurate with its information assets and risk assessment. In the event of a security breach, it should be able to demonstrate that its practices were in conformance with its written security policy, and that its controls were adequate. It is, however not that easy, since enterprises are outsourcing some of their work, and they must manage information risk across a vast global network of Service Providers. Outsourcing thus brings into focus the practices followed by Service Providers, and their accountability.
Service Providers are subjected to ongoing assessments and on-site audits, which are labor-intensive and costly for both the sides. Likewise, Service Providers with hundreds of Clients distributed in various geographies must submit themselves to several audits by the Clients. Moreover, the multiple assessments are based on different frameworks, questionnaires and audit approaches – clearly they result in wasted effort and time; and, of course, higher costs. It is the wish of both - Clients and Service Providers - that third-party evaluations that are standards-based, or framework-based, may ease the assessment burden. But how do they view the implementation of a standard, or best practices for security; and an assessment framework to validate that this has indeed made the organization secure? Again both of them will have a different perspective on this.
Can enterprises take a methodical approach to assessing and managing the risks through frameworks like ISO 27001; BITS Shared Assessment Program, Moody’s Vendor Information Risks ratings, Information Security Forum, COSO, NIST or COBIT? Will attestation of a Service Provider’s practices necessarily be in the form of a third-party certification, or a maturity rating of its practices?
Foreword
With DSCI best practices and data-centric methodology, we’ve rolled out a solution for adoption by Service Providers to make them secure. DSCI Security Framework (DSF©) is based on a number of security principles, that help make the security program of an organization dynamic, instead of a static checklist approach that relies on bulky documentation. We wanted to review the available assessment frameworks, to see how DSF© could fit into them, and how rating of practices may give a sense of security to organizations, and also show them the direction for improvements. In short, it’ll help realize an effective security program, and transparent assessment framework, that may address the concerns of both Clients and Service Providers. In the process, ‘reasonable security practices’ will get implemented.
It is with this in view that DSCI partnered with Ernst & Young Pvt. Ltd. (EY) in this study, which required extensive knowledge and experience in the domain, to review the existing frameworks and think through the advantages of certification/ratings. Survey of Clients and Service Providers, based on an in-depth questionnaire gives key pointers to the concerns of both the groups, and points towards a possible third-party ratings approach that may be useful and acceptable to both, namely Clients and Service Providers.
I would like to acknowledge the great team effort of DSCI and EY in conducting this study, and creating a useful analysis. I hope this report will generate sufficient interest among Clients, Service Providers, and even governments and regulators that will help DSCI arrive at the right decisions in taking the next steps in certification/rating of Service Providers.
Kamlesh BajajCEO, DSCI
Service Provider Assessment Framework
The study teamData Security Council of India
Mr. Vinayak Godse Director – Data ProtectionMr. Vikram Asnani Senior Consultant – Security PracticesMr. Rahul Jain Senior Consultant – Security Practices
Ernst & Young Pvt. Ltd.
Ms. Nity Singh Manager – Advisory ServicesMr. Taslimm Quraishi Manager – Advisory ServicesMr. Lalit Kalra Consultant – Advisory Services
DSCI Project Advisory GroupProf. N. Balakrishnan Chairman DSCI and Associate Director, Indian Institute
of Science (IISc), BangaloreMr. B.J. Srinath Senior Director, Indian Computer Emergency Response
Team (CERT-In)Prof. Anjali Kaushik Management Development Institute, GurgaonMr. Akhilesh Tuteja Executive Director, KPMGMr. Kartik Shahani Country Manager, India & SAARC, RSAMr. Satish Das CSO, CognizantMr. Baljinder Singh Global Head of Technology, Information Security &
Business Continuity, EXL Service (I) Pvt. Ltd.Mr. Vishal Salvi CISO, HDFC Bank Pvt. Ltd.Mr. Ashwani Tikoo CIO, Computer Sciences Corporation India Pvt. Ltd.Mr. PVS Murthy Global Head – Information Risk Management
Advisory, TCSMr. Deepak Rout CISO, UninorMs. Seema Bangera DGM – Information Security, Intelenet Global
Service Provider Assessment Framework
Executive summaryBusinesses today are global, complex and fast evolving, and technology has made business transactions independent of space and time. This has enabled businesses to focus on its core competencies and outsource non-core business operations to Service Providers, who are capable of providing services to the businesses from around the world round the clock. Information Security and Privacy becomes crucial when it comes to outsourcing as technology enables free flow of information across borders between Clients and Service Providers. This information could be business sensitive information and / or sensitive personal information of the Clients’ end customers including but not limited to health related information, credit card details, social security number, etc. Also, stringent global data protection regulations make the businesses liable for loss, misuse, wrongful disclosure of any personal information of any citizen – irrespective whether the failure is at outsourcers’ end or Service Provider’s end.
The Indian IT/BPO Service Providers are striving hard to ensure that security and privacy of data is well maintained. They follow stringent security controls specified by the Clients through contractual obligations. The Clients conduct regular Information Security and Privacy assessments of the Service Providers to ensure compliance with the contractual obligations and / or regulatory requirements or to simply assess the security posture of Service Providers. In this outsourcing ecosystem, many Clients have developed and applied their own proprietary assessment frameworks for evaluating their Service Providers. Service Providers, on the other hand, strain their resources to respond to diverse client information requests. This isolated approach proves to be an inefficient and costly affair, both for the Clients and the Service Providers. Inconsistencies arising from use of different assessment methodologies cause delays, resulting in inefficient use of time and resources. Aggravating the problem is the unavailability of generally accepted standard for Service Provider assessments. To overcome these issues and challenges, DSCI as an industry initiative seeks to establish a well defined Service Provider Assessment Framework in order to have a common assessment approach that can be used to assess different Service Providers.
This study especially through its survey attempts to understand the perspective of Client and Service Provider organizations with respect to Service Provider assessments and takes inputs to define a Service Provider Assessment Framework.
Service Provider Assessment Framework
The survey results reveal that:
DSCI should play a vital role in conducting Service Provider assessments and sharing •
the outcome in the ecosystem. It should:have an Service Provider assessment program that comprises of framework, processes, •
and methodology for assessmentsprovide an organization wide security and privacy maturity rating, and domain specific •
maturity rating that may be shared in the ecosystem after taking the due permission of the Service Providers
A new standard mapped to prevalent standards should be considered as a potential •
assessment standard for third party assessments of Service Providers
DSCI as an industry initiative and a Self Regulatory Organization having •
representation from both the Client and Service Provider organizations should empanel auditing firms for conducting independent third party assessments of Service Providers
The study also focused on understanding of various assessment models which included Malcolm Baldrige Framework, Capability Maturity Model Integration (CMMI), CRISIL Ratings, BITS framework, e-Sourcing Capability Model (eSCM), Moody’s assessment framework. The study of assessment models reveals that:
Service Provider Assessment Framework should be easy to comprehend and •
adaptable regardless of size of the organization and nature/ complexity of its processes
The framework assessment areas should be outlined in the form of best practices •
rather than a stringent set of controls. This would provide an opportunity to organizations for implementing / performing the control activities according to the needs of the organization specific environment
The framework should follow a process-approach and outline measurable •
assessment areas
It should be reviewed and updated (if required) on a periodic basis•
The maturity criteria should be transparent, and should help in assigning a formal •
maturity rating to a Service Provider
Overall, DSCI may develop a Service Provider Assessment Framework that is aligned to DSF© Best Practices & the maturity criteria defined for each of its sixteen security disciplines and the study results elucidated in this report; and make it popular in the ecosystem by performing pilot runs. The framework may follow a CMMI-like rating methodology which is assessment of the security and privacy practices at both the layers – capability/ maturity of the business processes, and maturity of the organization.
Introduction .................................................................................................................. 1
Survey Highlights......................................................................................................... 5
Detailed Survey Results ...............................................................................................7
Key drivers for Service Provider assessments ....................................................7
Scale of Service Provider assessments ..............................................................9
Current assessment program/ mechanism .......................................................11
Focus on Data Privacy in Service Provider assessments .................................13
Types of Service Provider assessments ...........................................................14
Level of perceived risk – IT services .................................................................15
Level of perceived risk – BPO services .............................................................17
Risk profiling of Service Providers .....................................................................18
Frequency of Service Provider assessments ....................................................19
Budget and cost for Service Provider assessments ..........................................21
Modes of Service Provider assessments ..........................................................23
Service Provider assessment challenges ..........................................................25
Service Provider assessments – solutions and future landscape .....................27
Influence of IT (Amendment) Act, 2008 on Service Provider assessments ......29
Third party assessments ...................................................................................31
Third party assessors ........................................................................................33
Standards for Service Provider assessments ...................................................35
Role of DSCI in Service Provider assessments ................................................37
Outcome of Service Provider assessments ......................................................39
Sharing of Service Provider assessment results ...............................................41
Recommendations......................................................................................................43
Annexure ....................................................................................................................45
Glossary .....................................................................................................................57
References .................................................................................................................57
Content
1
Service Provider Assessment Framework
BackgroundAs buyers of Information Technology (IT) and Business Process Outsourcing (BPO) services become increasingly sophisticated and demanding, Service Providers are challenged to achieve new levels of efficiency, agility and transparency in service delivery and protection of information. Clients increasingly expect real evidence of robust process management, continuous improvement, effective governance, and measures adopted for ensuring Information Security and Privacy.
ObjectiveDSCI engaged EY to study the current landscape of Service Provider (IT/BPO organizations) assessments conducted by the Client organizations, and assist in documenting the assessment approach that may be adopted in order to minimize the challenges of both, Client and Service Provider organizations, with an intent of evaluating and reporting on Information Security and Privacy posture of the Service Providers.
ApproachIn order to achieve the project objectives, the joint study team undertook the following steps:
Primary research• : A survey of Client and Service Provider organizations was undertaken to gain an insight into the current Service Provider assessment program. The survey covered the following aspects:
Business drivers for Client organizations to conduct Service •
Provider Assessments
Introduction
2
The value that various Service Provider assessments conducted by •
Client organizations bring to the Service Providers
Investments made, and challenges faced by the Service Provider •
and Client organizations in driving such assessments
Possible solutions for overcoming the current challenges•
Role of DSCI and third parties in Service Provider assessments•
Secondary research• : A study was undertaken to document the pros and cons of prevalent assessment frameworks like Capability Maturity Model Integration (CMMI), BITS shared assessment program, Carnegie Mellon University e-Sourcing Capability Model (eSCM), etc. The list of assessment frameworks was documented on the basis their widespread use, and international recognition in performing assessments. The study areas included the following:
Assessment areas / ease of use by the organization being •
assessed
Assessment methodology / scoring pattern / process of sharing •
assessment results
Acceptability / popularity of the framework•
Independence of examiners•
Frequency of framework update to cater to future requirements•
The team also studied DSCI Security Framework (DSF©) Best Practices and maturity rating criteria for each of its sixteen disciplines to gather inputs (in addition to the inputs provided by primary and secondary research) for defining Service Provider Assessment Framework.
3
Service Provider Assessment Framework
Profile of participantsThe survey respondents were a set of Client and Service Provider organizations. The respondents were majorly from Information Technology (IT), Business Process Outsourcing (BPO), Telecommunications and Financial Services verticals. Correspondingly, the survey results have been divided into two perspectives – Clients’ perspective and Service Providers’ perspective, and may be read accordingly.
Industry wise distribution
KPOBPO IT Services
42.00%50.00%
8.00%
Client organizations
Service Provider organizations
37.00%
18.00%
36.00%
9.00%
Industry wise distribution
Telecommunication BankingTechnology Financial Services
4
Service Provider Assessment Framework
6$ 1 billion to $ 9 billion
4$ 100 million to $ 249 million
1$ 10 billion to $ 24 billion
1Less than $ 100 million
Number of Service Providers
Number of Client organizations
3
3
1
2
More than $ 24 billion
$ 1 billion to $ 24 billion
$ 100 million to $ 249 million
Less than $ 100 million
Client organizations
Service Provider organizations
The sample size selected for the survey was limited and this should be taken into consideration when interpreting the survey results.
5
Service Provider Assessment Framework
Service Provider assessments are conducted by Client organizations •
in order to protect business sensitive information, and mitigate security & privacy risks while outsourcing work to Indian IT/ BPO companies. These assessments help Service Provider organizations to align security & privacy initiatives to their Client’s requirements and build on the existing relationship with the Clients
Comprehensive risk based assessments covering all the domains of •
security are carried out annually by majority of Client organizations. Vulnerability assessments and penetration testing continue to display strong acceptance (100%) by Client organizations in Service Provider assessment programs
Most of the Service Provider organizations reported that ISO 27001 •
controls checklist is used as a mechanism by their Clients for conducting assessments. On the other hand, Client organizations revealed that a proprietary Service Provider assessment program has been developed to conduct Service Provider assessments
Provisions of the IT (Amendment) Act, 2008 (ITAA 2008) need to be •
appropriately incorporated in the Client-Service Provider contracts
High number of assessments around the year is the most critical •
challenge faced by Service Providers at the time of assessments, followed by meeting diverse & varied assessment. Whereas for Clients, rising legal liabilities, regulatory requirements, level of security awareness in the Service Providers, ensuring compliance by Service Providers, and Service Provider commitment to ensure Information Security & Privacy are some of the critical challenges faced in assessing Service Providers
Survey highlights
6
Currently, Service Provider assessments are majorly conducted •
onsite by Client’s internal staff. Majority of the Client organizations indicated that auditing firms empanelled by a joint industry consortium of outsourcers and the Service Providers could act as the third party assessors for conducting independent Service Provider assessments
More than half of the Service Provider respondents suggested that •
DSCI should have a Service Provider assessment program that comprises of framework, processes, methodology for assessments
Clients and Service Providers reveal that third parties should conduct •
Service Provider assessments, based on a standardized assessment methodology. This would save costs and efforts by avoiding the need for conducting assessments of multiples Service Providers
Both Client and Service Provider respondents suggested a new •
standard mapped to ISO 27001, NIST –SP, COBIT, ITIL etc. that meets all the regulatory requirements like GLBA, HIPAA, PCI DSS etc., as a potential assessment standard for third party assessments of Service Providers
DSCI should provide organization wide security and privacy maturity •
rating, and also domain specific maturity rating
7
Service Provider Assessment Framework
Key drivers for Service Provider assessmentsThe survey results reflect that majority of the Clients consider protecting business sensitive information, and mitigating security & privacy risks as the critical business drivers for conducting Service Provider assessments. On the other hand, Service Providers report that Client’s corporate policy requirements, and achieving end customer confidence are the main reasons which drive their Clients to conduct assessments.
Detailed survey results
Clients’ perspective
Business drivers for conducting Service Provider assessments
88.89%
88.89%
77.78%
77.78%
55.55%
44.44%
33.33%
Protecting business sensitive information includingintellectual property
Mitigating security and privacy risks that exist in outsourcing arrangements
To address the security and privacy concerns of some of the key stakeholders within our organization
Strengthening of data protection regime in the geographies where we operate, stipulating stringent
requirements and heavy fines for a data breach
Use Service Provider assessments as a mechanism to foster a culture of compliance at all Service Providers and
introducing a sense of competition among them with regardsto fulfillment of their data security and data privacy needs
Addressing security and privacy risks that arise from use of emerging technologies
Data protection regulations demand ourorganization to undertake regular assessments of
third parties
44.44%Our corporate policies require us to undertake a
comprehensive vendor risk assessment
Achieving end customer confidence and preventing loss of reputation by mitigating risks of privacy/
information leakage that may arise at Service Provider end
55.55%
8
‘Service Provider assessment as a mechanism to foster a culture of compliance’ was selected by the least number (thirty three percent) of Clients while the same response was selected by fifty percent of the Service Provider organizations, as a reason for conducting assessments.
Reasons that drive Clients to conduct Service Provider assessments
66.67%
66.67%
58.33%
58.33%
50.00%
50.00%
41.67%
41.67%To address the security and privacy concerns of
some of the key stakeholders in Client organization
Protecting business sensitive information includingintellectual property
Clients use Service Provider assessments as a mechanismto foster a culture of compliance at all its Service Providers
and introducing a sense of competition among them with regards to fulfillment of their data security and data privacy needs
Clients’ corporate policies require them to undertake a comprehensive vendor risk assessment
Achieving end customer confidence and preventing loss of reputation by mitigating risks of privacy/
information leakage that may arise at Service Provider end
Mitigating security and privacy risks that exist in outsourcing arrangements
Data protection regulations demand Client organization to undertake regular assessments of third parties.
Strengthening of data protection regime in the Client geographies that stipulate stringent
requirements and heavy fines for a data breach
Service Providers’ perspective
Protecting business sensitive information and mitigating security and privacy risks are the major drivers for conducting Service Provider assessments
9
Service Provider Assessment Framework
Scale of Service Provider assessmentsThe survey results show that the number of Service Provider assessments is directly proportional to the number of Clients or Service Providers that an organization is engaged with. This is proven by the fact that Clients working with 500 Service Providers conduct more than 100 Service Provider assessments annually, and those with 200 & 300 Service Providers conduct 10-50 and 50-100 Service Provider assessments respectively. Also, Service Providers engaged with 800 Clients undergo 100-200 assessments annually, and those with 700 & 600 Clients undergo 50-100 assessments respectively 50-100 assessments respectively.
Number of Service Providers the organization is engaged with
0
100
200
300
400
500
600
1 2 3 4 5 6 7 8 9
Num
ber o
f S
evic
e P
rovi
ders
Clients
11.11%0-5
22.22%5-10
44.44%10-50
Number of Service Provider assessments conducted
11.11%
11.11%Above 100
Num
ber o
f an
nual
ass
essm
ents
50-100
Clients’ perspective
10
Service Provider Assessment Framework
Service Providers’ perspective
Number of Clients serviced by the organization
0100200300400500600700800900
1 2 3 4 5 6 7 8 9 10 11 12
Num
ber o
f Clie
nts
Service Providers
Number of Service Provider assessments faced
9.09%
Num
ber o
f ann
ual a
sses
smen
ts
36.36%50-100
Above 400
27.27%200-400
9.09%100-200
18.18%10-50
0.00%0-10
11
Service Provider Assessment Framework
Current assessment program/ mechanism Proprietary Service Provider assessments followed by SAS 70 and ISO 27001 checklist are the most commonly adopted assessment programs/ mechanisms by Client organizations.
On the other hand, more than ninety percent Service Providers reported that their Clients use ISO 27001 checklist for conducting assessments. This is closely followed by proprietary assessment programs and assessment programs of Client appointed external auditors (sixty seven percent each).
The survey further revealed that majority of the Client organizations do not consider ISO 27001 certification as an alternative to conducting Service Provider assessments.
Interestingly, the survey also highlighted that BITS Shared Assessment Program is not used by any of the Client organizations for conducting Service Provider assessments.
77.78%
44.44%
44.44%
33.33%
22.22%
22.22%
11.11%
0.00%
ISO 27001 controls checklist
BITS shared assessment program
Assessment program developed by ourorganization (proprietary)
Reliance on Statement on Auditing Standards(SAS) No. 70 report provided by the auditing
firm assessing your Service Providers
Asking the Service Providers to get ISO 27001 certified thereby eliminating the need for
getting assessed
Use pre-defined controls list provided by anassessment tool
Asking the Service Providers to provide self declaration/attestation for compliance to our
security policies/requirements
Assessment program of the appointed external auditor
Service Provider assessment program/mechanism used by the organization
Clients’ perspective
12
Service Provider Assessment Framework
78% Client organizations use proprietary assessment programs for conducting Service Provider assessments. However, the Service Providers report that their Clients use ISO 27001 checklist for conducting security and privacy assessments
Programs / mechanisms used by Clients for conducting assessments
91.67%
66.67%
66.67%
41.67%
25.00%
16.67%
16.67%
0.00%
ISO 27001 controls checklist
BITS shared assessment program
Others
Use pre-defined controls list provided byan assessment tool
Providing self declaration / attestation for compliance to client security policies/
requirements
Getting ISO 27001 certification eliminates the need for getting assessed
Assessment program of the client appointed external auditor
Assessment program developed by the client (proprietary)
Service Providers’ perspective
13
Service Provider Assessment Framework
Focus on Data Privacy in Service Provider assessmentsThe survey reveals that majority of the Client organizations cover privacy during Service Provider assessments. Contrastingly, majority of the Service Providers report that privacy is not covered as part of the assessments.
Eleven percent of the Client organizations also revealed that privacy is not covered as part of Service Provider assessments. Also, Client organizations seem to be satisfied with the current focus on privacy as no Clients foresee the need for a change in the privacy focus in Service Provider assessment programs.
Privacy is not covered
11.00%
56.00%33.00%
Coverage of privacy in Service Provider assessments
StronglyModerately Needs improvement (0%)
Coverage of privacy in Service Provider assessments
41.67%
25.00%
33.33%
Minority of clients’ Service Providers assessment programs cover Privacy
Majority of clients’ Service Providersassessment programs cover Privacy
Nearly half of the clients assessment programs cover Privacy
None of the clients’ Service Provider assessment programs cover Privacy (0%)
Clients’ perspective
Service Providers’ perspective
Majority of the Service Providers report that their Clients do not cover Privacy during assessments while Clients strongly perpetuate the coverage of Privacy in Service Provider assessments
14
Service Provider Assessment Framework
Clients’ perspective
Service Providers’ perspective
Types of Service Provider assessmentsVulnerability Assessment and Penetration Testing as a methodology of Service Provider assessments has a strong acceptance (100%) from Client organizations.
While only twenty five percent of Service Providers reveal line of service specific assessments is considered important by their Clients, Client organizations give more importance to these assessments.
Service Providers reveal that Client organizations display a strong propensity towards undertaking comprehensive risk-based assessments, and compliance based assessments
Different types of Service Provider assessments conducted by the organization
100.00%
88.89%
77.78%
77.78%
Risk based assessments
Lines of Service specific assessment (e.g.conducting application security assessment for
application development services)
Technical: vulnerability assessment and penetration testing
Regulatory / Compliance: Assessments to check compliance with applicable regulations (e.g. HIPAA,
GLBA) or Assessments based on compliance to Standards like ISO 27001 and PCI DSS
Different types of assessments conducted by Client organizations
100.00%
83.33%
75.00%
75.00%
25.00%
Comprehensive risk based assessmentcovering all the domains of security
Assessment based on well-knownstandards like ISO 27001
Comprehensive compliance basedassessment
Line of Service specific assessment (e.g.conducting application security assessment for
application development services)
Technical assessment of the IT systems including vulnerability assessment and penetration testing
15
Service Provider Assessment Framework
Level of perceived risk – IT servicesResults indicate that Client organizations perceive that outsourcing Custom Application Development services (seventy eight percent) involves high risk. This is distantly followed by Infrastructure, Network and Desktop Outsourcing and Software Deployment and Support at sixty seven percent each.
Service Providers cited Infrastructure Outsourcing followed by Network and Desktop Outsourcing as the critical risk areas for Service Provider assessments.
Clients as well as Service Provider organizations do not attach importance to IT Education and training services for assessments.
33.33% 11.11% 33.33%Hardware deployment and support
Level of perceived risks in the services outsourced by Client organizations: IT services
High Medium Low
77.78% 11.11%Custom application development
55.56% 33.33%Application management
66.67% 11.11% 11.11%Infrastructure services outsourcing
66.67% 11.11% 11.11%Software deployment and support
44.44% 22.22% 22.22%System integration
44.44% 44.44%Software testing
66.67% 22.22%Network and desktop outsourcing
44.44% 33.33%Hosted application management
44.44% 33.33%Hosted infrastructure services
33.33% 22.22% 22.22%Network consulting and integration
11.11% 22.22% 44.44%IT education and training
22.22% 22.22% 33.33%IT consulting
Custom Application Development, Network and Desktop Outsourcing together with Infrastructure outsourcing are current watchwords in the context of Service Provider assessments
Clients’ perspective
16
Service Provider Assessment Framework
41.67%
33.33%
25.00%
25.00%
25.00%
16.67%
8.33%
8.33%
8.33%
16.67%
8.33%
33.33%
8.33%
16.67%
8.33%
Infrastructure services outsourcing
Level of perceived risks in the services outsourced by Client organizations: IT services
Network and desktop outsourcing
Application management
Hosted application management
Hosted infrastructure services
System integration
Software testing
Custom application development
8.33% 16.67%Software deployment and support
8.33% 16.67%Hardware deployment and support
16.67% 8.33%Network consulting and integration
8.33% 16.67%IT education and training
16.67% 8.33%IT consulting
High Medium Low
Service Providers’ perspective
17
Service Provider Assessment Framework
Level of perceived risk – BPO servicesThe survey results indicate that sixty seven percent of Client organizations and forty two percent of Service Provider organizations consider that Finance and Accounting services involve high risk.
66.67% 11.11%Finance and accounting
44.44% 22.22%Customer interaction and support
44.44% 33.33% 11.11%Human resource management
44.44% 22.22% 11.11%Knowledge services
44.44% 22.22% 11.11%Vertical specific BPO services
22.22% 33.33% 11.11%Procurement services
High Medium
Level of perceived risks in the service outsourced by Client organizations: BPO services
Low
41.67%
25.00%
25.00%
25.00%
16.67%
8.33%
8.33%
8.33%
16.67%
16.67%
8.33%
8.33%
8.33%
8.33%
Finance and accounting
Level of perceived risks in the service outsourced by Client organizations: BPO services
Customer interaction and support
Human resource management
Knowledge services
Vertical specific BPO services
Procurement services
High Medium Low
Clients’ perspective
Service Providers’ perspective
Finance and Accounting services are considered important by majority of the organizations in the context of Service Provider assessments
18
Service Provider Assessment Framework
Clients’ perspective
Risk profiling of Service ProvidersThe growing awareness of the risk management in the Indian IT/BPO industry was clearly evident from the survey, which displayed that almost ninety percent of the Client organizations undertake a risk profiling for their Service Providers.
The survey results also emphasize the importance of Information Security and Privacy with nature and criticality of the business outsourced along with sensitivity of the data exported to Service Providers being given the most important criterion for risk profiling.
Undertake risk profilingDo not undertake risk profiling
11.00%
89.00%
88.89%
88.89%
88.89%
66.67%
55.56%
44.44%
44.44%
Sensitivity of data exported to the Service Providers
Type of connectivity with the Service Providers
Dependency on the Service Providers
Size and maturity of the Service Providers
ISMS certification achieved by the Service Providers
Nature & criticality of the business/services outsourced
Security incidents/breaches in the past
Criteria used for risk profiling of Service Providers
89% of the Client organizations rely on risk profiling to determine the frequency of Service Provider assessments
19
Service Provider Assessment Framework
Frequency of Service Provider assessmentsThe fact established in the previous question gets reestablished by the frequency of Service Provider assessments undertaken by Client organizations that perform risk profiling; the survey results show that the Service Providers identified under critical risk category undergo quarterly assessments.
A similar trend is observed for the Service Providers identified under ‘Medium’ and ‘Low’ risk categories, undergoing half yearly and yearly assessments by fifty six and forty five percent respondents respectively.
Organizations that do not undertake risk profiling, yearly assessments are preferred by almost twenty three percent of the organizations. Also eleven percent of Client organizations believe that the frequency depends on the trust and relationship between Client and Service Providers.
Frequency of assessing the Service Providers
33.33%
22.22%
11.11%
22.22%
55.56%
22.22%
11.11%
22.22%
22.22%
11.11%
44.44%
33.33%
Critical risk
High risk
Medium risk
Low risk
Negligible
Quarterly Half yearly Yearly
Clients’ perspectiveThe Service Providers with critical risk undergo quarterly assessments as per thirty three percent of Client organizations
21
Service Provider Assessment Framework
Clients’ perspectiveThe cost of periodic Service Provider assessments is built into the service delivery cost of Service Providers, and is a part of the contractual terms
Budget and cost for Service Provider assessmentsThis question was aimed at identifying the cost impact of Service Provider assessments on Clients and Service Providers.
Results highlight that majority of the Client organizations allocate only a small portion of IT security budget for Service Provider assessments. Only one of the respondents indicated that the organization allocates significant portion of IT security budget for Service Provider assessments. On the other hand, majority of the Service Providers allocate a considerable portion of the IT security budget towards assessments. This is because the cost for periodic Service Provider assessments is built into the service delivery cost of Service Providers and is part of the contract with the Service Providers.
Service Provider respondents in the ‘Others’ category indicated that cost of the assessment could be borne by either party, and it depends on the relationship and understanding between the Client and the Service Provider.
Portion of the IT security budget allocated for conducting Service Provider assessments
44.44%
22.22%
22.22%
11.11%
Small
Considerable
Negligible
Significant
Cost of Service Provider assessments
55.56%
44.44%
22.22%
22.22%
11.11%
11.11%
11.11%The cost is borne at the time of the Service Provider assessments by the Service Provider
Efforts spent by the Service Provider resources in coordinating / facing the assessments are
billed by the Service Providers
Significant cost of the Service Provider assessmentscomprises of overhead expenses like travel, etc
for our assessors
The cost is borne at the time of the ServiceProvider assessments and is shared between
Client and Service Provider as per the contract
The cost is borne at the time of the Service Provider assessments by the Client
We allocate a portion of our IT security budget for conducting Service Provider assessments
The cost for periodic Service Provider assessments is a part of the contract
22
Service Provider Assessment Framework
Portion of IT security budget allocated for facing assessments
66.67%
25.00%
8.33%
0.00%
Considerable
Small
Significant
Negligible
Cost of Service Provider assessments
66.67%
33.33%
33.33%
25.00%
16.67%
8.33%
8.33%
16.67%Others
The cost for periodic Service Provider assessments is apart of the contract
The cost is borne at the time of the Service Providerassessments by the Client
Efforts spent on Service Provider assessments is billed to the clients
The cost is borne at the time of the Service Providerassessments and is shared between Client and Service
Provider as per the contract
The cost is borne at the time of the Service Providerassessments by the Service Provider
Significant cost of the assessments comprises ofoverhead expenses like travel,and stay arrangements
for clients and/ or their sourced assessors We allocate a portion of our IT security budget for
Service Provider assessments
Service Providers’ perspective
23
Service Provider Assessment Framework
Modes of Service Provider assessmentsClient organizations prefer conducting onsite assessments post a self assessment by the Service Provider organizations either by their internal staff or by sourced assessors.
The survey results highlight that higher the risk perceived during risk profiling, more is the focus on assessments. Majority of the respondents conduct onsite assessments for critical, high and medium risk Service Providers. For low risk category of Service Providers, majority of the Client organizations adopt offshore self assessments.
Client organizations that do not perform the risk profiling of their Service Providers prefer to undertake onsite assessment by sourced assessors from auditing firms.
Type Self Assessment
(offshore)
Telephonic (offshore)
Onsite by Org Internal
staff
Onsite by org internal staff and sourced assessors
from auditing firms
Onsite by sourced
assessors
Third Party AssessmentsCategory
Critical risk 2 2 6 4 3 3
High risk 2 2 6 3 4 3
Medium risk 4 2 6 3 1 2
Low risk 5 3 4 1 0 0
Negligible 3 2 4 1 0 0
Clients’ perspective*For Client organizations that undertake risk profiling of Service Providers
*This data table is for eight Clients. Eight out of nine Clients interviewed undertake risk profiling.
24
Service Provider Assessment Framework
*This data table is for eight Clients. Eight out of nine Clients interviewed undertake risk profiling.
Modes adopted by Clients for conducting Service Provider assessments
100.00%
75.00%
66.67%
66.67%
25.00%
25.00%
8.33%
8.33%
Onsite assessments are conducted by client’s internal staff
Onsite assessments are conducted by sourced assessors
Onsite assessments are conducted by an independent Third party
Telephonic assessments are conducted by sourced assessors hired by client organization
Telephonic assessments are conducted by client’s internal staff
Self assessment questionnaire are provided in an assessment tool available online; we directly
upload our responses and evidences inthe tool without any intervention of the client
Onsite assessments jointly conducted by sourced assessors and client’s internal staff
Self assessment questionnaire are sent through email and we revert with the filled questionnaire and
evidences without any intervention of the client
Service Providers’ perspective
Onsite assessments by Client’s internal staff or sourced assessors is the preferred mode of assessment by Clients
25
Service Provider Assessment Framework
Service Provider assessment challengesThe survey results provide insight into the factors that influence Information Security and Privacy assessments in IT/BPO organizations.
Subcontracting by Service Providers and comfort provided by certifications like ISO 27001 are the critical challenges faced by Client organizations in assessing Service Providers on Information Security & Privacy according to forty four percent of Client organizations. This is one of the reasons why Client organizations do not consider ISO 27001 certification as an alternative to Service Provider assessments.
44.44%
44.44%
33.33%
22.22%
22.22%
22.22%
22.22%
22.22%
11.11%
11.11%
11.11%
11.11%
11.11%
11.11%
11.11%
33.33%
11.11%
33.33%
44.44%
33.33%
55.56%
22.22%
55.56%
44.44%
22.22%
22.22%
22.22%
22.22%
33.33%
55.56%
44.44%
44.44%
11.11%
44.44%
22.22%
11.11%
33.33%
55.56%
11.11%
22.22%
55.56%
55.56%
44.44%
55.56%
44.44%
11.11%
44.44%
44.44%
Comfort/ assurance provided by certifications like ISO 27001
Challenges faced by Client organizations
Subcontracting by the Service Providers
Inadequate budget
Auditor accreditation and Auditors’ management
Service Provider commitment
Meeting multiple customer requirements
Quantum of assessments
Rising legal liabilities/regulatory requirements
Level of security awareness in the Service Providers
Ensuring compliance by your Service Provider
Sensitizing key resources of Service Providers
High direct and indirect costs
Nature of outsourced work
Tracking and closure of assessment findings
High Medium Low
Adoption of Non standardized Information Security and Privacy framework
Availability of skilled resources for conducting the assessments
Multiple Service Providers for different lines of services in multiple geographies
Clients’ perspective
Subcontracting by the Service Providers and comfort provided by certifications like ISO 27001 are most significant assessment challenges faced by Client organizations
26
Service Provider Assessment Framework
Service Providers’ perspective
Factors such as cost, quantum of assessments were the least important challenges as perceived by Client organizations. Whereas, majority of Service Providers perceive high number of assessments around the year as one of the most significant challenges.
This difference in opinion regarding the challenges faced by Client and Service Provider organizations clearly indicates development of a robust assessment solution that meets the requirements of both parties.
50.00%
33.33%
33.33%
25.00%
16.67%
8.33%
33.33%
41.67%
25.00%
50.00%
33.33%
8.33%
8.33%
33.33%
33.33%
16.67%
25.00%
High number of assessments around the year
High Medium Low
Meeting diverse and varied assessment requirements of different clients
Closing the findings by providing evidences andsatisfying the client / auditors
High direct and indirect costs associated withgetting assessed multiple times
Ensuring availability of time and resources for coordinating/facing the assessments
Aligning to different areas of assessment/assessment methodologies adopted by
different clients High number of assessments around the year, and meeting diverse Client requirements are critical challenges faced by most of the Service Providers
27
Service Provider Assessment Framework
Service Provider assessments – solutions and future landscapeAn attempt was made to identify the possible solutions for the challenges faced by organizations. The survey results reveal that approximately thirty three percent of Clients and forty two percent of Service Provider organizations prefer the development and adoption of an international standard for Service Provider assessment. Also, usage of BITS shared assessment program was selected by forty four Client Organizations as a first preference among solutions.
Results indicate that more than forty percent of Service Providers regard development and adoption of an internal standard as a first preference. Independent third party assessments conducted by Self Regulatory Organizations (SRO) promoted by the Service Providers tops the chart for Service Providers as a second preference.
Clients’ perspective
Possible solution to overcome identified challenges
44.00%
33.00%
22.00%
22.00%
11.00%
11.00%
11.00%
11.00%
33.00%
22.00%
11.00%
11.00%
11.00%
11.00%
First reference Second preference Third preference
Industry & Service Provider promoted and standardized third party assessment
programs like BITS
Development and adoption of international standards for Service Provider
Assessment
There is no need for Service Provider assessments as data security and privacy
risks are already addressed through contracts
Self declaration by Service Providers for complying / fulfilling clients’ security requirements, thereby
making them liable for any security incident/databreach / violation should suffice
ISO 27001 certification should be accepted globally as a seal of trust and assurance; eliminating the need for Service Provider assessments
Independent third party assessments conducted by Self Regulatory
Organizations (SRO) promoted by the Service Providers
As per Client organizations, industry and Service Provider promoted and standardized third party assessment program can be used for assessments. This is closely followed by development and adoption of an international standard
28
Service Provider Assessment Framework
Development and adoption of an international standard is the first preference chosen by Service Providers
Possible solution to overcome identified challenges
41.67%
33.33%
25.00%
8.33%
8.33%
0.00%
8.33%
25.00%
8.33%
41.67%
25.00%
8.33%
8.33%
33.33%
8.33%
16.67%
First preference Second preference Third preference
Development and adoption of international standards for Service Provider assessment
ISO 27001 certification should be accepted by all the clients globally as a seal of trust and
assurance; eliminating the need for Service Provider assessments
Industry & Service Provider promoted and standardized third party assessment programs
like BITS
Independent Third Party assessments conducted by Self Regulatory Organizations (SRO)
promoted by the Service Providers
There is no need for Service Provider assessments as data security and privacy risks
are already addressed through contracts
Self declaration by Service Providers for complying/fulfilling clients’ security requirements, thereby
making them liable for any security incident/data breach/violation should suffice
Service Providers’ perspective
29
Service Provider Assessment Framework
Influence of IT (Amendment) Act, 2008 on Service Provider assessmentsThere is widespread awareness about IT (Amendment) Act, 2008 in the industry.
More than fifty percent of Service Provider and thirty three percent of Client organizations report that IT (Amendment) Act, 2008 will assist in strengthening the data protection initiatives of Indian Service Providers, and would provide greater assurance to the Clients. Approximately thirty three percent of Client organizations believe that IT (Amendment) Act, 2008 will have no impact on their Information Security and Privacy needs as they need to comply with their country’s regulations outside of India.
A similar number of Service Provider organizations revealed that they were not sure about the impact/ influence of IT (Amendment) Act, 2008 on Clients’ assessment strategy.
Influence of IT (Amendment) Act, 2008 on Service Provider assessment strategy
66.67%
33.33%
33.33%
11.11%
0.00%
Provisions of IT (Amendment) Act, 2008 need to be appropriately incorporated in
the client-Service Provider contracts
IT (Amendment) Act, 2008 will have noimpact as we need to comply with regulations
we are subjected to
IT (Amendment) Act, 2008 will strengthen the data protection initiatives of Indian
Service Providers and therefore will help provide greater assurance to us for
outsourcing our work to India
I’m not aware of IT (Amendment) Act, 2008
Self declaration by Service Providers for complying/fulfilling clients’ security requirements,
thereby making them liable for any security incident/data breach/violation should suffice
IT (Amendment) Act, 2008 needs to be incorporated in Client-Service Provider contracts this would assist in strengthening the data protection initiatives of Service Providers
Clients’ perspective
30
Service Provider Assessment Framework
Service Providers’ perspective
Influence of IT (Amendment) Act, 2008 on Service Provider assessment strategy
58.33%
33.33%
0.00%
8.33%Others
Not sure what will be the impact of IT(Amendment) Act, 2008
IT (Amendment) Act, 2008 will strengthen the dataprotection initiatives of Indian Service Providers andtherefore will help provide greater assurance to the
clients outsourcing their work to India
IT (Amendment) Act, 2008 will have no impactas clients need to comply with regulations
they are subjected to
31
Service Provider Assessment Framework
Third party assessmentsThird party assessments have gained importance in the Indian IT/BPO industry. Both Clients and Service Providers revealed that third parties should conduct Service Provider assessments based on a standardized assessment methodology.
Majority of respondents emphasized that use of third parties would not only help in ensuring transparency and independence of the assessments but also save cost and efforts.
A few Clients also reported that their organization’s Executive Management may not approve/ recognize third party assessments.
Options for third party assessments
66.67%
66.67%
55.56%
55.56%
55.56%
55.56%
22.22%
11.11%
11.11%
Third parties can conduct assessments of theService Providers, based on a standardized
assessment methodology, at a defined frequency
Third Party assessments would save costs andefforts by avoiding the need for conducting
assessments of multiple Service Providers
Our regulators / customers may not approve or recognize Third Party assessments
Third party assessments can be successful only if it is accepted by the outsourcing community
and regulators
Third Party assessments will bring transparencyand independence
Adopting Third Party assessments may raisetrust and accountability issues
My organizations’ Executive Management may not approve or recognize Third
Party assessments
The Third Party assessments will ensure that our resources are able to focus on improving security
& privacy posture
Third Party assessments may not be able to address the specific assessment requirements arising out
of a particular Client-Service Provider relationship
Majority of Clients and Service Providers report that third parties should conduct Service Provider assessments, based on a standardized assessment methodology at a defined frequency
Clients’ perspective
32
Service Provider Assessment Framework
Third party assessments would save cost and efforts by avoiding multiple assessments from different Clients
Service Providers’ perspective
Options for third party assessments
66.67%
41.67%
41.67%
41.67%
33.33%
16.67%
16.67%
Third party assessments may not be able to address the specific assessment requirements arising out a particular client– Service Provider relationship
Adopting Third Party assessments may raise trust and accountability issues
Third parties can conduct assessments of theService Providers, based on a standardized
assessment methodology, at a defined frequency
Third Party assessments would save costs andefforts by avoiding multiple assessments from
different clients
Third Party assessments will bring transparencyand independence
Third party assessments can be successful only ifall our clients accept it, irrespective of industry,
geography, Line of Service, etc.
The Third Party assessments will ensure that our resources are able to focus on improving security &
privacy posture instead of supporting multiple assessments
Thirty three percent of Service Providers expressed their concerns regarding the use of third party assessments stating that they third party assessments could be helpful if their Clients accept these.
33
Service Provider Assessment Framework
Third party assessorsThe survey highlighted that the auditing firms empanelled by a joint industry consortium of outsourcers and the Service Providers are the most potential third party assessors for conducting independent Service Provider assessments, seemingly acceptable to both the Client and Service Provider organizations. This option was selected by sixty six and fifty percent of the Client and Service Provider organizations respectively. Such an industry consortium will represent the interests and challenges of both the sides – the Clients and Service Providers.
Potential entity acting as third party for conducting independent Service Provider assessments
66.67%
55.56%
33.33%
0.00%
Auditing firms empanelled by a joint industry consortium of outsourcers and the
Service Providers
Auditing firms empanelled by the outsourcers’ industry consortium
Self Regulatory Organizations (SRO)promoted by the Service Providers
Auditing firms empanelled by the Service Providers
Clients’ perspective
Auditing firms empanelled by a joint industry consortium of Client and Service Providers can serve as third party assessors for conducting Service Provider assessments
34
Service Provider Assessment Framework
Service Providers’ perspectivePotential entity acting as third party for conducting independent Service Provider assessments
58.33
50.00%
25.00%
8.33%
Self Regulatory Organizations (SRO)promoted by the Service Providers
Auditing firms empanelled by a jointindustry consortium of outsourcers and the
Service Providers
Auditing firms empanelled by the outsourcers’ industry consortium
Auditing firms empanelled by the Service Providers
35
Service Provider Assessment Framework
Standards for Service Provider assessmentsNew domains of Information Security and Privacy have evolved. The domains which were not perceived to be critical are now among the most important security domains. Organizations have to comply with various models/standards/frameworks to adhere to the changing domains/rules and regulations. The organizations do no prefer to comply with so many standards and frameworks and this perception of the organizations was clearly evident from the survey results.
The survey results highlighted that Client organizations are keen on adopting a new standard mapped to ISO 27001, NIST – Special Publications, COBIT, ITIL etc. that meets all the regulatory requirements like GLBA, HIPAA, PCI DSS etc., as a potential standard for third party assessments. While this view was common amongst Client organizations and Service Provider organizations with eighty nine and sixty seven percent respondents respectively selecting this option, in reality Clients may be more inclined towards new standard than the Service Providers because they demonstrate compliance to different regulations. Though this has an indirect impact on Service Providers but they are primarily driven by contractual obligations.
Both Client and Service Provider organizations have similar number of respondents who selected ISO 27001 (sixty six percentages). It seemed that organizations are satisfied with the acceptance of ISO 27001 as a
A new standard mapped to ISO 27001, NIST-SP, COBIT, ITIL etc. that meets all the regulatory requirements like GLBA, HIPAA, PCI DSS etc. as a standard for third party assessments
Potential assessment standards for third party assessments of Service Providers
88.89%
66.67%
22.22%
22.22%
11.11%
ISO 27001 standard
A new standard mapped to ISO 27001, NIST– SP, COBIT, ITIL, etc. that meets
all the regulatory requirements like GLBA,HIPAA, PCI DSS, etc.
Others
Security and Privacy practices defined by SRO
BITS shared assessment framework
Clients’ perspective
36
Service Provider Assessment Framework
Potential assessment standards for third party assessments of Service Providers
66.67%
66.67%
16.67%
8.33%
ISO 27001 standard
Security and Privacy practices defined by SRO
BITS shared assessment framework
A new standard mapped to ISO 27001,NIST-SP, COBIT, ITIL, etc. that meets all the regulatory
requirements like GLBA, HIPAA, PCI DSS, etc.
Service Providers’ perspective
standard bearing in mind the challenge faced by Client organizations with respect to the comfort/ assurance provided by Service Providers through ISO 27001 certification.
Respondents in the ‘Others’ category also suggested the use of a unified compliance framework for assessments.
37
Service Provider Assessment Framework
Role of DSCI in Service Provider assessmentsThe question aimed to identify the role that DSCI could play as a Self Regulatory Organization (SRO), representative of both Client and Service Provider organizations, for conducting Service Provider assessments.
Majority of the Client organizations (sixty seven percent) indicated that DSCI should create a panel of competent auditors to conduct Service Provider assessments on behalf of DSCI, develop code of practices for Data Security and Privacy that should be adopted by the industry and define some criteria for assessing the maturity of the Service Providers.
Fifty eight percent of the Service Provider organizations indicated that DSCI should develop a Service Provider assessment program that comprises of framework, processes and methodology for conducting Service Provider assessments. This option was also highlighted by a similar number of Client organizations (fifty six percent).
Clients’ perspective
Role of DSCI in Service Provider assessments
66.67%
66.67%
66.67%
55.56%
55.56%
55.55%
33.33%
11.11%
DSCI should establish a mechanism to managethe assessment results including sharing of results
with clients and respective Service Providers
Others
DSCI should have code of practices for securityand privacy that need to adopted by its members
The code of practices should have some criteria for assessing the maturity of the
Service Providers
Code of practices should take a note of existingpreparedness and initiatives of Service
Providers in the areas of security and privacy
DSCI should have mechanism to reviewthe Service Provider assessments results
on a regular basis
DSCI should have a Service Provider assessmentprogram that comprises of framework, processes,
methodology for the assessment
DSCI should create a panel of competentauditors who will conduct the assessments on
behalf of DSCI
Majority of Clients and Service Providers perpetuated that DSCI should have a Service Provider assessment program that consists of framework, processes and methodology of assessments
Service Provider Assessment Framework
Role of DSCI in Service Provider assessments
58.33%
33.33%
25.00%
25.00%
25.00%
8.33%
8.33%
8.33%Others
DSCI should have a Service Provider assessmentprogram that comprises of framework, processes,
methodology for the assessmentsDSCI should create a panel of competent
auditors who will conduct the assessmentson behalf of DSCI
DSCI should have code of practices for security andprivacy that need to adopted by its members
The code of practices should have some criteria forassessing the maturity of the Service Providers
DSCI should have mechanism to review the ServiceProvider assessments results on a regular basis
DSCI should establish a mechanism to manage theassessment results including sharing of results with
clients and respective Service Providers Code of practices should take a note of existing
preparedness and initiatives of Service Providersin the areas of security and privacy
Service Providers’ perspective
39
Service Provider Assessment Framework
Outcome of Service Provider assessmentsThe survey results have unequivocally established that there should be organization-wide Security and Privacy maturity ratings, and domain specific ratings.
It was also indicated that the both Client organizations as well as Service Provider organizations prefer ratings over certifications.
Outcome of Service Provider assessments — Data Security
77.78%
55.56%
44.44%
DSCI should provide organizationwide security maturity rating
DSCI should provide domain specificmaturity rating (e.g. Application
security maturity rating)
DSCI should provide organization wide security certification to
Service Providers
Outcome of Service Provider assessments — Data Privacy
88.89%
44.44%
DSCI should provide organizationwide privacy certification to
Service Providers
DSCI should provide organizationwide privacy maturity rating
Clients’ perspective
Organization-wide security and privacy maturity ratings may be provided as a result of Service Provider assessments
40
Service Provider Assessment Framework
Service Providers’ perspective
Outcome of Service Provider assessments — Data Security
58.33%
33.33%
16.67%
DSCI should provide organization wide security maturity rating
DSCI should provide organization wide security certification to Service Providers
DSCI should provide domain specific maturity rating (e.g. Application security maturity rating)
Outcome of Service Provider assessments — Data Privacy
75.00%
50.00%
DSCI should provide organization wide privacy maturity rating
DSCI should provide organization wide privacy certification to Service Providers
41
Service Provider Assessment Framework
Sharing of Service Provider assessment resultsMajority of Client organizations (sixty seven percent) confirm that if DSCI assumes the role of a third party assessor, DSCI should conduct the assessment of the targeted Service Provider and share the report with the Client. Client organizations are also in favor of DSCI conducting assessments of the Service Providers and sharing the report with Service Providers’ Clients based upon the authorization of Service Provider (thirty three percent), while only eleven percent of the Client organizations suggested DSCI conducting the assessment of the Service Provider and submitting its report to the Service Provider.
In case DSCI assumes the role of a third-party assessor, Client and Service Provider organizations strongly support DSCI conducting the assessments of the targeted Service Provider and sharing the report with the Client on receiving requests from the Client
Most suitable assessment process in case DSCI assumes the role of a third party assessor
66.67%
33.33%
11.11%
11.11%
On receiving request from the client, DSCIconducts the assessment of the targeted
Service Provider and shares the report with the client
On receiving request from the ServiceProvider, DSCI conducts the assessment of the
Service Provider and based on theauthorization of Service Provider, DSCI shares
the report with Service Providers’ clients
On receiving request from the Service Provider,DSCI conducts the assessment of the Service
Provider and submits its report to the Service Provider.Service Provider then shares this report with his
clients when requested or otherwise
Based on DSCI assessments, ServiceProviders are benchmarked against defined
parameters and the report is made public
Clients’ perspective
42
Service Provider Assessment Framework
Most suitable assessment process in case DSCI assumes the role of a third party assessor
41.67%
41.67%
33.33%
8.33%
8.33%Others
Based on DSCI assessments, Service Providers are benchmarked against defined parameters and
the report is made public
On receiving request from the Service Provider, DSCIconducts the assessment of the Service Provider and
based on the authorization of Service Provider,DSCI shares the report with SPs’ clients
On receiving request from the client, DSCI conductsthe assessment of the targeted Service Provider and
shares the report with the client
On receiving request from the Service Provider, DSCIconducts the assessment of the Service Provider and
submits its report to the Service Provider. ServiceProvider then shares this report with his clients
Service Providers’ perspective
More than forty percent of the Service Provider respondents suggested that in case DSCI assumes the role of a third party assessor, DSCI should conduct the assessment of the targeted Service Provider on receiving request from the Client and share the report with the Client. Same number of Service Provider organizations also supports the process of DSCI conducting the assessment on receiving request from the Service Provider and submitting the report to Service Providers’ Clients upon authorization.
43
Service Provider Assessment Framework
The survey revealed some interesting findings and facts, both from Client and Service Provider perspective which were further validated by the secondary research. Based on the study of different assessment frameworks and findings of the survey, following are some of the salient preliminary recommendations for developing a Service Provider Assessment Framework:
DSCI should play a vital role in conducting Service Provider •
assessments and sharing the outcome in the ecosystem. It should:
Have an Service Provider assessment program that comprises of •
framework, processes, and methodology for assessments
Provide an organization wide security and privacy maturity rating, •
and domain specific maturity rating that may be shared in the ecosystem after taking the due permission of the Service Providers
A new standard mapped to prevalent standards should be •
considered as a potential assessment standard for third party assessments of Service Providers
DSCI as an industry initiative and a Self Regulatory Organization •
having representation from both the Client and Service Provider organizations should empanel auditing firms for conducting independent third party assessments.
The advantages of prevalent assessment frameworks like •
adaptability, flexibility, comprehensibility of assessment areas, process-driven, and measurement-based assessment process should be the characteristics of the Service Provider assessment framework that may be developed.
Recommendations
44
The assessment model should not become an overhead for an •
organization. It should be able to provide specific improvement opportunities that an organization should be able to imbibe. The assessment criteria should be transparent to the extent possible. The framework should be reviewed at least on an annual basis by a competent set of technical and process experts, preferably comprising DSCI members, members from third party assessors, and the industry.
The assessment framework should be applicable regardless of •
size of the organization and nature/ complexity of its processes. For this purpose, the assessment methodology adopted should contain a preliminary set of questions that can be self-assessed by an organization.
The Service Provider assessment framework should provide •
opportunities to organizations for implementing / performing the control activities according to the needs of the organization’s specific environment.
The framework should follow a process-approach and outline •
measurable assessment areas. The assessment areas that would link to specific business processes in an organization will be easy to align with overall business goals and objectives.
The framework should provide both – assessment area/ domain •
based maturity rating and organization-wide security and privacy maturity rating that summarizes the appraisal results and permit comparison amongst organizations.
The assessment model should be easy to comprehend and •
companies should be able to adopt on their own. All assessment areas should be broken down into a detailed list of specific and measureable steps that are easy to comprehend for assessment purposes.
45
Service Provider Assessment Framework
AnnexuresThe study team analysed the shortlisted assessment frameworks for their advantages and disadvantages when applied to Client-driven Service Provider assessments. A summary of each framework is provided below:
Malcolm Baldrige National Quality ProgramThe Malcolm Baldrige National Quality Award program uses Malcolm Baldrige Assessment framework to assess the Quality of applying organization on seven critical areas for an organization. The framework is based on the processes implemented and the results achieved. The assessment methodology requires a self-assessment by the company applying for the award which assists in dissolving the disparities between small, medium and large sized companies, and the way the control is implemented at the organization level. The framework has assigned separate weights to each individual area. However, the framework does not provide quantitative requirements for the criteria laid down. The requirements are subjective, and there are chances that the results when examined by different examiners may not be reproducible.
Capability Maturity Model Integration (CMMI)The framework was developed by Software Engineering Institute (SEI) in an attempt to integrate several disciplines such as Process and Product development, Acquisition and Supplier Sourcing. The framework is focused towards software development organization however the framework can be implemented across various organizations. The framework can be implemented using Staged or Continuous representation. The Staged representation provides the Maturity level for organizations and the Continuous representation provides Capability Levels for as a measure assigned individually against each process area. The framework is flexible and provides opportunities to organizations for undertaking the
46
activities according to their organization specific environment. For assessments the framework undertakes a Process based approach thereby adding value to the organization in the process of being assessed for maturity.
BITS Shared Assessment ProgramThe financial services industry increasingly relies on information technology (IT) Service Providers to support the delivery of financial services. The BITS shared assessment framework was developed by BITS IT Service Providers Working Group to address the concerns, arising out of increased regulatory scrutiny of financial institution risk assessment and management of outsourced IT services. The framework adopts a risk based approach for conducting the assessments. The framework can be used as a reference by to create a common understanding of the financial services industry’s needs among Service Providers and help to address known control weaknesses in outsourced IT services, resulting in more consistent and appropriate levels of management by financial services companies that outsource IT services.
The eSourcing Capability Model for Client Organizations (eSCM-CL)The eSCM was developed by a consortium led by Carnegie Mellon University’s Information Technology Service Qualification Center (ITSqc). The eSCM is best practices capability models with two purposes (1) to give Client organizations guidance that will help them improve their capability across the sourcing life-cycle, and (2) to provide Client organizations with an objective means of evaluating their sourcing capability. The model aims at assisting Client organizations to continuously evolve, and improve their capabilities to develop stronger, enduring and more
47
Service Provider Assessment Framework
trusting relationships with other Service Providers, and to meet the dynamic demands of business. The eSCM model provides the organizations the flexibility to choose from framework based (using the framework as best practices) or evaluation based (using the framework to undertake a formal assessment). The eSCM for Client organizations is composed of 95 practices covered under three dimensions Sourcing Life-cycle, Capability Area, and Capability Level.
Crisil Rating MethodologyCRISIL rates companies in variety of sectors. Since each sector has its own nuances, CRISIL has customized rating criteria and methodology for each sector to make the ratings exercise apt and meaningful. Extensive research is undertaken by Crisil before assigning rating to an organization. The rating methodology adopts a “risk-based” approach thereby helping the organization to align its activities in-line with the risks that matter.
Moody’s Working PaperMoody’s rates companies/covenants in various sectors/industries. Moody’s also report on market norms which are dynamic and may vary over time, by sector and by region. Moody’s covenant framework focuses on providing “point in time” assessment, with ongoing monitoring of covenants. Moody’s assessment is also based on the risk associated with the covenant.
The following criteria were chosen for comparing the assessment frameworks/ models:
Assessment Areas/ ease of use by the organization being •
assessed
Assessment Methodology/ scoring pattern•
Sharing of assessment results•
Acceptability/ popularity of the framework•
Independence of examiners•
Frequency of framework update to cater to future requirements•
49
Service Provider Assessment Framework
Advantages and disadvantages of prevalent assessment frameworks/ models
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
1. Assessment areas/ Ease of use by the organization being assessed
Advantages
The assessment criteria are non-•
prescriptive and adaptable by any organization regardless of its nature and size. It covers adequate details of the assessment criteria for any company to implement in a manner they desire and yet meet the expectations of the framework. In other words, the framework is not dictative or stringent in nature, thus making it easily usable by any organization.
The assessment areas are designed to •
cover the breadth of the organization which would be an expectation from any good assessment framework for greater adaptability in the industry regardless of the nature or size of the organization.
Disadvantages
The flip side is that the framework does •
not provide quantitative requirements for the criteria laid down. The requirements are subjective and hence there are chances that the results when examined by different examiners may not be reproducible.
The implementation of the assessment •
areas is person-dependent even for the organizations applying for the award. In other words, the judgment of the Point of Contact (POC) from the company applying for the award would greatly matter.
Advantages
The framework is flexible and provides •
opportunities to organizations for undertaking the activities according to their organization specific environment. The assessment model uses areas of assessment that are process driven namely, process management, project management, engineering, and support.
The framework is beneficial in aligning the •
organization’s business processes to business goals and objectives.
Disadvantages
Apart from specific goals and practices against •
each Process Area (PA) being assessed, the model contains generic goals and generic practices which an organization is required to meet and implement. This leads to unnecessary redundancies and are a major source of complaints on the standard in its current form.
The ease of use of the CMMI model by smaller •
organizations is a challenge due to its very nature of being long drawn. The model was originally designed for large companies for them to be able to streamline the processes surround the software development and engineering.
Advantages
CRISIL rates companies •
in variety of sectors. Since each sector has its own nuances, CRISIL has customized rating criteria and methodology for each sector to make the ratings exercise apt and meaningful. Therefore, these are easy to use and understand by the organization being assessed. Moreover, the rating criteria are publicly available.
CRISIL performs extensive •
research when finalizing the rating criteria for an industry.
Disadvantages
The assessment areas are •
limited to specific risks and issues, and do not help in tying to the overall business goals and objectives of an organization.
Advantages
The BITS assessment is non-•
prescriptive and is intended for consideration in conjunction with overall risk map of the organization.
The framework serves as •
a best-practices guide for financial organizations to implement. The guidelines are fairly detailed and easily used/ understandable without the need of a formal training.
The BITS shared assessment •
program has proved to be a boon for the Service Providers because it allows the companies to offset the cost, time and resources which would otherwise be spent in catering to the need of multiple financial companies.
Disadvantages
The BITS framework does •
not provide quantitative criteria, and therefore the implementation carried out by organizations using BITS may be difficult to measure or compare against another organization.
Advantages
The model not just helps in •
evaluating the capability of Service Providers; sourcing capability of Clients but also helps both, Service Provider and Client organizations to improve their capability across the sourcing lifecycle.
The assessment/ capability •
areas are fairly detailed and serve as best practices guidelines for the organizations to adapt.
The model clearly defines •
the purpose, outcome, team, sponsor, and model scope of the assessment which makes it clear and easy to use by the organization adopting the model.
Disadvantages
The eSCM model defines •
capability areas and capability levels in a subjective fashion. The demarcation between different capability levels cannot be measured in absolute terms.
The assessment areas are •
way too many and this may make the implementation/ evaluation process time consuming.
Advantages
The assessment is specific •
to a particular instrument/ covenant and the outcome may be used as a comparison factor with other covenants.
Disadvantages
The assessment is point-in-•
time, and for new securities only. There is no ongoing monitoring of the assessed securities.
The assessment model does •
not draw inference from specific assessments to be able to rate the maturity/ capability of the organization issuing the instrument/ covenant.
50
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
1. Assessment areas/ Ease of use by the organization being assessed
Advantages
The assessment criteria are non-•
prescriptive and adaptable by any organization regardless of its nature and size. It covers adequate details of the assessment criteria for any company to implement in a manner they desire and yet meet the expectations of the framework. In other words, the framework is not dictative or stringent in nature, thus making it easily usable by any organization.
The assessment areas are designed to •
cover the breadth of the organization which would be an expectation from any good assessment framework for greater adaptability in the industry regardless of the nature or size of the organization.
Disadvantages
The flip side is that the framework does •
not provide quantitative requirements for the criteria laid down. The requirements are subjective and hence there are chances that the results when examined by different examiners may not be reproducible.
The implementation of the assessment •
areas is person-dependent even for the organizations applying for the award. In other words, the judgment of the Point of Contact (POC) from the company applying for the award would greatly matter.
Advantages
The framework is flexible and provides •
opportunities to organizations for undertaking the activities according to their organization specific environment. The assessment model uses areas of assessment that are process driven namely, process management, project management, engineering, and support.
The framework is beneficial in aligning the •
organization’s business processes to business goals and objectives.
Disadvantages
Apart from specific goals and practices against •
each Process Area (PA) being assessed, the model contains generic goals and generic practices which an organization is required to meet and implement. This leads to unnecessary redundancies and are a major source of complaints on the standard in its current form.
The ease of use of the CMMI model by smaller •
organizations is a challenge due to its very nature of being long drawn. The model was originally designed for large companies for them to be able to streamline the processes surround the software development and engineering.
Advantages
CRISIL rates companies •
in variety of sectors. Since each sector has its own nuances, CRISIL has customized rating criteria and methodology for each sector to make the ratings exercise apt and meaningful. Therefore, these are easy to use and understand by the organization being assessed. Moreover, the rating criteria are publicly available.
CRISIL performs extensive •
research when finalizing the rating criteria for an industry.
Disadvantages
The assessment areas are •
limited to specific risks and issues, and do not help in tying to the overall business goals and objectives of an organization.
Advantages
The BITS assessment is non-•
prescriptive and is intended for consideration in conjunction with overall risk map of the organization.
The framework serves as •
a best-practices guide for financial organizations to implement. The guidelines are fairly detailed and easily used/ understandable without the need of a formal training.
The BITS shared assessment •
program has proved to be a boon for the Service Providers because it allows the companies to offset the cost, time and resources which would otherwise be spent in catering to the need of multiple financial companies.
Disadvantages
The BITS framework does •
not provide quantitative criteria, and therefore the implementation carried out by organizations using BITS may be difficult to measure or compare against another organization.
Advantages
The model not just helps in •
evaluating the capability of Service Providers; sourcing capability of Clients but also helps both, Service Provider and Client organizations to improve their capability across the sourcing lifecycle.
The assessment/ capability •
areas are fairly detailed and serve as best practices guidelines for the organizations to adapt.
The model clearly defines •
the purpose, outcome, team, sponsor, and model scope of the assessment which makes it clear and easy to use by the organization adopting the model.
Disadvantages
The eSCM model defines •
capability areas and capability levels in a subjective fashion. The demarcation between different capability levels cannot be measured in absolute terms.
The assessment areas are •
way too many and this may make the implementation/ evaluation process time consuming.
Advantages
The assessment is specific •
to a particular instrument/ covenant and the outcome may be used as a comparison factor with other covenants.
Disadvantages
The assessment is point-in-•
time, and for new securities only. There is no ongoing monitoring of the assessed securities.
The assessment model does •
not draw inference from specific assessments to be able to rate the maturity/ capability of the organization issuing the instrument/ covenant.
51
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
2 Assessment methodology/ Scoring pattern
Advantages
The assessment methodology requires •
a self-assessment by the company applying for the award which assists in dissolving the disparities between small, medium and large sized companies, and the way the control is implemented at the organization.
There are weights assigned to each •
criterion of the framework, which gives the organization a sense of direction on emphasis or focus for implementation/ assessment purposes.
The calculations involve an aggregate •
rather than an average of the evaluation dimensions, thus giving the overall picture more accurately despite being “result-focused”.
Disadvantages
The framework lays down the criteria •
for assessment but it is silent (in other words not transparent) about the method of assessment. Therefore, an organization applying for the award may not know what it missed vis-à-vis other companies, and why it did not receive the award.
The assessment methodology is •
subjective, and the scoring patterns assigns discrete percentage values to the evaluation dimensions. Which of the percentage would be chosen for a particular company is left to the judgment of the examiner.
In the self assessment process, the •
guidelines for High, Medium, and Low levels of importance are not defined.
The highest weights are assigned to •
the Results criteria and therefore, the framework is very “result-focused” rather than “process focused”. A highly result oriented approach may lead to dissolution of importance that processes play in an organization.
Advantages
The maturity model adopts a process •
approach, thereby adding value to the organization in the process of being assessed for maturity. An organization being assessed against this model would see increased number of processes under statistical control.
The model adopts a measurement based •
process improvement.
Individual processes operating in the company •
can be assessed or assigned capability levels namely, Optimizing, Quantitatively managed, etc.
Disadvantages
The assessment model may be complicated for •
companies to comprehend and adopt on their own. Hence, training by Software Engineering Institute (SEI’s) licensed consultants becomes imperative for companies. Consequently, the associated costs and administrative overheads required to undergo and maintain the assessment increases.
The model consists of two representations •
– continuous and staged which may lead to conflicting situations at some point during the assessment.
The numbering scheme of goals and practices •
is complicated. Example, goals of a process area are numbered “1 to n”. A specific goal of a process area therefore may have the label “SG 3” and a generic goal “GG5”. The practices corresponding to a goal use the index of the related goal and another index “1 to m” numbering it as one of the practices of this respective goal. Specific practices also have a capability rating indicated by a dash and the respective capability level. A specific practice therefore may have the label “Service Provider 3.3-1” and a generic practice “GP 5.1”.
The rating is subjective to the extent of the •
examiner’s understand of the business processes. The framework is rigid as the organization has to complete level 3 before completing level 4.
Advantages
The rating methodology •
adopts a “risk-based” approach as compared to CMMI that adopts a “process-based” approach and Malcolm Baldrige that adopts a “results- focused” approach. The advantage of risk based approach is the prioritization of the organization’s activities in-line with the risks that matter or high risks.
The rating methodologies and •
criteria are clearly spelt out, published and consistently applied. The rating/ grading scales are also publicly available.
CRISIL gives a detailed •
analytical report on the company’s strengths and weaknesses along with the ratings. The report helps in strengthening operations and improving the working of the company.
The rating scale used •
is objective and easily understood.
Disadvantages
The assessment methodology •
does not focus on overall strengths, weaknesses, processes operating in the organization, rather specific instruments.
Advantages
The framework adopts a risk-•
based approach and unlike CRISIL, applies to the overall risk map of the company.
The assessment is asset •
driven, helps in improving processes and controls in the organization. A by-product for financial organizations is compliance to the requirements laid down by the Federal Financial Institutions Examination Council (FFIEC).
The BITS shared assessment •
framework is flexible and can be as detailed and as objective driven as required by an organization.
Disadvantages
There is no scoring pattern •
or rating guidelines provided to measure the effectiveness of implementation of the BITS framework. In other words, the assessor’s knowledge, competence and understanding of processes would play a big role in the deciding the final results.
While the assessment has •
resulted in lowering the overall cost of assessments. However, the assessments are fairly detailed and carry on for 3 to 8 weeks. The time and resources that an organization would spend may be difficult to offset for smaller organizations.
Advantages
The assessment model •
helps Service Providers to objectively differentiate themselves from the rest.
The model provides for •
two methods of adoption – framework-based (which is using the practice area details as best practice guidelines for an organization to implement), and measurement-based (which allows the organization to undertake a formal assessment of its practices and obtain certification).
The assessment model •
contains evaluation types namely mini self-appraisal and mini evaluation which allows the flexibility to an organization to use only a sub-set of the practice areas.
Disadvantages
The model uses five different types of assessment methods, and does not require the organization to undertake a formal assessment by an external party. This may lead to the model being used mainly as a reference guide and lead to non-seriousness in adoption by many companies.
The mini evaluation and mini self-appraisal assessment strategies may give comfort over existing controls to the management of the organization without realizing that in reality these may be covering just some aspects of the overall areas of assessments.
Advantages
The framework adopts a •
risk-based approach, and looks after the needs of the investors.
The framework uses a •
qualitative rating scale of Strong/ good/ weak across a number of parameters associated with the level of protection offered to an investor. The definitions of Good, Strong, etc. levels are objectively defined and easy to understand.
The framework’s assessment •
methodology in assigning a rating is transparent and objective.
The assessments are •
quantitative in nature.
Disadvantages
The assessment methodology •
does not relate back to the business goals and objectives of the company issuing it.
52
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
2 Assessment methodology/ Scoring pattern
Advantages
The assessment methodology requires •
a self-assessment by the company applying for the award which assists in dissolving the disparities between small, medium and large sized companies, and the way the control is implemented at the organization.
There are weights assigned to each •
criterion of the framework, which gives the organization a sense of direction on emphasis or focus for implementation/ assessment purposes.
The calculations involve an aggregate •
rather than an average of the evaluation dimensions, thus giving the overall picture more accurately despite being “result-focused”.
Disadvantages
The framework lays down the criteria •
for assessment but it is silent (in other words not transparent) about the method of assessment. Therefore, an organization applying for the award may not know what it missed vis-à-vis other companies, and why it did not receive the award.
The assessment methodology is •
subjective, and the scoring patterns assigns discrete percentage values to the evaluation dimensions. Which of the percentage would be chosen for a particular company is left to the judgment of the examiner.
In the self assessment process, the •
guidelines for High, Medium, and Low levels of importance are not defined.
The highest weights are assigned to •
the Results criteria and therefore, the framework is very “result-focused” rather than “process focused”. A highly result oriented approach may lead to dissolution of importance that processes play in an organization.
Advantages
The maturity model adopts a process •
approach, thereby adding value to the organization in the process of being assessed for maturity. An organization being assessed against this model would see increased number of processes under statistical control.
The model adopts a measurement based •
process improvement.
Individual processes operating in the company •
can be assessed or assigned capability levels namely, Optimizing, Quantitatively managed, etc.
Disadvantages
The assessment model may be complicated for •
companies to comprehend and adopt on their own. Hence, training by Software Engineering Institute (SEI’s) licensed consultants becomes imperative for companies. Consequently, the associated costs and administrative overheads required to undergo and maintain the assessment increases.
The model consists of two representations •
– continuous and staged which may lead to conflicting situations at some point during the assessment.
The numbering scheme of goals and practices •
is complicated. Example, goals of a process area are numbered “1 to n”. A specific goal of a process area therefore may have the label “SG 3” and a generic goal “GG5”. The practices corresponding to a goal use the index of the related goal and another index “1 to m” numbering it as one of the practices of this respective goal. Specific practices also have a capability rating indicated by a dash and the respective capability level. A specific practice therefore may have the label “Service Provider 3.3-1” and a generic practice “GP 5.1”.
The rating is subjective to the extent of the •
examiner’s understand of the business processes. The framework is rigid as the organization has to complete level 3 before completing level 4.
Advantages
The rating methodology •
adopts a “risk-based” approach as compared to CMMI that adopts a “process-based” approach and Malcolm Baldrige that adopts a “results- focused” approach. The advantage of risk based approach is the prioritization of the organization’s activities in-line with the risks that matter or high risks.
The rating methodologies and •
criteria are clearly spelt out, published and consistently applied. The rating/ grading scales are also publicly available.
CRISIL gives a detailed •
analytical report on the company’s strengths and weaknesses along with the ratings. The report helps in strengthening operations and improving the working of the company.
The rating scale used •
is objective and easily understood.
Disadvantages
The assessment methodology •
does not focus on overall strengths, weaknesses, processes operating in the organization, rather specific instruments.
Advantages
The framework adopts a risk-•
based approach and unlike CRISIL, applies to the overall risk map of the company.
The assessment is asset •
driven, helps in improving processes and controls in the organization. A by-product for financial organizations is compliance to the requirements laid down by the Federal Financial Institutions Examination Council (FFIEC).
The BITS shared assessment •
framework is flexible and can be as detailed and as objective driven as required by an organization.
Disadvantages
There is no scoring pattern •
or rating guidelines provided to measure the effectiveness of implementation of the BITS framework. In other words, the assessor’s knowledge, competence and understanding of processes would play a big role in the deciding the final results.
While the assessment has •
resulted in lowering the overall cost of assessments. However, the assessments are fairly detailed and carry on for 3 to 8 weeks. The time and resources that an organization would spend may be difficult to offset for smaller organizations.
Advantages
The assessment model •
helps Service Providers to objectively differentiate themselves from the rest.
The model provides for •
two methods of adoption – framework-based (which is using the practice area details as best practice guidelines for an organization to implement), and measurement-based (which allows the organization to undertake a formal assessment of its practices and obtain certification).
The assessment model •
contains evaluation types namely mini self-appraisal and mini evaluation which allows the flexibility to an organization to use only a sub-set of the practice areas.
Disadvantages
The model uses five different types of assessment methods, and does not require the organization to undertake a formal assessment by an external party. This may lead to the model being used mainly as a reference guide and lead to non-seriousness in adoption by many companies.
The mini evaluation and mini self-appraisal assessment strategies may give comfort over existing controls to the management of the organization without realizing that in reality these may be covering just some aspects of the overall areas of assessments.
Advantages
The framework adopts a •
risk-based approach, and looks after the needs of the investors.
The framework uses a •
qualitative rating scale of Strong/ good/ weak across a number of parameters associated with the level of protection offered to an investor. The definitions of Good, Strong, etc. levels are objectively defined and easy to understand.
The framework’s assessment •
methodology in assigning a rating is transparent and objective.
The assessments are •
quantitative in nature.
Disadvantages
The assessment methodology •
does not relate back to the business goals and objectives of the company issuing it.
53
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
3 Outcome of the assessment/ Sharing of assessment results
Advantages
An organization, upon receiving the •
award becomes ineligible to apply for it after for a period of 5 years. This means that the framework allows more and more companies to become eligible to receive the award.
Disadvantages
The award restriction is 18 in a year •
and looking at the subjectivity or, non-transparency of the assessment methodology, this number may not be a good picture of the processes implemented by an organization applying for the award.
No communication is sent to the •
organizations whose applications are not selected by the Board of examiners.
Advantages
The model provides a single maturity rating •
that summarizes the appraisal results and permits comparisons across and among organizations.
The process of sharing results is transparent •
and the maturity ratings for assessed organizations are publicly available.
Disadvantages
The assessment model contains a •
unidirectional mapping between staged and continuous representations. However, if an organization chooses a maturity rating (staged representation); it does not indicate a reverse mapping to the capability levels at which the processes are operating.
Advantages
The ratings are under constant •
surveillance. They are revised as and when circumstances so warrant, and disseminated through the news media.
A rating history is maintained •
for a period of 5 years which gives an idea of an organization’s progression/ regression in the risks that matter.
Disadvantages
The outcome of the CRISIL •
ratings are issued in the form of an opinion on specific instruments, which does not have a fixed format that highlights what are the positives versus the negatives. Therefore, the results of the rating exercise cannot be directly used to improve on processes in the organization.
Relating the opinion issued •
back to the business goals, objectives and processes may be a tedious task.
Advantages
The outcome of the •
assessment is an Agreed Upon Procedures report which the Service Provider is free to share with all its Clients or outsourcers.
The outcome of the •
assessment also includes a remediation plan in-line with the requirements of the organization.
Disadvantages
There is no maturity rating •
or score provided to the organization that can be used to compare the company’s practices to other organizations.
Advantages
The outcome of each •
assessment method is clearly defined.
The outcome of the •
assessment gives the interested parties the comfort that the sourcing capabilities are indeed working fine.
The outcome of the •
assessment may be used as a differentiating factor by the organization thereby supporting the outsourcing process.
Disadvantages
The model focuses on capabilities and does not give a clear indication of the maturity of those processes.
Advantages
The outcome of each •
assessment is a rating assigned to a covenant which is easy to understand basis the guiding factors issued.
Disadvantages
The results are available in •
the form of opinion and do not always brings out the positives.
4 Acceptability / Popularity of the framework
Advantages
The results are conferred by the •
highest office in the United States which helps in making the framework widely acceptable.
The framework is acclaimed by the •
President of the country, thus making it popular and trusted.
The framework is promoted by public-•
private partnership, thus making it more acceptable.
Disadvantages
The flip side of the popularity is the •
bureaucracy and red-tapism that the adoption of the assessment framework may bring with it.
The value brought by the assessment •
framework to the organization while going through the assessment process cannot be measured or, is not visible. It does not influence or change the way things happen in an organization. Thus, many organizations may construe it as an overhead of documentation rather than being able to use it to bring about improvements in the organization.
Advantages
While going through the assessment process, •
the organization sees direct benefits / improvements being brought about in the organization. Thus, the assessment process is not seen as an overhead but something that adds value.
Disadvantages
The bodies of knowledge captured in CMMI •
models are only available for systems engineering, software engineering, product/ process development, supplier sourcing. Consequently, the maturity model is system/ software focused, and as such its tenets may not apply to industry sectors other than Services/ Technology/ Hardware.
Advantages
The CRISIL ratings are widely •
acceptable owing to the strong independence, right amount of confidentiality, and transparency of the final ratings.
Disadvantages
None noted.•
Advantages
The assessment is popular •
among financial institutions and helps them regulate the environment they operate in.
Disadvantages
The BITS shared assessment •
program is not popular among industries other than financial.
Advantages
The model’s acceptability •
may be reasonable in the outsourcers-outsourcing community.
Disadvantages
The model may not gain as •
much popularity because it does not force a formal evaluation or certification.
The model may not be very •
popular because of the cost, time and resources required to be spent in reading, interpreting, implementing, and then evaluating (if need be) the detailed requirements.
Advantages
The objective and quantitative •
nature of the assessment framework makes it popular and acceptable in the community.
Disadvantages
The framework cannot be •
extended to other industries.
54
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
3 Outcome of the assessment/ Sharing of assessment results
Advantages
An organization, upon receiving the •
award becomes ineligible to apply for it after for a period of 5 years. This means that the framework allows more and more companies to become eligible to receive the award.
Disadvantages
The award restriction is 18 in a year •
and looking at the subjectivity or, non-transparency of the assessment methodology, this number may not be a good picture of the processes implemented by an organization applying for the award.
No communication is sent to the •
organizations whose applications are not selected by the Board of examiners.
Advantages
The model provides a single maturity rating •
that summarizes the appraisal results and permits comparisons across and among organizations.
The process of sharing results is transparent •
and the maturity ratings for assessed organizations are publicly available.
Disadvantages
The assessment model contains a •
unidirectional mapping between staged and continuous representations. However, if an organization chooses a maturity rating (staged representation); it does not indicate a reverse mapping to the capability levels at which the processes are operating.
Advantages
The ratings are under constant •
surveillance. They are revised as and when circumstances so warrant, and disseminated through the news media.
A rating history is maintained •
for a period of 5 years which gives an idea of an organization’s progression/ regression in the risks that matter.
Disadvantages
The outcome of the CRISIL •
ratings are issued in the form of an opinion on specific instruments, which does not have a fixed format that highlights what are the positives versus the negatives. Therefore, the results of the rating exercise cannot be directly used to improve on processes in the organization.
Relating the opinion issued •
back to the business goals, objectives and processes may be a tedious task.
Advantages
The outcome of the •
assessment is an Agreed Upon Procedures report which the Service Provider is free to share with all its Clients or outsourcers.
The outcome of the •
assessment also includes a remediation plan in-line with the requirements of the organization.
Disadvantages
There is no maturity rating •
or score provided to the organization that can be used to compare the company’s practices to other organizations.
Advantages
The outcome of each •
assessment method is clearly defined.
The outcome of the •
assessment gives the interested parties the comfort that the sourcing capabilities are indeed working fine.
The outcome of the •
assessment may be used as a differentiating factor by the organization thereby supporting the outsourcing process.
Disadvantages
The model focuses on capabilities and does not give a clear indication of the maturity of those processes.
Advantages
The outcome of each •
assessment is a rating assigned to a covenant which is easy to understand basis the guiding factors issued.
Disadvantages
The results are available in •
the form of opinion and do not always brings out the positives.
4 Acceptability / Popularity of the framework
Advantages
The results are conferred by the •
highest office in the United States which helps in making the framework widely acceptable.
The framework is acclaimed by the •
President of the country, thus making it popular and trusted.
The framework is promoted by public-•
private partnership, thus making it more acceptable.
Disadvantages
The flip side of the popularity is the •
bureaucracy and red-tapism that the adoption of the assessment framework may bring with it.
The value brought by the assessment •
framework to the organization while going through the assessment process cannot be measured or, is not visible. It does not influence or change the way things happen in an organization. Thus, many organizations may construe it as an overhead of documentation rather than being able to use it to bring about improvements in the organization.
Advantages
While going through the assessment process, •
the organization sees direct benefits / improvements being brought about in the organization. Thus, the assessment process is not seen as an overhead but something that adds value.
Disadvantages
The bodies of knowledge captured in CMMI •
models are only available for systems engineering, software engineering, product/ process development, supplier sourcing. Consequently, the maturity model is system/ software focused, and as such its tenets may not apply to industry sectors other than Services/ Technology/ Hardware.
Advantages
The CRISIL ratings are widely •
acceptable owing to the strong independence, right amount of confidentiality, and transparency of the final ratings.
Disadvantages
None noted.•
Advantages
The assessment is popular •
among financial institutions and helps them regulate the environment they operate in.
Disadvantages
The BITS shared assessment •
program is not popular among industries other than financial.
Advantages
The model’s acceptability •
may be reasonable in the outsourcers-outsourcing community.
Disadvantages
The model may not gain as •
much popularity because it does not force a formal evaluation or certification.
The model may not be very •
popular because of the cost, time and resources required to be spent in reading, interpreting, implementing, and then evaluating (if need be) the detailed requirements.
Advantages
The objective and quantitative •
nature of the assessment framework makes it popular and acceptable in the community.
Disadvantages
The framework cannot be •
extended to other industries.
55
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
5 Independence of Examiners Advantages
The Panel of judges for the award work •
without compensation and nothing except limited travel expenses are reimbursed to them, which serves s a step towards ensuring unbiased assessments.
The Panel chairperson is reviewed •
every year by the Secretary of Commerce in the US.
Disadvantages
Panel member has a chance of serving •
a term of 6 years at a stretch, and yet continue to work with the company to which they belong. Hence, the independence of assessment process may be jeopardized.
There are no licensed personnel made •
available for spreading training and awareness on the framework.
Advantages
The Software Engineering Institute (SEI) •
provides a list of licensed consultants who can provide appraisal/ training services.
Disadvantages
None noted.•
Advantages
The rating committee •
comprises members who have the professional competence to meaningfully assess the credit analysis that underlies the rating, and have no concern with the company being rated. This makes the rating highly independent.
Disadvantages
None noted.•
Advantages
There are a set of listed •
companies that conduct the assessment of organizations.
Disadvantages
None noted.•
Advantages
The assessors are trained •
and certified by Carnegie Mellon University’s IT Services Qualification Center (ITSqc).
Disadvantages
Apart from the university’s •
ITSqc, the success of the model depends largely on the industry adopting it. Therefore, independence may be construed as an issue.
Advantages
The assessment makes use of •
analytics software which gives a sense of reliability to the assessees.
Disadvantages
In case the software solutions •
were not to work, the people trained in carrying out the assessment may not be available.
6 Frequency of framework update to cater to future requirements
Advantages
Over the period time, the framework •
has matured to keep in mind the future and dynamism associated with the marketplace, and is therefore now slated for a review once every two years.
Disadvantages
None noted.•
Advantages
The assessment framework has matured •
from the time of its inception, and released guidelines for implementation, and list of licensed consultants, since then.
Disadvantages
The framework does not have a set frequency •
of review/ revision.
Advantages
The ratings provide both •
short-term and long-term ratings, and the rating are re-evaluated as and when required.
CRISIL evaluates its rating •
criteria, methodologies, and procedures on a regular basis; and modifies or enhances them as necessary to respond to the needs of the markets.
Disadvantages
There is no specific frequency •
of review/ revision of CRISIL rating methodology.
Advantages
The Agreed Upon Procedures •
(AUPs) are updated at least once annually.
Disadvantages
None noted.•
Advantages
None noted.•
Disadvantages
The eSCM model does not •
have a fixed frequency for revision. It was last updated in the year 2004.
Advantages
None noted.•
Disadvantages
The model does not have a •
fixed frequency for revision.
56
Service Provider Assessment Framework
Sl. no. Comparison Criteria Malcolm Baldrige Framework Capability Maturity Model Integration (CMMI) CRISIL Ratings BITS framework e-Sourcing Capability Model (eSCM)
Moody’s assessment framework
5 Independence of Examiners Advantages
The Panel of judges for the award work •
without compensation and nothing except limited travel expenses are reimbursed to them, which serves s a step towards ensuring unbiased assessments.
The Panel chairperson is reviewed •
every year by the Secretary of Commerce in the US.
Disadvantages
Panel member has a chance of serving •
a term of 6 years at a stretch, and yet continue to work with the company to which they belong. Hence, the independence of assessment process may be jeopardized.
There are no licensed personnel made •
available for spreading training and awareness on the framework.
Advantages
The Software Engineering Institute (SEI) •
provides a list of licensed consultants who can provide appraisal/ training services.
Disadvantages
None noted.•
Advantages
The rating committee •
comprises members who have the professional competence to meaningfully assess the credit analysis that underlies the rating, and have no concern with the company being rated. This makes the rating highly independent.
Disadvantages
None noted.•
Advantages
There are a set of listed •
companies that conduct the assessment of organizations.
Disadvantages
None noted.•
Advantages
The assessors are trained •
and certified by Carnegie Mellon University’s IT Services Qualification Center (ITSqc).
Disadvantages
Apart from the university’s •
ITSqc, the success of the model depends largely on the industry adopting it. Therefore, independence may be construed as an issue.
Advantages
The assessment makes use of •
analytics software which gives a sense of reliability to the assessees.
Disadvantages
In case the software solutions •
were not to work, the people trained in carrying out the assessment may not be available.
6 Frequency of framework update to cater to future requirements
Advantages
Over the period time, the framework •
has matured to keep in mind the future and dynamism associated with the marketplace, and is therefore now slated for a review once every two years.
Disadvantages
None noted.•
Advantages
The assessment framework has matured •
from the time of its inception, and released guidelines for implementation, and list of licensed consultants, since then.
Disadvantages
The framework does not have a set frequency •
of review/ revision.
Advantages
The ratings provide both •
short-term and long-term ratings, and the rating are re-evaluated as and when required.
CRISIL evaluates its rating •
criteria, methodologies, and procedures on a regular basis; and modifies or enhances them as necessary to respond to the needs of the markets.
Disadvantages
There is no specific frequency •
of review/ revision of CRISIL rating methodology.
Advantages
The Agreed Upon Procedures •
(AUPs) are updated at least once annually.
Disadvantages
None noted.•
Advantages
None noted.•
Disadvantages
The eSCM model does not •
have a fixed frequency for revision. It was last updated in the year 2004.
Advantages
None noted.•
Disadvantages
The model does not have a •
fixed frequency for revision.
57
Service Provider Assessment Framework
Glossary
References
BPO Business Process Outsourcing
Client organization or Client
Organizations that outsource work to Service Providers
CMMI Capability Maturity Model Integration
COBIT Control Objectives for Information and related Technology that provide a set of best practices for Information Technology management
COSO Committee of Sponsoring Organizations
DSF DSCI Security Framework
GLBA Gramm-Leach-Bliley Act
HIPAA Health Insurance Portability and Accountability Act
ISO 27001 An Information Security Management System standard published in October 2005 by International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC)
IT Information Technology
ITIL Information Technology Infrastructure Library is a set of concepts and practices for Information Technology Services Management (ITSM), Information Technology (IT) development and IT operations.
NIST National Institute of Standards and Technology
PCI DSS PCI DSS or Payment Card Industry Data Security Standard is a worldwide information security standard defined by the Payment Card Industry Security Standards Council. The standard was created to help payment card industry organizations that process card payments prevent credit card fraud through increased controls around data and its exposure to compromise.
SAS 70 Statement on Auditing Standards No. 70
Service Provider or SP Organization that work for Client organizations in an outsourcing model
Malcolm Baldrige Framework: •
http://www.baldrige.com/
Capability Maturity Model Integration (CMMI): •
http://www.sei.cmu.edu/cmmi/
Crisil rating methodology: •
http://www.crisil.com
BITS framework for managing technology risks •
for IT service provider relationships: http://www.bitsinfo.org/
Carnegie Mellon University-e Sourcing •
Capability Model: http://itsqc.cmu.edu/models/escm-cl/downloads.asp
Moody’s Working Paper: •
www.moodys.com
DSCI Security Framework: •
http://www.dsci.in/
Ernst & Young Pvt. Ltd.
Assurance | Tax | Transactions | Advisory
About Ernst & YoungErnst & Young is a global leader in assurance, tax, transaction and advisory services. Worldwide, our 141,000 people are united by our shared values and an unwavering commitment to quality. We make a difference by helping our people, our clients and our wider communities achieve their potential.
Ernst & Young refers to the global organization of member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients. For more information about our organization, please visit www.ey.comErnst & Young Pvt. Ltd. is one of the Indian client serving member firms of EYGM Limited. For more information about our organization, please visit www.ey.com/india
Ernst & Young Pvt. Ltd. is a company registered under the Companies Act,1956 having its registered office at 22 Camac Street, 3rd Floor, Block C,Kolkata - 700016
In line with Ernst & Young’s commitment to minimize its impact on the environment, this document has been printed on paper with a high recycled content.
This publication contains information in summary form and is therefore intended for general guidance only. It is not intended to be a substitute for detailed research or the exercise of professional judgment. Neither EYGMLimited nor any other member of the global Ernst & Young organizationcan accept any responsibility for loss occasioned to any person acting orrefraining from action as a result of any material in this publication. Onany specific matter, reference should be made to the appropriate advisor.
50%
DATA SECURITY COUNCIL OF INDIA
L: Niryat Bhawan, 3rd Floor, Rao Tula Ram Marg, New Delhi-110057, India P: +91-11-26155071 | F: +91-11-26155070 | E: [email protected] | W: www.dsci.in