testing and monitoring vendors...vendor management system approved supplier list –authorized...

1

Testing and Monitoring

Vendors

SCCE Regional Conference – Boston March 24th, 2017

SCCE Regional Conference – Boston 2017 – Testing and Monitoring Vendors 2

Name Titles

Pascal Marat BT Global Services Security Compliance and

Assurance Director

[email protected]

Bill Cameron BT Global Services Security Risk and Compliance

Assurance Manager

[email protected]

Vin Scimeca BT Global Services Security Risk and Compliance

Assurance Manager

[email protected]

Introducing our team

2


1. Separate the facts from fiction - how to find out more about your vendors and the impact they have on

your operations – Risk Assessment

2. Learn the benefits of a centralized risk-based assurance program

3. Understand the other side of the table, how a vendor may prepare for assessments

4. Next step - how to leverage vendor assurance information for the best business outcomes

Objectives


• When you outsource know your business risks and potential mitigations

• Test risk areas prior to contract with vendor

• Build risk mitigations into your vendor agreements

• Monitor risk areas during the contract period

• Determine what types of assessments best fit the risk assigned to a vendor.

• In Person Assessments = High Risk Situations but at higher cost

• Hybrid Questionnaire / Remote Assessments = Medium Risk situations at moderate cost.

• Self Assessment Questionnaire = Low Risk and low cost

• Both Assessors and Auditees have different perspectives during the process. Understanding both sides is

very important for successful outcomes.

• Plan for remediation efforts to improve the business relationship.

Session Summary - Starting with the end

3

Part 1 Getting to know

your suppliers


Testing vs. Monitoring

• Testing (Vetting / Due Diligence) is performed before a contract is signed

• Monitoring ( Vendor Assessments or Audits) occur after a contract is in place.

Triggers :

Testing vs Monitoring

Event Triggers

Testing New business need to outsource (expertise, risk, cost, poor

performance of old vendor, contract expiration etc.)

Monitoring Periodic or contract defined assessments

Changes to your Contract or services provided

• Regulatory changes

• New business risks

• Contract Extension / renewal

Performance Issues or Incidents

4


Procurement – Testing: communicate your requirements and perform your diligence

Centralized Procurement function

Procurement Policy – Purchasing, Authority

Visible internally and accessible externally

Audit plan to meet internal and external requirements

Set of rules for external parties to engage with company

Online access to information

Generic Standards

Specific requirements could take the form of a Statement of

Requirements, Requests For Information / Proposals

Vendor Management system

Approved Supplier list – authorized contact

Exception process

Methodology to select and test or vet vendor

New vendors

Existing vendors

Accreditations

Risk assessment

Vendor Assessments

Procurement

Testing

Contract Language

Monitoring


• Primary Requirements – apply to all vendors

• Insurance, professional body status (legal proceedings…)

• Quality expectations

• Financial Viability

• Ability to scale up

• Ethical standards

• Ability to meet generic standards / control environment and governance framework – which ones?

• Sometimes done through a third party – supplier qualification process

• Risk Assessment – specific to a vendor and based on purchaser’s need

• Select the risk category for supplier, for example Insignificant, Low, Medium or High. How is this third party

interfacing with company in provisioning goods and services? What is the resulting risk exposure?

• Medium and high risk suppliers may require an independent assessment/qualification process, some of which may

include an on-site audit for relevant controls and procedures

• Lower risk suppliers may be able to provide a recognized relevant certificate and answer a supplier questionnaire.

Testing steps

5


Setting contractual obligations that the supplier would agree to…

• Expected Performance: include SMART measurements (SLA, KPI’s) – Specific, Measurable, Attainable, Relevant,

Timebound

• Required independent assessments and periodicity

• Consider including Quality Management System requirements, be explicit on reporting requirements

• Include controls that are important to your organization including compensatory controls and mitigations from risk

assessment

• Include Rights for Audit / Assessments, frequency and triggers (“may exercise right…”) and who bears the cost!

Have Procurement maintain a collection of approved templates such as clauses and Terms & Conditions

Challenges – novation, assignments, no contract…(?)

• Monitor the relationship and identify opportunities to bring the supplier in line with Procurement policies: changes to

services, expiration/renewal, incidents/performance issues

Getting to a contract – Supplier Management

Part 2– Methods

6


Methods

High

Risk Suppliers

Medium Risk Suppliers

Low Risk Suppliers

Self-Assessment Questionnaire

Hybrid - Remote Based

Auditing following

completion and submission of

evidence

In Person Audit


Best for: Time / Cost Benefit Risk

In Person

Assessments

High Risk

Situations

Relatively

High

Can have full details of

a vendor. Valid for

high value contracts.

Expensive

Cost and effort may

not yield a ROI

Hybrid –

Questionnaire /

Remote

Assessment

Medium

Risk

Situations

Medium Lower cost than in

person.

Can do more with

limited budget

Medium – will get

sense of risk without

cost of full Audit /

Assessment

Self Assessment

Questionnaire

Low Risk

Situations

or Follow

ups

Relatively

Low

Can do many at a time.

Low Cost.

May miss details.

Responses are often

sporadic requiring

follow up.

Questions could be

poorly interpreted

Methods – In detail -

7

Part 3 – The

assessments –

Understanding both

sides of the table


• Make sure requirements are clearly defined.

• Both parties should at least know what the requirements are even if it is before a contract.

• If you have a contract, know your contract –

• It is in both parties best interest to ensure that they know the contract to which they have signed.

• Understand all schedules or attachments, or subsequent amendments.

• Understand audit / assessment rights and rules which may have been agreed to.

TIP: We suggest to index your contracts to ensure that there is a clear understanding of requirements and

deliverables for all parties.

Seeing both sides of a vendor assessment (Testing and Monitoring) – Start

8


Step Assessor (Contracting Co.) Auditee (Vendor )

Determine Scope,

Communicate intent to

Assess, Open up

communication channels.

Establish scope and validate rights

Plan assessment highlighting

relevance to services provided

Open up channels of

communication

Is the request legitimate?

Is the Scope within the boundaries

of the contract?

What is the impact to operations of

the service?

Mobilize resources to assist with

request

Communication of Scope Assessment



Make the Assessment

Method Determination

Weigh risks and establish most

appropriate way to Audit

Communicate the audit method

Is the audit method proposed

reasonable?

What resources will be required to

address the assessment?

Assessment Method Determination

9



Self Assessment

Questionnaire

Are the questions relevant?

Have we provided sufficient

guidance on what we are expecting

back?

Have we given appropriate time for

completion?

Have we provided a mean for

vendor to ask for clarification?

Is evidence provided sufficient for

assessment?

Is there an NDA in place?

Do you have a contact for

clarification or extension request?

Can timeframe be met?

Are the questions being asked

relevant to the scope and relevant

to the services provided?

Have I documented all responses

and provided competent evidence?

Can information be disclosed within

policies?

Self Assessment Questionnaire



Hybrid – Questionnaire /

Remote Assessment

Everything from Self assessment

questionnaire plus…

Send a request for information

Schedule interviews

Clarify what evidence is acceptable

Timing of audit does it coincide with

business critical time periods?

What is the time impact to my

service?

Everything from Self assessment

questionnaire plus…

Verify availability of SMEs and

provide audit guidance

Be mindful of auditors scope creep

Ensure that documentation is

delivered within 24 hrs in the

proper format

Redact requested documents to

protect information and limit

information disclosure to strictly

necessary

Hybrid – Questionnaire / Remote assessment

10



In Person Audit Everything above plus…

Selection of in-house vs external

auditor

Travel coordination

Planning a site inspection,

coordination of contracting

company SMEs

Agree schedule with the vendor

including joint reviews of results

Everything above plus…

Establish a response team based on

scope

Verify availability of Key Personnel

and coordinate travel arrangements

Provide adequate facilities to host

audit team, inclusive of permissible

access (with sufficient notification)

In Person Audit Considerations



Documentation of the

Assessment

Confirm that all information has

been received

Verify that information is sufficient

and if not notify auditee

Develop draft report of

observations and findings

Review jointly report and amend as

needed

Schedule and conduct a final review

with key personnel for acceptance

Respond to information requests

and protect documentation as

applicable

Archive all provided documentation

in a searchable knowledge base

Review reports with Assessor and

management teams

Prepare responses or plans to

remediate findings

Develop lessons learned for future

audits

Documentation of the Assessment

11

Part 4 Next Steps


• Observation - Items which are in violation of the contract, but could be considered a violation of a best

practice or an item which could be an area of concern. These may also be sources of items which may be

sought to be added into contract language. An observation may be used at a later time during contract re-

negotiations as well.

• Finding - Something which is not an expected outcome based upon the criteria within the contract. This

could be process, procedure, work instructions, SLA’s other documentation, training issues or other items.

• Depending upon severity Findings may be sought to be resolved by the vendor or be a cause to terminate the

agreement.

• Grades of Reports - Satisfactory = A, Satisfactory with Caveats = B-C, and Unsatisfactory = D/E (Fail)

Observations vs Findings and Grades

12


Analyzing the Data

Category Examples of Findings Remediation

Training Lack of training on processes,

procedures or work instructions

Ensure all processes are documented up to

date and relevant staff are trained with

evidence.

Documentation Out of date documentation of

processes / procedures

Ensure all processes are documented and

up to date.

Personnel Lack of sufficient background

checks

Ensuring all staff have background checks

which are current.

Off boarding of staff Access still active of employees

who have left the company

Ensure that all access is removed for all

staff who leave the company within 24 hrs.

Financials Lack of details or inconsistent

records

Ensure proper details are provided and

records are up to date.

Security / Privacy Poor management of sensitive

personal information allowing

access to unauthorized staff

Ensure that personal data is properly

protected to ensure no unauthorized

access.


Remediation Phase


Remediation Setting a realistic remediation

schedule

Agree on timeline and deliverables

– plan, milestones, evidence and

closure criteria

Understand who the points of

contact are.

Communicate expectations and

concerns throughout the

remediation phase

Schedule re-assessment of the

vendor

Ensure management buy in of all

remediation efforts including

availability of resources

Assign a audit remediation program

manager and individual owners

Ensure relevant controls are

deployed and procedural changes

are documented and acknowledged

by impacted personnel

Provide regular updates to Assessor

with evidence of each remediation

to Assessor

13

Summary


• When you outsource know your business risks and potential mitigations

• Test risk areas prior to contract with vendor

• Build risk mitigations into your vendor agreements

• Monitor risk areas during the contract period

• Determine what types of assessments best fit the risk assigned to a vendor.

• In Person Assessments = High Risk Situations but at higher cost

• Hybrid Questionnaire / Remote Assessments = Medium Risk situations at moderate cost.

• Self Assessment Questionnaire = Low Risk and low cost

• Both Assessors and Auditees have different perspectives during the process. Understanding both sides is

very important for successful outcomes.

• Plan for remediation efforts to improve the business relationship.

Session Summary - Starting with the end

14

Questions?

testing and monitoring vendors...vendor management system approved supplier list –authorized...

Documents