prepared by: project management office (pmo) · the project charter should include the engagement...

Governance of

Evaluation, Proof of Concept and Pilot Projects

Office of the Chief Information Officer (OCIO) Government of Newfoundland & Labrador

Prepared By: Project Management Office (PMO)

Government of Newfoundland and LabradorOffice of the Chief Information Officer

Project Management Office

Governance of Evaluations, Proof of Concepts and Pilot Projects

Table of Contents

1. Governance of Evaluations, Proof of Concepts and Pilot Projects ............................................... 2

2. Definition of an Evaluation ........................................................................................................ 2

2.1 Example .......................................................................................................................................... 2 2.2 Concept .......................................................................................................................................... 2 2.3 Initiation and Planning ..................................................................................................................... 3 2.4 Analysis and Evaluation .................................................................................................................... 3 2.5 Close Out ........................................................................................................................................ 4 2.6 Duration ......................................................................................................................................... 4 2.7 Critical Successful Factors ................................................................................................................. 4 2.8 Control Documents .......................................................................................................................... 4 2.9 RACI ................................................................................................................................................ 4 2.10 Evaluation Workflow Diagram........................................................................................................... 6

3. Definition of a Proof of Concept ................................................................................................. 7

3.1 Example .......................................................................................................................................... 7 3.2 Initiation ......................................................................................................................................... 7 3.3 Planning and Analysis ....................................................................................................................... 8 3.4 Design to Close Out .......................................................................................................................... 8 3.5 Duration ......................................................................................................................................... 8 3.6 Critical Success Factors ..................................................................................................................... 8 3.7 Control Documents .......................................................................................................................... 9 3.8 RACI ................................................................................................................................................ 9 3.9 Proof of Concept Workflow Diagram ............................................................................................... 10

4. Definition of a Pilot .................................................................................................................. 11

4.1 Example ........................................................................................................................................ 11 4.2 Concept ........................................................................................................................................ 11 4.3 Initiation to Close ........................................................................................................................... 11 4.4 Duration ....................................................................................................................................... 11 4.5 Critical Success Factors ................................................................................................................... 12 4.6 Control Documents ........................................................................................................................ 12 4.7 Pilot Project Methodology .............................................................................................................. 12 4.8 Pilot Initiative Workflow Diagram .................................................................................................... 13




Page 2

1. Governance of Evaluations, Proof of Concepts and Pilot Projects This document outlines the governance for Evaluations, Proof of Concepts and Pilot Projects for the OCIO. The objective of governance is to ensure consistency, accountability and proper authorization to undertake any of the above activities. It ensures that the structure, oversight and management processes are in place for the delivery of the expected benefits from an Evaluation, Proof of Concept or Pilot Project. Accountability is one of the key concerns as it needs to address the needs of all stakeholders.

2. Definition of an Evaluation1 The purpose of an evaluation is to perform an assessment that seeks to determine if a technology or a combination of technologies is the best possible fit for the needs of a given business problem. Based on a prepared list of criteria along with some practical experimentation, a technology evaluation makes it possible to determine if a technology would be helpful in solving the business problem identified. The actual evaluation is limited to a very small number of subject matter experts within a particular team.

2.1 Example

An example of an evaluation would be the Entrust Two‐Factor Authentication product. This product was evaluated to see if it would be a good fit for the HRMS project which had a requirement for a Two‐Factor Authentication technology. Entrust provided the OCIO with a 90 day evaluation license of their product. The product was installed and evaluated by a couple of subject matter experts, after which the environment was de‐commissioned.

2.2 Concept

A product evaluation can originate from various sources. It can be driven by the need of a project in flight, a perceived use in an upcoming project, or a desire from clients or other branches within the OCIO to evaluate a product for potential future adoption. In the future, a product evaluation could also be undertaken by the Enterprise Architecture Division in accordance with its Enterprise Architecture roadmap, currently being developed. All requests for evaluations must identify the requestor, the authorization for such a request, the sponsor (owner), and source of funding if there are any costs associated with the evaluation. If funding is required, funds must be secured before proceeding. These requests should also be accompanied by a business case. Requests from clients and other Branches

1 Defined by wiseGEEK (http://www.wisegeek.com/what-is-software-evaluation.htm)




Page 3

within the OCIO should be channeled through Client Services. This applies to evaluations that may be done by individual branches utilizing their own internal resources. Requests for evaluation will be submitted to the Project Management Office (PMO) where it will be assigned a project number (DTC) and have the appropriate ISC folders set up. Requests will be forwarded to the Executive Director of the Branch that will be doing the evaluation for approval to proceed.

2.3 Initiation and Planning

The Initiation and Planning phase of the evaluation identifies the resources required. In planning for the evaluation, all resources, including those from other branches, should be identified. This resource identification will be useful for Solution Delivery as well as other OCIO branches in managing its resource planning and capacity, and to determine the impact on operational requirements and projects. These resources should be included in an evaluation plan which also identifies the start and end dates, the key stakeholders, the key milestones and any dependencies and constraints. In this phase the Sponsor is confirmed and the goals, objectives and scope are defined and agreed upon in a Project Charter. The Project Charter does not have to be completed in the level of detail as in a project, but with the relevant information to obtain the approval of the Project Sponsor to proceed with the evaluation. It is also important that out‐of‐scope items are clearly documented and included in the Project Charter. A Steering/Working Committee is established with the Chair being the Sponsor. Depending on the complexity of the evaluation or the impact it has on a project in flight, a Project Manager may be assigned. Otherwise, an internal resource will be assigned to provide oversight. The Project Charter should include the engagement and communication to the other branches of the OCIO and other stakeholders. Monthly reporting as a minimum will be submitted to the PMO and a Dashboard will be created and distributed to the OCIO management.

2.4 Analysis and Evaluation

In this phase, the analysis and evaluation take place in accordance with the scope of work as outlined in the Project Charter. Any changes to the scope of work or time frames will follow the normal change‐control procedures. The project manager must then ensure that all documentation has been approved and stored in TRIM in the assigned folder.




Page 4

2.5 Close Out

In this phase, an Evaluation Report that has been approved by the Director of Enterprise Architecture and accepted by the Sponsor will be compiled and the final results communicated to all key stakeholders.

2.6 Duration

The total duration of an evaluation shall be no more than 45 business days.

The duration of the product installation, configuration and usage should be no more than 35 business days; and

The written evaluation of the product, including all documentation, should take no more than 10 days.

2.7 Critical Successful Factors

A Documented Request for an Evaluation;

A Sponsor/Owner;

Steering / Working Committee;

Commitment of Key Stakeholders;

Regular Status Reporting; and

Approval and Acceptance of Final Report.

2.8 Control Documents

Request for an Evaluation;

Status Reports; and Steering / Working Committee Meeting Minutes.

2.9 RACI

EvaluationRACIChart

Task# TaskDescription EA

PMO

Executive

Director

orSLT

Client

Services

ISC/IM

AS

OPS

Project

Sponsor

1.1 IdentifynewEvaluation R I R,C R R,C R,C R,C A

1.2 SubmitnewEvaluationrequestwithBusinessCaseandScopeDocument

R,A I I R R R R C

1.3 ConfirmfundingfornewEvaluation

I R A I

1.4 AcknowledgeEvaluationRequests I R,A I I

1.5 PrioritizeEvaluationRequests C R,A C C C C I




RACI R = Responsible A = Accountable C = Consulted I = Informed

Page 5

EvaluationRACIChart


PMO

Executive

Director

orSLT

Client

Services

ISC/IM

AS

OPS

Project

Sponsor

1.6 AssignProjectNumberandcreatenewTRIMfolder

I R,A R

2.1 CreatetheEvaluationPlan R,A I C C C C

2.2 CreateProjectCharter R,A I I C

2.3 EstablishSteering/WorkingCommittee(chairedbysponsor)

I I R,A I C

3.1 ConductAnalysisandEvaluation R,A I I I

3.2 EnsurealldocumentationanddeliverablesareapprovedandsubmittedtoTRIM

R,A I I I I I I I

4.1 CreateEvaluationReportandCommunicateResults

R,A I I I




Page 6

2.10 Evaluation Workflow Diagram




Page 7

3. Definition of a Proof of Concept2 A Proof of Concept is the delivery of a working system to prove that the technology works and functions as intended. It is usually small and may or may not include all functionality; instead, it refers to a partial solution that involves a relatively small number of users. Once complete, the PoC should be dismantled and considered complete after the results have been documented.

3.1 Example

An example of a PoC would be the OCIO Wireless Network. This PoC resulted in the installation of a wireless infrastructure into three boardrooms within the Higgins Line building. There were a small number of executive members that were given access to this network for testing after which a lessons learned document was created. That document along with other documented recommendations on how to proceed with a full wireless network implementation were delivered at the end of the PoC.

Concept A Proof of Concept can be the result of the need to test a product to ensure that it is performing as it is marketed in a controlled environment. In most cases PoCs originate from other branches of the OCIO or the SLT. All PoCs must have a Business Case and these should be submitted to the PMO by Client Services including requests from the OCIO Executives. If the POC is not a funded project, then the funding must be identified and funding secured prior to initiating a PoC Project. The PMO will assign a project number (DTC) and set up the appropriate ISC folders. The PoC shall follow the normal SDLC and the deliverables for each phase, which will be determined in the planning exercise.

3.2 Initiation

In this phase, the funding source is confirmed by the PMO, a Project Initiation Document (PID) is created, the Solution Delivery Director, Delivery Manager, and Project Manager are assigned and the engagement of the other branches of the OCIO is completed. The Project Sponsor, Owner, Vendors and key stakeholders are identified. A preliminary scope document is created and high level costs and time frames are estimated. Resources are identified and, if professional services are required to supplement OCIO staff, then a Project Opportunity (PRO) or Staff Augmentation Request (SAR) will be issued subject to funding availability. 2 Defined by Claromentis (http://www.claromentis.com/wiki/help:general:proof_of_concep)t




Page 8

3.3 Planning and Analysis

A Project Manager will be assigned from the Initiation phase and a detailed plan will be developed. At a minimum, a project plan, a resource plan, a stakeholder analysis and a Project Charter should be delivered from the planning exercise. The Project Charter must be signed by the Project Sponsor before any further work is done and would include the Communication Plan, the final agreed scope of work, the out of scope items, cost estimates and time frames. In the analysis, the Project Manager would have determined the tasks that need to be done in the planning of the PoC. These tasks would have been reviewed with the project sponsor and included in the Project Charter. The deliverables, as identified in the SDLC, may not apply to all PoCs, thus a preliminary list of deliverables ‐ specific to the PoC ‐ should be created during the planning process. These deliverables will then be finalized in the Design to Close phases of the PoC. It should also be noted that there are no mandatory deliverables for PoC projects, as it will depend on the type of PoC; however, once the list of deliverables is finalized the team must complete them.

3.4 Design to Close Out

In the Design to Close phases of the PoC, the Project Manager, in consultation with the project team and sponsor, will finalize the deliverables that apply. The rigor that applies to a project may not apply to a PoC; however, the details will be determined in consultation with the stakeholders, which includes, at a minimum, the OCIO branches.

3.5 Duration

The total duration of a PoC shall be no more than 100 business days.

Duration of the product installation, configuration and usage should be no more than 90 business days; and

The written evaluation of the product, including all documentation, should take no more than 10 business days.

3.6 Critical Success Factors

Business Case;

Approved Funding;

Project Sponsor;

Commitment of Sponsor/Owner;

Scope Control;

Steering / Working Committee;

Management of Risks & Issues; and




Page 9

Compliance to Processes.


Project Charter;

Stakeholder Analysis;

Project Change Requests;

Status Reports;

Project Plan; and

3.8 RACI

ProofofConceptRACIChart


PMO

Executive

Director

SLT

Client

Services

ISC/IM

Project

Manager

AS

OPS

Project

Sponsor

1.1 IdentifynewPoCrequest R,A I R,C R,C R R,C R,C R,C C

1.2 DevelopBusinessCaseandsubmitwithnewPoCrequest

R,A I I R,I R R R R I

1.3 ConfirmfundingfornewPoC I R,A I I I I I I I

1.4 AcknowledgePoCRequests R,A

1.5 AssignProjectNumberandcreatenewTRIMfolder

I R,A I R,I I I

2.1 CreateProjectInitiationDocument(PID)

I R I I

2.2 CreatePreliminaryScopeDocument I I R,A

2.3 Determineifprofessionalservicesarerequired I I I C I I R,A I I I

3.1 ConductPoCPlanningC C C R,A C C C

4.1 FinalizeandcompleteallapplicableSDLCdeliverables C C R,A C

4.2 DocumentallfindingsandsubmitalldeliverablestoTRIM R,A

4.3 FinalizeresultsofPoCandcommunicatetoallstakeholders I I I I I I R,A I I I




Page 10

3.9 Proof of Concept Workflow Diagram




Page 11

4. Definition of a Pilot3 A pilot is an initial prototype of a system into production, targeting a limited scope of the intended final solution. The purpose of a pilot project is to test, often in a production environment, whether the system is working as it was designed while limiting business exposure.

4.1 Example

An example of pilot would be the Windows 7 Upgrade Pilot. This pilot included a small user base within the OICO to help the OCIO discover any unknown impacts to existing user hardware and desktop applications. The results of this pilot were used to determine challenges expected to be encountered during a corporate deployment and help develop solutions to these challenges.

4.2 Concept

A pilot is a project and is subject to all the rigors of the Project Management Life Cycle as well as the OCIO’s SDLC. At the end of a pilot, all the deliverables would have been met. The key difference is that in a pilot, the solution is rolled out into production to a limited set of users. All pilot projects will start with a Business Case submitted to the PMO from Client Services. These projects would have been on the OCIO’s funded project list. If a request for a pilot is received and it is not funded, then the funding must be secured prior to project initiation.

4.3 Initiation to Close

The activities from Initiation to Close will follow the OCIO SDLC. All deliverables, control documents and processes can be found on the PMO website.

4.4 Duration

The total duration is generally no more than 180 business days depending on scope of work and business impacts.

Duration of the product installation, configuration and usage should be no more than 170 business days depending on scope of work and business impacts; and

The written recommendation as to how to proceed, including all documentation, should take no more than 10 business days to compile.

3 As defined Yahoo Answers - Neeraj Bihari




Page 12

4.5 Critical Success Factors

Business Case;

Approved Funding;

Project Sponsor;

Commitment of Sponsor;

Qualified Project Manager;

Approved Project Charter;

Project Plan;

Steering Committee; and

Compliance with the SDLC.


A complete list of control documents that may be applicable to pilot projects can be viewed on the OCIO website at: http://www.ocio.gov.nl.ca/ocio/pmo/control.html

4.7 Pilot Project Methodology

Pilot projects must follow the normal SDLC methodology as outlined on the OCIO website. For a detailed breakdown of the SDLC methodology and related processes, please visit the following URL: http://www.ocio.gov.nl.ca/ocio/pmo/sdlc.html




Page 13

4.8 Pilot Initiative Workflow Diagram