it project monitoring report · 2019. 7. 9. · it monitoring report: kandrive 2 legislative...

20
IT PROJECT MONITORING REPORT Kansas Department of Revenue KanDrive IT Project Quarter Ending September 30, 2016 A Report to the Legislative Post Audit Committee By the Legislative Division of Post Audit State of Kansas December 2016 R-16-012.2 CURRENT STATUS: CAUTION

Upload: others

Post on 10-Sep-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT PROJECT MONITORING REPORT

Kansas Department of Revenue KanDrive IT Project

Quarter Ending September 30, 2016

A Report to the Legislative Post Audit Committee By the Legislative Division of Post Audit

State of Kansas December 2016

R-16-012.2

CURRENT STATUS: CAUTION

Page 2: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

Legislative Division of Post Audit

The Legislative Division of Post Audit is the audit arm of the Kansas Legislature. Created in 1971, the division’s mission is to conduct audits that provide the Legislature with accurate, unbiased information on the performance of state and local government. The division’s audits typically examine whether agencies and programs are effective in carrying out their duties, efficient with their resources, or in compliance with relevant laws, regulations and other requirements. The division’s audits are performed at the direction of the Legislative Post Audit Committee, a bipartisan committee comprising five senators and five representatives. By law, individual legislators, legislative committees, or the Governor may request a performance audit, but the Legislative Post Audit Committee determines which audits will be conducted. Although the Legislative Post Audit Committee determines the areas of government that will be audited, the audits themselves are conducted independently by the division’s professional staff. The division’s reports are issued without any input from the committee or other legislators. As a result, the findings, conclusions, and recommendations included in the division’s audits do not necessarily reflect the views of the Legislative Post Audit Committee or any of its members. The division conducts its audit work in accordance with applicable government auditing standards set forth by the U.S. Government Accountability Office. These standards pertain to the auditor’s

professional qualifications, the quality of the audit, and the characteristics of professional and meaningful reports. The standards also have been endorsed by the American Institute of Certified Public Accountants (AICPA) and adopted by the Legislative Post Audit Committee.

LEGISLATIVE POST AUDIT COMMITTEE Senator Michael O’Donnell, Chair Senator Anthony Hensley Senator Laura Kelly Senator Jeff Longbine Senator Julia Lynn

Representative Virgil Peck, Jr., Vice-Chair Representative John Barker Representative Tom Burroughs Representative Peggy Mast Representative Ed Trimmer

LEGISLATIVE DIVISION OF POST AUDIT 800 SW Jackson Suite 1200 Topeka, Kansas 66612-2212 Telephone: (785) 296-3792 Fax: (785) 296-4482 Website: http://www.kslpa.org Scott Frank, Legislative Post Auditor

HOW DO I REQUEST AN AUDIT? By law, individual legislators, legislative committees, or the Governor may request an audit, but any audit work conducted by the division must be directed by the Legislative Post Audit Committee. Any legislator who would like to request an audit should contact the division directly at (785) 296-3792.

The Legislative Division of Post Audit supports full access to the services of state government for all citizens. Upon request, the division can provide its audit reports in an appropriate alternative format to accommodate persons with visual impairments. Persons with hearing or speech disabilities may reach the division through the Kansas Relay Center at 1-800-766-3777. The division’s office hours are 8:00 a.m. to 5:00 p.m., Monday through Friday.

Page 3: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million
Page 4: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

This work was conducted by Alex Gard, PMP, CISA; Clyde-Emmanuel Meador, SSCP, PMM; and Katrin Osterhaus, PMP, CIA, CGAP. If you need any additional information about the findings, please contact Katrin Osterhaus at the Division’s offices.

Legislative Division of Post Audit 800 SW Jackson Street, Suite 1200

Topeka, Kansas 66612

(785) 296-3792 Website: www.kslpa.org

Page 5: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 1 Legislative Division of Post Audit R-16-012.2 December 2016

The Kansas driver’s license and identification system is responsible for issuing and tracking driver’s licenses and identification cards for the entire state. It also tracks driving records for roughly two million individuals, including any restrictions, suspensions or revocations for those individuals. The system is one of KDOR’s most critical systems and must be available to law enforcement at all times. The KanDrive project includes the conversion of the current Kansas driver’s license system from a legacy mainframe to a modern, web-based software framework. The new driver’s license system will continue to issue and manage driver’s licenses and identification cards. According to the project plan, new technology will allow better access to ongoing and ad-hoc reporting needs. In 2007, the Kansas Department of Revenue (KDOR) initiated a project to replace its mainframe systems. That project, referred to as the Division of Motor Vehicles (DMV) Modernization Project, had a total estimated budget of $40 million. In 2009, KDOR awarded a contract to the 3M Company to develop and implement the system. The goal of the project was to consolidate the Division of Motor Vehicle’s three older information systems into one. The new system would process motor vehicle titles and registrations and track and issue driver licenses. The DMV project was to be rolled out in two phases: Phase one – New Motor vehicle titling and registration system Phase two – New Drivers’ license and identification system Phase one of the DMV project, which included the new motor vehicle titling and registration system, was deployed in May 2012. Originally scheduled to be deployed in July 2011, this system was implemented in May 2012. Because of a number of problems at that time (including long delays at some county treasurer’s offices, complaints about corrupted files, and clerks being routinely disconnected from the system). LPA completed a performance audit in October 2014 (R-14-010). However, KDOR officials report those issues have been resolved in recent years.

In November 2015, the department ended the DMV project before completing phase two—the driver’s license system. Originally scheduled to be deployed by January 2012, this portion was significantly behind schedule, leading to KDOR terminating its contract with 3M in May 2014. At that time, KDOR planned to complete the driver’s license system of the DMV project in-house

The Current KanDrive Project Was Originally Part of KDOR’s DMV Modernization Project Which Began in 2007

Overview of the KanDrive Project

The Purpose of the KanDrive Project Is to Replace KDOR’s Old Mainframe Kansas Driver’s License System

Page 6: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016

by November 2015, for about $2.1 million. Summary progress reports published through the Kansas Information Technology Office (KITO) described the project on hold and eventually as “stopped” by August 2015. By then, the department had “de-scoped” phase two from the DMV project, thus officially ending the DMV project.

The KanDrive project was created to complete the driver’s license portion of the former DMV project at an estimated $6 million with a scheduled completion date of December 2017. The KanDrive project received approval from the executive Chief Information Technology Officer (CITO) in November 2015, with a project cost of $6.1 million and a planned completion date of December 27, 2017. According to the original KDOR project budget, the project cost is fully state-funded, including about $1.9 million in internal and $4.2 million in external costs. Those external expenditures represent contracting costs for Allied Global Services, Inc. to supplement KDOR’s in-house IT staff for the completion of this project. K.S.A. 46-1135, directs our office to conduct continuous audits of ongoing information technology projects by state agencies, including systems development and implementation. Our primary objective is to identify, as early as possible, when a project is at risk of failure due to scope, schedule, cost, or quality problems, and to communicate that risk to the appropriate level of project leadership, legislative bodies, or other stakeholders to get those projects back on track. Our secondary objective is to evaluate whether monitored IT projects have adequately planned for the implementation of required security controls. In February of this year, we selected the KanDrive project from a total of 21 active projects across state agencies as our first monitoring project. We chose the project for several reasons, including prior problems on the DMV project, the estimated cost of KanDrive, and its criticality for KDOR and other agencies.

We Selected the KanDrive Project for Continuous Monitoring Due to Its Prior Problems, Criticality, and Cost

Page 7: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 3 Legislative Division of Post Audit R-16-012.2 December 2016

Authorized under K.S.A. 46-1135, our audits of ongoing information technology projects evaluate the health of the project regarding system development and implementation, and the project’s adherence to relevant state statutes, Information Technology Executive Council (ITEC) policies and guidelines, Kansas Information Technology Office (KITO) templates and instructions for IT projects, and international project management standards and guidelines. As part of our monitoring efforts for the most recent calendar quarter, we reviewed project documentation, read relevant KITO reports, and attended many of the key communication meetings (steering committee, monthly status, and bimonthly progress meetings, contractor status meetings) from July 1 through September 30, 2016. Lastly, we interviewed members of the project team and steering committee as necessary. Due to their continuous nature, these audits are not conducted in accordance with generally accepted government auditing standards. We provided the draft report for the period ending September 30, 2016 to KDOR on November 3, 2016. The department’s response is included as Appendix A on page 11. Our previous monitoring report included an assessment of the project’s planning and initiation phase which started last year, as well as execution activities through June 30, 2016. The report was published in September 2016. We considered scope to be satisfactory, but called out several issues within schedule, cost and quality categories. We evaluate the project across four major areas – project scope, schedule, cost and quality. The following summarizes our findings in those areas from our monitoring report: Scope (satisfactory): We found the scope for the project to be clearly

defined, with major improvement ideas set aside for a separate project.

Schedule (caution): We found the project had designated project management staff, but certain work segments were behind schedule while other work had been delayed.

Cost (caution): We found project management appropriately used change

control processes to reduce or increase project cost baselines, but those changes were not sufficiently disclosed outside the agency.

Our Quarterly Reports Evaluate the System Development and Implementation Status of the Project

The KanDrive Project Was in Caution Status in Our Previous Monitoring Report, Resulting in Several Changes

Methodology and Results of Prior Monitoring Periods

Page 8: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 4 Legislative Division of Post Audit R-16-012.2 December 2016

Quality (unsatisfactory): We found the project did not include sufficient consideration for IT security, and the decision to eliminate independent verification and validation assessments created a risk that quality could suffer.

Officials acknowledged issues related to cost and quality and planned on several improvements in response to those findings. KDOR staff planned to incorporate all required information in the quarterly reporting to KITO to remedy our cost-related finding. Additionally, KDOR planned to take a number of actions to better incorporate IT security into the project. These included adding security requirements into the project plan and creating several specific security-focused deliverables, such as a security plan, a security assessment, a risk assessment, and a business impact analysis.

Page 9: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 5 Legislative Division of Post Audit R-16-012.2 December 2016

LPA Assessment of the KanDrive Project (as of September 30, 2016)

We determined the overall project health for KanDrive to be in caution status. The status is unchanged from our previous monitoring report. We evaluated the project across four major areas —project scope, schedule, cost, and quality. With the exception of quality, these areas also are tracked by the Kansas Information Technology Office, or KITO. Appendix B on page 14 contains a glossary of frequently used abbreviations in this report. The scale below describes the categories we established for our assessment: Satisfactory status: The project generally meets applicable state laws,

policies, and guidelines, generally complies with project management best practices, and has no material issues in scope, schedule, cost or quality.

Caution status: The project does not meet several state laws, policies

or guidelines, has deviations or unrealistic milestones in scope, schedule, cost, or quality, or has weak or insufficient mitigation plans for known issues which could result in project failure.

Unsatisfactory status: The project is not in compliance with many

state laws, policies or requirements, or has scope, schedule, cost, or quality deviations that are sufficiently material and no mitigation plans, thus causing the project to be at significant risk of failure.

The table below provides a summary of our findings for this reporting period.

Area Summary of Assessment Satisfactory Caution Unsatisfactory Informational

The scope is clearly defined and major improvement ideas are set aside for a separate project.

x

KDOR started planning and executing a separate project with system enhancements for KanDrive.

x

KDOR holds weekly conference calls to discuss progress with the contractor.

x

The contract has insufficient penalty clauses to ensure MorphoTrust meets intermediate milestones.

x

Certain work segments are behind schedule and other work has been delayed.

x

Since inception, the project has been re-baselined quarterly, with a net cost increase of $370,000.

x

As is the case for all KITO-approved IT projects, the project does not budget for costs associated with staff spending less than 50% of time on the project.

x

QualityThe project increased its efforts on IT security this quarter, but the first deliverable is not due to be completed until the end of November.

x

Kansas Department of Revenue's KanDrive IT ProjectSummary of LPA Monitoring Findings as of September 30, 2016

Cost

Source: LPA review of project documents, interview s, and attendence of periodic project meetings

Scope

Schedule

Overall Project Status: CAUTION

Page 10: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 6 Legislative Division of Post Audit R-16-012.2 December 2016

As the table on the previous page shows, several findings are informational in nature. These items relate to problems we identified that cannot be fixed going forward, or issues with state policies affecting all KITO-approved IT projects. Therefore, these items do not affect our assessment of the current status of this project. The remainder of the report provides additional information for items listed in the table. We determined the project scope for KanDrive to be satisfactory. The status is unchanged from our previous monitoring report.

The scope is clearly defined and major improvement ideas are set aside for a separate project (satisfactory). A clear description of what a project includes and does not include is important to ensure everyone understands the end goal. Additionally, project management standards require that scope is properly controlled throughout a project’s life cycle (planning, execution, close-out). The scope for KanDrive involves a clearly defined replication of the current system’s functionality in a different platform. Additionally, based on our interviews and documentation we reviewed, the KanDrive project team prevents scope expansion by tracking ideas for system improvements separately and only adding enhancements that do not affect cost or time as part of a change management process.

KDOR started planning and executing a separate project with system enhancements for KanDrive (informational). The KanDrive project converts the current driver’s license system from a legacy mainframe to a modern, web-based software framework. In addition, KDOR has begun a separate project which builds on KanDrive, adding enhancements such as a sanctions engine and a cashiering function. These enhancements are intended to automate and speed up the current licensing process, increase its accuracy, and allow for better ad-hoc reporting. Draft materials for the separate enhancement project were submitted to KITO in August 2016. The project has an estimated cost of $3.3 million and is scheduled to be completed at the same time as the main KanDrive project (December 2017). KDOR began executing several tasks on the enhancement project prior to receiving formal approval from the executive-branch Chief Information Technology Officer. This does not appear to comply with state law or ITEC policy, which requires formal approval before a project can begin. However, the absence of this approval does not appear to create any additional risk for the project, as staff from KDOR and KITO are in regular communication about this enhancement project.

Project Scope: SATISFACTORY

Page 11: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 7 Legislative Division of Post Audit R-16-012.2 December 2016

We determined the project schedule for KanDrive to be in caution status. The status is unchanged from our previous monitoring report. Although KDOR officials are confident the project will meet the completion date of December 2017, we continue to think there is a significant risk the project will not be completed on time without further budget or scope changes.

KDOR holds weekly conference calls to discuss progress with its contractor (satisfactory). Since 2009, KDOR has used a contractor (MorphoTrust) to provide the state with its driver’s licenses and identity cards. The latest contract amendment (signed in May 2016) requires MorphoTrust to integrate the current Kansas driver’s license information and issuance system with the KanDrive project. The two parties conduct weekly conference calls to discuss and review key topics, action items, accomplishments, upcoming work, and other items. Information from these conference calls as well as other communications and site visits feed into the monthly and bimonthly meetings for the benefit of other KDOR project members. The contract has insufficient penalty clauses to ensure MorphoTrust meets intermediate milestones (caution). KanDrive project management staff identified potential delays and accuracy issues with the MorphoTrust portion of this project as a major risk early on, and identified it as one of the top five risks for the project to KITO. To reduce this risk, staff developed several mitigation steps, including listing specific timelines, an expectation of the work to be completed, and identifying escalation points of contact with MorphoTrust. KDOR’s contingency plan included using the escalation process and imposing penalties. Although the current contract amendment contains a general "liquidated damages" section, it does not include specific penalties for missing milestones along the way.

There may be good reasons not to include penalties for missing intermediate deadlines, including the long-term nature of the parties' contractual relationship (MorphoTrust is responsible for providing driver’s license production maintenance and support until 2025). However, should the contractor miss upcoming intermediate milestones associated with the KanDrive integration, the project could experience a schedule delay, and KDOR may not have any financial recourse until the end of the integration phase in December 2017 or later.

Certain work segments are behind schedule and other work has been delayed (caution). The Joint Committee on Information Technology (JCIT) has outlined a number of project monitoring measures, including several critical task and task completion rate

Project Schedule: CAUTION

Page 12: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 8 Legislative Division of Post Audit R-16-012.2 December 2016

measures. For example, JCIT considers a project in caution status when the task completion rate is between 80%-90%. A list of measures the committee adopted is shown in Appendix C on page 15. We could not easily determine how much the project was behind schedule, but noted the following issues:

The bi-weekly status report for the period ending September 23, 2016 showed two major sections behind schedule. The project team produces bi-weekly reports which evaluate the progress of the project’s various work sections. This September 2016 report showed the website and user interface application development (“UX, UI, and .Net Development”) improved from the start of the quarter but is still behind. The quality assurance and testing segment continued to lag behind schedule. For both segments, project staff have created mitigation plans. Project staff explained that several key staff changes as well as physically relocating to another building contributed to the delays with the quality assurance and testing work. Other work segments were shown as progressing on schedule.

The completion of the “data conversion and fallback option” work segment has been pushed back by three months. Project management staff identified the roll-out process (either a “flip the switch” or sequential approach) as a high risk. Additionally, project staff identified not knowing the level of effort needed to clean and convert data as one of two issues in its two recent quarterly reports to KITO. Despite these issues, a critical work segment called “data conversion and fallback option” has been pushed back to start seven weeks later, and finish three months later, without any increases in resources.

Delaying the completion of this work until June 2017 is risky because the final six months of the project are dedicated to getting approval from an outside entity—the American Association of Motor Vehicle Administrators (AMVAA) and final roll-out activities—before the system can be used. Although staff have expressed optimism that it will take less time than originally planned, they also identified the AMVAA approval process taking longer than six months as a project risk.

The overall schedule performance index (SPI) indicates the project is somewhat behind schedule. This project management statistic measures the ratio of completed tasks to planned tasks. A project is considered on schedule when the index is equal or greater than 1.0. As of September 30, 2016, the project’s overall SPI had slipped from 0.96 to 0.91 based on the newly revised baseline. As described above, the new baseline reflects certain tasks having been moved to the separate enhancement project, and revisions to several start and completion dates. Appendix D on page 16 summarizes key statistics for current quarter as well as previous quarters for comparison.

We determined the project cost for KanDrive to be satisfactory. The status is an improvement from our previous monitoring report. This is the result of improvements KDOR has made to include the

Project Cost: SATISFACTORY

Page 13: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 9 Legislative Division of Post Audit R-16-012.2 December 2016

change log history information to KITO as part of its quarterly reporting information. This allows KITO officials to understand the project’s current cost status.

Project management standards require staff to make reasonable cost estimates to complete project activities, including labor, material, travel, and other costs. The KITO project management plan template supports this best practice by requiring agencies to estimate project cost at completion. This section summarizes our assessments regarding cost estimates for the project. Since inception, the project has been re-baselined quarterly, with a net cost increase of $370,000 (satisfactory). Project management standards require projects to follow formal change control processes when a project cannot meet its planned scope constraints, schedule, or cost estimates. Change requests may be necessary when productivity or resources were estimated incorrectly or unforeseen issues emerge. When approved changes affect scope, schedule, or cost, project plans may need to be “re-baselined” to create revised schedule or cost estimates. KDOR has followed the proper process for discussing and approving changes to the KanDrive project budget. During this quarter, two change requests reduced the previous $7.1 million cost baseline to $6.5 million. Most of the reduction reflects costs that were shifted to the enhancement project mentioned earlier. KDOR officials submitted a historical change log to KITO officials as part of the required reporting materials for the end of this quarter. Appendix D on page 16 shows the quarterly cost changes since project inception. As is the case for all KITO-approved IT projects, the project does not budget for costs associated with staff spending less than 50% of time on the project (informational). Based on our prior report, project staff added several security-related tasks and deliverables (e.g. develop a security plan, conduct a vulnerability assessment) to the detailed project plan. However, the assigned resources have no additional dedicated funding for those tasks, indicating this work is done “for free.” Because the state’s planning guidelines only require agencies to include staff costs when they are associated with a main task at least 50% of the time, it is likely that the project’s costs are understated. Because this affects all IT projects that are subject to ITEC reporting, this is informational only. We do not expect to the agency to take any action.

Page 14: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 10 Legislative Division of Post Audit R-16-012.2 December 2016

The project increased its efforts on IT security this quarter, but the first deliverable is not due to be completed until the end of November 2016 (caution). ITEC policies 7230 and 7230A list the minimum security requirements expected to be implemented for state information systems. Examples include:

Conducting a risk assessment Ensuring access control (e.g. identification and authentication,

proper password construction, account controls) Setting up systems configuration standards (e.g. change control,

malicious code protection mechanisms) Setting up systems operations standards (e.g. fault tolerant

settings, restoration processes) Providing for physical security standards (e.g. secure storage or

transportation of media) We expressed concerns about this and informed project staff informally in April. Because we did not see substantial progress, we considered this area unsatisfactory in our previous monitoring report. Project documents continued to lacked information on how staff planned to become compliant with those and other IT security requirements, and a draft security plan was incomplete and inadequate. During this quarter, project management increased communication with security personnel and have added several new deliverables to the project plan. However, the earliest deliverable is not due until the end of November 2016 (development of a security plan). Several deliverable deadlines also coincide with the “go live” date for the project (verification of the security assessment, documentation of the business impact analysis). Additionally, none of these new tasks include resource estimates. Therefore, we cannot evaluate the progress or quality of this work.

Project Quality: CAUTION

Page 15: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 11 Legislative Division of Post Audit R-16-012.2 December 2016

APPENDIX A

Agency Response

On November 3, we provided copies of the draft audit report to the Department of Revenue. We have made several changes based on its informal and formal response. Its response is included as this Appendix. The agency does not believe the findings we have raised to be an issue for the project to be completed on time and within budget and as such indicated no further action plans to be necessary.

Page 16: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 12 Legislative Division of Post Audit R-16-012.2 December 2016

Page 17: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 13 Legislative Division of Post Audit R-16-012.2 December 2016

Agency Action Plan

1 SCHEDULE - The contract has insufficient penalty clauses to ensure MorphoTrust meets intermediate milestones (caution)

This is a long term contract with compensation to the vendor based on documents produced rather than milestones associated with system upgrade. KDOR has dedicated project management resources monitoring vendor progress on a weekly basis to ensure synchronization with the KanDrive schedule.

2 SCHEDULE - Certain work segments are behind schedule and other work has been delayed (caution)- The biweekly status report for the period ending 9/23/2016 showed two major sections behind schedule.- The completion of the "data conversion and fallback option" work segment has been pushed back by three months.- The overall schedule performance index (SPI) indicates the project is somewhat behind schedule.

Development & QA have been making terrific progress under documented recovery plans for each. Development is planning to wrap-up on-schedule within the next 60-days, and QA is currently planned to complete the backlog of testing by June 2017 as planned. The LOE required by the data conversion team has been revised downward as discovery & work has progressed, and the planned completion will not impact any dependent activities. The current SPI of .91 is within a nominal margin of 10% variance for a perfect project SPI of 1.00

3 QUALITY - The project increased its efforts on IT security this quarter, but the first deliverable is not due to be completed until the end of November 2016 (caution)

The November security deliverable will show as "completed on time", and per the agency Security Officer the majority of the remaining deliverables must be completed when the solution is at or near completion and have therefore been scheduled appropriately in the KanDrive project plan.

LPA FINDING

Itemized Response to LPA Recommendations

Audit Title: IT Project Monitoring Report - Kansas Department of Revenue KanDrive

Agency: Kansas Department of Revenue

Page 18: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 14 Legislative Division of Post Audit R-16-012.2 December 2016

APPENDIX B

Glossary of Frequently Used Terms and Abbreviations

The following list contains various abbreviations and a definition of those terms.

CITO - Chief Information Technology Officer. K.S.A 75-7205 through K.S.A. 75-7207, established a CITO for each of the executive, judicial, and legislative branches of government. The respective CITO reviews and consults with each their branch agencies regarding information technology plans, monitors compliance with all information technology policies, and coordinates implementation of new information technology, among other duties.

DMV - Division of Motor Vehicles. The Motor Vehicles Program administers Kansas law relating to vehicle titling and registration, motor vehicle dealer licensing, and driver’s licenses. The Division of Vehicles has three subprograms which include Administration, Vehicle Services, and Driver Services. The Administration subprogram oversees policy and procedure to ensure a safe, fair and equitable customer service atmosphere for Kansas citizens.

ITEC - Information Technology Executive Council. The 17-member Information Technology Executive Council is responsible for approval and maintenance of all information technology policies, IT project management procedures, the statewide technical architecture, and the state's strategic information management plan.

KDOR - Kansas Department of Revenue. The Department is a Kansas Cabinet-level agency and is responsible for collecting taxes and fees, administering Kansas tax laws, issuing a variety of licenses, and providing assistance to Kansas citizens and local governments.

KITO - Kansas Information Technology Office. KITO supports the statutory

responsibilities of the Executive, Judicial, and Legislative Branch CITOs and the state’s Chief Information Technology Architect by providing enterprise services across state government.

SPI - Schedule Performance Index. A measure of schedule efficiency expressed as the

ratio of earned value (how much work has been completed by a certain date) to planned value (how much work was supposed to have been completed by that date).

Page 19: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 15 Legislative Division of Post Audit R-16-012.2 December 2016

APPENDIX C

JCIT Project Measurement Guidelines

JCIT Policy 2, approved by the committee in 1998, establishes a number of specific measures to evaluate state projects in active status. The table below enumerates those measures.

Area JCIT threshold Condition

Critical Path10%-20% behind schedule

The project will be considered in a caution status

Critical Path20% or more behind schedule

The project will be considered in a red or alert status.

Task Completion RateCompletion rate of 80%-90%

The project will be considered in a caution status

Task Completion RateCompletion rate of 80% or less

The project will be considered in a red or alert status.

Deliverable Completion Rate

Completion rate of 80%-90%

The project will be considered in a caution status

Deliverable Completion Rate

Completion rate of 80% or less

The project will be considered in a red or alert status.

Cost10%-20% deviation from plan

The project will be considered in a caution status

Cost20%-30% deviation from plan

The project will be considered in a red or alert status.

Cost30% or more deviation from plan

If costs are 30% higher than planned, serious consideration should be given to stopping the project. JCIT should find specific approval of the agency head and approval of a rationale that strongly supports continuation of the project. JCIT should consider recommending that an independent 3rd party be obtained to conduct a project review and make recommendations to the agency head and JCIT regarding causes for the project deviation from plan, corrective actions needed, expected outcomes, and whether the project the project should be continued.

Actual vs. Planned Resources

Deficiency gap of 15%-20%

The project manager should be acting with the project sponsor to correct this condition. For some projects, the impact of this level of deficiency may be greater than indicated and be reflected in the other measures as well.

Actual vs. Planned Resources

Deficiency gap of 20%-25%

There should be a plan to show a compensatory change n resources or a plan to reduce the scope, costs and objectives for the project with approval of the agency head. For some projects, the impact of this level of deficiency may be greater than indicated and will be reflected in the other measures as well.

Actual vs. Planned Resources

Deficiency gap of 25% or more

A deficiency of this magnitude places project in jeopardy and 3rd party review should be considered if the impact is reflected in other measures. The project should not be permitted to drift awaiting a compensatory resources plan or a new reduced project scope plan. If a new project plan is developed, the new financial plan, return on investment and objectives to be achieved must recalculated and presented for review as well.

JCIT Project Measurement Guidelines

Page 20: IT PROJECT MONITORING REPORT · 2019. 7. 9. · IT MONITORING REPORT: KANDRIVE 2 Legislative Division of Post Audit R-16-012.2 December 2016 by November 2015, for about $2.1 million

IT MONITORING REPORT: KANDRIVE 16 Legislative Division of Post Audit R-16-012.2 December 2016

APPENDIX D

Summary Schedule and Cost Statistics For KanDrive

This table includes original and quarterly statistics for the KanDrive Project based on our review of internal project management reports for the quarterly time periods through September 30, 2016.

Calendar Year Quarter ending 12/31/2015 3/31/2016 6/30/2016 9/30/2016Cost Baseline - the approved version of the project budget.

6,355,372$ 6,629,524$ 7,156,869$ 6,505,518$

Planned Value (PV) - the authorized budget assigned to scheduled work (also known as Budgeted Cost of Work Scheduled)

4,139,459$ 4,386,321$ 3,221,845$ 4,537,634$

Earned Value (EV) - the measure of work performed expressed in terms of the budget authorized for that work (also known as Budgeted Cost of Work Performed)

1,536,518$ 2,438,435$ 3,094,942$ 4,108,142$

Actual cost (AC) - the realized cost incurred for the work performed on activity during a specific time period.

1,467,415$ 2,302,664$ 2,842,990$ 3,785,416$

Schedule variance (SV) - a measure of schedule performance expressed as the difference between the earned value and the planned value.

(2,602,941)$ (1,947,886)$ (126,903)$ (429,492)$

Schedule Performance Index (SPI) -a measure of schedule efficiency expressed as the ratio of earned value to planned value (a ratio of 1.0 or better is good).

0.37 0.56 0.96 0.91

Cost Variance (CV) - the amount of budget deficit or surplus at any given point in time, expressed as the difference between earned value and actual cost.

69,103$ 135,772$ 251,951$ 322,725$

Cost Performance Index (CPI) - a measure of the cost efficiency of budgeted resources expressed as the ratio of earned value to actual cost (a ratio of 1.0 or better is good).

1.05 1.06 1.09 1.09

Summary Schedule and Cost Statistics