pacses preliminary turnover plan - pa - emarketplace 02-15... · this preliminary plan is intended...

APPENDIX O

PACSES Client Infrastructure Network Maintenance Preliminary Turnover Plan

PACSES Preliminary Turnover Plan

Prepared for the Commonwealth of Pennsylvania

Prepared by Lockheed Martin

03/17/2015

This preliminary plan is intended to help facilitate the transition and turnover of all PACSES

client network services and infrastructure to the PA commonwealth or its agent.

of 12

Purpose:

Prepare Preliminary Turnover Plan at direction of Commonwealth and submit plan to the

Commonwealth for approval.

Scope:

Lockheed Martin understands the overall PACSES mission of supporting the children of

Pennsylvania through effective and efficient automated child support enforcement services. Our

involvement with the PACSES program, from its inception and continuously since then, gives us

a solid understanding of the evolving needs and purpose of the program. The scope of this

specific effort is the maintenance of the PACSES network. A high level overview of the basic

PACSES network is presented below, showing connectivity relationships among the primary

PACSES user community.

PACSES NETWORK

County

Offices

IBM

UNIVERSITY

SCDU

OCSE

FPLS

PACSES

Host

Willow Oak Building

CIS Host

(Web TS)

CTNTB

Internet

ISP

NOC Router

BP RouterEdge

Router

BP Router

DPW

Firewall

Server

Farm

CSWS

Data Warehouse

Docushare/PTS

Adobe LiveCycle

PIM

PennDot

Commonwealth

Enterprise

Network

Labor &

Industry

Exchange Servers (Email)

PACSES Home Page

Web Servers

PACSES TV (Master)

FTI / E-Reports

QUICK

OCSE

IVR

Network Support Staff

Ollie Bldg. Connection (CW)

Code 1

QPlex

AVR

TRACE

KidStar

FXR

OTC

BCSE

Comptroller

DPW

1 Tech

ParkState.pa.us (State Home Page)

State Mail Servers (CWOPA)

BPOA

Credit

Bureaus

FCR

BIS

DOR

FIDM

Treasury

February 16, 2015

Remote

Users(Connectra)

File Servers

PACSES TV

M & H Drives

DPW

Router

CAO

Sungard

Recovery

Services

Firewall

Camp Hill

(Apps Vendor)

PACSESUNISYS

** Note: Connectivity to the PACSES Host (Unisys) is only via

the CEN through DPW. No inner-building connection exists.

**

File Server

PACSES TV

H Drives

Understanding:

Lockheed Martin is cognizant that we must continue PACSES maintenance and operational

support systems at the optimum level while executing our turnover plan. In addition, we will

work collaboratively with the Commonwealth or its agent to ensure that staff time is utilized

efficiently and that all turnover deliverables are provided expeditiously.

The methodology to transition the tasks to the Commonwealth or its agent will ensure continued

production and operation as well as effective management of the business functions. We will

include the tasks required to turn over the PACSES network and associated IT Services while

avoiding any disruption of services for the user community. In addition, we will provide detailed

of 12

procedures required to turnover system operations and support for the ongoing production

environment. Some of these include application development, testing and training procedures,

data conversion, system documentation, requirements definitions, and tools and utilities.

Approach:

First, our staff must develop and maintain a schedule to ensure that the appropriate takeover staff

is trained, and that their knowledge of the program is sufficiently institutionalized to operate and

maintain the PACSES network independently and without disruption. Second, we must work

closely with the takeover management staff to ensure that they understand the full scope of their

responsibilities to the many entities of the PACSES users, management, and oversight boards.

At a very high level, we will turn over the largest and most critical and complex systems first,

descending through all functions to the smallest and least complex systems and services. This

approach will permit the incoming contractor the longest time possible during the turnover

period to correct any knowledge transfer deficiencies associated with the particular system

turnover. We are most concerned that the vital services associated with these large and complex

systems can be maintained by the incoming contractor and turning these over first allows us

additional support time with that contractor’s staff.

A snapshot of the knowledge transfer process will include the following:

Documentation Study – review all project & system documentation

Discussion Forums – Q & A in group settings to reinforce study, internalize information,

and acquire further knowledge

Walkthroughs – Participate in system and technical overviews as well as detailed

technical functions

Observations – Watch demonstrations of specific operations and functions

Shadowing One-on-One – Follow incumbent contractor’s staff “hand-in-hand” and

“looking over the shoulder”

Reverse Shadowing – Commonwealth or its agent will manage, operate, and provide

support with incumbent looking on

High-level outline of the turnover approach and plan objectives are listed below:

Functional Areas Turnover Approach and List of Objectives

Project Management Discuss the following with new functional manager:

Organizational Charts

Review all roles and responsibilities

List of stakeholders & Points of Contact (POCs)

IT Operations

Review task objectives

Deliverables

Scheduled Milestones (PC refresh, Server upgrades…etc.)

SLA and SLO transition criteria

Hardware Inventory Walk through documented list of all hardware.

Network: Routers, Switches, Circuits

of 12

IVR: Servers and circuits

IT Security: Firewalls, Proxy Servers

End Users: Servers, Desktops, Laptops, Printers, Scanners

Network Service &

Support

Demonstrate access to and share administration and current

configuration of routers, switches, and circuits.

IVR Management Demonstrate access to and share administration and current

configuration of IVR servers and circuits.

IT Security Services &

Support

Demonstrate access to and share administration and current

configuration of firewalls and proxy servers

End User Services Demonstrate access to and share administration and current

configuration of Servers, Desktops, Laptops, Printers, and

Scanners

Turnover administrative

access to Commonwealth

Create admin accounts for Key Personnel and apply access to all

essential hardware.

Verify admin level of access on all critical hardware

Commonwealth takeover

Client Infrastructure

Operations

LM will be available as back-up support while the

commonwealth’s key personnel manage and support the client

infrastructure network, IVR, IT security and end user services.

Preliminary Turnover Plan:

The Preliminary Turnover Plan provides an outline of the turnover approach. It will include a

breakdown of the functional areas for turnover, an inventory of assets (documents, materials and

equipment), the schedule, staffing and key resource requirements, responsibilities of the various

parties, identify any risks and mitigation strategies, identify regular interim assessment tracking

and reporting activities, and list criteria for assessing successful knowledge transfer. For the

purpose of assuring the security and confidentiality of the network, the preliminary plan will not

include any detailed network documents containing such information as IP addresses, system

names, and operating systems. Instead this information will be sent to the DPW project manager

securely under separate cover.

of 12

1. Project Management

1.1. Organization – Key Technical Leadership

1.2. Turnover Roles & Responsibilities

1.2.1. PACSES Project Manager (BIS)

Oversee turnover activities, coordinate support, and provide PA staff to support

the turnover plan.

1.2.2. PACSES Program Manager (LM)

Oversee turnover activities, coordinate support, and provide staff to support the

turnover plan. Report turnover completion to PACSES Project leads.

1.2.3. Turnover Team Lead – (LM Project Manager)

Ensures delivery and has overall technical responsibility for turnover of all

PACSES client infrastructure network and associated services.

1.2.4. Network Lead

Ensures technical delivery and turnover of Network (Routers, Switches, Circuits)

1.2.5. IVR Lead

Ensures technical delivery and turnover of IVR (Servers & Circuits)

1.2.6. IT Security Lead

Ensures technical delivery and turnover of IT Security (Firewall Servers &

Appliances)

1.2.7. End User Services Lead

Ensures technical delivery and turnover of all Servers, Desktops, and Printers

(Servers include Windows Domain Controllers, E-mail, and county file servers)

1.2.8. Service Desk Support

Ensure technical delivery and turnover of the PACSES help desk

1.2.9. Content Delivery Services

Ensure technical delivery and turnover of Database & Remote Access (Citrix)

1.3. IT Operations

1.3.1. Task Objectives

of 12

Task Task Objectives

Task 1. - Knowledge Transition Ensure a smooth transition of responsibilities;

Complete knowledge transfer;

Establish strong accountability controls;

Mitigate risk to the Commonwealth, DPW, clients, and

taxpayers; and Establish operations in the facility

provided by the DPW.

Task 2. - Project Management,

Communications and

Documentation

Create and maintain task lists and project plans

Maintain up-to-date inventory information and

documentation of all hardware (including county

equipment such as desktops and printers), software,

telecommunications lines and circuits for the PACSES

infrastructure

Coordinate activities with other PACSES suppliers

Coordinate activities with Commonwealth and county

staff

Manage user accounts (Adds, Changes, and Deletes) for

client infrastructure network, firewall, servers, e-mail

system and client platform

Task 3. - Consolidated Management

and Support for PACSES Client

Infrastructure Network Disaster

Recovery, Backup and Restoration

for PACSES Owned Servers

The disaster recovery plan kept current and validated

Backups created and systems restored from backup when

needed

Task 4. - Information Necessary to

Procure Service / Maintenance

Contracts, Client Infrastructure

Network Hardware and Software

Provide technical support, information and assistance on

various options for client infrastructure network hardware

and software procurements, and service/maintenance

services, as well as the annual budget planning process

Task 5. - Options for Client

Infrastructure Network Upgrades

and Application of Emerging

Technologies, Implement and

Support Approved Changes and

Upgrades

Options identified for client infrastructure network

upgrades and application of emerging technologies

Approved changes and upgrades implemented

Client Infrastructure Network usage monitored and

expansion needs identified

Task 6. - Knowledge Transfer of

Infrastructure Management and

Support to Commonwealth and

County Technical Staff and End

Users

Written instructions on the management and use of

WAN/LAN and Desktop for end users

Orientation provided to Commonwealth and County

Technical Staff on newly installed equipment or upgrades

as needed

Task 7. - Level II and III Service

Desk Functions, Including Account Efforts to resolve problems coordinated with Level I

Service Desk

of 12

Administration PACSES client infrastructure network support calls

triaged (Level II)

PACSES client infrastructure network support call

problems resolved (Level II)

Efforts to resolve client infrastructure network problems

coordinated with other suppliers, when needed (Level III)

To ensure problems escalated to other suppliers are

satisfactorily and timely resolved


and Support for All PACSES Client

Infrastructure Network Equipment

Installations / Relocations at

PACSES Remote Sites, PACSES

Network Operations Center and

County Offices

Management of all PACSES client infrastructure network

equipment installations and relocations

Obsolete, unneeded or replaced equipment disposed of

properly


and Support for All PACSES Client

Infrastructure Network Devices

Ongoing operational monitoring and appropriate response

to solve client infrastructure network problems

Task 10. - Consolidated

Management and Support for All

PACSES Servers Across the State

Server performance monitored and tuned

Server(s) upgraded / expanded

Servers to be secure


Management and Desktop Support

for PCs in the County DRS Offices

“Default” workstation configuration for each of the

desktop platforms, for example Windows XP and

Windows 7

Client platform performance monitored and tuned

Client platform(s) upgraded / expanded

Client platforms to be secure


Management and Support of All

Client Infrastructure Network

Software, Including Release

Upgrades

Software updated with current support releases

Software problems fixed

Documentation (log) of Software configuration and all

changes

Software media to be secure


Management and Support for the

County LAN Interfaces to Local

County Data Processing Client

Infrastructure Networks

Interfaces with the PACSES client infrastructure network

reviewed

Interfaces added or expanded, as needed


Management and Support for All

Management and support of the PACSES e-mail system

Support of the internet access

of 12

PACSES Internet, E-Mail, Web

Access, Dial-up Connectivity and

Monitoring of Client Infrastructure

Network Activity

Management of Citrix connectivity

E-mail and web access to be secure


Management and Support of All

DRS, and PACSES Sites Printing

Operations

Assistance with resolving county printing issues and

answering county printing questions



Voice Gateways

Monitor the voice gateways

Troubleshoot problems with county Telephone companies



Cisco Call Managers and IVR

Platform

Verification of all batch downloads to the State Collections &

Disbursement Unit (SCDU)

Scripts created, changed and updated


Management and Support for

Content Delivery Network (CDN)

Support for IPTV

Task 19. - Client Infrastructure

Network Support for PACSES

Applications Testing and

Production Environments

Support for PACSES applications testing and production

environments


Maintenance, Management, and

Support of the Virtual Private

Network (VPN), the Internet

Firewall, and the Business Partner

Firewall

Serve as a liaison with County DRS Administrators and their

data processing operations to ensure child support program

security

Ongoing management of Infrastructure Security

Plans for Infrastructure security developed and maintained

Approved infrastructure security projects / upgrades

implemented


Maintenance, Management, and

Support of the Client Infrastructure

Network Anti-virus Software and

Definitions, Including Release

Upgrades

Anti-virus and security systems maintained and hot fixes and

patches implemented

Anti-virus software installed on all new / replacement servers,

workstations and AVRs

1.4. Deliverables

Review each deliverable for customer expected content and schedule for delivery.

Capture process for all gathering of information and share access to systems that auto-

generate reports.

1.4.1. Daily Network & Server Status report

This report is delivered daily via e-mail. Sample report and process to construct it

will be outlined during one-on-one training sessions.

of 12

1.4.2. Weekly Activity Report

This report contains detailed status of all activities and current projects. The

report is generated by the on-site project manager and delivered the DPW project

manager.

1.4.3. Monthly SLA/SLO report

This report updates the DPW project manager on all SLA/SLO status. See sample

below:

1.4.4. Internet Use & Abuse report

This detailed report outlines inappropriate and non-business traffic on the

network.

1.4.5. Post IRS audit corrective action plan

On a cycle of every three years, the PACSES network is audited for security

standards and compliance. Corrective actions are documented and implemented

Target

%

T

y

p

e

Actual %

99.9%

99.8%

# Open Closed # Que

11 5 6

100.0%

97.0%

99.0%

96.0%

# Incid #-Rpt #-Res

0 0 0

# Req

0

Opened Met N-Met

260 258 2

Opened Assign Closed

91 169 85

Level III Vendor Assign Closed

Microsoft 0 0

Cisco 0 0

Dell 15 13

Pomeroy 154 132

IBM 0 0

Svys Psb #-Resp Av-Resp

91 3 3

77 0 NA

Network Service (NSR's)

PACSES Network (NOC)

NSR's

Speed to Respond

Remedy (Monthly)

90.0%S

L

O

100.0%

S

L

O

100.0%100.0%

100.0%S

L

O

98.0%

County Circuit/Routers

Network Security Services

93.4%Level II/III Time to

Coordinate, Assign & Resolve95.0%

S

L

O

S

L

O

AV/Intrusion Protection (SVR's)

AV/Intrusion Protection (PC's)

Updates & Patch Mngmt (SVR's)

Updates & Patch Mngmt (PC's)

Security Incident Reporting

(ITB-SEC024)

Incident Management

Firewall Service Requests

TitleStatus Level

(Avg.)

Status Level

98.0%S

L

O

91.0%

99.5%S

L

A

Network Availability

Network Servers

NSR's

99.9%

Status Level

Closed

0

Current Month

Previous Month

90.0%

Customer Satisfaction/

Date Survey85.0%

S

L

O

100.0%

99.2%

of 12

throughout the contract period of performance. Any outstanding actions will be

documented and turned over to the commonwealth.

1.5. Scheduled Milestones

Significant projects are documented and scheduled with the DPW project manager.

1.5.1. PC/Desktop refresh

PC/Desktop hardware and software are continuously evaluated to avoid

obsolescence. At the direction of the DPW project manager and LM project

manager, PC/Desktops are upgraded and thoroughly tested to ensure compatibility

with the PACSES application. Any planned upgrades will be documented and

turned over to the commonwealth.

1.5.2. Server upgrades

Server hardware and software are continuously evaluated to avoid obsolescence.

At the direction of the DPW project manager and LM project manager, Servers

are upgraded and thoroughly tested to ensure compatibility with the hosted

applications. Any planned upgrades will be documented and turned over to the

commonwealth.

1.5.3. Network & Circuit upgrades

Network hardware and software are continuously evaluated to avoid obsolescence

and to ensure appropriate level of bandwidth to support performance

requirements. At the direction of the DPW project manager and LM project

manager, Routers, Switches, and circuits are upgraded to maintain network

availability and support utilization demands from the many PACSES applications.

Any planned upgrades will be documented and turned over to the commonwealth.

1.6. SLA/SLO transition

A thorough review of all SLA/SLOs will be conducted with key staff to ensure a smooth

turnover and to minimize downtime or impact to daily operations.

1.6.1. Network Availability (SLA)

Network engineers will share monitoring tools that capture the raw data and

ultimately used to document network availability.

1.6.2. Network Service Requests (SLO)

Project management will share the method of data capture and procedure used to

measure performance to meet the objective.

1.6.3. Network Security Services (SLO)



1.6.3.1. Anti-Virus & Intrusion Protection

1.6.3.2. Security patch management

1.6.4. Security Incident reporting (SLO)



1.6.5. Firewall Service Requests (SLO)



1.6.6. Help Desk Response Time (SLO)



of 12

1.6.7. Help Desk Level 2/3 response (SLO)



2. Hardware Inventory

The list below contains a current snapshot of the hardware supporting the PACSES

network

Components Totals Manufacturers Operating

Systems

Notes

Routers 79 Cisco Systems Cisco IOS

Switches 222 Cisco Systems Cisco IOS Core, 12, 24, 48 Port

Firewalls 4 Cisco Systems Cisco IOS ASA

Servers 126 IBM, Dell, Cisco Windows 2008 R2

Server SP2

AD, File, & Print

Content/WA

Engines

85 Cisco Systems Cisco IOS Audio/Video Content

Delivery

Workstations 3251 Dell, HP Windows 7 SP1 w/dual monitors

Printers 924 Kyocera, Lexmark N/A

3. Network Service & Support

Turn over administration and current configuration of routers, switches, and circuits.

3.1. Review configurations and maintenance procedures for network support

3.2. Create accounts for access to routers, switches, and circuits

3.3. Demonstrate access and share administration

4. IVR management

Turn over administration and current configuration of IVR and circuits.

4.1. Review configurations and maintenance procedures for IVR support

4.2. Create accounts for access to IVR and circuits


5. IT security services and support

Turn over administration and current configuration of IT security

5.1. Review configurations and maintenance procedures for IT security support

5.2. Create accounts for access to IT security services and appliances


6. End user services

Turn over administration and current configuration of all servers, desktops, printers, and

scanners

6.1. Review configurations and maintenance procedures for servers, desktops, printers, and

scanners

6.2. Create accounts for access to servers and local admin accounts for desktops


of 12

7. Turnover administrative access to Commonwealth

7.1. Create admin accounts for Key Commonwealth Personnel and apply access to all

essential hardware.

7.2. Verify admin level of access on all critical hardware

8. Commonwealth takeover Client Infrastructure Operations

8.1. LM will be available as back-up support while the commonwealth’s key personnel

manage and support the client infrastructure network, IVR, IT security and end user

services.

of 12

Relative 60 day schedule Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9

Project Management

Organizational Roles & Responsibilities

IT Operations

Review Task Objectives

Deliverables

Milestones

SLA / SLO review

Hardware Inventory

Network

IVR

IT Security

End User

Network Service and Support

Demonstrate access to and share

administration and current configuration

of routers, switches, and circuits.

IVR Management



of IVR servers and circuits.

IT Security Services & Support



of firewalls and proxy servers

End User Services



of Servers, Desktops, Laptops, Printers,

and Scanners

Turnover administrative access

Commonwealth takeover

Timeline for Preliminary Turnover Plan

pacses preliminary turnover plan - pa - emarketplace 02-15... · this preliminary plan is intended...

Documents