Created by: J. Stokey 1

Division of Career ServicesQuality Assurance Annual Report

Fiscal Year 2005 Fiscal & Program Annual

Quality Assurance Reviews

2

Division of Career ServicesQuality Assurance Annual Report

Part I

Fiscal Year 2005 Fiscal Quality

Assurance Reviews

3

DCS 2005 Fiscal Quality Assurance Review Sites

Berkshire: Berkshire Training & Employment Program (BTEP) -City of Pittsfield Boston: Boston Private Industry Council; EDIC - City of BostonBrockton: Brockton Area Private Industry Council (BAPIC)Cape Cod & Islands: Job Training & Employment Corp (JTEC)Central Mass: Workforce Central; City Manager’s Office of Employment and Training-City of

WorcesterFall River: Bristol County Training Consortium (BCTC); City of Fall RiverFranklin/Hampshire: Franklin Hampshire Employment & Training Consortium (F/HETC)Greater Lowell: Career Center of Lowell (CCL) - City of LowellGreater New Bedford: Department of Workforce Development Administration (DWDA) -

City of New BedfordHampden: Hampden REB; Hampden County Employment and Training Consortium

(HCETC) - City of SpringfieldMerrimack Valley: Department of Training & Development (DTD) -City of Lawrence Metro North: Metro North REB; Employment Resources, Inc. (ERI)Metro South/West: Metro South West Employment and Training Administration (MS/WETA)North Central: Employment Links, Inc. (ELI); Mount Wachusett Community

College (MWCC)North Shore: North Shore Career Centers; Southern Essex WIB (SEWIB); City of SalemSouth Coastal: South Coastal Career Development Administration (SCCDA) - City of Quincy

4

Fiscal Systems Reviewed for FY 2005

Accounting Systems and ReportingAccounting Systems and Reporting Cash and Grant Management and Grant Management Cost Allocation Plan and Cost Allocation Plan and Cost

Classification MethodologyClassification Methodology AuditAudit Property/Inventory LogProperty/Inventory Log Procurement and ContractingProcurement and Contracting Fiscal Policy and ProceduresFiscal Policy and Procedures Personnel Policies and ProceduresPersonnel Policies and Procedures Fiscal Monitoring Fiscal Monitoring

5

Quality Assurance Ratings System

NO FINDINGS

RECOMMENDATIONS/CONCERNS

FINDINGS

6

Snapshot of all Workforce Investment Areas by Fiscal Review Status

0%

20%

40%

60%

80%

100%

120%

No Findings 94% 88% 94% 100% 100% 94% 94% 94% 100%

Recommendations/Concerns 38% 25% 12% 0% 12% 38% 6% 25% 12%

Findings 6% 12% 6% 0% 0% 6% 6% 6% 0%

Accounting Systems

Cash & Grant

Management

Cost Allocation

PlanAudit

Property & Inventory

Procurement &

Contracting

Fiscal Policy & Procedure

Personnel Policy &

Procedures

Fiscal Monitoring

7

Fiscal Findings and

Recommendations/Concernsby

Category

8

Accounting Systems Detail

Program Income - No system in place to account for program income earned at career centers; program income not recorded on FSR. A system for accounting of program income must be established and program income reported on FSRs.

Financial Status Reports - Ensure that FSRs are submitted to DCS timely to comply with reporting policies.

80% Expenditure Requirement (2) – Expenditure requirement for WIA Dislocated Worker funds may not be achieved. Ensure that at least 80% of funds obligated/expended in year one of program period. Straight-line projection indicated that FY 05 WIA Adult, Dislocated Worker, and section 173 funds may not reach the 80% goal.

NEG Program Income - NEG program income should be reported in the quarter in which it is earned.

Reporting of Stand-in Costs - Incorrect Stand-in costs reported on March FSR; costs were those of a sub-recipient.

9

Cash & Grant Management Detail

Bonding for Fiscal Employees - Increase bonding amounts for employees with financial responsibility.

Monthly Reconciliation of Accounts - Accounts should be fully reconciled each month. The fiscal operator should not allow for up to $25 write-off of un-reconciled account

80% Expenditure Requirement - Unmet expenditure requirement for WIA Youth funds. Multi-year contracting and low rate of expenditure/performance by youth vendors impacted ability to meet requirement to expend 80% of grant amount in the first year of appropriation. Ensure that funds are expended within the allowable two-year funding period and meet expenditure requirements.

OJT Payment Procedures - OJT reimbursement (payment) procedures were not followed. Ensure that OJT payments are made in accordance with appropriate procedures.

10

Cost Allocation Plan & Methodology Detail

Tax-exempt purchases - Ensure that all purchases are tax-exempt.

Expenses not allocated to benefiting grants - Expenses in the amount of $424,800 have not been allocated to benefiting grants. Ensure that all expenses are allocated to grants.

11

Audit

There were no fiscal findings related to OMB A-133 Single Audit requirements.

12

Property / Inventory LogDetail

Fully Documented Property Inventory - A full inventory must be complete and fully documented.

Disposal of Property - Request technical assistance from DCS on how to properly dispose of outdated and unused property and equipment.

13

Procurement & ContractingDetail

Procurement Procedures - The policies of each entity (Administrative Entity and WIB) need to be revised to include language defining their particular role with regard to the youth contract RFP process. A formal agreement between the two entities should have a formalized “Statement of Work” or “Memorandum of Understanding” outlining the roles and responsibilities of each entity.Contract of $3,500 contains amendment clause that could result in contract exceeding the $5,000 procurement mark that requires a formal bid.

RFP Complaint Procedures – A formal written local policy on RFP protest procedures is needed.

RFP and Contract Award Process - RFP and contract award process currently in place should be strengthened. A policy on protest procedures should also be formalized.

WIA Youth Procurement and Contracting - Insufficient documentation of FY 05 Title I Youth procurement and contracting activity.

14

Fiscal Policies & ProceduresDetail

Fiscal Policies and Procedures - Some fiscal policies and procedures are not being followed; others need to be updated and fully implemented.

Petty Cash - Petty cash system should be closed out or a formal petty cash system with written procedures needs to be developed.

15

Personnel Policies & ProceduresDetail

Workers’ Compensation - Ensure that employees are covered under the City's’ Workers’ Compensation policy.

Pre-determined Time Allocations/Budgeting on Employee Timesheets - Develop an acceptable cost allocation method consistent with the requirements of A-122. Charges must be based on hours worked on a benefiting grant, not based on budget projections or other distribution determined before services are performed.

16

Fiscal MonitoringDetail

Sub-recipient Monitoring - Monitor sub-recipients’ revenue and expenses related to program income. Ensure that sub-recipients are in compliance with applicable OMB circulars and other Federal regulations.

17

Division of Career ServicesQuality Assurance Annual Report

Part II

Fiscal Year 2005 Program

Quality Assurance Reviews

18

Berkshire: BerkshireWorks; Berkshire Training & Employment Program (BTEP) Boston: JobNet, The Work Place; Boston Career Link; Mayor’s Office of Jobs and Community

Services (JCS)Brockton: CareerWorks; Brockton Area Private Industry Council (BAPIC) Cape Cod & Islands: Career Opportunities; Job Training & Employment Corp (JTEC)

Central Mass: City Manager’s Office of Employment & Training ; Workforce Central

Fall River: Bristol County Career Centers; Bristol County Training Consortium (BCTC)Franklin/Hampshire: Franklin/Hampshire Career Centers; Franklin Hampshire Employment & Training Consortium (F/HETC)

Greater Lowell: Career Center of Lowell (CCL) - City of Lowell Greater New Bedford: Greater New Bedford Career Centers; Department of Workforce Development Administration (DWDA)Hampden: FutureWorks; CareerPoint; Hampden County Employment and Training Consortium (HCETC)Merrimack Valley: ValleyWorks Career Centers; Department of Training & Development (DTD) Metro North: CareerSource; The Career Place; Employment Resources, Inc. (ERI)Metro South/West: Employment and Training Resources (ETR); Metro South West Employment and Training Administration (MS/WETA) North Central: Career Centers of North Central Mass.;Employment Links, Inc. (ELI); Mount Wachusett Community College (MWCC)North Shore: North Shore WIB (NSWIB); North Shore Career CentersSouth Coastal: South Coastal Career Centers & Career Development Administration (SCCDA)

DCS 2005 Program Quality Assurance Review Sites

19

Program Systems Reviewed for FY’05

Eligibility Determination Management Information System & MOSES Individual Training Account (ITA) EO/Non-discrimination / ADA / Grievance &

Complaint Workforce Investment Act Requirements Monitoring Systems - Sub-recipients /

Vendors Performance Measures and Goals Case Management On-the-Job Training Career Center Seminars Wagner-Peyser Services

20

Snapshot of all Workforce Investment Areas by Program Quality Assurance

Review Status

0%

20%

40%

60%

80%

100%

120%

No Findings 82% 100% 100% 100% 100% 100% 94% 100% 94% 100% 100%

Recommendations/Concerns 75% 63% 0% 38% 6% 18% 44% 44% 6% 0% 0%

Findings 18% 0% 0% 0% 0% 0% 6% 0% 6% 0% 0%

EligibilityMIS & MOSES

ITAEO/ADA/ Griecance

&

WIA Require-ments

Monitoring Sub-

recipients

Monitoring Perfor-mance

Case Manage-

mentOJT

Career Center

Seminars

Wagner-Peyser

21

Program Findings and

Recommendations/Concerns by

Category

22

Eligibility Determination Detail

Eligibility Verification Quality Control – Use Eligibility Verification worksheets for eligibility determination and income calculation and conduct quality control over the files to ensure customers’ verification documents are obtained.

Quality Control Youth Eligibility Records - Continue to provide training and technical assistance to youth providers and maintain the quality control procedures.

Income Calculation – An income computation form should be utilized to consistently record and document how income determinations are calculated.

Staff Training on Income Eligibility – Train staff in use of forms and develop quality control of files. Staff training recommended to review eligibility policy, procedures, and requirements on income inclusion and exclusion policy.

Citizenship/Work Authorization - Advised use of documentation other than voter registration for proof of authorization to work in the US. Customer file lacked appropriate documentation of authorization to work in the U.S. Work permits for Youth were used as proof of citizenship/work authorization without verifiable documentation.

Veteran Status Verification – Ensure DD214 documentation obtained for Veterans’ case files.

23

Eligibility Determination Detail (cont.)

Applicant Statements – Ensure applicant statement forms used for any required eligibility criteria contain all required elements.

Selective Service Documentation – Incomplete and inconsistent record of Selective Service documentation in customer file.

Labor Market Research - Ensure Labor Market Information is consistent with the justification for approved training and documented in the MOSES case plans. Labor market information sources such as CIS or the OOH to support training justification should be noted

Follow-up Services – Implement process to ensure that follow-up services are documented in the case file and/or MOSES.

Best Practices Guide - Develop a Best Practices Guide to maintain a consistent, comprehensive approach to eligibility determination and verification, including income calculation, forms, checklists, and procedures.

24

Management Information & MOSES Detail

Data Entry -Evaluate MOSES quality control systems; determine and implement modifications necessary to ensure complete and current data. Implement consistent, uniform approach to recording narrative service information.

Recording Services – Ensure career center staff record regular periodic services,

especially Case Management in MOSES General Services as well as in MOSES Notes. Develop standardized procedures, and increase oversight for recording data in MOSES, including documentation of case plans and corresponding services.

Completeness of Records - Increase quality control over MOSES records and Case Plans to ensure services are recorded accurately and effectively in MOSES. Ensure that Youth records reflect EEO/Grievance notification, resolve alerts, include assessment data, case plan, and services provided.

Title I Enrollments - Identify inactive Title I enrollments, review cases and determine appropriate action to take; accurately reflect participants’ status in MOSES system.

Youth Vendor Reporting – Develop appropriate system to gather, record and maintain timely required information from vendors on youth customers’ services. Enter MIS data, at least monthly, from youth sub-recipients.

25

Management Information & MOSES Detail (cont.)

MOSES Case Management and Case Plans – Use Case Plans and Case Management in MOSES for all Title I customers. Develop a system for using MOSES Case Management and Case Plans for WIA Youth; include procedures that identify the roles of youth vendors and career center youth staff responsible to record information in MOSES. Develop a plan to train staff and implement quality control measures to ensure use of MOSES Case Management and Case Plans.

Alerts – Increase quality control over MOSES alerts. Identify and resolve alerts timely. Alerts should be resolved on a continuing basis.

Follow-up Services – Implement consistent process to ensure that follow-up is offered and available to customers and recorded in MOSES.

Use of Labor Market Information - Provide staff training in LMI on both utilization of appropriate LMI sources and effective documentation. Increase the level of quality control and oversight over the usage and documentation of LMI.

Staff Training - Provide additional instruction and training in use of MOSES to record information, services, and notes.

26

Individual Training Accounts (ITAs)Findings & Recommendations/Concerns

There were no program findings related to Individual Training Accounts.

27

EO/Non-Discrimination/ADA/ Grievance & Complaint Detail

Recording EEO/Complaint Process Notification - Notification of EEO/Grievance Notification needs to be recorded as provided in all customer MOSES records. Ensure that the EEO/Complaint Process service is captured in MOSES for each customer. Develop a crystal report to identify any customers whose MOSES records are missing the EEO/Complaint Notification service and record where needed.

EEO/Complaint Process Notification Acknowledgement Forms – Notice of the Grievance/Complaint process form should be a standard component of each customer orientation event. Forms should be signed by customer, recorded in MOSES, and maintained in file.

28

Workforce Investment Act Requirements Detail

Year-round Youth Services – Ensure that the WIA Youth sub-recipients comply with contract agreements that require year-round youth services.

29

Monitoring Systems Sub-recipients and Vendors Detail

Monitoring Policy and Procedures – Provide a copy of local monitoring policy of sub-recipients. Maintain documentation of monitoring results, corrective action, follow-up and resolution as required.

Monitoring of Youth Program - Develop formal documentation of the performance review and oversight that occurs, especially regarding corrective action to achieve performance measures and goals.

Youth Program Services – Youth should be recruited into the vendor (year-round) programs with participation in those programs and other appropriate youth elements/activities as a pre-requisite for participation in the summer work experience to avoid participation in summer work experience, only.

30

Monitoring SystemsPerformance Measures & Goals

Detail

Adult and Youth Performance Measure Achievement – Implement corrective action plan to address achievement of Title I performance measure for Adults and Youth. Follow-up on Adults after exit and review status of current participants to work toward exit is needed.

Youth Performance on Attainment of Diploma and GED –Review youth goals and make adjustments to improve the performance measures.

Entered Employment for Persons with Disabilities and UI Claimants - Continue holding job fairs, expand employer contacts, and implement additional steps to increase placements.

Adult and Dislocated Worker Enrollments - Adult customers were above and dislocated workers were below planned enrollment goals; implement strategy to reallocate funding which would allow more dislocated workers to be served.

Youth Enrollments – Unmet Youth enrollment goals for FY2005, affecting 80% expenditure requirement; corrective action necessary to ensure that WIA Youth performance is consistent with program goals in sub-recipient contract agreement.

31

Monitoring SystemsPerformance Measures & Goals

Detail (cont.)

Enrollment and Performance Goals –Monitor effectiveness of corrective actions to address enrollment and performance issues; revise as needed to ensure meeting performance goals.

Title I Adult Enrollment – Monitor effectiveness of the corrective action that was implemented to assure achievement of Title I Adult enrollment goal.

Enrollments and Entered Employment Rates – Increase Title I Adult enrollments and Adult and Dislocated Worker entered employment rates. Employer Services Unit should continue working with employers and customers to improve outreach and services that will help increase the number of entered employments.

Youth Performance – Work with Youth vendors to develop plans for improving performance to meet goals, especially Older Youth Employed with Credentials and Younger Youth GED/Diploma rate.

32

Case Management Detail

Quality Control of Case Plans – Provide additional quality control over Goal and Assessment sections of Case Plans to ensure appropriate completion. Develop and implement standard operating procedures to address adequacy of case plans and case management as well as quality control over this function.

Case Records - Ensure case notes and case management documentation details are uniform, complete, and detail customers’ progress. Conduct increased quality control over completion of case plans, case management services, and utilization of MOSES to record information.

Adult, Dislocated Workers, and Youth Case Management - Ensure Case Management is accurately recorded in case records, and MOSES, to demonstrate adherence to required contact intervals. Ensure Case Management services are provided to youth by vendors and accurately recorded in MOSES in accordance with the required contact intervals.

Documentation of Vendor Services - Systems must be implemented to ensure Youth providers report case management information to demonstrate that services are being provided.

33

Case Management Detail (cont.)

Assignment of Case Managers – Ensure that all customers requiring case management are assigned a case manager, and the assignment is recorded in the MOSES.

Record Keeping - Case Management services are being provided as reflected in notes or other services, record of “Case Management” services in MOSES needs to be consistently recorded.

Training Justification – Conduct increased quality control over the completion of the case plans to ensure that the labor market information and justification for training is completed appropriately. ITA training justification should be relative to the specific skills of the customer and requirements of selected training complete with reference to the Labor Market Information.

34

On-the-Job Training (OJT)Detail

Compliance with WIA OJT Policy - Procedures in WIA OJT & Customized Training Policy (00-41) are not being followed especially Operational Guidelines #3, #4, #5. Must come into compliance with Policy 00-41.

Documentation of Local OJT Policies and Procedures – No manual describing policies & procedures was available. Must develop and implement a policy and procedures manual covering all aspects of the program.

Staff Training in LMI – Provide training for staff on the effective use of Labor Market Information.

Completion of OJT Customer Case Plans – All individual career development strategy forms (Case Plans) that support the need for OJT must be properly completed.

Occupational Classification – Utilize the O*NET SOC which has replaced the DOT.

35

Career Center SeminarsFindings & Recommendations/Concerns

There were no program findings, recommendations, or concerns related to Career Center Seminars.

36

Wagner-Peyser ServicesFindings & Recommendations/Concerns

There were no program findings, recommendations, or concerns related to Wagner-Peyser Services.