afmd-12.19.3 guide to federal agencies' procurement of audit … · accepted government...

Preface

Passage of the Chief Financial Officers Act of 1990 (Public Law 101-576) was a significant step for- ward in improving financial management in the federal government. Besides establishing a chief financial officer (CFO) for 23 federal agencies, the act provides for improvements in accounting systems, internal controls, and financial information. One of the act’s major impacts is the requirement for the annual preparation and audit of financial statements.

The CFO Act provides that the financial audits are to be conducted by the agencies’ inspectors general (IGs) or by independent external auditors. At the discretion of the Comptroller General, GAO may audit federal agency financial statements in lieu of audits by IGs or independent external auditors.

This guide provides advice to IGs and other federal officials on contracting with independent public accountants (IPAs) for audit services. It is intended to give officials who are unfamiliar with federal procurements a basic understanding of how IPA contracts should be awarded. The guidance is largely based on GAO’s experience in contracting for audit services and on discussions with government personnel about the special requirements of the CM3 Act. It supplements the more general information provided in the National Intergovernmental Audit Forum’s 1988 publication “How to Avoid a Substandard Audit: Suggestions for Procuring an Audit” and should be read in conjunction with policy guidance issued by the Office of Management and Budget and GAO for the preparation and audit of annual financial statements. The guide is not intended to substitute for consultation with agency

Page 1 GAO/AFMB12.19.3

Preface

officials, including procurement counsels, who are responsible for ensuring that contracting actions are consistent with the requirements of applicable stazQ

Werner Grosshans Assistant Comptroller General

for Policy

Donald H. Chapin Assistant Comptroller General Accounting and Financial

Management Division

Page 2 GAO/AFMLS12.19+3

GAO,‘AFMD12.19.3

Cunknts

Preface 1

An Overview of 6 Federal IPA Highlights of the Federal Procurement 7

Service Process

Procurements

Developing the Request for Proposal

Deciding the Contract Form and Price Arrangements

Specifying Services and Deliverables Other Audit Services Background Information Included in the

RFP Timing of Audit Work and Milestones Independence Frequency of Competition Preproposal Conferences

14 14

16 17 18

18 25 25 25

Evaluating Proposals and Selecting a Contractor

The Technical Evaluation Panel Technical Evaluation Price Evaluation Best and Final Offers

27 27

;; 30

Monitoring and Evaluating

32

Contractor Work

Appendixes Appendix I: Information to Include in an 34 RFP

Appendix II: Example of a Technical Evaluation Form

36

Appendix III: Example of a Final Evaluation Worksheet

41

Page 4 GAO/AFMDl2.19.3

Contents

Figures Figure 1: Key Segments of the Federal Procurement Process

Figure 2: First-Year Audit Time Frames Figure 3: Second-Year Audit Time Frames Figure 4: Audit Tasks

10

20 22 24

Abbreviations

BARDR

CF-O CQTR

FAR FMFIA

GSA IPA IG OMB RFP TFR

basic audit requirements determination report

chief financial officer contracting officer’s technical

representative Federal Acquisition Regulation Federal Managers’ Financial Integrity

Act General Services Administration independent public accountant inspector general Office of Management and Budget request for proposal transaction flow review

Page 5 GAO/AFMB12.19.3

An Overview of Federal IF!A Service l+ocurements

A number of federal agencies may need to contract for audit services to comply with the audit requirements of the CFO Act. Fiscal year 1991 financial statements are due to the Director of the Office of Management and Budget (OMB) by March 31,1992, for (1) revolving funds and trust funds, and (2) to the extent practicable, offices, bureaus, and activi- ties that performed substantial commercial functions during the preceding fiscal year. The CFO Act also provides a schedule for the submission of audited agencywide statements by 10 agencies during 1991,1992, and 1993.

The CFQ Act specifies that the statements be audited in accordance with applicable generally accepted government auditing standards by the agency’s IG or an independent external auditor, as determined by the IG. Agencies without IGs are to be audited by independent external auditors, as determined by the agency head. The audit reports are to be submitted to the agency head by June 30 of the year following the audited fiscal year. The agency’s annual report containing the audit report and other financial information is due to OMB and the agency head 60 days after the audit report is submitted.

Contracting with a qualified independent public accountant (IPA) can help an agency meet specific deadlines established by the act. An IPA can be hired to perform all the work necessary to prepare the required audit reports or to work jointly with the IG (auditing some segments of an agency while the IG audits other segments). However, GAO’s experience in contracting for audit services suggests that obtaining quality, timely audits at a reasonable price could be a challenge to many agencies because they are unfamiliar with audit requirements and criteria for evaluating IPA qualifications.

The General Services Administration (GSA) is CUP- rently working to procure governmentwide financial support services, including financial audit

Page 6 GAO/AFMD-12.19.3

An Overview of Federal LPA Service Procnrementa

services similar to those discussed here. Agencies may be able to arrange for audit services under GSA’s contract by October 1991. Further information can be obtained by contacting GSA’s Federal Supply Service, Office and Scientific Equipment and Commodity Center, at (703) 667-2292.

Highlights of the Federal agency procurement requirements are spec-

Federal ified in various statutes and the Federal Acquisition

Procurement Process

Regulation (FAR). Contracting officers, contract specialists, and procurement attorneys cm provide invaluable guidance on these complex requirements. However, it is important for federal managers who will be involved in procuring IPA audit services to be familiar with a few important aspects of these regulations, which are discussed below.

Key Agency Officials

Three key officials directly involved in planning, awarding, and monitoring the contract are the contracting officer; the contracting officer’s technical representative (COTR), sometimes referred to as the government technical representative or project officer; and the contract specialist. The contracting officer has the highest level of responsibility, since his or her signature legally binds an agency to a contract. The COTR generally has day-to-day responsibility over all aspects of the contract, including writing work requirements, providing technical assistance to procurement officials, evaluating proposals, monitoring contractor progress, resolving problems, and handling other administrative matters, such as certifying invoices for payment. The CUI’R needs to be involved from the day the procurement process begins and, for IPA procurements, must possess the necessary accounting/auditing background to successfully perform these important functions. It is likely that the agency IG or the IG’s representative would

Page 7 GAO/AFMD12.l9.2

An Overview of Federal IPA !kvice Procurements

serve as the COTR. The contract specialist is responsible for ensuring that solicitation documents, contracts, and contract modifications are prepared in accordance with the FAR.

Publicizing Full and open competition is basic to federal pro- Contracts and curement. Encouraging many qualified IPAs to

Vendor Competition submit audit proposals increases the likelihood of obtaining a good quality audit at a fair price. To promote competition, the solicitation should be publicized through an announcement in the Commerce Business Daily well before a request for proposal is issued. Consideration should also be given to providing the announcement to a list of potential auditors compiled from general and professional directories.

The requirements regarding both publicizing contracts and obtaining competition vary according to the cost of the services involved. For contracts over $26,000, federal regulations generally require advertising the contract requirements in the Com- merce Business Daily to attract prospective offerors, Contracts between $10,000 ($5,000 within the Department of Defense) and $25,000 must be publicized at least through the posting of a solicitation synopsis in a public place. There are no specific publicizing requirements for contracts below these levels.

Contracts over $26,000 are also generally required to be fully competed prior to award. Limited competition should be used for contracts under $25,000 (known as small purchases), whereby vendors who offer the desired goods or services are contacted and asked to submit quotations. Quotations must be solicited from a reasonable number of vendors to provide the maximum practicable extent of competition, and to ensure that the purchase is advanta- geous to the government. For contracts under $2,600, an award may be made without soliciting

Page 8 GAO/AFMDIZ.19.8

K

Y

Y

An Overview of Federal IPA Service Procurements

competitive quotations, if the price is considered to be reasonable.

Under emergency conditions, or if a particular vendor has unique qualifications, agencies may be able to award sole source contracts without obtaining competition. Agencies are required to maintain written justification for sole source procurements.

Since most contracts for IPA services under the CFQ Act will exceed $25,000, they will generally be subject to the maximum publicity and competition requirements. However, small contracts may be appropriate to obtain technical expertise regarding a specific issue or product.

Method of Procurement

The primary method of federal procurement for professional services is negotiation, and this method is appropriate for IPA contracts under the CIQ Act. A negotiated procurement allows the government to conduct discussions with offerors regarding their proposals and provides an opportunity for offerors to revise their initial offer. Under negotiation, agencies can more thoroughly consider an offeror’s technical qualifications as well as its price.

Figure 1 presents an overview of the federal process for negotiated procurements from start to finish. This process, from the decision to contract to contract award, typically takes from 9 months to a year at most federal agencies.

Page 9 GAO/AFMIM2.1D.3

An Overview of Federal JPA E&rYice procnrements

Figure 1: Key Segments of the Federal Procurement Process

Designate Key Designate Key Slatf Slatf

. COTR . COTR

m Technical m Technical Evaluation Evaluation Panel Panel

c

Preproposal Conference (Optional) I

I

Technical and Price

Evaluation

- Compile All Evaluation Scores and Total

- Determine Competitive Range

l Notify Offerors Not in Competitive Range

Negotiations

l Request Best and Final OHer

l Final Evaluation

Monitor and Evsluate Contract Work

9 Hold Progress Meetings

. Review Workpapers

l Assess Compliance With Contract Provisions

* Review Draft Deliverable Products

Page 10 GAO/AFMIM2.lDd

An ihwrvkw of Federal IFA service Pltmcnmmen~

Contract Tsfpes Two primary types of contracts can be used to procure audit services.

+ Fixed-price: This type of contract provides for a ‘firm price that is generally not subject to adjust- ment. A fixed price contract places maximum risk on the contractor who must control costs to achieve a profit or avoid a loss. This type of contract is appropriate for purchases of goods and services where contract specifications are well-defined and unknown factors or problems are believed to be minimal. This type of contract might be appropriate for (1) initial contracts under the CM3 Act for agencies that have a history of preparing financial statements and subjecting them to audit and (2) subsequent years when agencies have gained experience with the expanded requirements of the act.

l Labor-hour: This type of contract obligates the government to pay the contractor on the basis of direct labor hours at specified fixed hourly rates that include wages, overhead, general and administrative expenses, and profit. Such contracts should only be used when the extent or duration of required work cannot be reasonably estimated. Under labor-hour contracts, the government and the contractor share the risk of exceeding antici- pated costs. The contract must include a ceiling price that is set at an appropriate level to promote contractor efficiency. Contractors are reimbursed for other direct charges, such as travel, in accordance with federal regulations.

Relationship of Contractor to Agency Personnel

Under the FAR, contractors cannot supervise gov- emment employees, and a government employee cannot supervise the detailed work of contractor staff+ This is to prevent government agencies from inappropriateIy supplementing their work force and to prevent contractors from inappropriately assuming responsibility for the work of government employees

Page 11 GA0/AFMD12.19.3

An Overview of Federal IPA Service Procurements

Socioeconomic Programs

The FAR also encourages procurement opportum- ties for small, women-owned, and minority-owned businesses, and certain procurements may be set aside for them. Since many of these businesses may not have the resources to serve as prime contractors on contracts as large as those required under the CFO Act, such procurements may specify their par- ticipation through subcontracts or joint ventures with larger firms. However, the use of subcon- tracting cannot be so extensive as to prevent the prime contractor from qualifying as the principal auditor under generally accepted government auditing standards.

The remainder of this guide provides information on developing a request for proposal, evaluating proposals and selecting a contractor, and monitoring and evaluating contract work.

Page 12 GAO/AFMDltl%.3

GAO/AFMD12.19.3

Developing the Request for Pmposal

In order to begin the process of negotiating with potential contractors, an agency issues a written request for proposal (RFP). RFPs set forth all terms, conditions, and evaluation criteria as well as the scope of the work required. These requirements must be agreed upon by agency officials and pre- cisely described. General or incomplete specifications in RFPs will undermine the quality of the final products. Appendix I contains a list of the specific information that agencies should consider including in their RFPs for audit services.

Based on GAO’s experience in dealing with both agency and contractor personnel, GAO has identified several areas related to developing RFPs for audit services that require particular attention by responsible agency officials. These are discussed in the following sections.

Deciding the Contract Form and Price Arrangements

As a part of acquisition planning as prescribed in the FAR, agencies must carefully consider the type of contract and contract pricing they will use before they prepare solicitations. The nature of the audit services and the requirements of the CFQ Act make these planning decisions even more critical. GAO’s experience in procuring audit services suggests some issues that agencies may wish to consider in planning IPA service procurements.

In the initial year of an audit under the CFO Act, agencies may not be in a position to reliably estimate the hours-either in total or by level of auditor personnel-required to complete the work in a professional manner. Under these circum- stances, it is difficult to determine in advance the estimated cost of CM) audits. Accordingly, agencies may wish to consider using labor-hour contracts as described in the FAR. Price competition for labor- hour contracts is based on hourly direct labor rates rather than on total contract price. It is, therefore, important that this type of contract specify the

r

Page 14 GAO/[email protected]

Developing the EC+quest for Propod

hourly rates to be charged for each level of personnel to be used in the job (e.g., partner, manager, senior, junior, and industry specialist). Under the FAR, appropriate government monitoring of contractor performance is required to give reasonable assurance that efficient methods and effective cost controls are being used.

Whether agencies use fixed-price or labor-hour contracts, they may wish to consider using indefiite- quantity contract provisions, which allow agencies to issue specific task orders. Task orders allow agencies to have segments of audit work performed separately and give agencies the opportunity to reassess their needs and their IPA’s performance before ordering additional work.

In this regard, task order contracts may be espe- cially appropriate for agencies that are developing statements and having audits performed for the first time and, therefore, cannot accurately estimate the time needed to complete the entire audit. Under such a contract, these agencies could issue a task order for the initial scoping and planning effort prior to issuing further task orders for the remainder of the first-year audit. The task order deliverable for the scoping work would be a report that describes the financial systems, identifies problems, specifies the audit tasks, and estimates the number of hours by level of personnel required to accomplish the tasks. The scoping report accepted by the agency could then serve as the basis for further statements of work for task orders to complete the audit. If the time needed to complete a two-step process would prevent the agency from meeting the statutory deadlines, GAO can provide assistance to agencies in determining labor- hour ceilings in lieu of the first step.

Agencies may also wish to consider including annual renewal options in their initial contracts. Such options may minimize the cost of successive contracting efforts. They may also result in lower

Page 16 0A0,‘AFMD12.19.3

Developing the Request for Proposll

prices in option years, since the IPA will be able to work more efficiently after learning about the agency and its internal control systems during the first year of work and plan the use of audit resources over a longer period of time. In the second year of an audit under the CFO Act, the agency should expect a substantial reduction in the number of hours required to accomplish the tasks. GAO’s experience with audits of major government entities is that about 30 percent fewer hours are needed in the second year. Some further decrease in audit hours occurs in the third year.

All agency solicitations, whether for full audit contracts or task order contracts, should provide for a formal review and reevaluation of the IPA’s work not later than the point at which one-third of the estimated hours have been expended. At this point, the hours and milestones and, if appropriate, the contract pricing, could be renegotiated and adjusted based on conditions found which were not contem- plated in the audit plan or the RFP. Modifications increasing the contract or task order price or ceilings should be avoided after the one-third point has been reached.

Specifying Services and Deliverables

It is extremely important that potential contractors know as specifically as possible what is expected of them. The RFP should contain enough information to provide offerors with a common basis on which to prepare and submit proposals. Offerors need to know the entity to be audited, the scope of work (as best as it can be determined), and the reports required. In addition, the RFP should provide that deliverable drafts be submitted for agency review and comment, including legal review.

GAO has issued various guides related to the preparation and audit of annual financial statements. In addition, OMB, with GAO’s input, is developing guidance for audits under the CFO Act. Agency officials should refer to current guidance to determine

Page 16 GAO/AF’MD12.lB.3


the segments of the agency that need to prepare financial statements, the scope of the audits to be done, and the required reports. This guidance should be used as the basis for the RFP coverage of services and deliverables for audits under the CFD Act and for accounting and auditing standards to be applied. Further information regarding current GAO guidance can be obtained by contacting GAO’s Accounting and Financial Management Division at (202) 276-9461.

Other Audit Sk-vices

Services other than those required by the CFO Act might be suitable for inclusion under the same contract. Such services might include audits of

l segments of financial statements beyond those required by the act;

l financial reports of other kinds, e.g., statement of cash receipts and disbursements; and

+ financial information, e.g., expenditures for specific programs or services.

Agencies may include work beyond that needed to comply with current OMB and GAO guidance on financial statement audits and related deliverables and on auditing standards. For example, agencies may want IPAs to examine the assertions made in agency reports required by the Federal Managers’ Financial Integrity Act (FMFIA) of 1982 (31 U.S.C. 3612 (b) and (c)) and the agency’s process for arriving at these assertions. The CFO Act requires CFos to include a summary of the agency’s FMFIA report in their annual reports to agency heads.

Agencies should also consider requiring, as another deliverable, a training program on the audit meth- odology employed by the IPA in carrying out the work. Such training would be useful to the IG in conducting future audits.

Page 17 GAO/APMD12.19.3

Developing the Repeat for Propneal

Information Included in the RFP

services is limited to those required by the CKI Act or includes additional services, the agency should provide as much useful background information as possible with the RFP. In addition to the items listed in appendix I, the agency should consider including the following in the RFP package:

l recent agency reports to the Congress; l a list of pertinent IG, GAO, and internal review

reports issued within the past 3 years; . previous financial audit reports, if the agency has

been audited before and the auditor has reported on the results of the internal control and accounting systems work performed as part of the audit; and

l the agency’s most recent report to the President and the Congress under FMFIA.

Timing of Audit Work and Milestones

The RFP should include a schedule by which certain milestones in the audit process must be reached to ensure the timely preparation and issuance of financial statements and related reports. The RFP should also specify that the IPA notify the appropriate agency officials as soon as possible if the IPA identifies unanticipated problems that could delay completion of the audit work.

Planning and conducting ClW audits are year-round efforts and, thus, generally do not require that work be concentrated at peak times. Figure 2 illus- trates the statutory milestones and the ideal timing for various phases of work in a first-year audit. However, to meet CFO Act milestones, some agencies will be required to greatly compress the fiit- year time frames shown in figure 2. This compres- sion could increase the cost of the audit. Figure 3 provides a similar illustration for a second-year audit.

As figure 2 shows, for first-year audits, develop ment of an agency profile and transaction flow reviews (TFRs) should begin at least several months

Page 18 GAO/AFMD1219.3

DevelopingtheEequest for Proposal

in advance of the start of the audit year. However, as indicated in figure 3, in subsequent years, the agency profile and TF’Rs need only be updated and, therefore, require less time to complete. Figure 4 provides a more detailed description of the tasks included in each audit step.

The following terms are used in the three figures.

s Basic audit requirements determination report (BARDR’I: A BARDR is used to document significant accounting deficiencies that must be correckd before a successful financial audit can be completed. Items that may be reported include (1) devi- ations from generally accepted accounting principles, (2) critical internal control weaknesses, and (3) failure to comply with applicable laws and regulations. The auditors should provide a BARDR to management as soon as deficiencies are identified. Subsequent BARDRs should be issued if additional deficiencies are found.

l Transaction flow reviews (TFRs): TF’Rs consist of determimng, through interviews, observations, and reviews of procedures manuals and processing documents, the transaction processing steps for each of the agency’s major accounting cycles. Through TFRs, auditors can identify critical internal control points and develop plans for the subsequent testing phases of the audit.

Page 19 GAO/AFMIM2.191


Figure 2: First-Year Audit Time Frames

CFO Act Milestones

Fiscal year- end

Audit Work Milestunes

Audit Work Time Framw

One4hlrd assessment

A

A Develop Agenay Prufife A

A Drwelop and V&date Initial TFRs A

4 * Issue BARDFIG to Mgmt.

A Audit Plan A

A Contra1 Test A

6130 9130 12131 3131 6/30 9RO

I Audit Year

r

Page 20 GAO/AF’MD12.19.2


t

Agency Flnanclal FMFIA statements report to OMB

A A

Audit report to agency head

A

Agency’s annual report Issued

A

Finalize opinion and related reports

Y

c Substantive Test A

b Draft l/C and A A Audit Report A Compliance Report5

A FMFIA Summary (Assist) A A Financial Analysis (Assist) A

A Annual Report (Assist) A

9mo 12/31 3131 6130 9130

Page 21 GAO/AFMDl2.l9.3

Y

Developing the Request for Propod

Figure 3: Second-Year Audit Time Frames

CFO Act Milestones

Audit Work Milestones

Audit Work Time Frames

Estimate hoursto 0Wlplete audit

A

One-third assessmm

A

6130 9130 12131 3131 6/30 9130

czl Audit Year

Page 22 GAo/AfwB12.19.9

Developing the Request for proposal

Agency Financial FMFIA statements I@POrl to OMB

Audit Agency’s report to annual agency report head Issued

Finake opinion and related reports

b Substantive Test A

A Draft I/C and A A Audit Report A Compliance Reports

A FMFIA Summary (Assist) A

r

A Annual Report (Assist) A

9130 12131 3/31 6l30 9130

Page 23 GAO/AFMBl2.19.3


Figure 4: Audit Tasks

Audit Scope and Plan (First Year)

* Develop profile of agency financial management organization. systems. and operations.

* Document internal control structure

. Make a prelimltxy assessment of the reliability of the internal confrol structure.

a Prepare a basic audit re uirements determination report (BA a OR).

- Develop initial transaction flow (TFAs) and validate.

- Develop control test plan.

Audlt Plan (Subsequent Years)

* Update agency profile.

- Update documentation of the internal control structure.

- Make a preliminary assessment of the reliability of the internal control structure.

- Prepare EIARDR.

- Update TFRs.

- Update control test plan.

I Control Test

- Test the Internal mntrols for reliability.

l Assess the FMFIA process for evaluating accounting systems and Internal controls.

* Test controls for ensuring compliance with laws and regulations.

l Develop substantive test plan.

Substantive Test

m Test the process for year-end adjustments and closings.

m Evaluate the process for reporting FMFIA weaknesses.

- Field test account balances and compliance with laws and regulations.

- Test the consolidation and processing of financial statements.

Prepare Audit Report

- Opinion on finandal statements, - lnlernai control report, and

- Reoort on comollance wiih laws and reoulations. I

Agency Annual Reporl - Assist in preparing FMFIA summary. - Assist In the analvsis of the financial statements.

Page 24 GAO,‘-12.193

Developing the Eequm3t for Pmpo6aI

Independence A general standard of the audit profession is that an auditor shall be independent both in fact and in appearance. For this reason, an agency must reserve the right to disqualify an IPA for what it determines to be a potential or actual conflict of interest. To avoid misunderstandings, the initial RFP should require the IPA to stipulate work performed for the agency during the last 3 years and to mention any relationships or situations that might be viewed as a conflict.

Frequency of Competition

While the accounting profession’s guidance states that providing both audit and management services does not in itself present a conflict of interest, agency officials should consider how others outside the organization might view this relationship. GAO has disqualified auditors whose management services work was so extensive and fundamental to the accounting operations as to raise doubts about their independence.

As discussed above, using contracts with options to conduct later-year audits can reduce the time and expense of contracting for audits. However, there are other concerns which may influence how fre- quently agencies wish to compete their audit contracts. On one hand, new competition may result in new auditors, which may infuse the audit process with fresh views and new perspectives. On the other hand, maintaining an ongoing long-term relationship with an IPA may increase audit efficiency and provide agency managers with added assurance that the audit is being conducted by a proven firm.

Preproposal Conferences

A preproposal conference (sometimes referred to as a bidders’ conference), usually held about 10 days after RFP issuance, provides offerors with an opportunity to ask questions about the procurement. Preproposal conferences are not mandatory but, because of the new requirements of the Cl33 Act aml possible misinterpretations, they may be

Page 26 GAO/-12.198

helpful for these procurements. Information provided at the conference by the agency must be made available to all potential contractors regardless of their attendance.

t

Once the deadline for submission of proposals has passed, the agency can begin evaluating them. The technical segments of proposals should be evaluated first, independent of price. This evaluation must be carefully conducted and thoroughly documented in an orderly manner to provide a systematic frame- work for selecting an auditor on the basis of the agency’s established RF’P criteria.

The Technical To bring varied perspectives to the evaluation pro-

Evaluation Panel cess, a technical evaluation panel of three to five people should be established. The panel should be comprised of individuals with experience ln accounting, auditing, budgeting, contracting, or any specialty field pertinent to the required audit work. Panel members should be designated early in the procurement process so they can gain a thorough understanding of what is to be evaluated and how, and plan their time accordingly+ The CUI’R should be one of the panel members.

Technical Evaluation

To facilitate the technical evaluation of proposals, the RFP should elicit specific information from offerors, including

l their capabilities, l their work plan and technicaI approach, l their technical expertise, and l the qualifications of their personnel.

The agency should carefully consider what details to request for each of the above categories and should ensure that the RF’P clearly states what is required. Appendix II presents an example of a technical evaluation form that sets forth the kind of information that might be solicited and a related numerical rating system. The form should be tai- lored to the specific RFP by the IG and others who will be involved in the evaluation process.

Page 27 GAo/JtFJKLM2.1g.a

Evaluating PFopaeals and Selecdng a C4mtract4m

The RPP should state that failure to meet the following criteria will render the proposal seriously deficient.

1. Capabilities of the IPA

Does the proposal show that the offeror has sufficient resources to perform the audit?

2. Work Requirements and Technical Approach

Does the proposal demonstrate that the IPA has an understanding of the audit work’s objective(s), the agency’s needs, the established milestones, and the fiial deliverables? Does the proposal contain a sound technical plan and realistic estimates of time required to complete the various phases of the work? Does the proposal commit the IPA to start the audit work when required and indicate the contractor’s intent to complete it on time? Does the proposal describe (1) a systematic approach to examining and reporting on systems and internal controls and (2) effective procedures, including consideration of risk and materiality, to determine the extent of audit testing and review necessary?

3. IPA’s Technical Experience

Does the proposal show that the IPA has the experience to perform the required work?

4. Qualifications of the Audit Team

Does the proposal clearly show that the assigned audit team has sufficient individual and collective experience and professional qualifications? Does the proposal specify that any changes in key personnel must be approved by the agency and that substitutes of equivalent competence and exper- iences will be offered?

r

Page 28 GAO/-12.19.3

Evahating I’mpade and SekctLngaCun~r

Initial evaluations should be based on the proposals as submitted. Each member of the technica.I evaluation panel should (1) independently review the technical proposals using the criteria specified in the RFP, (2) make a list of strengths and weaknesses by criteria, (3) note any questions about the proposals, and (4) complete a technical evaluation form, which includes scoring each proposal. Mem- bers should then meet to compare lists of strengths and weaknesses and ratings made using a technical evaluation form. They should discuss any unusual aspects of the proposals and prepare a composite technical evaluation using a point system.

Price Evaluation The RFP must state that price will be considered in the final selection. After offerors have been ranked according to their technical evaluation, cost should be considered. For labor-hour contracts, the contract price will be determined by the hourly rates specified for audit personnel. The RFP must specify how the agency will evaluate the hourly rates included in the proposals. For example, prices could be evaluated using estimates of the total contract cost based on the hourly rates and estimated or assumed hours of work required to complete the audit. The following considerations are important when evaluating prices included in labor-hour contract proposals.

l IPA firms are service businesses whose hourly rates are set according to the level of personnel, most commonly partner, manager, senior, and junior. In some cases, IPA firms also have industry specialists-senior partners and managers-who are rec- ognized as experts in specific fields.

l Standard hourly rates are calculated to include both direct and indirect costs as well as profit.

l Compensation drives standard hourly rates. Rates for junior staff are usually comparable among firms in the same locality. However, partner compensation may vary widely between junior and senior partners of a firm and between firms.

Page 29 GAO/AF’MDl2.lB.3

EvalnatlpgRopoda and Sele.cijng ~Contractor

. Prices reflect the laws of supply and demand. For example, as more federal agencies seek audit services, hourly rates may rise. Conversely, in order to obtain a federal client and gain experience, IPA firms may provide price incentives by discounting their standard hourly rates or not billing for all incurred hours.

l Proposals that reflect unreasonably low prices should be carefully considered as (1) they may show a lack of understanding of the work to be done or (2) they may have been offered to gain an advantage in obtaining add-on work at higher fees. This could lead to low quality or incomplete work.

l If the proposal includes options for later-year audits, the approach for pricing work efforts in the option year(s) should include decreasing costs due to experience with and expected improvements in agency accounting systems and controls.

Best and F’inal Offers

The agency may now be prepared to select the best proposal(s). If, however, more information is needed before final selection, the contracting officer must determine which of the IPAs fall within the competitive range based on price and other factors included in the solicitation. Individual discussions can be held with these offerors to allow them to respond to questions and to submit additional information by a specified date. During these discussions, agency personnel must take care not to reveal any information submitted by other offerors. The agency then requests each IPA within the competitive range to submit its “best and final offer.” The agency should then evaluate this additional information and make adjustments to the original evaluation, as appropriate.

Appendix III contains an example of a final evaluation worksheet that can be used to combine technical and price considerations and determine the best qualified firm(s). In determining how much to weight technical and price considerations, agencies should bear in mind that a low price is no substitute

Page 30 GAO/AF’Mb12.19.3

Evaluating Proposals and Selecting a t3drachr

for the capability needed to meet the compiex technical requirements of supplying the services called for by the CFO Act.

Page 31 GAO/AFMD12.198

Monitming and Evahatig tin-r Work

Once the contract has been awarded, the COTR must monitor the contractor’s work to ensure that it is progressing on schedule and that all related problems are resolved. Progress should be moni- tored through periodic meetings where problems can be quickly identified and resolved. A 1988 GAO publication entitled, Guide For Review Of Indepen- dent Public Accountant Work (Technicaldeline B 1 l), includes a detailed program for reviewing . . the work of independent public accountants for financial statement audits. Key aspects of monitoring include

l attending an opening conference with the contractor and other agency officials to start the contract or task order;

l attending key meetings with contractor and agency officials;

l monitoring and evaluating contractor progress; l reviewing and approving contractor progress bill-

ings for payments; l resolving problems that may result in reduced audit

quality, missed deadlines, or additional costs; l reviewing contractor work and draft and final

deliverable products for technical completeness and compliance with the contract terms and conditions;

9 attending an exit conference with the contractor and other agency officials to complete the contract or task order; and

l evaluating overall compliance with contract requirements.

If changes in work scope or unforeseen problems affect the contractor’s hours and ability to complete the work within the quoted price and time frames, the contractor may seek to increase the contract’s price. Roth the contractor and the federal agency need to recognize this possibility. However, alterna- tives to requests for contract increases exist. Some aspects of the work may be scaled back or other contractors’ work or government-performed work can be substituted. Any modifications, however, that would restrict the contractor’s scope or reduce

Page 32 GAO/AFMIH2.lB.3

Monitoring and Evaluating CbMract.or Work

the contractor’s role so that it could no longer be considered the principal auditor should be avoided. Under professional auditing standards, these conditions may result in a disclaimer or qualified opinion. So that agency managers have time to consider alternative responses to unforeseen problems, any modifications should be negotiated before the contractor has completed one third of the originally estimated hours.

Page 33 GAO/AFBiD12.19.3

Appendix I

ISormation ID Include in an RFP

.

.

.

l

.

.

.

.

.

.

.

.

.

.

.

.

.

The prime consideration in preparing an RFP is to provide offerors with a common baseline of information to enable them to prepare proposals that address all agency audit needs. However, agency officials should consult with their purchasing office and/or legal counsel to ensure that the RF’P con- forms with applicable laws and regulations. At a minimum, the RFP should contain the following:

name and address of the federal contracting organization; entity to be audited, the scope of audit services to be provided, and specific reports and other products to be delivered (See OMB and GAO guidance for the preparation and audit of annual financial statements.>; period to be audited (with an explanation if the RF’P calls for option-year proposals); name and telephone number of an agency contact person; format in which proposals should be prepared; address to which proposals should be delivered or sent; date and time proposals are due; number of proposal copies to be submitted; requirements for establishing the IPA’s independence; criteria to be used in evaluating the proposal and their relative importance; expected schedule of work (completing field work, issuing reports, etc.); provisions for confidentiality and nondisclosure of work; notification of any security clearances needed or requirement not to remove agency documents from a site; requirements for workpaper retention and for making the workpapers available to the entity as well as to GAO, if they request them; policies on changes in key personnel; the method and timing of payment; provisions for reimbursing contractors for travel (government travel regulations apply) or other

Page 34 GAO/APMM8l93

1

-1 Information to Inelude ln anItPP

.

.

direct out-of-pocket expenses, such as computer time; and any other important points, including the conse- quences if due dates are missed or work does not meet audit standards.

The chances of obtaining high quality proposals will be enhanced if the RF’P

explains the work that the organization does; describes in some detail the organization’s accounting system, administrative controls, records, and procedures; informs prospective offerors if data from prior years (audit reports, management letters, etc.) will be available and whether major audit findings remain open from prior years; describes expected audit products in detail, the required format of the audit report, the format of any required progress reports, the addressee for deliverables, and the number of copies and their distribution; explains any assistance that the organization will offer, such as staff support to assist the auditor (which could materially reduce audit costs}; requests resumes of the key contractor employees to be assigned to the audits; provides requirements for periodic status meetings or written reports on the work’s progress; provides requirements for training of IG staff on the methodologies used in the audit; and notifies IPAs of required entrance and exit conferences to communicate pertinent factors for starting and completing work.

Page 35 GAO/‘AF’MD12.19.3

Appendix I’r:

Example of a Technical Evaluation Form

I. Cambilitv of Cuntracbr

'Iheoontractorhastheresouroesto performtheauditsasevidencedby:

Possible mmlkr of JJoints Awarded

A)

B)

(3

D)

El

Size-Offices #- Persomel # O-5 (Over 10 offices - 5 points) (3 to 10 offioes - 3 points) (uMer3offices--Opoint)

Washingtan, D.C., office o-1

Cmpubreqxxtise

Personal ccwplters for field auditors o-1 minfran~cmq&erauditspecialists o-2 ccrmprehensivecanplterauditpragrams o-2

Statisticalsa@ingexpehiseti experience o-2

qUality controlpmgram

peerreviewresults(obtainmstrecmtcqy) O-l Refemalsofsubstardard mrk within past 4years. (Deduct1pointforea&referral to il!z&wm of 5.)

Page 26 GAO/‘AFMD12.19.2

Apped~ II: ExampleofaTechnical Evahation Form

F) -1 presentation

- organization ard ccanpleteness in follming contract specifications

- Plan to use minority mntractmr(s)

Total points for se&ion I

II. Work@ouirmentsand?&chni~l&moach

lXec0ntractorunAerstar& the soJp of worktobedonearxldenmstrates~ t.echnicalapproach.

A) Demonstratedknowledge of work objectives,agercyn~,milestoraes, and deliverables.

8) Techicalplan andtilmz esthtes

-FstabLihejawlitmethcdology -Useofgov ermnmtaudithg

- PL=lnniq - Entranoe/exit: conference -InInternalmntrols -Ca@iancework -Wxtantivetesting -Risk-t -Statisticalsamplirq -Quality control - Repxt deliverables -cIlx?puteruse - Handling of audit/job problems -F.mlistictim&imatesand

cmnihznttOcm@letethework

Total points for section II

Possible l-llmkr of points Awarded

o-3

0-3

O-20

O-10

O-10 (circle)

YN

YN YN YN YN YN YN YN YN YN YN YN YN

YN

O-20

Page37 GAO/APMD12.19.3

Appt?~~li~ II: b8lIlQh of a Technical EvalnatloIl Form

Pcssible nulnber of points Awarded

III. Ted-mica1 Emerience of Contractor

Cmtra~~the~ienceto performasevidmcedkypastau3its, irdustq specialization, and kncwledge offfderalfimnciallrELrEq~ problermgai.ne3thrcu@atitsof federal agencies, ccopcorati-, grants, and loans.

A) Audits of:

- Federal agencies ad carporations o-5 -Federalgrants o-2 -stateandlocalgmmment o-1 - Idrye private .seztor clients o-4

3) Orqanized i.rdMxy specialist groups ina~licableareas o-3

(circle) -- YN - Finxcialinstitutions YN -- YN -0thother YN

C) Experiemehperfomingfinamial analysis o-2

D) Experience incmprhmsive examination 0fintlm?a1con~1ardreporting(sAs30) o-3

!Itbl points for section III O-20

Page 38 GAO/AF’MD12.l9.3

Appendix II: Ekample of a Technical Eihhation Form

IV. pualifications of Audit Team

Resumes indicate appropriate qualifications at industry specialist, partmr, manager, senior and junior levels:

A) Years with firm/other relevant fxpzrifmce

--Lndustry specialist

-Pa?TtnfT

- MaEager

- senior

N-(s)

B) Industry audit expzience

Possible - of pints Awarded

O-20

Average Y-S

Average years Partner Manager senior

- Federal agencies and corporations

-Federalgrants -stateandlocal

goverrmwt ~ - - - Uryeprivate

sector clients - ____ -

Cl Fducation/trainiq

-Accountiqdeqrees ---degrees/m - Special training

o-15

o-3

Page 39 GAO/AF‘?vUbl2.19.3

Appemdix II: Example of a Technical Jhduation Form

D) Professiond acccm'plishments

- Professional oqanizations - Books, articles

rz=

Possible mlrbsr of pints Awarded

o-2

Total pints for section IV O-40

'IOTALWINTSEDRSlX!ITONSITHRouwIV n=uLp -

V. Fh .S't.rm and th&l-ESSffi ( CTurmmrize Briefly)

vI. puestions/Comments

Page 40 GAO/AFMDlbl9.3

Appendix III

Ekmnple of a Final Evaluation Worksheet

The following is a sample formula for assigning final scores to proposals. After final scores have been assigned, proposals can be ranked.

A key element of the formula is the weight that price versus technical score will have on the final overall evaluation. In GAO’s opinion, the technical score should carry much more weight, since technical expertise is what is being purchased.

The final score for each proposal should be determined by adding the weighted technical score to the weighted price score.

Technical score

Weight given to Weighted

X technical = technical score score

Price score Weight X given to = Weighted

price score price score ~.___ Total Points (Final Score)

iZ U.S. GOVERNMENT PRINTINGOFFICE: 1994 X00-869,‘00006

(910009) Page 41 GAO/AFMD12.1B.3

,

U.S. General AasxmMi@ Once P.O. Box 6015 Gaithersburg, IyI[D BB384-6016

or visit:

Room 1000 706 4th St. NW (tornmof 4th & G SW U.S. General Aeoomti~ 0iT’ice 1 I’,, wasbington, DC ijl 1 ,j

I/ 1.;/,t

Orders may also be placed by c&Ii&g, 1:; $1

(202) 612-6066 or by ldlg fax Iluiutm~ I;,, iii/

(301) 258-4066. ;:j /I.; :,; # ! i!. i: ,ii I

afmd-12.19.3 guide to federal agencies' procurement of audit … · accepted government...

Documents