request for proposals for administration/delivery, scoring ... · administration and online...

1 RFP 33111-03819

STATE OF TENNESSEE

Department of Education

REQUEST FOR PROPOSALS FOR

Administration/Delivery, Scoring, and Reporting of State Assessments

RFP # 33111-03819

RFP CONTENTS

SECTIONS:

1. INTRODUCTION

2. RFP SCHEDULE OF EVENTS

3. RESPONSE REQUIREMENTS

4. GENERAL CONTRACTING INFORMATION & REQUIREMENTS

5. EVALUATION & CONTRACT AWARD

ATTACHMENTS:

6.1. Response Statement of Certifications & Assurances

6.2. Technical Response & Evaluation Guide

6.2.A Evaluation Scoring Attachment

6.3. Cost Proposal & Scoring Guide

6.3.A Cost Proposal Workbook

6.3.B Cost Option Workbook

6.4. Reference Questionnaire

6.5. Score Summary Matrix

6.6. Pro Forma Contract

6.6.A Reporting Attachment

2 RFP 33111-03819

1. INTRODUCTION

The State of Tennessee, Department of Education, hereinafter referred to as “the State,” issues this Request for Proposals (RFP) to define minimum contract requirements; solicit responses; detail response requirements; and, outline the State’s process for evaluating responses and selecting a contractor to provide the needed goods or services.

Through this RFP, the State seeks to procure necessary goods or services at the most favorable, competitive prices and to give ALL qualified respondents, including those that are owned by minorities, women, service-disabled veterans, persons with disabilities and small business enterprises, an opportunity to do business with the state as contractors, subcontractors or suppliers.

1.1. Statement of Procurement Purpose

The State seeks to procure services for the administration/delivery, scoring, and reporting of the Tennessee Comprehensive Assessment Program (TCAP) state assessments to students in grades 3-8 and high school end-of-course for English Language Arts, mathematics, social studies, and science and alternate assessments for students in grades 3-8 in science and social studies and high school alternate assessment in science. The administration of these assessments will be a combination of traditional paper and pencil administration and online administration. The high-level goals of the assessment program are on-time, uninterrupted delivery, timely and accurate scoring, and timely, useful reporting.

1.1.2. The Department of Education estimates the maximum liability to be $20 million each year.

1.2. Scope of Service, Contract Period, & Required Terms and Conditions

The RFP Attachment 6.6., Pro Forma Contract details the State’s requirements: Scope of Services and Deliverables (Section A); Contract Period (Section B); Payment Terms (Section C); Standard Terms and Conditions (Section D); and, Special Terms and Conditions (Section E). The pro forma contract substantially represents the contract document that the successful Respondent must sign.

1.3. Nondiscrimination

No person shall be excluded from participation in, be denied benefits of, or be otherwise subjected to discrimination in the performance of a Contract pursuant to this RFP or in the employment practices of the Contractor on the grounds of handicap or disability, age, race, creed, color, religion, sex, national origin, or any other classification protected by federal, Tennessee state constitutional, or statutory law. The Contractor pursuant to this RFP shall, upon request, show proof of such nondiscrimination and shall post in conspicuous places, available to all employees and applicants, notices of nondiscrimination.

1.4. RFP Communications

1.4.1. The State has assigned the following RFP identification number that must be referenced in all communications regarding this RFP: RFP # 33111-03819

3 RFP 33111-03819

1.4.2. Unauthorized contact about this RFP with employees or officials of the State of Tennessee except as detailed below may result in disqualification from consideration under this procurement process.

1.4.2.1. Prospective Respondents must direct communications concerning this RFP to the

following person designated as the Solicitation Coordinator: Tamara Byrd State of Tennessee, Central Procurement Office 312 Rosa L. Parks Ave., 3

rd Floor WRS Tennessee Tower

Nashville, TN 37243 615-532-2314 [email protected]

1.4.2.2. Notwithstanding the foregoing, Prospective Respondents may alternatively contact:

a. staff of the Governor’s Office of Diversity Business Enterprise for assistance available to minority-owned, woman-owned, service-disabled veteran-owned, businesses owned by persons with disabilities, and small businesses as well as general, public information relating to this RFP (visit https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/governor-s-office-of-diversity-business-enterprise--godbe--/godbe-general-contacts.html for contact information); and

b. the following individual designated by the State to coordinate compliance with the

nondiscrimination requirements of the State of Tennessee, Title VI of the Civil Rights Act of 1964, the Americans with Disabilities Act of 1990, and associated federal regulations:

Helen Crowley State of Tennessee, Central Procurement Office 312 Rosa L. Parks Ave. 3rd Floor WRS Tennessee Tower Nashville, TN 37243 615-741-3836 [email protected]

1.4.3. Only the State’s official, written responses and communications with Respondents are binding with regard to this RFP. Oral communications between a State official and one or more Respondents are unofficial and non-binding.

1.4.4. Potential Respondents must ensure that the State receives all written questions and comments,

including questions and requests for clarification, no later than the Written Questions & Comments Deadline detailed in the RFP Section 2, Schedule of Events.

1.4.5. Respondents must assume the risk of the method of dispatching any communication or response

to the State. The State assumes no responsibility for delays or delivery failures resulting from the Respondent’s method of dispatch. Actual or digital “postmarking” of a communication or response to the State by a specified deadline is not a substitute for the State’s actual receipt of a communication or response.

1.4.6. The State will convey all official responses and communications related to this RFP to the

prospective Respondents from whom the State has received a Notice of Intent to Respond (refer to RFP Section 1.8).

1.4.7. The State reserves the right to determine, at its sole discretion, the method of conveying official,

written responses and communications related to this RFP. Such written communications may

https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/governor-s-office-of-diversity-business-enterprise--godbe--/godbe-general-contacts.html



mailto:[email protected]

4 RFP 33111-03819

be transmitted by mail, hand-delivery, facsimile, electronic mail, Internet posting, or any other means deemed reasonable by the State. For internet posting, please refer to the following website: https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/supplier-information-/request-for-proposals--rfp--opportunities.html.

1.4.8. The State reserves the right to determine, at its sole discretion, the appropriateness and

adequacy of responses to written comments, questions, and requests related to this RFP. The State’s official, written responses will constitute an amendment of this RFP.

1.4.9. Any data or factual information provided by the State (in this RFP, an RFP amendment or any

other communication relating to this RFP) is for informational purposes only. The State will make reasonable efforts to ensure the accuracy of such data or information, however it is the Respondent’s obligation to independently verify any data or information provided by the State. The State expressly disclaims the accuracy or adequacy of any information or data that it provides to prospective Respondents.

1.5. Assistance to Respondents With a Handicap or Disability

Prospective Respondents with a handicap or disability may receive accommodation relating to the communication of this RFP and participating in the RFP process. Prospective Respondents may contact the Solicitation Coordinator to request such reasonable accommodation no later than the Disability Accommodation Request Deadline detailed in the RFP Section 2, Schedule of Events.

1.6. Respondent Required Review & Waiver of Objections

1.6.1. Each prospective Respondent must carefully review this RFP, including but not limited to, attachments, the RFP Attachment 6.6., Pro Forma Contract, and any amendments, for questions, comments, defects, objections, or any other matter requiring clarification or correction (collectively called “questions and comments”).

1.6.2. Any prospective Respondent having questions and comments concerning this RFP must provide

them in writing to the State no later than the Written Questions & Comments Deadline detailed in the RFP Section 2, Schedule of Events.

1.6.3. Protests based on any objection to the RFP shall be considered waived and invalid if the

objection has not been brought to the attention of the State, in writing, by the Written Questions & Comments Deadline.

1.7. Pre-Response Conference

A Pre-response Conference will be held at the time and date detailed in the RFP Section 2, Schedule of Events. Pre-response Conference attendance is not mandatory, and prospective Respondents may be limited to a maximum number of attendees depending upon overall attendance and space limitations. The conference will be held at: William R. Snodgrass Bldg.-Tennessee Tower 312 Rosa L. Parks Ave., 3

rd Floor- Conference Room F

Nashville, TN 37243 Web-Ex will be available Meeting link: https://tngov.webex.com/tngov/j.php?MTID=m566c88c585746a76c3a7d816e47d84c5

Meeting number (access code): 649 692 691 The purpose of the conference is to discuss the RFP scope of goods or services. The State will entertain questions, however prospective Respondents must understand that the State’s oral response to any question at the Pre-response Conference shall be unofficial and non-binding. Prospective Respondents

https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/supplier-information-/request-for-proposals--rfp--opportunities.html

https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/supplier-information-/request-for-proposals--rfp--opportunities.html

5 RFP 33111-03819

must submit all questions, comments, or other concerns regarding the RFP in writing prior to the Written Questions & Comments Deadline date detailed in the RFP Section 2, Schedule of Events. The State will send the official response to these questions and comments to prospective Respondents from whom the State has received a Notice of Intent to respond as indicated in RFP Section 1.8 and on the date detailed in the RFP Section 2, Schedule of Events.

1.8. Notice of Intent to Respond

Before the Notice of Intent to Respond Deadline detailed in the RFP Section 2, Schedule of Events, prospective Respondents should submit to the Solicitation Coordinator a Notice of Intent to Respond (in the form of a simple e-mail or other written communication). Such notice should include the following information: the business or individual’s name (as appropriate); a contact person’s name and title; and the contact person’s mailing address, telephone number, facsimile number, and e-mail address. A Notice of Intent to Respond creates no obligation and is not a prerequisite for submitting a response, however, it is necessary to ensure receipt of any RFP amendments or other notices and communications relating to this RFP.

1.9. Response Deadline

A Respondent must ensure that the State receives a response no later than the Response Deadline time and date detailed in the RFP Section 2, Schedule of Events. The State will not accept late responses, and a Respondent’s failure to submit a response before the deadline will result in disqualification of the response. It is the responsibility of the Respondent to ascertain any additional security requirements with respect to packaging and delivery to the State of Tennessee. Respondents should be mindful of any potential delays due to security screening procedures, weather, or other filing delays whether foreseeable or unforeseeable.

6 RFP 33111-03819

2. RFP SCHEDULE OF EVENTS

2.1. The following RFP Schedule of Events represents the State’s best estimate for this RFP.

EVENT

TIME

(Central Time zone)

DATE

1. RFP Issued March 5, 2019

2. Disability Accommodation Request Deadline 2:00 p.m. March 8, 2019

3. Pre-response Conference 10 a.m. March 14, 2019

4. Notice of Intent to Respond Deadline 2:00 p.m. March 15, 2019

5. Written “Questions & Comments” Deadline 2:00 p.m. March 20, 2019

6. State Response to Written “Questions & Comments” March 29, 2019

7. Response Deadline 2:00 p.m. April 11, 2019

8. State Schedules Respondent Oral Presentation April 18, 2019

9. Respondent Oral Presentation 8 a.m. - 4:30 p.m. April 22, through April 26, 2019

10. State Completion of Technical Response Evaluations

April 30, 2019

11. State Opening & Scoring of Cost Proposals 2:00 p.m. May 6, 2019

12. Negotiations May 13, 2019

13. State Notice of Intent to Award Released and RFP Files Opened for Public Inspection

2:00 p.m. May 30, 2019

14. End of Open File Period June 6, 2019

15. State sends contract to Contractor for signature June 10, 2019

16. Contractor Signature Deadline 2:00 p.m. June 13, 2019

2.2. The State reserves the right, at its sole discretion, to adjust the RFP Schedule of Events as it

deems necessary. Any adjustment of the Schedule of Events shall constitute an RFP amendment, and the State will communicate such to prospective Respondents from whom the State has received a Notice of Intent to Respond (refer to section 1.8).

7 RFP 33111-03819

3. RESPONSE REQUIREMENTS

3.1. Response Form

A response to this RFP must consist of two parts, a Technical Response and a Cost Proposal.

3.1.1. Technical Response. RFP Attachment 6.2., Technical Response & Evaluation Guide provides

the specific requirements for submitting a response. This guide includes mandatory requirement items, general qualifications and experience items, and technical qualifications, experience, and approach items all of which must be addressed with a written response and, in some instances, additional documentation.

NOTICE: A technical response must not include any pricing or cost information. If any pricing or cost information amounts of any type (even pricing relating to other projects) is included in any part of the technical response, the state may deem the response to be non-responsive and reject it.

3.1.1.1. A Respondent must use the RFP Attachment 6.2., Technical Response & Evaluation

Guide to organize, reference, and draft the Technical Response by duplicating the attachment, adding appropriate page numbers as required, and using the guide as a table of contents covering the Technical Response.

3.1.1.2. A response should be economically prepared, with emphasis on completeness and

clarity. A response, as well as any reference material presented, must be written in English and must be written on standard 8 ½” x 11” pages (although oversize exhibits are permissible) and use a 12 point font for text. All response pages must be numbered.

3.1.1.3. All information and documentation included in a Technical Response should

correspond to or address a specific requirement detailed in the RFP Attachment 6.2., Technical Response & Evaluation Guide. All information must be incorporated into a response to a specific requirement and clearly referenced. Any information not meeting these criteria will be deemed extraneous and will not contribute to evaluations.

3.1.1.4. The State may determine a response to be non-responsive and reject it if:

a. the Respondent fails to organize and properly reference the Technical Response

as required by this RFP and the RFP Attachment 6.2., Technical Response & Evaluation Guide; or

b. the Technical Response document does not appropriately respond to, address, or

meet all of the requirements and response items detailed in the RFP Attachment 6.2., Technical Response & Evaluation Guide.

3.1.2. Cost Proposal. A Cost Proposal must be recorded on an exact duplicate of the RFP Attachment

6.3., Cost Proposal & Scoring Guide.

NOTICE: If a Respondent fails to submit a cost proposal exactly as required, the State may deem the response to be non-responsive and reject it.

3.1.2.1. A Respondent must only record the proposed cost exactly as required by the RFP

Attachment 6.3., Cost Proposal & Scoring Guide.

8 RFP 33111-03819

3.1.2.2. The proposed cost shall incorporate ALL costs for services under the contract for the total contract period, including any renewals or extensions.

3.1.2.3. A Respondent must sign and date the Cost Proposal. 3.1.2.4. A Respondent must submit the Cost Proposal to the State in a sealed package

separate from the Technical Response (as detailed in RFP Sections 3.2.3., et seq.). 3.2. Response Delivery

3.2.1. A Respondent must ensure that both the original Technical Response and Cost Proposal

documents meet all form and content requirements, including all required signatures, as detailed within this RFP, as may be amended.

3.2.2. A Respondent must submit original Technical Response and Cost Proposal documents and

copies as specified below.

3.2.2.1. One (1) original Technical Response paper document labeled: “RFP # 33111-03819 TECHNICAL RESPONSE ORIGINAL” and Eight (8) digital copies of the Technical Response in the form of one (1) digital document in “PDF” format properly recorded on its own otherwise blank, USB flash drive labeled: “RFP # 33111-03819 TECHNICAL RESPONSE COPY” The digital copies should not include copies of sealed customer references, however any other discrepancy between the paper Technical Response document and any digital copies may result in the State rejecting the proposal as non-responsive.

3.2.2.2. One (1) original Cost Proposal paper document labeled:

“RFP # 33111-03819 COST PROPOSAL ORIGINAL” and one (1) copy in the form of a digital document in “PDF/XLS” format properly recorded on a separate, blank, USB flash drive labeled: “RFP # 33111-03819 COST PROPOSAL COPY” In the event of a discrepancy between the original Cost Proposal document and the digital copy, the original, signed document will take precedence.

3.2.3. A Respondent must separate, seal, package, and label the documents and copies for delivery as

follows: 3.2.3.1. The Technical Response original document and digital copies must be placed in a

sealed package that is clearly labeled: “DO NOT OPEN… RFP # 33111-03819 TECHNICAL RESPONSE FROM [RESPONDENT LEGAL ENTITY NAME]”

3.2.3.2. The Cost Proposal original document and digital copy must be placed in a separate,

sealed package that is clearly labeled: “DO NOT OPEN… RFP # 33111-03819 COST PROPOSAL FROM [RESPONDENT LEGAL ENTITY NAME]”

9 RFP 33111-03819

3.2.3.3. The separately, sealed Technical Response and Cost Proposal components may be

enclosed in a larger package for mailing or delivery, provided that the outermost package is clearly labeled: “RFP # 33111-03819 SEALED TECHNICAL RESPONSE & SEALED COST PROPOSAL FROM [RESPONDENT LEGAL ENTITY NAME]”

3.2.4. A Respondent must ensure that the State receives a response no later than the Response Deadline time and date detailed in the RFP Section 2, Schedule of Events at the following address: Tamara Byrd State of Tennessee, Central Procurement Office 312 Rosa L. Parks Ave., 3

rd Floor WRS Tennessee Tower

Nashville, TN 37243 615-532-2314 [email protected]

3.3. Response & Respondent Prohibitions

3.3.1. A response must not include alternate contract terms and conditions. If a response contains such terms and conditions, the State, at its sole discretion, may determine the response to be a non-responsive counteroffer and reject it.

3.3.2. A response must not restrict the rights of the State or otherwise qualify either the offer to deliver

goods or provide services as required by this RFP or the Cost Proposal. If a response restricts the rights of the State or otherwise qualifies either the offer to deliver goods or provide services as required by this RFP or the Cost Proposal, the State, at its sole discretion, may determine the response to be a non-responsive counteroffer and reject it.

3.3.3. A response must not propose alternative goods or services (i.e., offer services different from

those requested and required by this RFP) unless expressly requested in this RFP. The State may consider a response of alternative goods or services to be non-responsive and reject it.

3.3.4. A Cost Proposal must be prepared and arrived at independently and must not involve any

collusion between Respondents. The State will reject any Cost Proposal that involves collusion, consultation, communication, or agreement between Respondents. Regardless of the time of detection, the State will consider any such actions to be grounds for response rejection or contract termination.

3.3.5. A Respondent must not provide, for consideration in this RFP process or subsequent contract

negotiations, any information that the Respondent knew or should have known was materially incorrect. If the State determines that a Respondent has provided such incorrect information, the State will deem the Response non-responsive and reject it.

3.3.6. A Respondent must not submit more than one Technical Response and one Cost Proposal in

response to this RFP, except as expressly requested by the State in this RFP. If a Respondent submits more than one Technical Response or more than one Cost Proposal, the State will deem all of the responses non-responsive and reject them.

3.3.7. A Respondent must not submit a response as a prime contractor while also permitting one or

more other Respondents to offer the Respondent as a subcontractor in their own responses. Such may result in the disqualification of all Respondents knowingly involved. This restriction does not, however, prohibit different Respondents from offering the same subcontractor as a part


10 RFP 33111-03819

of their responses (provided that the subcontractor does not also submit a response as a prime contractor).

3.3.8. The State shall not consider a response from an individual who is, or within the past six (6)

months has been, a State employee. For purposes of this RFP:

3.3.8.1. An individual shall be deemed a State employee until such time as all compensation for salary, termination pay, and annual leave has been paid;

3.3.8.2. A contract with or a response from a company, corporation, or any other contracting entity in which a controlling interest is held by any State employee shall be considered to be a contract with or proposal from the employee; and

3.3.8.3. A contract with or a response from a company, corporation, or any other contracting entity that employs an individual who is, or within the past six (6) months has been, a State employee shall not be considered a contract with or a proposal from the employee and shall not constitute a prohibited conflict of interest.

3.4. Response Errors & Revisions

A Respondent is responsible for any and all response errors or omissions. A Respondent will not be allowed to alter or revise response documents after the Response Deadline time and date detailed in the RFP Section 2, Schedule of Events unless such is formally requested, in writing, by the State.

3.5. Response Withdrawal

A Respondent may withdraw a submitted response at any time before the Response Deadline time and date detailed in the RFP Section 2, Schedule of Events by submitting a written request signed by an authorized Respondent representative. After withdrawing a response, a Respondent may submit another response at any time before the Response Deadline. After the Response Deadline, a Respondent may only withdraw all or a portion of a response where the enforcement of the response would impose an unconscionable hardship on the Respondent.

3.6. Additional Services

If a response offers goods or services in addition to those required by and described in this RFP, the State, at its sole discretion, may add such services to the contract awarded as a result of this RFP. Notwithstanding the foregoing, a Respondent must not propose any additional cost amounts or rates for additional goods or services. Regardless of any additional services offered in a response, the Respondent’s Cost Proposal must only record the proposed cost as required in this RFP and must not record any other rates, amounts, or information. NOTICE: If a Respondent fails to submit a Cost Proposal exactly as required, the State may deem the response non-responsive and reject it.

3.7. Response Preparation Costs

The State will not pay any costs associated with the preparation, submittal, or presentation of any response.

11 RFP 33111-03819

4. GENERAL CONTRACTING INFORMATION & REQUIREMENTS

4.1. RFP Amendment

The State at its sole discretion may amend this RFP, in writing, at any time prior to contract award. However, prior to any such amendment, the State will consider whether it would negatively impact the ability of potential Respondents to meet the response deadline and revise the RFP Schedule of Events if deemed appropriate. If an RFP amendment is issued, the State will convey it to potential Respondents who submitted a Notice of Intent to Respond (refer to RFP Section 1.8). A response must address the final RFP (including its attachments) as amended.

4.2. RFP Cancellation

The State reserves the right, at its sole discretion, to cancel the RFP or to cancel and reissue this RFP in accordance with applicable laws and regulations.

4.3. State Right of Rejection

4.3.1. Subject to applicable laws and regulations, the State reserves the right to reject, at its sole discretion, any and all responses.

4.3.2. The State may deem as non-responsive and reject any response that does not comply with all

terms, conditions, and performance requirements of this RFP. Notwithstanding the foregoing, the State reserves the right to waive, at its sole discretion, minor variances from full compliance with this RFP. If the State waives variances in a response, such waiver shall not modify the RFP requirements or excuse the Respondent from full compliance, and the State may hold any resulting Contractor to strict compliance with this RFP.

4.4. Assignment & Subcontracting

4.4.1. The Contractor may not subcontract, transfer, or assign any portion of the Contract awarded as a result of this RFP without prior approval of the State. The State reserves the right to refuse approval, at its sole discretion, of any subcontract, transfer, or assignment.

4.4.2. If a Respondent intends to use subcontractors, the response to this RFP must specifically identify

the scope and portions of the work each subcontractor will perform (refer to RFP Attachment 6.2., Section B, General Qualifications & Experience Item B.14.).

4.4.3. Subcontractors identified within a response to this RFP will be deemed as approved by the State

unless the State expressly disapproves one or more of the proposed subcontractors prior to signing the Contract.

4.4.4. After contract award, a Contractor may only substitute an approved subcontractor at the

discretion of the State and with the State’s prior, written approval. 4.4.5. Notwithstanding any State approval relating to subcontracts, the Respondent who is awarded a

contract pursuant to this RFP will be the prime contractor and will be responsible for all work under the Contract.

4.5. Right to Refuse Personnel or Subcontractors

The State reserves the right to refuse, at its sole discretion and notwithstanding any prior approval, any personnel of the prime contractor or a subcontractor providing goods or services in the performance of a contract resulting from this RFP. The State will document in writing the reason(s) for any rejection of personnel.

12 RFP 33111-03819

4.6. Insurance The State will require the awarded Contractor to provide a Certificate of Insurance issued by an insurance company licensed or authorized to provide insurance in the State of Tennessee. Each Certificate of Insurance shall indicate current insurance coverages meeting minimum requirements as may be specified by this RFP. A failure to provide a current, Certificate of Insurance will be considered a material breach and grounds for contract termination.

4.7. Professional Licensure and Department of Revenue Registration

4.7.1. All persons, agencies, firms, or other entities that provide legal or financial opinions, which a Respondent provides for consideration and evaluation by the State as a part of a response to this RFP, shall be properly licensed to render such opinions.

4.7.2. Before the Contract resulting from this RFP is signed, the apparent successful Respondent (and

Respondent employees and subcontractors, as applicable) must hold all necessary or appropriate business or professional licenses to provide the goods or services as required by the contract. The State may require any Respondent to submit evidence of proper licensure.

4.7.3. Before the Contract resulting from this RFP is signed, the apparent successful Respondent must

be registered with the Tennessee Department of Revenue for the collection of Tennessee sales and use tax. The State shall not award a contract unless the Respondent provides proof of such registration or provides documentation from the Department of Revenue that the Contractor is exempt from this registration requirement. The foregoing is a mandatory requirement of an award of a contract pursuant to this solicitation. For purposes of this registration requirement, the Department of Revenue may be contacted at: [email protected].

4.8. Disclosure of Response Contents

4.8.1. All materials submitted to the State in response to this RFP shall become the property of the State of Tennessee. Selection or rejection of a response does not affect this right. By submitting a response, a Respondent acknowledges and accepts that the full response contents and associated documents will become open to public inspection in accordance with the laws of the State of Tennessee.

4.8.2. The State will hold all response information, including both technical and cost information, in

confidence during the evaluation process. 4.8.3. Upon completion of response evaluations, indicated by public release of a Notice of Intent to

Award, the responses and associated materials will be open for review by the public in accordance with Tenn. Code Ann. § 10-7-504(a)(7).

4.9. Contract Approval and Contract Payments

4.9.1. After contract award, the Contractor who is awarded the contract must submit appropriate documentation with the Department of Finance and Administration, Division of Accounts.

4.9.2. This RFP and its contractor selection processes do not obligate the State and do not create

rights, interests, or claims of entitlement in either the Respondent with the apparent best-evaluated response or any other Respondent. State obligations pursuant to a contract award shall commence only after the Contract is signed by the State agency head and the Contractor and after the Contract is approved by all other state officials as required by applicable laws and regulations.

4.9.3. No payment will be obligated or made until the relevant Contract is approved as required by

applicable statutes and rules of the State of Tennessee.

13 RFP 33111-03819

4.9.3.1. The State shall not be liable for payment of any type associated with the Contract resulting from this RFP (or any amendment thereof) or responsible for any goods delivered or services rendered by the Contractor, even goods delivered or services rendered in good faith and even if the Contractor is orally directed to proceed with the delivery of goods or the rendering of services, if it occurs before the Contract Effective Date or after the Contract Term.

4.9.3.2. All payments relating to this procurement will be made in accordance with the Payment

Terms and Conditions of the Contract resulting from this RFP (refer to RFP Attachment 6.6., Pro Forma Contract, Section C).

4.9.3.3. If any provision of the Contract provides direct funding or reimbursement for the

competitive purchase of goods or services as a component of contract performance or otherwise provides for the reimbursement of specified, actual costs, the State will employ all reasonable means and will require all such documentation that it deems necessary to ensure that such purchases were competitive and costs were reasonable, necessary, and actual. The Contractor shall provide reasonable assistance and access related to such review. Further, the State shall not remit, as funding or reimbursement pursuant to such provisions, any amounts that it determines do not represent reasonable, necessary, and actual costs.

4.10. Contractor Performance

The Contractor who is awarded a contract will be responsible for the delivery of all acceptable goods or the satisfactory completion of all services set out in this RFP (including attachments) as may be amended. All goods or services are subject to inspection and evaluation by the State. The State will employ all reasonable means to ensure that goods delivered or services rendered are in compliance with the Contract, and the Contractor must cooperate with such efforts.

4.11. Contract Amendment

After Contract award, the State may request the Contractor to deliver additional goods or perform additional services within the general scope of the Contract and this RFP, but beyond the specified Scope, and for which the Contractor may be compensated. In such instances, the State will provide the Contractor a written description of the additional goods or services. The Contractor must respond to the State with a time schedule for delivering the additional goods or accomplishing the additional services based on the compensable units included in the Contractor’s response to this RFP. If the State and the Contractor reach an agreement regarding the goods or services and associated compensation, such agreement must be effected by means of a contract amendment. Further, any such amendment requiring additional goods or services must be signed by both the State agency head and the Contractor and must be approved by other state officials as required by applicable statutes, rules, policies and procedures of the State of Tennessee. The Contractor must not provide additional goods or render additional services until the State has issued a written contract amendment with all required approvals.

4.12. Severability

If any provision of this RFP is declared by a court to be illegal or in conflict with any law, said decision will not affect the validity of the remaining RFP terms and provisions, and the rights and obligations of the State and Respondents will be construed and enforced as if the RFP did not contain the particular provision held to be invalid.

4.13. Next Ranked Respondent

The State reserves the right to initiate negotiations with the next ranked Respondent should the State cease doing business with any Respondent selected via this RFP process.

14 RFP 33111-03819

5. EVALUATION & CONTRACT AWARD

5.1. Evaluation Categories & Maximum Points

The State will consider qualifications, experience, technical approach, and cost in the evaluation of responses and award points in each of the categories detailed below (up to the maximum evaluation points indicated) to each response deemed by the State to be responsive.

Evaluation Table

EVALUATION CATEGORY MAXIMUM POINTS POSSIBLE

General Qualifications & Experience (refer to RFP Attachment 6.2., Section B)

10

Technical Qualifications, Experience & Approach (refer to RFP Attachment 6.2., Section C)

60

Oral Presentation (refer to RFP Attachment 6.2., Section D)

0

Cost Proposal (refer to RFP Attachment 6.3.) 30

5.2. Evaluation Process

The evaluation process is designed to award the contract resulting from this RFP not necessarily to the Respondent offering the lowest cost, but rather to the Respondent deemed by the State to be responsive and responsible who offers the best combination of attributes based upon the evaluation criteria. (“Responsive Respondent” is defined as a Respondent that has submitted a response that conforms in all material respects to the RFP. “Responsible Respondent” is defined as a Respondent that has the capacity in all respects to perform fully the contract requirements, and the integrity and reliability which will assure good faith performance.)

5.2.1. Technical Response Evaluation. The Solicitation Coordinator and the Proposal Evaluation

Team (consisting of three (3) or more State employees and non-State employee experts) will use the RFP Attachment 6.2., Technical Response & Evaluation Guide to manage the Technical Response Evaluation and maintain evaluation records.

5.2.1.1. The State reserves the right, at its sole discretion, to request Respondent clarification

of a Technical Response or to conduct clarification discussions with any or all Respondents. Any such clarification or discussion will be limited to specific sections of the response identified by the State. The subject Respondent must put any resulting clarification in writing as may be required and in accordance with any deadline imposed by the State.

5.2.1.2. The Solicitation Coordinator will review each Technical Response to determine

compliance with RFP Attachment 6.2., Technical Response & Evaluation Guide, Section A— Mandatory Requirements. If the Solicitation Coordinator determines that a response failed to meet one or more of the mandatory requirements, the Proposal Evaluation Team will review the response and document the team’s determination of whether:

a. the response adequately meets RFP requirements for further evaluation;

b. the State will request clarifications or corrections for consideration prior to further

evaluation; or,

c. the State will determine the response to be non-responsive to the RFP and reject it.

15 RFP 33111-03819

5.2.1.3. Proposal Evaluation Team members will independently evaluate each Technical

Response (that is responsive to the RFP) against the evaluation criteria in this RFP and will score each in accordance with the RFP Attachment 6.2., Technical Response & Evaluation Guide.

5.2.1.4. For each response evaluated, the Solicitation Coordinator will calculate the average of

the Proposal Evaluation Team member scores for RFP Attachment 6.2., Technical Response & Evaluation Guide, and record each average as the response score for the respective Technical Response section.

5.2.1.5. The Solicitation Coordinator will invite the top three (3) ranked Respondents to make an

oral presentation.

5.2.1.5.1. The Solicitation Coordinator will schedule Respondent presentations during the period indicated by the RFP Section 2, Schedule of Events. The Solicitation Coordinator will make every effort to accommodate each Respondent’s schedules. When the Respondent presentation schedule has been determined, the Solicitation Coordinator will contact Respondents with the relevant information as indicated by RFP Section 2, Schedule of Events.

5.2.1.5.2. Respondent presentations are only open to the invited Respondent,

Proposal Evaluation Team members, the Solicitation Coordinator, and any technical consultants who are selected by the State to provide assistance to the Proposal Evaluation Team.

5.2.1.5.3. Oral presentations provide an opportunity for Respondents to explain

and clarify their responses. Respondents must not materially alter their responses and presentations will be limited to addressing the items detailed in RFP Attachment 6.2., Technical Response & Evaluation Guide. Respondent pricing shall not be discussed during oral presentations. Evaluators may adjust Respondents’ Technical Response scores based on Oral Presentations.

5.2.1.5.4. The State will maintain an accurate record of each Respondent’s oral

presentation session. The record of the Respondent’s oral presentation shall be available for review when the State opens the procurement files for public inspection.

5.2.1.6. Before Cost Proposals are opened, the Proposal Evaluation Team will review the

Technical Response Evaluation record and any other available information pertinent to whether or not each Respondent is responsive and responsible. If the Proposal Evaluation Team identifies any Respondent that does not meet the responsive and responsible thresholds such that the team would not recommend the Respondent for Cost Proposal Evaluation and potential contract award, the team members will fully document the determination.

5.2.2. Cost Proposal Evaluation. The Solicitation Coordinator will open for evaluation the Cost

Proposal of each Respondent deemed by the State to be responsive and responsible and calculate and record each Cost Proposal score in accordance with the RFP Attachment 6.3., Cost Proposal & Scoring Guide.

5.2.2.1 Clarifications and Negotiations: The State reserves the right to award a contract on the basis of initial responses received, therefore, each response shall contain the Respondent’s best terms and conditions from a technical and cost standpoint. The State reserves the right to conduct clarifications or negotiations with one or more Respondents.

16 RFP 33111-03819

All communications, clarifications, and negotiations shall be conducted in a manner that supports fairness in response improvement.

5.2.2.2. Clarifications: The State may identify areas of a response that may require further clarification or areas in which it is apparent that there may have been miscommunications or misunderstandings as to the State’s specifications or requirements. The State may seek to clarify those issues identified during one or multiple clarification rounds. Each clarification sought by the State may be unique to an individual Respondent, provided that the process is conducted in a manner that supports fairness in response improvement.

5.2.2.3. Negotiations: The State may elect to negotiate with one or more Respondents by requesting revised responses, negotiating costs, or finalizing contract terms and conditions. The State reserves the right to conduct multiple negotiation rounds or no negotiations at all.

5.2.2.4. Cost Negotiations: All Respondents, selected for negotiation by the State, will be given equivalent information with respect to cost negotiations. All cost negotiations will be documented for the procurement file. Additionally, the State may conduct target pricing and other goods or services level negotiations. Target pricing may be based on considerations such as current pricing, market considerations, benchmarks, budget availability, or other methods that do not reveal individual Respondent pricing. During target price negotiations, Respondents are not obligated to reduce their pricing to target prices, but no Respondent is allowed to increase prices.

5.2.2.5. If the State determines that it is unable to successfully negotiate a contract with the apparent best evaluated Respondent, the State reserves the right to bypass the apparent best evaluated Respondent and enter into contract negotiations with the next apparent best evaluated Respondent.

5.2.3. Total Response Score. The Solicitation Coordinator will calculate the sum of the Technical Response section scores and the Cost Proposal score and record the resulting number as the total score for the subject Response (refer to RFP Attachment 6.5., Score Summary Matrix).

5.3. Contract Award Process

5.3.1. The Solicitation Coordinator will submit the Proposal Evaluation Team determinations and scores to the head of the procuring agency for consideration along with any other relevant information that might be available and pertinent to contract award.

5.3.2. The procuring agency head will determine the apparent best-evaluated Response. To effect a contract award to a Respondent other than the one receiving the highest evaluation process score, the head of the procuring agency must provide written justification and obtain the written approval of the Chief Procurement Officer and the Comptroller of the Treasury.

5.3.3. The State will issue a Notice of Intent to Award identifying the apparent best-evaluated response and make the RFP files available for public inspection at the time and date specified in the RFP Section 2, Schedule of Events.

NOTICE: The Notice of Intent to Award shall not create rights, interests, or claims of entitlement in either the apparent best-evaluated Respondent or any other Respondent.

17 RFP 33111-03819

5.3.4. The Respondent identified as offering the apparent best-evaluated response must sign a contract drawn by the State pursuant to this RFP. The Contract shall be substantially the same as the RFP Attachment 6.6., Pro Forma Contract. The Respondent must sign the contract by the Contractor Signature Deadline detailed in the RFP Section 2, Schedule of Events. If the Respondent fails to provide the signed Contract by this deadline, the State may determine that the Respondent is non-responsive to this RFP and reject the response.

5.3.5. Notwithstanding the foregoing, the State may, at its sole discretion, entertain limited negotiation prior to Contract signing and, as a result, revise the pro forma contract terms and conditions or performance requirements in the State’s best interests, PROVIDED THAT such revision of terms and conditions or performance requirements shall NOT materially affect the basis of response evaluations or negatively impact the competitive nature of the RFP and contractor selection process.

5.3.6. If the State determines that a response is non-responsive and rejects it after opening Cost Proposals, the Solicitation Coordinator will re-calculate scores for each remaining responsive Cost Proposal to determine (or re-determine) the apparent best-evaluated response.

18 RFP 33111-03819

RFP ATTACHMENT 6.1.

RFP # 33111-03819 STATEMENT OF CERTIFICATIONS AND ASSURANCES

The Respondent must sign and complete the Statement of Certifications and Assurances below as required, and it must be included in the Technical Response (as required by RFP Attachment 6.2., Technical Response & Evaluation Guide, Section A, Item A.1.).

The Respondent does, hereby, expressly affirm, declare, confirm, certify, and assure ALL of the following:

1. The Respondent will comply with all of the provisions and requirements of the RFP.

2. The Respondent will provide all services as defined in the Scope of the RFP Attachment 6.6., Pro Forma Contract for the total Contract Term.

3. The Respondent, except as otherwise provided in this RFP, accepts and agrees to all terms and conditions set out in the RFP Attachment 6.6., Pro Forma Contract.

4. The Respondent acknowledges and agrees that a contract resulting from the RFP shall incorporate, by reference, all proposal responses as a part of the Contract.

5. The Respondent will comply with:

(a) the laws of the State of Tennessee;

(b) Title VI of the federal Civil Rights Act of 1964;

(c) Title IX of the federal Education Amendments Act of 1972;

(d) the Equal Employment Opportunity Act and the regulations issued there under by the federal government; and,

(e) the Americans with Disabilities Act of 1990 and the regulations issued there under by the federal government.

6. To the knowledge of the undersigned, the information detailed within the response submitted to this RFP is accurate.

7. The response submitted to this RFP was independently prepared, without collusion, under penalty of perjury.

8. No amount shall be paid directly or indirectly to an employee or official of the State of Tennessee as wages, compensation, or gifts in exchange for acting as an officer, agent, employee, subcontractor, or consultant to the Respondent in connection with this RFP or any resulting contract.

9. Both the Technical Response and the Cost Proposal submitted in response to this RFP shall remain valid for at least 120 days subsequent to the date of the Cost Proposal opening and thereafter in accordance with any contract pursuant to the RFP.

10. The Respondent affirms the following statement, as required by the Iran Divestment Act Tenn. Code Ann. § 12-12-111: “By submission of this bid, each bidder and each person signing on behalf of any bidder certifies, and in the case of a joint bid each party thereto certifies as to its own organization, under penalty of perjury, that to the best of its knowledge and belief that each bidder is not on the list created pursuant to §12-12-106.” For reference purposes, the list is currently available online at: https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/library-/public-information-library.html.

By signing this Statement of Certifications and Assurances, below, the signatory also certifies legal authority to bind the proposing entity to the provisions of this RFP and any contract awarded pursuant to it. If the signatory is not the Respondent (if an individual) or the Respondent’s company President or Chief Executive Officer, this document must attach evidence showing the individual’s authority to bind the Respondent.

DO NOT SIGN THIS DOCUMENT IF YOU ARE NOT LEGALLY AUTHORIZED TO BIND THE RESPONDENT

SIGNATURE:

PRINTED NAME & TITLE:

DATE:

RESPONDENT LEGAL ENTITY NAME:

https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/library-/public-information-library.html

https://www.tn.gov/generalservices/procurement/central-procurement-office--cpo-/library-/public-information-library.html

19 RFP 33111-03819

RFP ATTACHMENT 6.2. — Section A

TECHNICAL RESPONSE & EVALUATION GUIDE

SECTION A: MANDATORY REQUIREMENTS. The Respondent must address all items detailed below and provide, in

sequence, the information and documentation as required (referenced with the associated item references). The Respondent must also detail the response page number for each item in the appropriate space below.

The Solicitation Coordinator and the Department of Education will review the response to determine if the Mandatory Requirement Items are addressed as required and mark each with pass or fail. For each item that is not addressed as required, the Proposal Evaluation Team must review the response and attach a written determination. In addition to the Mandatory Requirement Items, the Solicitation Coordinator and the Department of Education will review each response for compliance with all RFP requirements.


Response Page #

(Respondent completes)

Item Ref.

Section A— Mandatory Requirement Items Pass/Fail

The Response must be delivered to the State no later than the Response Deadline specified in the RFP Section 2, Schedule of Events.

The Technical Response and the Cost Proposal documentation must be packaged separately as required (refer to RFP Section 3.2., et. seq.).

The Technical Response must NOT contain cost or pricing information of any type.

The Technical Response must NOT contain any restrictions of the rights of the State or other qualification of the response.

A Respondent must NOT submit alternate responses (refer to RFP Section 3.3.).

A Respondent must NOT submit multiple responses in different forms (as a prime and a subcontractor) (refer to RFP Section 3.3.).

A.1. Provide the Statement of Certifications and Assurances (RFP Attachment 6.1.) completed and signed by an individual empowered to bind the Respondent to the provisions of this RFP and any resulting contract. The document must be signed without exception or qualification.

A.2. Provide a statement, based upon reasonable inquiry, of whether the Respondent or any individual who shall cause to deliver goods or perform services under the contract has a possible conflict of interest (e.g., employment by the State of Tennessee) and, if so, the nature of that conflict. NOTE: Any questions of conflict of interest shall be solely within the discretion of the State, and the State reserves the right to cancel any award.

A.3. Provide a current bank reference indicating that the Respondent’s business relationship with the financial institution is in positive standing. Such reference must be written in the form of a standard business letter, signed, and dated within the past three (3) months.

A.4. Provide two current positive credit references from vendors with which the Respondent has done business written in the form of standard business letters, signed, and dated within the past three (3) months.

20 RFP 33111-03819


Response Page #


Item Ref.


A.5. Provide an official document or letter from an accredited credit bureau, verified and dated within the last three (3) months and indicating a satisfactory credit rating for the Respondent (NOTE: A credit bureau report number without the full report is insufficient and will not be considered responsive.)

A.6

Provide statement agreeing to adhere to the State’s Enterprise Information Security Policies. The State’s Enterprise Information Security Policies document is found at https://www.tn.gov/finance/strategic-technology-solutions/strategic-technology-solutions/sts-security-policies.html or as modified and agreed to by State.

A.7. Provide evidence that the Respondent can ingest Tennessee test content into the Respondent’s computer-based test delivery system.

Acceptable evidence:

The State will provide all Respondents non-secure “check sets” that include QTI 2.1 packages to render the various item types required on the TCAP Assessments. The Respondent must demonstrate that it can import the State’s check sets that include QTI 2.1 packages and render the TN test content as designed without error or unacceptable variance in its computer-based test delivery system. Evidence of the complete and successful ingestion must be provided by the response deadline.

A.8. Provide evidence that the Respondent has experience administering at least 1.5 million test sessions during at least one three- to four-week test administration window. During the testing window, the load should have included at least 100,000 concurrent users and the percent of down time should not have exceeded 0.5% of operational administration time across all users.


Authenticated log files from the Respondent’s server infrastructure, and

Verifiable statement(s) from the Respondent’s client(s) for whom the above criteria were met.

A.9 Provide evidence that the Respondent can provide assessments in both paper- and computer-based formats for operational use by Fall 2019 for the End of Course (EOC) content areas and by Spring 2020 for grades 2-8.


A timeline of deployment that includes a key dates schedule that outlines delivery of EOC content area assessments operational testing windows beginning on December 2, 2019 for fall EOC testing, on March 16, 2020 for grade 2, science and social studies ALT testing, and on April 13, 2020 for grades 2-8 and EOC spring testing.

A.10. Provide evidence that the Respondent can meet the data management requirements to provision students for the TCAP Assessments accurately, consistently and timely.

Acceptable Evidence:

https://www.tn.gov/finance/strategic-technology-solutions/strategic-technology-solutions/sts-security-policies.html


21 RFP 33111-03819


Response Page #


Item Ref.


Link(s) to the Respondent’s secure website(s) that include fully-functional demo prototypes, along with documentation, for its web-based provisioning applications. The documentation must include the minimum amount of data needed to provision a test by the system and describe how the State will provide student information in the system for assigning test events, reporting, and editing, and

A quality assurance plan for Respondent’s web-based provisioning applications. The plan should include a mitigation plan in case of severe issues such as outages or compromised data quality.

A.11. Provide evidence that the Respondent can accurately and efficiently score all TCAP Assessments.


Description of the Respondent’s quality assurance procedures and quality control checks for its machine and human scoring processes,

Documentation or description of procedure for independent verification of scoring, and

Verifiable statement(s) from the Respondent’s client(s) for whom automated (AI) scoring was implemented to score all or a subset of the client’s constructed responses items or tasks that the procedures employed successfully met the client’s objectives. Statement must include contact name, phone number and e-mail address to verify information.

A.12. Provide evidence that the Respondent can provide reporting for the TCAP Assessments accurately, consistently and timely.

Acceptable evidence includes all of the below:

Sample schedule(s) from client(s) for whom the Respondent has delivered reports of assessment results,

The Respondent’s plan for reporting that allow for comparisons at the student, class, teacher, school, district and state levels,

Link(s) to secure website(s) that include fully-functional demo prototypes, along with documentation, for the Respondent’s web-based reporting and report delivery system,

A quality assurance plan for the Respondent’s web-based reporting and report delivery system. The plan should include a mitigation plan in case of severe issues such as system outages,

Key Dates Schedule that outlines the delivery of raw scores to districts for Fall EOC assessments by January 6, 2020 and grades 3-8 and Spring EOC assessments by May 20, 2020,

Key Dates Schedule that outlines the delivery of cumulative data files (CDFs) for EOC assessments to the state by January 13, 2020 for Fall EOC testing and June 1, 2020 for Spring EOC testing,

Key Dates Schedule that outlines the delivery of CDFs for grades 2-8 and TCAP Alternate assessments to the state by June 1, 2020,

Key Dates Schedule that outlines delivery of reports for Fall EOC assessments to districts by February 11, 2020, and,

Key Dates Schedule that outlines delivery of reports for Spring EOC

22 RFP 33111-03819


Response Page #


Item Ref.


assessments to districts by June 29, 2020, and,

Key Dates Schedule that outlines delivery of reports for grades 2-8 and ALT assessments to districts by June 29, 2020.

A.13. Provide a copy of Respondent’s Tennessee Department of Revenue Certificate of Registration for sales and use tax. If it is determined the Respondent does not need to obtain a sales &use tax account, a letter from the Tennessee Department of Revenue can be issued to fulfill this requirement.

Respondents may contact the Tennessee Department of Revenue by email at [email protected]. In addition to their email address, vendors should provide a contact name and telephone number, as well as the answers to the following questions (some may not apply to every situation):

1. As an out-of-state vendor, what are you selling or providing to a Tennessee entity?

2. Do you have any employees in Tennessee? 3. Do you have a business location in Tennessee? 4. Do you have sales reps that enter Tennessee? 5. How is your product delivered to a Tennessee customer? (common

carrier or company vehicles, electronically, or by tangible method) 6. If installation is required, who installs the product?

State Use – Solicitation Coordinator Signature, Printed Name & Date:


23 RFP 33111-03819

RFP ATTACHMENT 6.2. — SECTION B


SECTION B: GENERAL QUALIFICATIONS & EXPERIENCE. The Respondent must address all items detailed below and

provide, in sequence, the information and documentation as required (referenced with the associated item references). The Respondent must also detail the response page number for each item in the appropriate space below. Proposal Evaluation Team members will independently evaluate and assign one score for all responses to Section B— General Qualifications & Experience Items.


Response Page #


Item Ref.

Section B— General Qualifications & Experience Items

B.1. Detail the name, e-mail address, mailing address, telephone number, and facsimile number of the person the State should contact regarding the response.

B.2. Describe the Respondent’s form of business (i.e., individual, sole proprietor, corporation, non-profit corporation, partnership, limited liability company) and business location (physical location or domicile).

B.3. Detail the number of years the Respondent has been in business.

B.4. Briefly describe how long the Respondent has been providing the goods or services required by this RFP.

B.5. Describe the Respondent’s number of employees, client base, and location of offices.

B.6. Provide a statement of whether there have been any mergers, acquisitions, or change of control of the Respondent within the last ten (10) years. If so, include an explanation providing relevant details.

B.7. Provide a statement of whether the Respondent or, to the Respondent's knowledge, any of the Respondent’s employees, agents, independent contractors, or subcontractors, involved in the delivery of goods or performance of services on a contract pursuant to this RFP, have been convicted of, pled guilty to, or pled nolo contendere to any felony. If so, include an explanation providing relevant details.

B.8. Provide a statement of whether, in the last ten (10) years, the Respondent has filed (or had filed against it) any bankruptcy or insolvency proceeding, whether voluntary or involuntary, or undergone the appointment of a receiver, trustee, or assignee for the benefit of creditors. If so, include an explanation providing relevant details.

B.9. Provide a statement of whether there is any material, pending litigation against the Respondent that the Respondent should reasonably believe could adversely affect its ability to meet contract requirements pursuant to this RFP or is likely to have a material adverse effect on the Respondent’s financial condition. If such exists, list each separately, explain the relevant details, and attach the opinion of counsel addressing whether and to what extent it would impair the Respondent’s performance in a contract pursuant to this RFP.

NOTE: All persons, agencies, firms, or other entities that provide legal opinions regarding the Respondent must be properly licensed to render such opinions. The State may require the Respondent to submit proof of license for each person or entity that renders such opinions.

B.10. Provide a statement of whether there are any pending or in progress Securities Exchange Commission investigations involving the Respondent. If such exists, list each separately, explain the relevant details, and attach the opinion of counsel addressing whether and to what extent it will impair the Respondent’s performance in a contract pursuant to this RFP.

NOTE: All persons, agencies, firms, or other entities that provide legal opinions regarding the

09-20-18 RFP

RFP ATTACHMENT 6.2. — SECTION B (continued)

24 RFP 33111-03819


Response Page #


Item Ref.


Respondent must be properly licensed to render such opinions. The State may require the Respondent to submit proof of license for each person or entity that renders such opinions.

B.11. Provide a brief, descriptive statement detailing evidence of the Respondent’s ability to deliver the goods or services sought under this RFP (e.g., prior experience, training, certifications, resources, program and quality management systems, etc.).

B.12. Provide a narrative description of the proposed project team, its members, and organizational structure along with an organizational chart identifying the key people who will be assigned to deliver the goods or services required by this RFP.

B.13. Provide a personnel roster listing the names of key people who the Respondent will assign to meet the Respondent’s requirements under this RFP along with the estimated number of hours that each individual will devote to that performance. Follow the personnel roster with a resume for each of the people listed. The resumes must detail the individual’s title, education, current position with the Respondent, and employment history.

B.14. Provide a statement of whether the Respondent intends to use subcontractors to meet the Respondent’s requirements of any contract awarded pursuant to this RFP, and if so, detail:

(a) the names of the subcontractors along with the contact person, mailing address, telephone number, and e-mail address for each;

(b) a description of the scope and portions of the goods each subcontractor involved in the delivery of goods or performance of the services each subcontractor will perform; and

(c) a statement specifying that each proposed subcontractor has expressly assented to being proposed as a subcontractor in the Respondent’s response to this RFP.

B.15. Provide documentation of the Respondent’s commitment to diversity as represented by the following:

(a) Business Strategy. Provide a description of the Respondent’s existing programs and procedures designed to encourage and foster commerce with business enterprises owned by minorities, women, service-disabled veterans, persons with disabilities, and small business enterprises. Please also include a list of the Respondent’s certifications as a diversity business, if applicable.

(b) Business Relationships. Provide a listing of the Respondent’s current contracts with business enterprises owned by minorities, women, service-disabled veterans, persons with disabilities, and small business enterprises. Please include the following information:

(i) contract description;

(ii) contractor name and ownership characteristics (i.e., ethnicity, gender, service-disabled veteran-owned or persons with disabilities);

(iii) contractor contact name and telephone number.

(c) Estimated Participation. Provide an estimated level of participation by business enterprises owned by minorities, women, service-disabled veterans, persons with disabilities and small business enterprises if a contract is awarded to the Respondent pursuant to this RFP. Please include the following information:

(i) a percentage (%) indicating the participation estimate. (Express the estimated participation number as a percentage of the total estimated contract value that will be dedicated to business with subcontractors and supply contractors having such ownership characteristics only and DO NOT INCLUDE DOLLAR AMOUNTS);

(ii) anticipated goods or services contract descriptions;

(iii) names and ownership characteristics (i.e., ethnicity, gender, service-disabled veterans, or disability) of anticipated subcontractors and supply contractors.

09-20-18 RFP


25 RFP 33111-03819


Response Page #


Item Ref.


NOTE: In order to claim status as a Diversity Business Enterprise under this contract, businesses must be certified by the Governor’s Office of Diversity Business Enterprise (Go-DBE). Please visit the Go-DBE website at https://tn.diversitysoftware.com/FrontEnd/StartCertification.asp?TN=tn&XID=9810 for more information.

(d) Workforce. Provide the percentage of the Respondent’s total current employees by ethnicity and gender.

NOTE: Respondents that demonstrate a commitment to diversity will advance State efforts to expand opportunity to do business with the State as contractors and subcontractors. Response evaluations will recognize the positive qualifications and experience of a Respondent that does business with enterprises owned by minorities, women, service-disabled veterans, persons with disabilities, and small business enterprises and who offer a diverse workforce.

B.16. Provide a statement of whether or not the Respondent has any current contracts with the State of Tennessee or has completed any contracts with the State of Tennessee within the previous five (5) year period. If so, provide the following information for all of the current and completed contracts:

(a) the name, title, telephone number and e-mail address of the State contact knowledgeable about the contract;

(b) the procuring State agency name;

(c) a brief description of the contract’s scope of services;

(d) the contract period; and

(e) the contract number.

NOTES: Current or prior contracts with the State are not a prerequisite and are not required for the

maximum evaluation score, and the existence of such contracts with the State will not automatically result in the addition or deduction of evaluation points.

Each evaluator will generally consider the results of inquiries by the State regarding all contracts noted.

B.17. Provide customer references from individuals who are not current or former State employees for projects similar to the goods or services sought under this RFP and which represent:

two (2) accounts Respondent currently services that are similar in size to the State; and

three (3) completed projects.

References from at least three (3) different individuals are required to satisfy the requirements above, e.g., an individual may provide a reference about a completed project and another reference about a currently serviced account. The standard reference questionnaire, which must be used and completed, is provided at RFP Attachment 6.4. References that are not completed as required may be deemed non-responsive and may not be considered.

The Respondent will be solely responsible for obtaining fully completed reference questionnaires and including them in the sealed Technical Response. In order to obtain and submit the completed reference questionnaires follow the process below.

(a) Add the Respondent’s name to the standard reference questionnaire at RFP Attachment 6.4. and make a copy for each reference.

(b) Send a reference questionnaire and new, standard #10 envelope to each reference.

(c) Instruct the reference to:

(i) complete the reference questionnaire;

(ii) sign and date the completed reference questionnaire;

(iii) seal the completed, signed, and dated reference questionnaire within the envelope provided;

https://tn.diversitysoftware.com/FrontEnd/StartCertification.asp?TN=tn&XID=9810

09-20-18 RFP


26 RFP 33111-03819


Response Page #


Item Ref.


(iv) sign his or her name in ink across the sealed portion of the envelope; and

(v) return the sealed envelope directly to the Respondent (the Respondent may wish to give each reference a deadline, such that the Respondent will be able to collect all required references in time to include them within the sealed Technical Response).

(d) Do NOT open the sealed references upon receipt.

(e) Enclose all sealed reference envelopes within a larger, labeled envelope for inclusion in the Technical Response as required.

NOTES: The State will not accept late references or references submitted by any means other than that

which is described above, and each reference questionnaire submitted must be completed as required.

The State will not review more than the number of required references indicated above. While the State will base its reference check on the contents of the sealed reference envelopes

included in the Technical Response package, the State reserves the right to confirm and clarify information detailed in the completed reference questionnaires and may consider clarification responses in the evaluation of references.

The State is under no obligation to clarify any reference information.

B.18. Provide a statement and any relevant details addressing whether the Respondent is any of the following:

(a) is presently debarred, suspended, proposed for debarment, or voluntarily excluded from covered transactions by any federal or state department or agency;

(b) has within the past three (3) years, been convicted of, or had a civil judgment rendered against the contracting party from commission of fraud, or a criminal offence in connection with obtaining, attempting to obtain, or performing a public (federal, state, or local) transaction or grant under a public transaction; violation of federal or state antitrust statutes or commission of embezzlement, theft, forgery, bribery, falsification or destruction of records, making false statements, or receiving stolen property;

(c) is presently indicted or otherwise criminally or civilly charged by a government entity (federal, state, or local) with commission of any of the offenses detailed above; and

(d) has within a three (3) year period preceding the contract had one or more public transactions (federal, state, or local) terminated for cause or default.

SCORE (for all Section B—Qualifications & Experience Items above):

(maximum possible score = 10)

State Use – Evaluator Identification:

09-20-18 RFP

27 RFP 33111-03819

RFP ATTACHMENT 6.2. — SECTION C


SECTION C: TECHNICAL QUALIFICATIONS, EXPERIENCE & APPROACH. The Respondent must address all items

(below) and provide, in sequence, the information and documentation as required (referenced with the associated item references). The Respondent must also detail the response page number for each item in the appropriate space below.

A Proposal Evaluation Team, made up of three or more State employees, will independently evaluate and score the response to each item. Each evaluator will use the following whole number, raw point scale for scoring each item:

0 = little value 1 = poor 2 = fair 3 = satisfactory 4 = good 5 = excellent

The Solicitation Coordinator will multiply the Item Score by the associated Evaluation Factor (indicating the relative emphasis of the item in the overall evaluation). The resulting product will be the item’s Raw Weighted Score for purposes of calculating the section score as indicated.


Response Page #


Item Ref. Section C— Technical Qualifications,

Experience & Approach Items Item

Score Evaluation

Factor

Raw Weighted

Score

C.1. Provide a narrative that illustrates the Respondent’s understanding of the State’s requirements and project schedule.

1

C.2. Provide a narrative that illustrates how the Respondent will complete the scope of services, accomplish required objectives, and meet the State’s project schedule.

1

C.3. Provide a narrative that illustrates how the Respondent will manage the project, ensure completion of the scope of services, and accomplish required objectives within the State’s project schedule.

2

C.4. Provide narrative that illustrates how the Respondent plans to fulfill the requirements for assessment services as outlined in Pro Forma Contract Section A.3.

Include how the Respondent has met similar requirements in the past, discuss any special circumstances that might affect this Contract, and how the Respondent intends to deal with those circumstances.

5

C.5. Provide narrative that illustrates a plan for transitioning from paper-based testing (PBT) to computer-based testing (CBT) for each grade span/content area in which such a change is indicated in the Pro Forma Contract Section A.3.c.

Include detailed descriptions of key activities and deliverables, discussion of potential challenges and how the Respondent plans to address each challenge and to implement quality control checks to support a successful transition.

6

C.6. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for program management as outlined in Pro Forma Contract Section A.4.

5

09-20-18 RFP

RFP ATTACHMENT 6.2. — SECTION C (continued)

28 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

Include an organization chart that shows the structure of the project team that the Respondent has proposed for this Contract; evidence of highly-skilled project management team members, especially for the Lead Program Manager; the proposed process by which the State may engage with authorities within the company to resolve issues; documentation of how the proposed process has been successfully used to support client satisfaction in other assessment programs with similar scope and timelines; and, how the Respondent plans to facilitate timely, responsive, engaged communication and decision-making for this Contract.

C.7. Provide a narrative that illustrates the process for the management and timely resolution of the issues received by the Respondent’s Call Center and other technical support issues as required in Pro Forma Contract Section A.4.(h)(1). The proposed process should allow the State and the Respondent to efficiently oversee the customer service operations as a means for sustaining and improving the quality of the assessment program and user satisfaction.

Include the proposed nature and extent of the technical support services to meet the needs of the State, and why the Respondent anticipates this level of support will be sufficient.

5

C.8. Provide narrative that illustrates how the Respondent plans to fulfill the State’s requirements for the test security procedures, including forensics analysis, as outlined in Pro Forma Contract Section A.5.

Include how the Respondent plans to enhance the test security procedures for the State’s assessments, including any policies, procedures, or analyses not listed in section A.5.

4

C.9. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for quality assurance as outlined in Pro Forma Contract Section A.6.

Include details about the Respondent’s risk mitigation processes and strategies. Also, if applicable, include any quality issues for which the Respondent has been responsible in the past five years with other assessment program(s). Without violating any nondisclosure or confidentiality agreements, the information should include the name of the assessment program(s), descriptions of the errors or defects, the magnitude and impact of the issues, findings from root cause analyses, processes or strategies implemented to correct the errors and prevent reoccurrence, and the timelines in which the mitigation processes or strategies were carried out.

6

09-20-18 RFP


29 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

C.10. Provide a narrative that illustrates how the Respondent plans to fulfill the requirements for psychometrics and research as outlined in Pro Forma Contract Section A.7.

Include the Respondent’s experience with similar development activities for programs on a similar scale and timelines.

5

C.11. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for assessment materials as outlined in Pro Forma Contract Section A.8.

Include the Respondent’s plan for handling straggler test documents and materials for paper-based tests.

Include the Respondent’s plan for overage for all printed materials to ensure that all schools have sufficient materials. If overage differs by assessment type or printed material type, specify the overage for each material type.

Include the Respondent’s plan for pre-id and whether it entails the use of an overspray (vendor applied) or labels (school applied).

Include the Respondent’s plan for shipment for all printed materials (e.g. whether ground, 2

nd day, or

other method will be used).

Include a description of all box types that may be used for shipment of materials, including sizes, colors, and bursting tests.

Include a sample of the proposed reports available for practice tests.

6

C.12. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for packaging and distributing assessment materials as outlined in Pro Forma Contract Section A.8.g.

Include the Respondent’s experience with paper-based testing (PBT) for assessment programs on a similar scale and timelines and how the Respondent typically handles the logistics of packaging and distributing paper test materials.

4

C.13. Provide access to a fully-functional demo site, along with documentation, for the Respondent’s computer-based testing (CBT) delivery system as outlined in Pro Forma Contract Section A.9. The State will provide non-secure “check sets” that include QTI 2.1 packages to render the various item types required on the TCAP Assessments. The Respondent must demonstrate that it can import the State’s check sets and render the TN test content in its CBT delivery system.

6

09-20-18 RFP


30 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

Include, as part of the demo site, all required item types, reference materials, navigation features, tools, accommodations and accessibility features. The goal of the demo site is to showcase the Respondent’s ability to ingest and render without error TCAP content from the development vendor on multiple types of devices, operating systems, and web browsers.

C.14. Provide comprehensive specifications of the devices, operating systems, and web browsers that the Respondent’s CBT delivery system supports on the client (user) side.

Include minimum requirements in terms of versions and technical specification for the CBT delivery system. Please note that the State places significant value in compatibility with the devices that already exist in Tennessee school districts which include:

Platform Operating System Versions

Windows Windows 7, 8, 10

Chromebook Chrome OS version 59+

Apple Mac OS 10+

Apple IOS IOS 11+

5

C.15. Provide a narrative that illustrates the Respondent’s available tools, utilities and processes that can help school districts evaluate the degree to which they have attained technical readiness for CBT, such as device compatibility, network performance and bandwidth as outlined in Pro Forma Contract Section A.9.

5

C.16. Provide a narrative that illustrates the Respondent’s methodology for providing field support to districts who experience technical difficulties during the operational testing windows as outlined in Pro Forma Contract Section A.9.

Include specifically the Respondent’s plan for hands-on customer support for troubleshooting or mitigating common technical problems.

5

C.17. Provide a narrative that illustrates the Respondent’s plan to conduct field trials of its CBT delivery system as outlined in Pro Forma Contract Section A.9. The goal of the field trials is to ensure that the system operates correctly and has adequate capacity to meet expected testing loads under the realistic conditions and use cases in the Tennessee assessment context.

Include the proposed frequency on which the field

5

09-20-18 RFP


31 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

trials will be conducted; how system analytics and district (user) feedback will be monitored and collected; and how results of the field trials will be communicated to the State.

C.18. Provide a narrative that illustrates the ability of the Respondent’s CBT delivery system to function in low bandwidth scenarios as outlined in Pro Forma Contract Section A.9.

Include the minimum bandwidth required for the Respondent’s system to operate viably from the user’s device to point of ingress at the data center; the minimum requirement to operate optimally; and whether the system requires constant network connectivity throughout a testing session. (If yes, include an explanation for this limitation; if no, include an explanation of how the Respondent has eliminated the need for constant connectivity.)

5

C.19. Provide a narrative that illustrates the measures that the Respondent has taken to build a CBT delivery system that is fault tolerant and able to minimize student impact during unexpected events, such as dropped network connections, broadband disruptions or other transients typical in education across a diverse state as outlined in Pro Forma Contract Section A.9.

Include specific requirements or rationale that have led to architectural and technical choices.

5

C.20. Provide a narrative that illustrates the Respondent’s solution to recovering student work in the event of a long-term network outage, device failure or other event that prevents a student from submitting a completed test as outlined in Pro Forma Contract Section A.9.

Include the Respondent’s approach to recovering constructed responses; specify limitations and constraints under which recovery is possible.

5

C.21. Provide a narrative that illustrates how the Respondent will dedicate system capacity to the Tennessee assessment program such that events and actions of its other customers and tenants of the same data center will not impact students in Tennessee as outlined in Pro Forma Contract Section A.9.

Include all aspects of the Respondent’s computer-based testing delivery system from point of ingress to its data center(s) back to point of egress.

5

C.22. Provide a narrative that illustrates the Respondent’s strategy for preventing denial of service (DOS and DDOS) attacks and other forms of “man in the middle” exploits as outlined in Pro Forma Contract Section

5

09-20-18 RFP


32 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

A.9.

Include the Respondent’s specific mechanisms for detection, defense and recovery.

C.23. Provide a narrative that illustrates where the Respondent’s CBT delivery system components are located.

If the Respondent uses a public or private cloud platform, include which cloud platform and include details on your use of Platform as a Service (PaaS) verses Infrastructure as a Service (IaaS) architectures; if the Respondent does not use PaaS infrastructure, include how it uses virtualization; also, include an overview of the Respondent’s use of a content delivery network (CDN), if applicable.

5

C.24. Provide narrative that illustrates an architectural overview of the Respondent’s CBT delivery system placing emphasis on the logical tiers that exist and the data flows between those tiers and external components as outlined in Pro Forma Contract Section A.9. For example, if the design includes a web front end (WFE), middleware layer that supports scoring and a database backend, each would be considered a tier.

Include descriptions of how the tiers are coupled, synchronous versus asynchronous processing, and the approach to implementing the Respondent’s message bus or equivalent queue processor. Please assume a deep technical understanding of contemporary cloud architecture and services-oriented architecture on the part of the reviewer. Submission of architectural diagrams is encouraged.

5

C.25. Provide a narrative that illustrates the Respondent’s method of understanding the expected load characteristics and usage profile of a new customer and how that understanding is used to conduct system capacity planning, design reviews and configuration optimizations in advance of operational testing as outlined in Pro Forma Contract Section A.9.

5

C.26. Provide a narrative that illustrates how the Respondent scales out and scales up its system capacity both in response to proactive capacity planning or reactively if reduced headroom is observed during operational testing as outlined in Pro Forma Contract Section A.9.

Include how the Respondent’s system scalability relates to each architectural tier described above; place specific emphasis on how the Respondent scales its web frontend, database tier and other components that are part of a synchronously

5

09-20-18 RFP


33 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

processed student event (i.e. impacts the student experience during testing); describe how the scale strategy ensures that the database component is not a bottleneck or single point of failure.

C.27. Provide a narrative that illustrates the Respondent’s backup protocols and disaster recovery plan as outlined in Pro Forma Contract Section A.9.

Include any other aspects related to redundancy and system recovery that is relevant.

5

C.28. Provide a description of how the Respondent ensures service availability and optimum performance during operational testing as outlined in Pro Forma Contract Section A.9.

Include whether the Respondent offers its customers a service level agreement (SLA); include any recovery point objective (RPO) and recovery time objective (RTO) that the Respondent is willing to commit to the State during operational testing and outside of operational testing.

5

C.29. Provide a narrative that illustrates the Respondent’s approaches to data protection as applied to ensuring appropriate security and privacy of State data in its possession, including data that is at rest and in motion, as outlined in Pro Forma Contract Section A.9.

If the Respondent conducts penetration testing, include information on who conducts the tests, the scope of the test and the frequency; include specifically technology and architectural choices.

6

C.30. Provide a narrative that illustrates the Respondent’s approach to monitoring and diagnostics during an operational window, in particular its strategy to detect and react quickly to system or environmental anomalies that could impact the student testing experience as outlined in Pro Forma Contract Section A.9.

5

C.31. Provide a narrative that illustrates how the Respondent’s organization ensures server operating systems, database and other software components are maintained at supported versions and patched appropriately in accordance with manufacturer recommendations and advisories as outlined in Pro Forma Contract Section A.9.

5

C.32. Provide a narrative that illustrates the Respondent’s approach to endpoint protection and threat monitoring (server-side) as outlined in Pro Forma Contract Section A.9.

5

09-20-18 RFP


34 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

C.33. Provide copies of the Respondent’s information systems policies, controls and procedures as they relate to the end-to-end delivery of a computer-based testing service.

Include description of the Respondent’s software development lifecycle (SDLC), quality assurance (QA), deployment & environment strategy, and change controls as they relate to the production environment; include additional policies that the Respondent feels best communicates its organization’s commitment to internal controls. Examples of other documents to include are:

Breech and Incident Response Management

Physical Security (building security)

HR Policies (as they relate to security, such as screening and background checks)

Data Retention and Destruction

Risk Management

6

C.34. Provide a list and copies of certifications that the Respondent holds as it relates to the end-to-end delivery of a CBT delivery system.

Emphasize those that verify the procedures and controls stated above; include procedural, infrastructure and audit related certifications. Please note that the State places significant emphasis on vendors obtaining SOC 2 TYPE II certifications. Examples of certifications, include:

SOC Audit

FEDRAMP

ISO

FIPS

AICPA

4

C.35. If the Respondent relies on business partners to help fulfill its CBT delivery responsibilities, provide a description of what roles are performed by third parties, how the Respondent manages the dependencies and the certifications held by its partners.

5

C.36. Describe your ability to ingest student data from the state to support provisioning of students for testing. Include your ability to accommodate new or transfer students within the operational testing window.

Include the Respondent’s system’s ability to accept partial data, changes and delta data (i.e. CRUD); include also protocols and standards that support the transmission of data, for example, JSON, XML, Ed-Fi, OneRoster etc.; indicate whether the system support pulling data from an API end point; and describe the system’s ability to provide data back to the State using

4

09-20-18 RFP


35 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

the same considerations.

C.37. Provide a narrative that illustrates the Respondent’s customer-facing software meets accessibility guidelines for education as outlined in Pro Forma Contract Section A.9.

Indicate the standards that are met, and the process used to verify that the software is accessible. Note that this requirement pertains to the Respondent’s overall user interface, navigation and design paradigms.

5

C.38. Provide a narrative that illustrates how long the Respondent has utilized its current system architecture (excluding minor improvements and upgrades).

Include any plans for a major shift, revision or redesign of the Respondent’s software or system architecture in the future and what the projected timeframe for any potential changes.

5

C.39. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for assessment provisioning activities as outlined in Pro Forma Contract Section A.10.

5

C.40. Provide access to a fully-functional demo site, along with documentation, for the Respondent’s web-based provisioning applications that are outlined in Pro Forma Contract Section A.10. The goal of the demo site is for the State to test the features and capabilities of the Respondent’s web-based provisioning applications on multiple types of devices, operating systems, and web browsers.

Include documentation that describes how the State will provide student information in the Respondent’s applications for assigning test events, reporting, and editing; also indicates the minimum amount of data needed to provision a test with the applications.

6

C.41. Provide a narrative that illustrates the Respondent’s quality assurance plan for data integrity in the web-based provisioning applications outlined in Pro Forma Contract Section A.10

Include a mitigation plan in case of severe issues such as application errors, system outages or compromised data quality.

6

C.42. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for assessment processing activities as outlined in Pro Forma Contract Section A.11.

Include how the Respondent plans to provide

6

09-20-18 RFP


36 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

accurate, consistent and timely processing of assessment responses to the State.

C.43. Provide a narrative that illustrates how the Respondent plans to fulfill the State’s requirements for scoring activities as outlined in Pro Forma Contract Section A.12.

Include the Respondent’s experience supporting scoring activities for other assessment programs on a similar scale and timeline; how the vendor will manage and monitor the effectiveness of scoring; how the Respondent plans to provide accurate, consistent and timely scores; the Respondent’s proposal for transitioning the TCAP Assessments to automated (AI) scoring; and evidence of the reliability and validity of the Respondent’s automated scoring engine. Also include a proposal for conducting range finding activities.

Complete the Technical Evaluation Scoring Criteria Attachment 6.2.A as a support to the narrative.

6

C.44. Provide narrative that illustrates how the Respondent plans to fulfill the State’s requirements for reporting activities as outlined in Pro Forma Contract Section A.13.

Include the Respondent’s experience providing reporting services for other assessment programs on a similar scale and timeline; and how the Respondent plans to ensure accuracy, consistency and timeliness of the cumulative data files (CDFs) for reporting.

6

C.45. Provide access to a fully-functional demo site, along with documentation, for its web-based reporting and report delivery system as outlined in Pro Forma Contract Section A.13.g. The goal of the demo site is for the State to test the features of the Respondent’s web-based reporting and report delivery system on multiple types of devices, operating systems, and web browsers.

6

C.46 Provide a narrative that illustrates how the Respondent will monitor and maintain cost control in the project, including specific information on procedures used for cost management. Note: No pricing may be provided in this response. The inclusion of any price proposal as part of the response to this evaluation criteria will result in the Respondent being deemed non-responsive.

4

C.47 List any requirements of the RFP that will result in Respondent incurring significant new development costs to either its test delivery, reporting, scoring or other software systems in order to satisfy the RFP

4

09-20-18 RFP


37 RFP 33111-03819


Response Page #




Score Evaluation

Factor

Raw Weighted

Score

requirements.

Additionally, RFP requirements in excess of industry standard that result in significant additional Respondent investment in other areas and increase the price of Respondent’s bid should be discussed.

Note: No pricing may be provided in this response. The inclusion of any price proposal as part of the response to this evaluation criteria will result in the Respondent being deemed non-responsive.

C.48 The Contractor shall provide a narrative demonstrating how it will provide permanent capacity for 250,000 concurrent users and its method for scaling for 500,000 concurrent users if needed.

6

The Solicitation Coordinator will use this sum and the formula below to calculate the section score. All calculations will use and result in numbers rounded to two (2) places to the right of the decimal point.

Total Raw Weighted Score: (sum of Raw Weighted Scores above)

Total Raw Weighted Score X 60

(maximum possible score)

= SCORE: Maximum Possible Raw Weighted Score

(i.e., 5 x the sum of item weights above)



09-20-18 RFP

38 RFP 33111-03819

RFP ATTACHMENT 6.2.— SECTION D


SECTION D: ORAL PRESENTATION (NOT SCORED). The Respondent, if ranked as a top three (3) Respondent, must address ALL Oral Presentation Items (below).

A Proposal Evaluation Team, made up of three or more State employees and non-state employees, will independently evaluate and score the presentation response to each item. Each evaluator will use the following whole-number, raw point scale for scoring each item:

0 = little value 1 = poor 2 = fair 3 = satisfactory 4 = good 5 = excellent

The Solicitation Coordinator will multiply the Item Score by the associated Evaluation Factor (indicating the relative emphasis of the item in the overall evaluation). The resulting product will be the item’s raw, weighted score for purposes of calculating the section score as indicated.


Oral Presentation Items Item Score

Evaluation Factor

Raw Weighted Score

D.1. Provide access to a fully-functional demo site, along with documentation,

for its web-based reporting and report delivery system as outlined in Pro Forma Contract Section A.13.g. The goal of the demo site is for the State to test the features of the Respondent’s web-based reporting and report delivery system on multiple types of devices, operating systems, and web browsers.

0 0 0

D.2. Additional questions that arise through the evaluation of the technical criteria, to be provided to Respondents in advance of the oral presentation date. The questions may be different for different respondents. Questions will be tailored based on the technical response received with goals to assure that the State understands the capabilities of the Respondent’s systems and to maintain a fair and competitive process.

0 0 0

Total Raw Weighted Score (sum of Raw Weighted Scores above):

The Solicitation Coordinator will use this sum and the formula below to calculate the score. Numbers rounded to two (2) places to the right of the decimal point will be standard for calculations.

0

total raw weighted score

X 0 (maximum section score)

= SCORE: 0 maximum possible raw weighted score

(i.e., 5 x the sum of item weights above)



09-20-18 RFP

39 RFP 33111-03819

RFP ATTACHMENT 6.3.

COST PROPOSAL & SCORING GUIDE NOTICE: THIS COST PROPOSAL MUST BE COMPLETED EXACTLY AS REQUIRED

Electronic Cost Proposals

a. Respondent shall submit the Cost Proposal to the State in a sealed package separate from the Technical Response (as detailed in RFP Sections 3.2.3., et seq.).

i. Respondent individual with the authority to bind the company must sign and date the Cost Proposal

in the Cost Proposal workbook. b. Cost Proposal Content. Respondent shall specifically include all costs including expenses to be charged for

performing the services necessary to accomplish the objectives of the contract. Pricing shall include all costs outlined in the pro-forma, which include but are not limited to all Respondent staff costs, administrative support costs, supplies, office supplies, pagers, cell phones, parking fees, meals, lodging, rents, mileage, travel expenses, training, after hours or weekend time, insurance, use of subcontractors, overhead, profit, and costs for all other items consumed, utilized, and/or required by Respondent’s staff or subcontractor’s staff. Unless otherwise specified, all hardware and software deemed necessary by the Respondent shall be included in the proposed costs. Items or costs required to provide the services and deliverables as proposed not identified in the Respondent’s cost proposal will be the sole responsibility of the Respondent.

c. Respondent shall discuss in its technical proposal response (Evaluation Criteria C.47) any requirements of the

RFP that shall result in Respondent incurring significant new development costs to either its test delivery, reporting, scoring or other software systems in order to satisfy the RFP requirements. Additionally, RFP requirements in excess of industry standard that result in significant additional Respondent investment in other areas and increase the price of Respondent’s bid shall be discussed in the applicable sections of Respondent’s response. The dollar amount of the Respondent investment that will be passed on to the State for each RFP requirement shall be detailed in the Respondent’s Cost Notes (see h. below) submitted along with its cost response.

d. Respondent shall propose their costs for the Administration, Scoring and Reporting of the Tennessee Assessments by completing the Assessment RFP Cost Proposal workbook.

i. Respondents shall complete the cost for each item in this tab. The cost for each row in the tab shall be input in column C for Year One (1), column G for Year Two (2), column K for Year Three (3), column O for Year Four (4), and column S for Year Five (5).

ii. The Base Evaluation Factors (Quantity) change from year two (2), to three (3) to account for the PBT to CBT transition of additional grades and content areas.

iii. If any row does not have a cost associated with it the Respondent shall indicate no cost by placing

a zero in the cell.

iv. For any row where the Respondent proposes additional quantities in their technical evaluation criteria, the Respondent shall enter the proposed additional quantity in the columns E, I, M, Q, and/or U (labeled "Additional Evaluation Factor (Qty)"), as appropriate for each year. The Respondent shall place a zero in this cell if no additional quantities are proposed for a given line item/row.

v. Each section of the Assessment RFP Cost Proposals (Cost or Services Description) column has

additional rows for the Respondent to capture items that are included in the technical evaluation criteria but not specified in the cost workbook. These rows contain the word "Other" (e.g. Row 16). If a Respondent needs to use these rows, they shall specify what the line item is and where in the technical criteria response the item is mentioned. The State reserves the right to change any additional row and the associated quantities to ensure they reflect the Respondent’s response in the RFP technical evaluation criteria.

vi. The State reserves the right to adjust the quantities in columns E, I, M, Q, and/or U (labeled "Additional Evaluation Factor (Qty)") and the associated cost for any row to ensure they reflect the Respondent’s response in the RFP technical evaluation criteria. Specifically, the overage provided

09-20-18 RFP

40 RFP 33111-03819

for assessment materials should match the response provided to technical evaluation criteria C.11. and the scoring quantities should match the response provided to C.43.

e. Respondents shall complete the Cost Option workbook to propose pricing for the following optional items. Note: This workbook will not be used to evaluate the pricing. While not scored, the contractor will be held to the pricing provided in the proposal for cost options if the State elects to exercise them.

i. Optional Formative Assessment Tool (Noted as OT in the Pricing Sheet – Pricing Summary tab) –

Described in the RFP in Section A.8. Only to be proposed if the vendor has an existing tool that will allow teachers to create low cost short “testlets” on the fly for classroom administration to students. The Respondent shall provide a description of the tool and pricing.

ii. Move the paper based ELA writing test earlier in the year (Noted as PW in the Pricing Proposal – Pricing Summary tab) – Described in the Pro Forma is section A.3 (c). Provide the costs for moving the ELA writing assessment for only those students taking the writing assessment on paper to earlier in the year in order to receive results faster.

iii. Implement CBT in year 2 vs. year 3 – (Noted as Y2 in the Pricing Proposal – Pricing Summary tab)

–Indicate the price for the assessments should the state move the applicable tests (Grade 5-8 ELA and mathematics and social studies of the general assessment) to online testing in year 2 of the contract instead of year 3 as currently planned.

f. Points awarded for vendor pricing shall be allocated based on the following formula:

The final pricing when calculating the vendor points awarded for this procurement will be determined by cell V152 in the Cost Proposal workbook.

(Lowest evaluation cost amount from all proposals / Evaluation cost amount being evaluated) X 30 = Respondent Points

g. The completed electronic Cost Proposals must be submitted to the State’s Procurement Office as separate files

along with the Respondent’s proposal on flash drives, as described in Section 3.2, of the RFP.

h. Cost Notes. In addition to the cost proposals, a Word document should be attached that outlines each

Respondent’s cost approach to administer the assessment program and details key cost assumptions. Explanations of cost approaches, either requested by Tennessee or proposed by the Respondent, shall be detailed in the cost notes and in a separate file on a flash drive. Some, but not all, areas that shall be addressed in the Cost Notes include:

Transition costs incurred in moving from the 2018-19 assessment program to the new assessment program

Any additional line items proposed (using the “Other” rows in the cost workbook) and why they are necessary to meet the Pro Forma scope.

A detailed description of the changes in the assessment program from year 1 to years 2 and 3.

Standard setting costs shown separately

Details of any artificial intelligence (AI) scoring plan including, but not limited to, the percentage of responses scored each year by AI, whether the scored responses replace human scores of first or second reads, subject area of scoring, etc.

Any proposed cost efficiencies

A detailing of any additional Respondent investment in either its existing software or other systems required to meet the requirements of the RFP and the amount passed on to the state. A discussion of the individual RFP requirements themselves that lead to the additional investment and any vendor proposed alternatives that would save costs is required.

Cost Notes for Cost Options

Discussion of Respondent’s online writing tool feature, functionality and pricing

Discussion of any risks, issues or concerns with moving the writing portion of the ELA assessment earlier in the year for those students taking the assessment via paper and pencil

Discussion of the risks, issues, pros and cons of moving to computer based assessment in year 2 vs. year 3.

09-20-18 RFP

41 RFP 33111-03819

RFP ATTACHMENT 6.4.

REFERENCE QUESTIONNAIRE

The standard reference questionnaire provided on the following pages of this attachment MUST be completed by all individuals offering a reference for the Respondent.

The Respondent will be solely responsible for obtaining completed reference questionnaires as required (refer to RFP Attachment 6.2., Technical Response & Evaluation Guide, Section B, Item B.17.), and for enclosing the sealed reference envelopes within the Respondent’s Technical Response.

09-20-18 RFP, 11-08-18 FA

42 RFP 33111-03819

RFP # 33111-03819 REFERENCE QUESTIONNAIRE

REFERENCE SUBJECT: RESPONDENT NAME (completed by Respondent before reference is requested)

The “reference subject” specified above, intends to submit a response to the State of Tennessee in response to the Request for Proposals (RFP) indicated. As a part of such response, the reference subject must include a number of completed and sealed reference questionnaires (using this form).

Each individual responding to this reference questionnaire is asked to follow these instructions:

complete this questionnaire (either using the form provided or an exact duplicate of this document);

sign and date the completed questionnaire;

seal the completed, signed, and dated questionnaire in a new standard #10 envelope;

sign in ink across the sealed portion of the envelope; and

return the sealed envelope containing the completed questionnaire directly to the reference subject.

(1) What is the name of the individual, company, organization, or entity responding to this reference

questionnaire? (2) Please provide the following information about the individual completing this reference

questionnaire on behalf of the above-named individual, company, organization, or entity.

NAME:

TITLE:

TELEPHONE #

E-MAIL ADDRESS:

(3) What goods or services does/did the reference subject provide to your company or organization? (4) What is the level of your overall satisfaction with the reference subject as a vendor of the goods

or services described above?

Please respond by circling the appropriate number on the scale below.

1 2 3 4 5

least satisfied

most satisfied

09-20-18 RFP, 11-08-18 FA

43 RFP 33111-03819

RFP # 33111-03819 REFERENCE QUESTIONNAIRE — PAGE 2

If you circled 3 or less above, what could the reference subject have done to improve that rating?

(5) If the goods or services that the reference subject provided to your company or organization are

completed, were the goods or services provided in compliance with the terms of the contract, on time, and within budget? If not, please explain.

(6) If the reference subject is still providing goods or services to your company or organization, are

these goods or services being provided in compliance with the terms of the contract, on time, and within budget? If not, please explain.

(7) How satisfied are you with the reference subject’s ability to perform based on your expectations

and according to the contractual arrangements? (8) In what areas of goods or service delivery does/did the reference subject excel? (9) In what areas of goods or service delivery does/did the reference subject fall short? (10) What is the level of your satisfaction with the reference subject’s project management structures,

processes, and personnel?


1 2 3 4 5

least satisfied

most satisfied

What, if any, comments do you have regarding the score selected above?

09-20-18 RFP, 11-08-18 FA

44 RFP 33111-03819

RFP # 33111-03819 REFERENCE QUESTIONNAIRE — PAGE 3

(11) Considering the staff assigned by the reference subject to deliver the goods or services described

in response to question 3 above, how satisfied are you with the technical abilities, professionalism, and interpersonal skills of the individuals assigned?


1 2 3 4 5

least satisfied

most satisfied


(12) Would you contract again with the reference subject for the same or similar goods or services?


1 2 3 4 5

least satisfied

most satisfied


REFERENCE SIGNATURE: (by the individual completing this

request for reference information)

DATE:

(must be the same as the signature across the envelope seal)

09-20-18 RFP, 11-08-18 FA

45 RFP 33111-03819

RFP ATTACHMENT 6.5.

SCORE SUMMARY MATRIX

RESPONDENT NAME RESPONDENT NAME RESPONDENT NAME

GENERAL QUALIFICATIONS & EXPERIENCE (maximum: 10)

EVALUATOR NAME

EVALUATOR NAME

REPEAT AS NECESSARY

AVERAGE: AVERAGE: AVERAGE:

TECHNICAL QUALIFICATIONS, EXPERIENCE & APPROACH (maximum: 60)

EVALUATOR NAME

EVALUATOR NAME

REPEAT AS NECESSARY


ORAL PRESENTATION (maximum: 0)

EVALUATOR NAME

EVALUATOR NAME

REPEAT AS NECESSARY


COST PROPOSAL (maximum: 30)

SCORE: SCORE: SCORE:

TOTAL RESPONSE EVALUATION SCORE:

(maximum: 100

Solicitation Coordinator Signature, Printed Name & Date:

09-20-18 RFP

46 RFP 33111-03819

RFP ATTACHMENT 6.6.

RFP # 33111-03819 PRO FORMA CONTRACT

The Pro Forma Contract detailed in following pages of this exhibit contains some “blanks” (signified by descriptions in capital letters) that will be completed with appropriate information in the final contract resulting from the RFP.

PRO FORMA CONTRACT DRAFTED IN COMPLIANCE WITH CURRENT APPLICABLE MODEL

1

C O N T R AC T (fee-for-goods or services contract with an individual, business, non-profit, or governmental entity of another state)

Begin Date End Date Agency Tracking # Edison Record ID

Contractor Legal Entity Name Edison Vendor ID

Goods or Services Caption (one line only)

Administration, Scoring, and Reporting of State Assessments

Contractor CFDA #

Contractor

Funding —

FY State Federal Interdepartmental Other TOTAL Contract Amount

TOTAL:

Contractor Ownership Characteristics:

Minority Business Enterprise (MBE):

African American Asian American Hispanic American Native American

Woman Business Enterprise (WBE)

Tennessee Service Disabled Veteran Enterprise (SDVBE)

Disabled Owned Business (DSBE)

Tennessee Small Business Enterprise (SBE): $10,000,000.00 averaged over a three (3) year period or employs

no more than ninety-nine (99) employees.

Government Non-Minority/Disadvantaged Other:

Selection Method & Process Summary (mark the correct response to confirm the associated summary)

Competitive Selection Request for Proposals

Other

Budget Officer Confirmation: There is a balance in the appropriation from which obligations hereunder are required to be paid that is not already encumbered to pay other obligations.

Speed Chart (optional) Account Code (optional)

2

CONTRACT BETWEEN THE STATE OF

TENNESSEE, Department of Education

AND CONTRACTOR NAME

This Contract, by and between the State of Tennessee, acting by and through the Tennessee Department of Education (“State”) and Contractor Legal Entity Name (“Contractor”), is for the provision of Administration, Delivery, Scoring and Reporting of State Assessments, as further defined in the "SCOPE." State and Contractor may be referred to individually as a “Party” or collectively as the “Parties” to this Contract.

The Contractor is a/an Individual, For-Profit Corporation, Non-Profit Corporation, Special Purpose Corporation Or Association, Partnership, Joint Venture, Or Limited Liability Company. Contractor Place of Incorporation or Organization: Location Contractor Edison Registration ID # Number

A. SCOPE:

A.1. The Contractor shall provide all goods, services, and deliverables as required,

described, and detailed below and shall meet all service and delivery timelines as specified by this Contract. The Contractor shall customize or modify any goods, services, or deliverables required under this Contract as requested by the State at the Contractor’s sole expense. All defined terms or acronyms shall have the meaning as set forth below or as otherwise set forth in the Contract:

a. Accessibility Features. The term “Accessibility Features” means Assessment Administration modifications allowable for use with any student. The included modifications to Accessibility Features shall include, without limitation: flexible setting or scheduling; visual, tactile, or auditory aids; scribe; marking in test books; oral presentation (read aloud or text to speech for computer); adjustments to background or contrast colors for CBT; and enlarged print online, but shall not include Individualized Education Plan (“IEP”) or 504 plan documentation.

b. Accommodations. The term “Accommodations” means testing Accommodations specified by IEP or 504 service plan or other allowable state documentation. These may include without limitation, Braille, the ability for the student to dictate responses, or an administrator providing prompting upon request and additional time.

c. Administration. The term “Administration” means the window in which an Assessment is given. The Administration of the End of Course (“EOC”) exams are given in the fall and spring and the Administration of the Achievement (“ACH”) is given in the spring during a Test Year.

d. Administration Materials. The term “Administration Materials” shall have the meaning ascribed to this term in Section A.8.d. of this Contract.

e. Anchor Papers. The term “Anchor Papers” means the sample essays that clearly exhibit the criteria for each Score point.

f. Anchor Text. The term “Anchor Text” means the primary text used in the

Constructed Response Items, which may have more than one text. Anchor Text provides the overall theme of the Item.

3

g. Annual Work Plan. The term “Annual Work Plan” shall have the meaning ascribed to this term in Section A.4.a. (1) of this Contract.

h. Approval by the State. The term “Approval by the State”, and any variations on this term, means approval by, at minimum, the Commissioner of Education or a point of contact specifically designated by the Commissioner as being the individual authorized to provide State approval.

i. Assessment. The term “Assessment” means an evaluation of student achievement related to knowledge and skills in a specific Content Area or a specific grade level and shall be used throughout this Contract to denote any test given under this Contract (e.g., CBT or PBT) at all grade levels and Content Areas.

j. Assessment Administration. The term “Assessment Administration” shall mean the Administration of the Assessment by the Contractor.

k. Assessment Administration Manual or TAM. The term “Assessment Administration Manual” or acronym “TAM” shall have the meaning ascribed to this term or this acronym in Section A.8. of this Contract.

l. Assessment Materials. The term “Assessment Materials” shall have the meaning ascribed to this term in Section A.8. of this Contract.

m. Assessment Program. The term “Assessment Program” shall have the meaning ascribed to this term in Attachment A of this Contract.

n. Assessment Reports. The term “Assessment Reports” shall have the meaning ascribed to this term in Section A.13., which shall include Student Reports grade 3-8, EOC Student Reports, Student Labels grade 3-8, EOC Student Labels, Raw Score Reports, Standard Analysis Reports, Summary Reports, Roster Reports, or any other Reports as directed by the State from time-to-time during the Term.

o. Authorized Use or Authorized User. The terms “Authorized Use” or “Authorized User” shall have the meaning ascribed to this term in Section A.10.a. of this Contract.

p. Automated Scoring. The term “Automated Scoring” means the use of Computer-Based programs to assign grades to Constructed Response Items.

q. Blind Scoring. The term “Blind Scoring” means the method of scoring that allows each student’s response to be Scored independently, without the reader having knowledge of any previous Scores.

r. Calibration. The term “Calibration” means placing Items on a common scale so that Items measuring the same underlying content represent a position on the scale that is also represented by other Items of Comparable difficulty.

s. Call Center. The term “Call Center” shall have the meaning ascribed to this term in Section A.4.h. (1) of this Contract.

t. College and Career Ready Performance Levels. The term “College and Career Ready Performance Levels” means the performance levels established to provide a measure of student work on eleventh grade Assessments, (e.g., English III, and Algebra II, Geometry, or Integrated Math III), that is aligned with the expectation that a student could succeed in credit bearing, entry level coursework in college.

u. Comparable, Comparability, or Comparability and Equating. The term “Comparable”, “Comparability” or “Comparability and Equating” means statistical evidence that prior Assessments are similar in content and

4

psychometric characteristics to the current Assessment.

v. Computer-Based. The term “Computer-Based” shall mean those goods, services, or deliverables of the Contractor that are accessed or delivered by electronic device such as a computer.

w. CBT Platform. The term “CBT Platform” shall have the meaning ascribed to this term in Section A.9.a. of this Contract.

x. CBT System. The term “CBT System” shall have the meaning ascribed to this term in Section A.9.a. of this Contract.

y. Comprehensive Data File or acronym CDF. The term “Comprehensive Data File” or acronym “CDF” shall have the meaning ascribed to this term or acronym in Section A.12.d. of this Contract.

z. Computer-Based Testing or CBT. The term “Computer-Based Testing” or acronym “CBT” shall have the meaning ascribed to this term in Section A.9. of this Contract.

aa. Constructed Response. The term “Constructed Response” or acronym “CR” means assessment Items that measure application-level cognitive skills and content knowledge. These Items may include, but not be limited to, Items in which a student provides a brief response to a posed question, completes a computation, creates a graphic representation, provides justification or explanation of his or her thoughts or work on an equation, a response to a text-based Prompt that can vary in length and text complexity based on grade level, or activities that are related and require students to construct a response, create a product, or perform a demonstration that is evaluated by a predetermined set of criteria.

bb. Content Areas. The term “Content Areas” means English Language Arts,

Mathematics, Science, or Social Studies.

cc. Cut Scores. The term “Cut Scores” means the minimum test Score necessary to demonstrate that a test taker has the knowledge or skills to perform at a certain level of achievement.

dd. English Learner or EL. The term “English Learner” or acronym “EL” is a student

whose first language is not English and who is Limited English Proficient as determined by the State.

ee. Field Test. The term “Field Test” shall mean to test the Contractor’s procedure

for testing the goods, services, or deliverables required under this Contract by the Contractor in actual situations reflecting intended use.

ff. Fiscal Year or FY. The term “Fiscal Year” or acronym “FY” means the State

accounting period of July 1 through June 30.

gg. Interoperability. The term “Interoperability” refers to the principle of using standardized data formats and data transport protocols to promote the effective exchange and utilization of data between two or more systems or system components. Interoperability is a core design principle for the State’s technology development and operations.

hh. Item or Items. The term “Item” or “Items” means test questions that may include

Constructed or Selected Responses.

ii. Item Bank. The term “Item Bank” means the test question repository or archive.

jj. Large Print or Large Print Test Books. The term “Large Print” shall mean print

5

sufficient in size to Accommodate students with vision limitations. The term “Large Print” shall also have the meaning ascribed to this term in Section A.8.a. of this Contract. The term “Large Print Test Books” shall have the meaning ascribed to this term in Section A.8.c. (2) of this Contract.

kk. Limited English Proficient or LEP. The term “Limited English Proficient” or

acronym “LEP” refers to a student whose first language is not English and who is limited in English proficiency as determined by the State (also referred to as English Learners).

ll. Linking Items. The term “Linking Items” means Items that have known

psychometric properties and that are common across forms for the purpose of equating Assessments.

mm. Liquidated Damages. The term “Liquidated Damages” shall have the meaning

ascribed to this term in Attachment B of this Contract.

nn. Local Education Agency or LEA. The term “Local Education Agency” or acronym “LEA” means a school district or school system that is the financial and administrative agency for schools in a certain region of the State.

oo. Paper-Based Testing or PBT. The term “Paper-Based Testing” or acronym

“PBT” shall mean testing rendered through a tangible medium such as paper or a similar material.

pp. Parent Brochure. The term “Parent Brochure” shall have the meaning ascribed

to this term in Section A.13. of this Contract.

qq. Performance Standards. The term “Performance Standards” means the student performance that is indicated or reported at multiple levels pursuant to an Assessment. The Performance Standards are established by educators at Performance Standards setting.

rr. Progress Report. The term “Progress Report” shall have the meaning ascribed to this term in Sections A.4.b. (1) ii. and D.13. of this Contract.

ss. Prompt. The term “Prompt” means text that defines the topic for a student’s writing. The different modes of writing assessed may include, but not be limited to, narrative, expository, informational, explanatory, analytic, opinion, or argument.

tt. Quality Assurance. The term “Quality Assurance” shall have the meaning ascribed to it in Section A.6. of this Contract.

uu. Quality Assurance Report. The term “Quality Assurance Report” shall have the meaning ascribed to this term in Section A.4.b. (1)vii.

vv. Question and Test Interoperability or QTI. The term “Question and Test Interoperability” or acronym “QTI” means a standard format for the representation of Assessment Content and results, supporting the exchange of this material between authoring and delivery systems, repositories, or other learning management systems. QTI allows Assessment Materials to be authored and delivered on multiple systems interchangeably.

ww. Reporting Category or Reporting Categories. The term “Reporting Category”

or “Reporting Categories” means divisions of the academic standards used in

6

Reporting student performance on the Assessments or as further defined by the State and the Contractor during the Term.

xx. Report, Reporting, or Reports. The term “Report” or “Reports” shall mean any

Report or Reports, including, without limitation, Assessment Reports, Quality Assurance Reports, Student Reports, or Technical Reports, required by the Contractor under this Contract or as directed by the State from time-to-time during the Term. T h e t e r m “ R e p o r t ” o r “ R e p o r t s ” s h a l l a l s o h a v e t h e m e a n i n g a s c r i b e d t o t h i s t e r m i n R F P A s s e s s m e n t R e p o r t i n g A p p e n d i x 6 . 6 . A . The term “Reporting” shall pertain to the Contractor’s reporting activities such as preparing or filing a Report or maintaining a Reporting system.

yy. Scale Score. The term “Scale Score” means the numerical Score, based upon

the number of Items a student correctly answers, which summarizes the overall level of performance attained by the student.

zz. Score, Scores or Scoring. The term “Score” shall mean a determination of the

number of correct versus incorrect responses a student achieves on an Assessment. The term “Scores” shall be mean more than one (1) Score. The term “Scoring” shall refer to the Contractor’s activities or process of determining student Scores on an Assessment. The term “Score”, Scores, or “Scoring” shall also have the meaning ascribed to any of these terms in Section A.12. of this Contract or as the context requires.

aaa. Score Card. The term “Score Card” shall have the meaning ascribed to this

term in Section A.14. of this Contract.

bbb. Selected Response or SR. The term “Selected Response” or acronym “SR” means a test question that requires the student to choose the correct or best answer from a list of possible responses.

ccc. Secure Browser. The term “Secure Browser” means a program specifically

created to allow the delivery of a “locked-down” testing environment for Web-Based online tests. A Secure Browser prevents students from accessing other computers, Internet applications, or copying test information, thus creating a safe test environment.

ddd. Service Anomaly. The term “Service Anomaly” means a confirmed outage,

slow down, or other disruption to testing service delivery, lasting greater than five (5) minutes, including: functional non- conformities, which negatively impact the student testing experience in one or more entire classes or is caused by any software or infrastructure provided or managed by the Contractor or a subcontractor of the Contractor. A Service Anomaly is measured based on the first verifiable occurrence of the event.

eee. State Special Schools. The term “State Special Schools” means a school that

is managed by the State or another State agency. The current State Special Schools include: include: Alvin C. York Institute, Tennessee Governor’s Academy, Tennessee School for the Blind, Tennessee School for the Deaf, West Tennessee School for the Deaf, and Department of Children’s Services.

fff. Student ID Number. The term “Student ID Number” means the student

identification number randomly generated from the statewide student management system.

ggg. Student Labels. The term “Student Labels” means a label that contains all

scores for a particular student for the Assessment Administration that can be affixed on the student Record or transcript. This term shall also have the meaning ascribed to it in Attachment B of this Contract.

7

hhh. Student Report. The term “Student Report” shall have the meaning ascribed to

this term in Section A.13. of this Contract.

iii. Student Response Document. The term “Student Response Document” shall

have the meaning ascribed to this term in Section A.8.e. (2) of this Contract.

jjj. Student Test Booklets. The term “Student Test Booklets” shall have the meaning ascribed to this term in Section A.8.e. (2) of this Contract.

kkk. TAC. The acronym “TAC” means the Technical Advisory Committee.

lll. TCAP. The acronym “TCAP” means the Tennessee Comprehensive

Assessment Program.

mmm. TCAP Assessments. The term “TCAP Assessments” refers to all State Assessments as a group.

nnn. TCAP/ACH. The acronym “TCAP/ACH” means the Achievement Assessment

mandatory for grades 3-8 in math, ELA, science, and social studies.

ooo. TCAP/Alternate. The acronym “TCAP/Alternate” means the Alternate

Assessments in science and social studies in grades 3-8 and Biology.

ppp. TCAP/EOC. The acronym “TCAP/EOC” means the end of course Assessments provided in specified secondary Content Areas.

qqq. TCAP/Grade 2. The acronym “TCAP/Grade 2” means the optional

Assessment that LEAs opt into in math and ELA.

rrr. Technical Report. The term “Technical Report” means the comprehensive summary Report as defined in Section A.7.h. of this Contract.

sss. Tennessee Academic Standards. The term “Tennessee Academic Standards”

means the common set of expectations for what students will know and be able to do at the end of a grade as determined by the Tennessee Academic Standards adopted by the State Board of Education. The content the Tennessee Academic Standards is located at http://www.tennessee.gov/education/standards/index.shtml, which is incorporated in this Contract by reference as though set forth verbatim.

ttt. Tennessee Value-Added Assessment System or TVAAS. The term

“Tennessee Value-Added Assessment System” or acronym “TVAAS” means the component of the State accountability program that utilizes a statistical methodology to measure student achievement gains.

uuu. Test Development Contractor. The term “Test Development Contractor” shall

have the meaning ascribed to this term in Section A.2. of this Contract. The term Test Development Contractor shall also include State contractors performing the same or similar duties as the current Test Development Contractor from time-to-time during the Term.

vvv. Test Materials. The term “Test Materials” shall have the meaning ascribed to

this term in Section A.8.e. of this Contract.

www. Test Year or TY. The term “Test Year” or acronym “TY” means the Assessment

Administration period from September 1 through August 31.

http://www.tennessee.gov/education/standards/index.shtml

8

xxx. Test Administrator Proctor Scripts or TAPS. The term “Test Administrator Proctor

Scripts” or acronym “TAPS” shall have the meaning ascribed to this term or acronym in Section A.8.d. (2) of this Contract.

yyy. Testing Window. The term “Testing Window” shall have the meaning ascribed to

it in Section A.3.b. of this Contract.

zzz. User’s Guide. The term “User’s Guide” shall have the meaning ascribed to this term in Section A.8.d. of this Contract.

aaaa. Wait Time. The term “Wait Time” means the elapsed time from connection of a

call to the time the caller receives a response from the Contractor’s personnel or the Contractor’s automated service.

bbbb. Web-Based. The term “Web-Based” means that portion of Contractor’s goods or

services that are delivered via Web-Based Applications.

cccc. Web-Based Applications. The term “Web-Based Applications” shall have the meaning ascribed to this term in Section A.10 of this Contract.

A.2. Assistance to State and Test Development Contractor.

As directed by the State, the Contractor shall provide goods, services, or deliverables to the State and assist the State’s test development contractor (hereinafter, the “Test Development Contractor”), currently Educational Testing Service (“ETS”), to provide delivery, Administration, Scoring, and Reports, as more particularly described in Attachment A this Contract, related to the State’s Assessment program. The Contractor’s goods, services, and deliverables in this regard shall allow all tested students in Tennessee to access Assessments in ways that produce meaningful results. The State has separately contracted with ETS for development of all test forms, including Item development and related activities for the 2019-2020 school year. The Test Development Contractor will provide the Contractor with print-ready forms for PBT and QTI packages containing test forms for CBT. The Test Development Contractor will also provide all necessary Scoring keys, rubrics, and Scoring guides for test Items. All results from the Assessments shall be valid, reliable, and fair measurements of College and Career Ready Performance Levels for all students and shall be accessible for use by the widest possible range of students, including without limitation, students with disabilities and English Learners. All Assessments implemented and maintained by the Contractor shall comply with Title I requirements of the Individuals with Disabilities Education Act (“IDEA”); the Every Student Succeeds Act (“ESSA”) of 2015; and applicable Tennessee law and administrative regulations. The Assessments to be included are described in Section A.3 and Attachment A of this Contract. The results of these Assessments shall provide Comparability information between Assessments given in other states as aligned to similar academic standards. Unless otherwise indicated, all requirements of this Contract shall apply to all Assessments. The quantity of deliverables for each Assessment shall be determined and approved in collaboration with the State and shall be finalized with the Contractor in its Annual Work Plan described in Section A.4.a.(1) of this Contract and the enrollments for each grade or Content Area Assessment for the two most recent school years are shown in Section A.3.g. of this Contract.

9

A.3. General Scope of Assessment Services.

a. Test Structure – The Assessment services provided by the Contractor shall include

delivering, administering, Scoring, and Reporting the following Assessments for the TCAP, based on the Tennessee Academic Standards. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.4 is incorporated in this Contract by reference as though set forth verbatim.

(1) Achievement (TCAP/ACH) Assessments shall be administered in grades 3-8

and optional in grade 2 in ELA and Mathematics; in grades 3-8 for Science; and for grades 3-8 in Social Studies.

(2) End of Course (TCAP/EOC) Assessments shall be required for specified

secondary Content Areas, which are English I, II, Algebra I, II, Geometry, Integrated Math I, II, III, Biology, and U.S. History.

(3) Alternate (TCAP/Alternate) Assessments in science shall be required in grades 3-8 and in social studies in grade 5-8. These Assessments shall be required once in science and once in social studies in high school.

(4) Grade Two (TCAP/Grade 2) Assessments are optional, subject to the State’s

written election, and made available in math and ELA for general education. LEAs have the discretion to opt into grade two (2) Assessments by November of each Test Year.

(5) Grade Two Alternate (TCAP/Grade 2 Alternate) Assessments are optional,

subject to the State’s written election, and shall be made available in math and ELA for those LEAs opting into the TCAP/Grade 2 general education Assessments.

b. The Contractor shall ensure that all Assessments adhere to existing or future changes to

federal or State legislation, rules, or policies. The Contractor shall maintain for review and audit purposes all documents, papers, or records relating to its work as required by the State or the United States Department of Education. The Contractor shall be responsible for all services provided by it under this Contract, whether performed by the Contractor or a subcontractor, delegate, or assignee of the Contractor. Unless otherwise specified in this Contract, all actions requiring State approval are set forth in Section A.4. All components of the goods, services, or deliverables provided by the Contractor shall comply with State Board of Education Rules and Policies located at https://www.tn.gov/sbe/rules--policies-and- guidance.html, which is incorporated in this Contract by reference as though set forth verbatim, or applicable Tennessee or federal law. The Contractor shall provide documentation of the components of the Assessment program as requested by the State. Subject to the State’s approval, the Contractor shall develop annual testing schedules, including development of computer and paper-based test forms and Assessment Materials, delivery of tests for Administration in schools, delivery of Assessment Administration Quality Assurance Reports to the State, delivery of Score Reports to the State, delivery of Reports to LEAs, and delivery of Field Test information to the Test Development Contractor. The current TCAP Assessment Administration schedule is set forth at https://www.tn.gov/content/dam/tn/education/testing/tst_tcap_assessment_calendar_2018- 19_2019- 20.pdf, which is incorporated in this Contract by reference as though set forth verbatim.

(1) TCAP/ACH and TCAP/Grade 2 is administered in late April – early May. At the State’s request, the writing portion on the 3-8 ELA Assessment as a PBT may be administered separately and earlier than the spring Testing Window. A separate Testing Window will allow LEAs to choose one (1) of three (3) two (2)-day Testing Windows in March.

(2) TCAP/EOC is administered two (2) times a Test Year as follows:

https://www.tn.gov/sbe/rules--policies-and-

https://www.tn.gov/sbe/rules--policies-and-

https://www.tn.gov/sbe/rules--policies-and-guidance.html

https://www.tn.gov/content/dam/tn/education/testing/tst_tcap_assessment_calendar_2018-

https://www.tn.gov/content/dam/tn/education/testing/tst_tcap_assessment_calendar_2018-

https://www.tn.gov/content/dam/tn/education/testing/tst_tcap_assessment_calendar_2018-19_2019-20.pdf

https://www.tn.gov/content/dam/tn/education/testing/tst_tcap_assessment_calendar_2018-19_2019-20.pdf

10

i. Fall: The Testing Window is generally from the end of November to mid- December. The tests are available for schools on block schedules and make-up testing.

ii. Spring: The Testing Window is generally from mid-April to early May

and coincides with the TCAP/ACH window.

(3) TCAP/Alternate is administered in a longer Testing Window, typically running from mid-March until early May.

(4) During the first year of the Term, the Contractor shall conduct an individualized readiness

visit with each LEA in the spring and a needs assessment and customer service visit each fall during the Term.

c. The State’s current and future plan for Administration mode on the TCAP Assessments

is shown in the table below. Please note that in middle school (grades 5-8), CBT and PBT depend on the Content Area.

Grade Span/Content Area 2019-2020 Starting in 2020-2021

3-4/ELA, Math, Science, Social Studies

PBT

PBT

5-8/ELA, Math, Social Studies

PBT PBT or CBT (at State’s

request)

5-8 Science CBT CBT

All End-of-Course Exams CBT CBT

Grade 2 Math and ELA PBT PBT

Science and Social Studies Alternate 3-8 and High School Science Alternate

PBT

PBT

The Contractor shall support the State’s transition from PBT to CBT in the necessary grade levels and Content Areas. The Contractor shall provide a full PBT back-up option each Testing Year during the Term if requested by the State. The right to exercise the PBT option shall be at the State’s sole and absolute discretion. The State will notify the Contractor of its decision to exercise the PBT option at least three (3) months prior to the beginning of the Testing Window. During the initial Term of this Contract, the Contractor shall conduct at least two (2) trial runs for PBT logistics to confirm its shipping logistics and accuracy. This shall include a fall and early spring shipment to each Assessment location for every potential student. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.5 is incorporated in this Contract by reference as though set forth verbatim.

d. Maintaining test validity, reliability, and Comparability of Assessment Score interpretations across prior Test Years is a fundamental requirement of this Contract. The Contractor shall assist the State with the design and control of these aspects of the Assessment Program. Contractor changes in the design of the Assessments, including without limitation, Administration, Scoring, or Reporting during the Term, must be approved in writing in advance by the State. The Contractor shall be responsible for providing to the State documentation regarding the technical quality of the Assessment Administration, Scoring, or Reporting that show how the Assessment results meet professional standards as outlined in the Standards for Educational and Psychological Testing and the U.S. Department of Education’s Assessment Peer Review process.

e. The primary purpose of the Assessment is to improve teaching and learning.

Additionally, Assessment results are critical for student grades, school accountability, and teacher evaluations. Because of the crucial nature of the TCAP, all aspects of test

11

delivery, Administration, Scoring, or Reporting shall meet the highest professional standards or best practices for test security and validity. High-quality test Administration and results shall be provided by the Contractor to meet this purpose.

f. The performance level labels, descriptors, and Cut Scores adopted by the State shall

be used by the Contractor in Scoring and Reporting, unless determined otherwise by the State in writing.

g. The Contractor shall provide testing materials based upon the number of students

tested in each LEA. Currently, Tennessee has approximately one hundred forty-seven (147) LEAs, four (4) State Special Schools, the state Achievement School District, and approximately one thousand seven hundred (1700) public schools. Private schools may also opt to administer TCAP Assessments. Approximately, five thousand seven hundred (5700) private school students are part of the Assessment each Test Year. Recent historical usage figures are provided in the following table:

TCAP Assessment

Test Year 2016-17 Test Year 2017-18

CBT PBT CBT PBT

ACH – Spring NA 1,783,539 371,961 1,418,474

ACH Large Print – Spring NA 2,245 NA 1,192

ACH Braille – Spring NA 323 NA 0

EOC – Spring 56,602 448,160 502,409 NA

EOC Large Print – Spring NA 367 NA 40

EOC Braille – Spring NA 77 NA 12

EOC – Fall NA 120,001 118,547 NA

EOC Large Print – Fall NA 106 NA 0

EOC Braille – Fall NA 20 NA 6

Alternate Gr 3-8 (science and social studies only) – Spring

NA

14,094

NA

13,430

A.4. Program Management

a. Administration Activities – The Contractor shall provide goods, services, or deliverables for

the delivery, Administration, Scoring, and Reporting of TCAP Assessments for all public, State Special Schools, approved private schools, or home-schooled students beginning in2019-2020 (“TY20”). The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.5 is incorporated in this Contract by reference as though set forth verbatim.

(1) Annual Work Plan – The Contractor shall prepare, within fifteen (15) calendar days of

the Contract Effective Date, for FY20 and each July 15 during the Term for each Administration of the Assessment thereafter, an Annual Work Plan. The Annual Work Plan shall generally include Contractor’s plan for the delivery, Administration, Scoring, or Reporting of each Assessment. The Annual Work Plan shall incorporate the schedules for each activity required by this Contract. Specifically, the requirements for the Annual Work Plan include the following:

i. The Annual Work Plan shall outline by task and due date each activity to be performed following the deliverable timelines established in the schedules required in Section A.4.b. of this Contract. The Annual Work Plan must describe all activities related to the delivery, Administration, Scoring, and Reporting of TCAP Assessments for the given year. The Annual Work Plan shall indicate

12

areas of responsibility for the Contractor and the State, and handoffs between the Contractor and other contractors providing relevant goods, services, or deliverables to the State. The Annual Work Plan shall serve as a monitoring document to be used by the State to track and ensure timely completion of tasks as scheduled. Each Annual Work Plan shall be comprised of all activities related to the Administration of each Assessment and may encompass more than a twelve (12) month timeframe.

ii. Each year, the Contractor shall coordinate with the Test Development Contractor

to ensure that the tasks in its Annual Work Plan associated with the handoff of deliverables between the Contractor and the other contractors are consistent with the timelines established in the schedules referenced in section A.4.b., specifically:

1) The Annual Work Plan for the Contractor shall begin with the delivery

of print PDFs (for PBTs) and QTI packages (for CBTs) from the Test Development Contractor.

2) The Contractor’s plan must include the delivery of field and

operational test Item analysis as outlined in Section A.7.d for the subsequent year’s development to the Test Development Contractor.

iii. The Annual Work Plan shall include all tasks related to TCAP-ACH, TCAP-Grade

2, TCAP Fall and Spring EOC, and 2-11 alternate testing.

iv. The Annual Work Plan for the initial 2019-2020 Assessment Administration shall indicate the essential steps leading to the transition between the Contractor and the Test Development Contractor and shall be submitted within fifteen (15) calendar days of the Contract Effective Date. The Annual Work Plan shall include the steps for all project work tasks and deliverables, including initiation and completion dates, and task responsibilities to avoid any disruption of services, requirements, or deliverables to the students, teachers, schools, LEAs, or the State.

v. The final Annual Work Plan for this Contract shall indicate the essential steps

leading to the transition between the Contractor and any contractor awarded subsequent contracts for the provision of the goods, services, or deliverables set forth in this Contract. The final Annual Work Plan shall also include the business processes and steps for all project work tasks and deliverables, including initiation and completion dates, and task responsibilities needed to avoid any disruption of services, requirements, or deliverables to the students, teachers, schools, LEAs, or the State.

vi. The Annual Work Plan shall be delivered in a format as determined by the

State, taking into account the needs of the State and State system limitations. Both the State and the Contractor shall have access to and the ability to track all dates in the Annual Work Plan. All changes shall be made as agreed to in writing as indicated by the State’s project management plan, which will be provided to the Contractor on or before the Contract Effective Date, with updates provided by the State from time-to-time during the Term.

vii. The Contractor shall establish secure websites for use in maintaining the Annual

Work Plan or other communications, including but not limited to, posting administrative materials for review with the State for Contract management. The Annual Work Plan and any related or supporting materials shall be considered delivered and final after written notification is given to the

Contractor via email or other electronic methods by the State as indicated in the State’s project management plan.

viii. The Contractor shall also convene the meetings more particularly described in

13

Section A.4.j. of this Contract.

b. Schedule of Goods, Services, or Deliverables – The State shall have the opportunity to review and approve all goods, services, or deliverables purchased under this Contract. The State reserves the right to determine specific details of goods, services, or deliverables under this Contract. Changes to any goods, services, or deliverables provided by the Contractor under this Contract shall only be made with State’s written approval. The Contractor shall not disseminate any written information, materials, or deliverables associated with this Contract to the field, public, or any other third party without the State’s prior written approval. The Contractor shall allow the State a minimum of five (5) business days to review materials or deliverables. The Contractor shall make all necessary corrections or modifications to goods, services, or deliverables as directed by the State and provide at least three (3) business days for the State to review and approve any corrected or modified goods, services, or deliverables. All modifications or corrections of goods, services, or deliverables shall be at the Contractor’s sole cost and expense.

(1) The Contractor shall provide the goods, services, or deliverables under

this Contract in accordance with the delivery schedule stated below.

i. Annual Work Plan – The Annual Work Plan, as more particularly described in Section A.4.a. (1) of this Contract, shall include all activities related to each Assessment Administration. The initial Annual Work Plan is due within fifteen (15) calendar days of the Contract Effective Date, which shall cover the period from the Contract Effective Date through the completion of testing for the first Assessment Administration during the initial Term of this Contract. Each subsequent Annual Work Plan shall be due to the State by July 15 each year during the Term and shall cover the full test cycle, including any required Reports for that year.

ii. Progress Reports – Quarterly Progress Reports, as more particularly described in sections A.4.f.(2) & (3) of this Contract, shall be provided to the State beginning July 15, 2019, and on each anniversary date thereafter during the Term.

iii. Data Management Activities – Web-Based provisioning applications, as more

particularly described in Section A.10. of this Contract, shall be provided to the State by July 15, 2019, and on each anniversary date thereafter during the Term. Specific dates for component deliverables shall be detailed in the Annual Work Plan.

iv. Technical Support Services – Technical support services for all Computer-Based

Applications (including Web-Based Reporting) as detailed in Section A.4.i. of this Contract shall be provided to the State on a schedule specified in the Annual Work Plan.

v. By July 15, 2019, and on each anniversary date thereafter during the Term, the

Contractor shall provide the State a written proposal for User Acceptance Testing (“UAT”) of the Assessment and Reporting platform for the State’s review and approval. By August 19, 2019, and each anniversary date thereafter during the Term, the Contractor shall successfully complete UAT to allow for student practice tests.

vi. By July 15, 2019, and on each anniversary date thereafter during the Term, the Contractor shall provide a written proposal for a comprehensive check that all Assessment content can be successfully imported and rendered error-free on all State authorized devices and systems that meet minimum specifications for the Assessment for State review and approval. A written Report documenting successful completion of the approved comprehensive check must be completed by the Contractor and delivered to the State by August 1, 2019, and on each anniversary date thereafter during the Term.

vii. By July 15, 2019, and on each anniversary date thereafter during the Term, the

14

Contractor shall provide for State review and approval a written proposal for a comprehensive check that the Assessment platform will successfully import, update, and incorporate all State user information from the State information system. A written Report documenting successful completion of the information import must be provided by the Contractor to the State by August 30, 2019, and on each anniversary date thereafter during the Term.

viii. The Contractor shall provide a Quality Assurance Report that details all Quality Assurance checks and outcomes prior to Scoring, including PBT or CBT check-sets to confirm data input matches any expected Scoring outcomes. Specific dates for this deliverable shall be detailed in the Annual Work Plan.

ix. Assessment Materials – The Contractor shall deliver all Assessment Materials

at least two (2) weeks prior to the Testing Window as detailed in Section A.8. of this Contract, to all LEAs.

x. Processing and Scoring Activities – The specific dates for component

deliverables of processing and Scoring, including, if applicable, automated Scoring, as more particularly described in Sections A.11 and A.12 of this Contract, shall be detailed in the Annual Work Plan.

xi. Web-Based Reporting Activities – Web-Based Reporting as more particularly described in Section A.13.g. of this Contract, shall be provided to the State on a date detailed in the Annual Work Plan or as determined by the State in writing.

xii. The Contractor shall provide all Assessment Reports on the timelines indicated in

Section A.13. of this Contract.

xiii. Student Demo – By August 31, 2019, and on each anniversary date thereafter during the Term, the Contractor shall complete a student demo that is operational in the CBT platform.

xiv. Practice Tests – By September 30, 2019, and on each anniversary date thereafter

during the Term, the Contractor shall provide practice testing that is operational in the CBT Platform.

(2) Schedule of One-time Deliverables:

i. Materials for Introductory Meetings – By September 1, 2019, the Contractor shall deliver all instructional materials to system coordinators, school coordinators, and provide materials for regional meetings as defined in Section A.4.k. of this Contract and thereafter provide these materials by the date set forth in the Contractor’s Annual Work Plan.

ii. Computer-Based Testing Delivery System – The confirmation of successful initial

set up of Computer-Based testing delivery system and Web-Based Applications, including evidence of comprehensive quality control checks, as more particularly described in Sections A.9 and A.10 of this Contract, shall be provided by the Contractor by the date set forth in the Annual Work Plan.

iii. Shipment of Test Archive Documents – The Contractor shall deliver all archived

documents to the State in accordance with Section A.8.g. (2) of this Contract.

(3) The State shall provide to the Contractor the following by July 15, 2019:

i. Contact information, including email, for LEA and school users of W eb-

Based goods, services, or deliverables provided under this Contract.

ii. Information and requirements to assist the Contractor in coordinating the Contractor’s activities with the State’s student information system,

15

including organization files and provisioning of file specifications.

c. Project Team – The Contractor shall provide the State with a project team to support

the delivery, Administration, Scoring, and Reporting of the TCAP Assessments as follows:

(1) The Contractor shall assign a single point of contact (the “Lead Program Manager”), to

manage all inquiries related to materials, training, and technical assistance required for this Contract. The Lead Program Manager shall have experience leading Assessment programs on a similar scale as the TCAP Assessment, shall have the authority to bind the Contractor, and who shall be located within twenty (20) miles of the Tennessee Department of Education’s offices in Nashville, Tennessee, on a regular basis for in-person meetings with State staff.

(2) The Contractor’s program management team shall be stationed located within twenty (20) miles of the Tennessee Department of Education’s offices.

(3) Contractor team members assigned to this Contract must have, at a minimum,

the necessary technical experience, knowledge, and operational experience in the following areas:

i. Managing or coordinating the development and implementation of large-

scale projects similar to the Assessment;

ii. Communicating effectively orally and in writing;

iii. Technical skills in the implementation of a large-scale testing program similar to the TCAP Assessment and a working knowledge of professional testing standards and practices;

iv. Academic, technical, and operational experience in working with a statewide

assessment, similar to the TCAP Assessment, for students with disabilities; and

v. Academic, technical, and operational experience in working with a statewide

assessment, similar to the TCAP Assessment, for LEP students.

(4) As requested by the State, the Contractor shall provide a personnel roster and resumes of key people who shall be assigned to perform duties or services under this Contract. The roster shall indicate the annual full-time equivalent (“FTE”) of each of the key people proposed for this Contract, by year. The State reserves the right to reject key personnel proposed for the project team in the State’s sole and absolute discretion. The State may request that key personnel submit to a performance exercise prior to Approval by the State. The performance exercise shall be a task created by the State that mirrors a task the State deems necessary for the key personnel to successfully execute services required under this Contract. The results of the performance exercise shall be used to verify that key personnel proposed by the Contractor are qualified to provide services in accordance with this Contract.

(5) The Contractor shall not remove, reassign, or reduce FTE responsibility for any key

personnel (including but not limited to: Lead Program Manager, Project Manager, Project Coordinator, Chief Psychometrician, Software Project Manager, Lead Software Developer, Content Area Directors, or other personnel in Lead or Director level positions) without prior written notification to and Approval by the State.

(6) The Contractor shall provide qualified replacements in the event that key personnel become unavailable to provide services at the levels specified in the Contract due to resignation, illness, or other factors affecting availability. The Contractor shall provide the State with written notice immediately upon determination of the need for replacement personnel. The Contractor shall also provide the State with written

16

notification and the resume or vitae of all replacement personnel proposed for the project team. The State reserves the right to reject key replacement personnel proposed for the project team in the State’s sole and absolute discretion.

d. Administrative Tasks – The Contractor shall provide administrative tasks including, but not limited

to, printing and distribution, Administration, Scoring, and Reporting for annual Assessments. The Contractor shall provide administrative support necessary for successful completion of these tasks, including timely communication with the State and coordination with other contractors as necessary and as authorized by the State.

e. Continuity of Contracts – The Contractor shall:

(1) Participate in joint meetings with other State contractors, the TAC, Assessment Logistics Advisory Committee or other groups identified by the State as needed. Participation shall include preparation of support materials and other documentation.

(2) Cooperate fully with the State in providing a transition between the Contractor

and the existing contractors to avoid any disruption of services, requirements, or deliverables to students, teachers, schools, LEAs, or the State.

(3) Cooperate fully with the State and any future contractor designated by the State to

transition to a potential new Contractor for the TCAP Assessments. The Contractor shall provide a transition plan, including essential data, materials, or other documentation a future contractor would need to implement similar responsibilities to those required of the Contractor under this Contract.

(4) Work with any State-appointed contractor in the handling of any student data

to support the State’s teacher evaluation system and any Reporting or other activities as requested by the State.

(5) Work with any State contractors in the development of Web-Based

Applications to ensure a smooth transition within existing services. f. Management Meetings – The Contractor shall:

(1) Convene at least one (1) weekly management meeting onsite at the State’s office in Nashville and one (1) weekly technical meeting between the Contractor and the State. These management meetings shall include a review of the key activities, deliverables, and dates and provide an opportunity to discuss task implementation and status. During a Testing Window, management meetings shall occur at least two (2) times per day on site at the State’s offices in Nashville.

(2) Produce quarterly and annual Progress Reports with relevant tasks and activities from

the schedule and progress noted for each. Progress Reports shall include a description of activities completed during the prior quarter (or year, for the annual Progress Report) and submitted to State. The Progress Reports shall provide a list of significant operational problems needing corrective action and shall address the following elements for each operational problem:

i. Identify the problem;

ii. Assign responsibility for taking corrective action;

iii. Evaluate the importance of the problem; iv. Investigate possible causes of the problem;

v. Analyze the problem;

vi. Recommend actions to prevent recurrence of this or similar problems;

17

vii. Implement new process controls as necessary;

viii. Determine what to do to correct the current problem and its consequences; and

ix. Record permanent changes in process documentation.

(3) Each quarterly Progress Report shall also contain:

i. A section that summarizes issues and responses of the Call Center as detailed in

Section A.4.h. (1) of this Contract. Pertinent data should include, but not be limited to: summaries and tallies of questions or complaints received by the Call Center, average hold time, percentage of dropped calls, average time to respond, maximum time to respond, average time to resolution, maximum time to resolution, or any other information agreed to between the Contractor and the State;

ii. A section that addresses issues or problems raised by the State;

iii. A section that addresses ongoing problems;

iv. A section that details the invoices submitted and paid; and

v. An executive summary that provides an informative and substantive description of

the major problems and recommendations

g. Unanticipated Issues or Problems - Unanticipated issues or problems shall be reported and addressed by the Contractor within twenty (24) hours of occurrence.

h. Records and Minutes – The Contractor shall take minutes and record lists of participants, including institutional affiliation and contact information, for all meetings including, but not limited to, management meetings and advisory groups convened under this Contract. All minutes, records, and lists of participants shall be provided by the Contractor to the State for review within two (2) business days after each meeting. All records and minutes shall be provided in a style and format as directed by the State. The Contractor shall review contact information for each meeting and update contact information if changed.

i. Technical Support Services – To support LEAs in the use of the goods, services, or

deliverables provided by the Contractor under this Contract, the Contractor shall perform the following:

(1) Operate a dedicated Call Center for the TCAP Assessments. The Call Center shall

provide a toll- free phone line and receive calls, chat messages, and e-mails from 6:30 a.m. to 4:30 p.m., Central Time on all State business days, except during the week preceding the Testing Window and every day of the Testing Window, when the Call Center shall be open weekdays from 5:00 a.m. to 6:00 p.m. Central Time The Call Center shall respond to all voicemails and e-mails within one (1) calendar day of receipt using the same delivery method with which the Call Center was contacted unless requested otherwise by the person requesting assistance. The Contractor shall provide the capacity for a caller to request the Call Center to call back instead of waiting on hold; the call back should be in the order received. The Call Center staff shall escalate a call or issue to someone who is able to handle it appropriately. The average Wait Time for phone and live chat shall be less than two (2) minutes during call center hours, with a maximum Wait Time of less than five (5) minutes during call center hours. Ninety-five percent of issues must be resolved by the Contractor within three (3) calendar days outside of Testing Windows. During the Testing Window, 95% of issues must be resolved by the Contractor within twenty-four (24) hours. The Call Center shall log, document, and summarize comments, complaints, and questions from schools and LEAs regarding the goods, services, or

18

deliverables provided by the Contractor. The Contractor shall establish a method of ensuring that issues identified via support tickets are analyzed to determine if further redress should be included in future technical documentation or software development updates to Web-Based Applications. The Contractor shall provide both detail and summary customer service logs on a weekly basis for four (4) weeks leading up to each Testing Window. The frequency shall increase to daily throughout the Testing Window. The Contractor shall provide a contact support ticket system that enables efficient management of the Call Center by supporting tracking of contacts, issue types, status, response (including escalation), time to response, and time to resolution for telephone, email, and Web- Based “chat” support, and any other information agreed to between the Contractor and the State. The Contractor shall provide a mechanism to measure customer satisfaction after the conclusion of each customer service interaction. The Contractor shall provide a summary of customer satisfaction on a weekly basis for four (4) weeks leading up to each Testing Window. The frequency shall increase to daily throughout the Testing Window.

(2) Develop scripts and referral guides for technical support personnel. Guides shall be created for the CBT delivery system described in Section A.9 of this Contract, Web- Based Applications as described in Section A.10 of this Contract, and for the Web- Based Reports or Reporting responsibilities described in Section A.13.g. of this Contract. All guides shall be submitted by the Contractor to the State for review and approval. The Contractor and the State shall develop and implement a process for timely updating these scripts and guides.

(3) During Testing Windows, the Contractor shall provide additional support to handle

calls regarding the Computer-Based Assessment Administration. Trained technical support staff shall be immediately available for any issues that cannot be resolved in a timely way by Call Center staff.

(4) During the Testing Windows, technical and program staff shall be available to jointly

staff a tactical operations open Call Center and hold daily webinars.

(5) The Contractor’s performance will be evaluated against the Score Card in Section A.14

of this Contract, which includes key customer service metrics for the Call Center.

The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.5 is incorporated in this Contract by reference as though set forth verbatim.

j. Various Meetings – The Contractor shall convene the meetings listed in the table below. The specific dates, times, and places for each meeting shall be detailed in the Annual Work Plan. These meetings will be at the Contractor's sole cost and expense. The Contractor shall pay expenses of all attendees. These expenses shall include but not be limited to: mileage, lodging, meals required during travel and overnight stays, and parking. All expenses born by the Contractor shall all be based on federal reimbursement guidelines. The Contractor shall provide and pay for the facility with Internet access, including room rental, breaks, and meals, and any necessary equipment. The Contractor shall be responsible for professional fees, consultant and teacher honorariums or substitute pay on a daily basis. The arrangements for each meeting shall be part of the Contractor’s Annual Work Plan, to be approved by the State.

Meeting Name Frequency Number of State

Attendees Food/Beverage Provided

Travel Reimburseme

nt Required? (Mileage,

hotel, per diem)

Substitute ($100/day) or Stipend ($150/day)

Reimbursement Required?

19

Introductory meetings

Up to twelve (12) meetings statewide

Approximately 195 attendees per

Yes No No

Regional Assessment Meetings

Up to six (6) locations statewide, twice annually prior to fall and spring

Approximately 195 attendees per location

Yes No No

Joint Planning Meeting

2 x annually Approximately 25 attendees (TDOE and the Contractor)

Yes No No

Assessment Logistics Advisory

3 x annually 15 attendees (TDOE and

Yes Yes No

Standard Setting Once per new test (science 3- 8, Biology, and alternate science 3-8 will standard set in summer 2020; social studies 3-5

10 attendees per test

Yes Yes Stipend only

New DTC training 1 x annually 45 attendees (TDOE and

Yes No No

Range finding 1 x annually 9 attendees per test (ELA 3-8, English 1-2, US History EOC)

Note: Printed materials will be required for this meeting for each attendee as outlined in

Yes Yes Both

In addition to the foregoing, the Contractor shall convene such additional training meetings to introduce the Computer-Based testing platform as required in Section A.9.a. of this Contract. These additional meetings shall occur annually over the summer at a date, time, and place proposed by the Contractor and approved by the State.

k. Materials for Introductory Meetings – The Contractor shall develop and produce the materials for

the introductory meetings (the “Introductory Materials”) as required in the table located at Section A.4.i. of this Contract, according to the timelines established in the Annual Work Plan. The Contractor shall utilize a registration process to determine the number of LEA participants attending each meeting, which shall not exceed the number of attendees set forth in Section A.4.i. of this Contract. The Introductory Materials shall include the following:

(1) Instructions for Training System Coordinators – The Contractor shall prepare

instructions related to key issues to be covered during system coordinator training with explanations accompanied graphics depicting relevant Items that include, without limitation, all Web-Based Applications, Administration protocols, and shipping instructions. The Contractor shall also conduct the training and provide training modules and resources, which shall also be made available in the State’s

20

learning management system (“LMS”). The Contractor may provide a training video as part of modules and resources.

(2) Instructions for Training School Coordinators – The Contractor shall prepare

training materials provided for system coordinators to use in training school coordinators. The Contractor shall conduct the training and provide training modules and resources, which shall also be made available in the State’s LMS. Training shall be conducted onsite and provided via webinar. The webinar shall be recorded and made available to be posted in the State’s LMS. The Contractor shall provide a training video as part of modules and resources.

(3) Materials for Regional Meetings – The Contractor shall prepare training materials

for system coordinators to use in training school level personnel on Administration activities for the upcoming Assessment Administrations. The Contractor shall conduct the training and provide training modules and resources that include art/graphics related to all steps of Assessment Administration and processing to include, without limitation order entry, shipping, test security, Assessment Administration, and post-test processing requirements. The modules and resources may be made available in the State’s LMS.

A.5. Test Security Procedures

a. The Contractor shall provide detailed, standardized test security procedures for review

and Approval by the State. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.8 is incorporated in this Contract by reference as though set forth verbatim. Approved security procedures shall be included in the annual Technical Report as required in Section A.7.h. of this Contract. These procedures must:

(1) Provide security procedures and safeguards to ensure the security of the Items,

test forms and all ancillary materials during development, production, Field Testing, Calibration, printing, and Reporting. The procedures shall ensure day-to-day security to prohibit unauthorized personnel access to testing materials through deliberate or unintentional actions.

(2) Include test security procedures and safeguards including, but not limited to,

serial numbered booklets, sealed booklets, and confidentiality agreements as necessary.

(3) Provide a secure architecture to protect processing, Scoring, and

Reporting environments from network-based attacks. (4) Provide security procedures and safeguards to ensure that electronic files and

data are developed, used, and maintained in a secure manner to protect the confidentiality of all student information, including all materials, records, demographic information, Reports, and files.

(5) Utilize encryption to ensure security of the Assessments and all

student/teacher/school/system information entered through all Web-Based Applications.

(6) Provide confidentiality agreements for participants in all meetings where confidential information is shared. All participants must express understanding of and agree to the expectation that they are not to discuss or reveal the contents of any Items, results of the Assessment, student responses, student demographic information, or any other information pertinent to the development, processing, Scoring, or Reporting of the Assessments. Confidentiality agreements may also be required when the State authorizes information to be shared that is confidential, for example, because it is embargoed or a draft.

21

(7) With the written approval of the State, implement procedures for the collection and secure destruction of secure materials (including unused tests, unused answer documents, Assessment Administration Manuals, and Scoring guides) each year following the Assessment Administration.

(8) Conduct data forensics analyses for CBT and PBT, with a rationale to monitor

and evaluate the Assessments for potential cheating and provide documentation to the State for recommended censure or invalidation of a test document. Data forensics shall include erasure analysis for PBT.

(9) Include the provision of documentation to be made available to school and

LEA personnel for Reporting of any potential breach of security procedures either in Assessment Administration or materials handling. Such occurrences reported to the Contractor shall be reported to the State within twenty-four (24) hours of notification. The State shall review and determine appropriate steps to be taken.

(10) Provide and cover the costs for any experts in the event the State is in need of factual or expert testimony in any litigation over any Assessment.

(11) Prohibit communication with the press or with LEAs regarding procedures or

outcomes for the Assessment, test delivery, Administration, Scoring, or Reporting. Any contact with the press or LEAs shall be the sole responsibility of the State.

(12) Provide thorough documentation of all test security procedures and results,

including a summary in the Technical Report each year.

b. The Contractor shall implement any additional measures directed by the State, as necessary to enhance the security of the State’s tests and ensure the credibility and utility of the test results.

A.6. Quality Control and Assurance

Accurate, consistent, timely fulfillment of all requirements in this Contract is of utmost importance to the integrity of the TCAP Assessment program and the reliability and validity of TCAP scores. As part of the Annual Work Plan described in Section A.4.a. (1) of this Contract, the Contractor shall provide a comprehensive and detailed Quality Assurance plan that is specific to the TCAP Assessments. The Annual Work Plan shall include specific safeguards, quality control, and Quality Assurance measures, along with error correction and mitigation strategies in the case of defective, inconsistent or late delivery. The Contractor‘s compensation will be tied to a Score Card in Section A.14. of this Contract or subject to Liquidated Damages as specified in Attachment B for the accurate, consistent and timely fulfillment of all delivery, Administration, Scoring, and Reporting requirements in this Contract. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.9 is incorporated in this Contract by reference as though set forth verbatim.

a. Quality Control – The Contractor shall provide detailed, standardized, quality control

procedures for review and Approval by the State. Approved quality control procedures shall be included in the Annual Work Plan for each Assessment. The Contractor shall provide quality control measures including but not limited to the following:

(1) A written proposal for a comprehensive check that all Assessment content has

been successfully imported and will be rendered error-free on all state authorized devices and systems that meet minimum specifications for the Assessment. Subsequently, a written Report documenting successful completion of the comprehensive check must be produced by the Contractor on an annual basis.

(2) A written proposal for a comprehensive check that the Assessment and Reporting platform will successfully import, update, and incorporate all user information from

22

the State information system. Subsequently, a written Report documenting successful completion of the information import must be provided by the Contractor on an annual basis.

(3) A written proposal for UAT of the Assessment and Reporting platforms. The

UAT procedures shall include key elements such as: Procedures for ensuring the quality and stability of CBT Administration, including, without limitation, benchmark or stress tests; a process for dealing with outages, test deck, checks for local capacities and compliance with Administration specifications; technical support procedures; etc. The Contractor is responsible for replacing or correcting defective portions of the CBT or Reporting system or platform at no cost to the State in such a time as not to impede the Administration, Scoring, or Reporting of Assessment data.

(4) The Contractor shall implement a plan approved by the State for monitoring and

maintaining the quality of printed materials, and their timely and accurate delivery and recovery associated with this Contract. The Contractor is responsible for replacing misprinted, erroneously delivered, or otherwise defective materials at no cost to the State in such a time as not to impede the Assessment, Scoring, or Reporting of Assessment data.

(5) The Contractor shall implement a plan approved by the State for monitoring and

maintaining the quality of Scoring (such as rater qualification and training, evaluating and maintaining acceptable Scoring quality, or evaluating the validity of automated Scoring).

(6) The Contractor shall implement a plan approved by the State for monitoring

and maintaining the quality of Reporting (such as verifying the accuracy of all reported information).

b. Error Correction – The Contractor is responsible for correcting any errors in goods,

services, or deliverables at the Contractor’s expense, arising from activities that are the responsibility of the Contractor including but not limited to, print errors, CBT outages or delivery issues, late or incorrect delivery of Assessment Materials, late or inaccurate Scoring, late, inaccurate, incomplete or inconsistent data files and analysis results, or late and inaccurate Reports. Such corrections may involve activities that include but are not limited to the following:

(1) Analyses to identify the root cause and extent of errors; (2) Edit, revise, or reprogram Computer-Based Applications; (3) Reprint or reproduce products or other materials; (4) Provide additional training to Contractor support staff, State staff, or LEA

personnel as needed via training materials, webinars, or regional meetings; (5) Replace or correct data files;

(6) Reproduce corrected Reports; (7) Ship replacement materials, products, or Reports to the State or LEAs by

using expedited shipping services; and (8) Purchase additional equipment as necessary (e.g., servers or dedicated

data circuits).

c. The Contractor shall provide documentation to the State annually of the procedures and results of the Contractor’s Quality Assurance and error correction activities. The Contractor shall include a summary in the annual Technical Report.

23

A.7. Psychometric and Research Activities

The Contractor is responsible for providing psychometric and research activities associated with Scoring and Reporting the results of the Assessment, including scaling, equating, and standard- setting. The Contractor is also responsible for generating information about field-test Items to inform actions by the Test Development Contractor. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.10 is incorporated in this Contract by reference as though set forth verbatim.

a. Psychometric Software

(1) The Contractor shall use commercially available software for Item Calibration and test Scoring to allow the State or its designee the opportunity to perform independent Quality Assurance.

(2) If the Contractor uses proprietary software for these functions, then the State or its

designee shall be granted a license to use the software at no cost to the State for the duration of this Contract. Any such proprietary software shall be installed in a way and at a location approved by the State.

b. Psychometric Model and Calibration

(1) The Contractor shall continue to utilize the Item response theory (“IRT”) models used in the TCAP Assessments in 2018 or models agreed to by the State to calibrate Items and score tests for the TCAP Assessments.

(2) The Contractor shall include documentation of Item Calibration procedures sufficient

for independent Quality Assurance and the quality of Item Calibration results.

(3) Prior to the spring 2020 Assessment Administration, the Contractor shall provide empirical evidence based on TCAP data that the results (e.g., Calibrations, assigned scores, etc.) from its psychometric software and procedures are comparable to the results obtained by the contractor who provided psychometric services in during the 2018 Administration.

c. Field Test Item Analyses

(1) The Contractor is responsible for conducting Field Tests and producing Field Test Item data files as specified in Attachment A.

d. Operational Item and Test Form Analyses

(1) The Contractor shall annually conduct analyses on operational Items as part of the Quality Assurance process to identify potential problems, such as error in the Scoring key, a problem with the display of an Item, etc.

(2) For Items that are scored by human scorers or automated Scoring programs, the

Contractor shall compute estimates of inter-rater reliability and Scoring accuracy, such as scorer agreement with validation papers or check sets.

(3) The Contractor shall conduct annual analysis of operational test forms for all TCAP

grades and Content Areas. The analysis results should be used to evaluate or confirm the psychometric characteristics of the operational test forms. The analysis shall include test information curves, conditional standard errors of measurement (“CSEM”) curves, reliability, classification accuracy, and classification consistency. The Contractor shall include the results of this analysis in the annual Technical Report.

(4) The operational Item and test form analysis shall be conducted for all students and

by demographic student groups, such as gender, ethnicity, socio economic status, and special populations (e.g.., English learners or students with disabilities).

24

(5) At least six (6) weeks prior to the spring 2020 Assessment Administration, the Contractor shall provide empirical evidence based on TCAP data that the results from its operational Item and test form analyses are comparable to those previously computed for prior TCAP Assessments.

e. Scaling and Equating

(1) The Contractor shall annually conduct scaling and equating analysis to place student performance on all operational test forms, including CBT, PBT, Braille and Large Print Test Books, onto the common base scale for each grade-level and Content Area TCAP Assessment. The annual scaling and equating process shall include:

i. Analysis for evaluating the stability of the common-Linking Item (i.e., stability check);

ii. Analysis for evaluating equating against the base year (including Item and scale

drift analyses);

iii. Internal quality control/Quality Assurance procedures for ensuring the accuracy of the scaling and equating outcomes. The Respondent shall implement procedures that ensure scaling and equating outcomes are checked for accuracy (e.g., number of replicators, independence of software/code, criteria for evaluating results, etc.); and

iv. The Contractor shall collaborate with other contractors engaged by the State to

independently verify scaling and equating results, should the State decide to do so.

(2) Details of the scaling and equating design for the TCAP Assessment can be found in

the TNReady ACH Operational 2016-2017 Technical Bulletin and TNReady EOC Operational 2016-2017 Technical Bulletin http://www.livebinders.com/play/play?id=2244559, which is incorporated in this Contract by reference as though set forth verbatim.

(3) Prior to the spring 2020 Administration of the Assessment, the Contractor shall

provide empirical evidence based on TCAP data that the results from its scaling and equating procedures are comparable to those previously produced for the TCAP program. The date shall be included in the Annual Work Plan.

f. Standard Setting – The Contractor shall implement a plan for conducting standard settings

for the TCAP Assessments as outlined below.

Timeline Assessments

Summer 2020 Science 3-8 Biology Science alternate Assessment 3-11

Summer 2021 Social Studies 3-5 Social Studies alternate Assessment 3-5

The standards setting workshops shall include, without limitation, the following:

A panel of educators, who are knowledgeable in the grade level/Content Areas, familiar with Tennessee academic standards and graduation requirements, and who are drawn from various stakeholder groups (including representatives for special education, English Learners, visually and hearing-impaired populations, colleges, universities and other community groups). The panelists shall be recruited in collaboration with the State to review, reset, or establish the performance levels on TCAP Assessments.

(1) Training for the selected panelists in standard setting methodology and the standard

setting process.

http://www.livebinders.com/play/play?id=2244559

25

(2) An established and empirically-based standard setting methodology and the opportunity for panelists to review reset or establish Cut Scores using Items or student work from the TCAP Assessments.

(3) A standard setting plan that shall be approved by the State’s TAC well in advance of

the standard setting workshop.

(4) The opportunity for representatives from the TAC or persons who are recognized nationally as having expertise in standard setting to observe the workshop and evaluate the standard setting process.

The Contractor shall cover expenses for the standard setting process, including attendees’ expenses and the TAC or professional evaluators as outlined in the meeting table in Section A.4.j. of this Contract. The Contractor shall document the standard setting process and results according to industry-accepted criteria.

g. Research Studies

(1) The Contractor shall design and conduct, as needed, mode (CBT versus PBT), device (desktop/laptop vs. tablet), Computer-Based testing delivery system interface (old vs. new, see Section A.9.a. of this Contract) Comparability studies. The study plan shall be presented to the State’s TAC for feedback prior to implementation. Upon request of the State, this deliverable shall be included in the Annual Work Plan.

(2) Additional research studies may be required of the Contractor to address

special issues, such as adverse impact or performance of demographic student groups, testing time, automated Scoring, or new Item types.

h. Technical Report

(1) The Contractor shall annually develop and publish a preliminary Technical Report for State review within four (4) months of release of Score Reports. Upon receipt of State approval, the Contractor shall provide a final Technical Report for each Assessment. Technical Reports shall be provided in hard copy and uploaded to a secure shared site.

(2) The Contractor shall conduct internal Item bias, reliability, validity, or other

technical studies as necessary to support the TCAP Assessments and provide study results to the State in the annual Technical Report within four (4) months of release of Score Reports to LEAs. The date this study shall be included in the Annual Work Plan.

(3) The Contractor shall conduct studies examining criterion-related validity in

relation to other test instruments (e.g., TCAP Achievement tests, Comparability to other state Assessments, etc.) and provide study results to the State in the annual Technical Report within four (4) months of release of Score Reports. The date shall be included in the Annual Work Plan.

(4) The Technical Report shall include, but not be limited to, the documentation of

procedures, analyses, and results in the most recently published Technical Report (i.e., the TNReady ACH Operational 2016-2017 Technical Bulletin and TNReady EOC Operational 2016-2017 Technical Bulletin.) covering activities similar to those undertaken in this Contract. See, http://www.livebinders.com/play/play?id=2244559, the contents of which are incorporated by reference as though set forth verbatim in this Contract.

A.8. Assessment Materials

The Contractor shall provide all required paper and electronic Assessment Materials including but not limited to: test books, response documents, test directions, Assessment

http://www.livebinders.com/play/play?id=2244559

26

Administration Manuals (“TAM”) (separate directions and manuals should be available for the practice test and operational Administrations, for CBT or PBT Assessments, and for test administrators and testing coordinators), and user guides for the CBT Platform. See Attachment A of this Contract for a list of required printed materials.

Samples of all Assessment Materials shall be provided to the State for review and revision prior to each Assessment Administration by the date specified in the Annual Work Plan. All Braille with ink print, Large Print Test Books, and Computer-Based versions of the test shall be delivered to the State at the same time as the regular PBT materials. The Contractor shall allow the State a minimum of five (5) business days for initial review. Upon completion of any necessary revisions or modifications of any Assessment Materials, the Contractor shall return materials to the State and provide not less than three (3) business days for final review and approval. The State shall have the opportunity to review and modify the design of Assessment materials prior to any printing or production for any Assessment Administration.

All electronic Assessment Materials (e.g., CBT) or files shall be compatible with the current version and the two (2) previous versions of the iOS and Windows operating systems (“N-2”) as specified in A.9.c. (1).vi. The electronic Assessment Materials should also be compatible with available web browsers including but not limited to Google Chrome, Microsoft Internet Explorer/Edge, Firefox, and Safari as specified in Section A.9.c. (1) of this Contract. The Contractor shall provide the State with technical specifications required for any Computer-Based Applications or programs, including the CBT Platform. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.14 is incorporated in this Contract by reference as though set forth verbatim.

All descriptions of materials shall apply to each TCAP Assessment unless otherwise noted. Some additional materials may be added as needed. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.11 is incorporated in this Contract by reference as though set forth verbatim. The following specifications apply to Assessment materials, which shall be developed by the Contractor and delivered to the State:

a. Technical Specifications for All Assessment Materials – The Contractor shall use the

following guidelines for all Assessment Materials developed for the TCAP program (Additional specifications for modified Assessment Materials (e.g., Braille and Large Print), which are more particularly described in Section A.8.c. of this Contract):

(1) Test books and manuals shall be 8 ½” x 11”, saddle-stitched or perfect bound;

(2) The State may review any colors used for printing test books, Student Response

Documents, ancillary materials and Reports;

(3) Materials defined in Sections A.8.b-e of the Contract shall be used for operational and field Assessment Administrations;

(4) Assessment Materials shall be available in an electronic format compatible for use by

the State or LEAs; and

(5) Materials and procedures shall be available for additional test forms in the case of security breaches or other needs during the Testing Window. The Contractor shall deliver the Assessment Materials for PBT as directed by the State within three (3) calendar days of notification by the State. The Contractor will be supplied by the Test Development Contractor with a backup test form for each Assessment administered on computer, in case of a security breach, etc. The Contractor shall “stand up” the backup test form for operational Administration within twenty-four (24) hours of notification by the State.

b. Preparation Materials – The Contractor shall develop and produce the preparation materials

(the “Preparation Materials”) according to timelines established in the Annual Work Plan for each Assessment. The Contractor shall refresh the Preparation Materials with new Items at a minimum of every other year for the duration of the Contract. The new Items for the Preparation Materials will be supplied by the Test Development Contractor as approved by the

27

State. The Preparation Materials shall contain the following:

(1) Student Demo – A student demo shall be available for each grade band and Content Area. The Contractor shall develop and provide a Computer-Based student demo tool that mirrors the CBT interface. The main purpose of the student demo is to allow students the opportunity to become familiar with the CBT environment, including without limitation, navigation features, reference materials, tools, Accessibility Features and Accommodations, and all possible Item types that students may encounter. The student demo should meet all technical requirements defined in Section A.9. of this Contract.

(2) Practice Tests – A practice test that can support up to two (2) per grade level/Content

Area shall be available. The Test Development Contractor will provide to the Contractor up to two test forms per grade/Content Area identical in format, style, and Item quality and similar in content to the operational test form for each grade level and Content Area, except that secure Items shall not be included, nor shall Linking Items, field-test Items, or Items reserved for future operation be used. The test forms will be in camera-ready PDF and QTI package format for CBT. The Contractor shall use the supplied test forms to deliver practice tests. For the grades and Content Areas requested by the state, the practice tests shall:

i. Include information related to understanding the Assessment and offer suggestions on ways to prepare for the Assessment;

ii. Be available in Computer-Based formats within the CBT test platform;

iii. Meet all technical requirements defined in Section A.9. of this Contract and mirror

the CBT delivery system interface, except that the Optional Formative Assessment Tool as specified in Section A.8.b.(3) of this Contract shall not be included;

iv. Be available for use throughout the entire school year beginning on September

15, 2019, and each anniversary date thereafter during the Term; and

v. Include Reporting of student results, including a summary of test performance and standard analysis performance.

(3) Optional Formative Assessment Tool– At the request of the State, the Contractor

shall develop a formative Assessment tool for each grade level and Content Area. With content provided by the Test Development Contractor, the tool developed by the Contractor shall meet all technical requirements defined in Section A.9. of this Contract and shall also:

i. Import Items that will be identified by academic standard and Reporting Category

by Content Area for each grade;

ii. Include both selected and CR Item types;

iii. Provide teachers access to Item keys;

iv. Be designed to allow teachers to set up their own Computer-Based practice tests for students;

v. Exclude all Items used in the common Linking-Item, practice tests as specified in

section A.8.b.(2), Field Test Item Data Files, or operational forms; and

vi. Be capable of providing Student Reports.

c. Modified Format Materials – The Contractor shall deliver certain modified format materials (the “Modified Format Materials”) to assist in the Administration of the TCAP Assessments. Braille

28

shall be reviewed by an independent expert and the form must be approved by the State’s Sight Review Committee prior to final printing. The Modified Format Materials shall meet the following requirements:

(1) Braille Test – Using camera ready Braille test forms from the Test Development

Contractor, the Contractor shall produce forms of the TCAP Assessments in Braille. The Contractor shall produce the number of Braille versions, by Assessment, specified in its Annual Work Plan, as approved by the State. The numbers administered in 2018 are shown in the table in Section A.3. of this Contract. The Braille materials shall be delivered with the Test Materials defined in Section A.8.e. of this Contract. The specifications for Braille versions include:

i. Ink print copy and scan-able answer document shall be provided on a 1:1 ratio;

ii. Teacher’s notes for the Braille version shall be developed to assist teachers with

transcriber’s notes in the Braille edition, use of special symbols, changes in wording, ink print to Brailed page references, and any other Administration information exclusively related to the Braille test;

iii. Braille test books shall be of appropriate weight and type of paper as determined

in collaboration with the State;

iv. Braille tests and directions shall be marked with ink print on the front cover to indicate the corresponding grade level or Content Area;

v. Pages shall be Brailed on one side of the paper;

vi. Items with graphics will be Brailed on facing pages whenever possible;

vii. Use of thermoform will be kept to a minimum;

viii. Individually identifying data for tracking purposes shall be included on all

materials; and

ix. Individually packaged Braille version with teacher’s notes, ink print copy, rulers or other required manipulatives, and a separate scan-able answer document for transference of student responses.

(2) Large Print Test Books – Using camera-ready Large-Print test forms from the Test Development Contractor, the Contractor shall produce test forms for paper-based testing in Large Print Test Books. Large Print Test Books shall be delivered with Test Materials defined in Section A.8.e. of this Contract. The online testing platform shall provide zoom capabilities for enlarged text onscreen. The specifications for printed Large Print Test Books include:

i. Each Large Print Test Book shall be in 18-point font size;

ii. Items requiring measurement shall be scaled for use with regular rulers, or appropriately scaled rulers shall be supplied by the Contractor;

iii. Grey-scale and shading shall be avoided;

iv. Highest possible contrast shall be used for text, art, illustrations, and graphics;

v. Paper with a dull finish in ivory, cream, or white with black print shall be used;

vi. Booklets shall be 9” x 12” or such other size as agreed upon by the State;

vii. Binding shall allow each page to lie completely flat for whole page

29

viewing and ease of handling; and

viii. The Contractor shall produce the number of Large Print Test Books, by

Assessment, as approved by the State. The numbers administered in 2018 are shown in the table in Section A.3. of this Contract.

(3) Pre-coded response documents or labels shall be provided for students requiring a

modified format test version whenever possible.

(4) Student responses recorded in the Braille and Large Print Test Books shall be capable of being transcribed onto regular Student Response Documents or entered into the CBT Platform by the test administrator of the Assessment.

d. Administration Materials – The Contractor shall develop and produce all Administration Materials to assist in the Administration of the TCAP Assessments. Separate Assessment Administration Materials shall be required for CBT and PBT for Field Test and operational Administrations. All Administration Materials shall be completed and delivered to the State, or to such persons, schools or LEAs as directed by the State, at least six (6) weeks prior to the Testing Window. Administration Materials shall meet the following requirements:

(1) User’s Guide for the CBT System – The Contractor shall develop and produce,

in collaboration with the State, a user’s guide (the “User’s Guide”) for each Assessment Administration. The User’s Guide shall provide technical specifications for use of the CBT Platform. Information shall include without limitation: hardware specifications, proctor caching requirements if needed, student data upload process, data editing information, detailed information on the use of the Assessment tools, and other technical guidelines as necessary. Separate guides may be provided with focuses for technical and Assessment staff. The guide shall be posted to State and Contractor websites.

(2) Printed Materials – The Contractor shall produce and print Assessment

Administration Manuals and building testing coordinator guides and print Assessment Administration Test Administrator Proctor Scripts (“TAPS”), secure content Assessment Administration TAPS, directions for test administrators, and Accommodated Assessment Administration TAPS. Information about the types and numbers of each document is provided in the table below.

Document Title # Versions Approx.

# Pages # Produced if Printed

Delivery Mode

Assessment Administration Manual (TAM)

1 30 1 per 50 students

Printed

Assessment Administration Proctor Script (TAPS)

1 for English EOCs Computer- Based 1 for Math EOCs Computer-Based 1 for US History EOC Computer-Based 1 for science 5-8 and Biology EOC Computer- Based

20 1 per 20 students

Printed

1 for Math EOCs paper-based 1 for US History EOC paper- based 1 for Biology EOC paper- based 4 for science 5-8 paper-based (1 per grade level) 5 for ELA 4-8 (1 per grade level) 6 for math 3-8 (1 per grade level)

Secure Content TAPS

15 for ELA grade 3 (1 per version) 3 for ELA grade 2 (1 per version)

60 1 per 20 students

Printed

30

Accommodated TAPS (Read Aloud)

2 for English EOCs paper-based (1 per test) 6 for Math EOCs paper-based (1 per test) 1 for US History EOC paper- based 1 for Biology EOC paper- based 6 for science 3-8 paper-based (1 per grade level) 6 for ELA 3-8 paper-based (1 per grade level)

60 Ordered by districts for students receiving read- aloud Accommodations for paper-

Printed

Directions for Test Administration

2 for ALT social studies 3-8 2 for ALT science 3-8 2 for ALT

60-70 1 per student test

Printed

Building Testing Coordinator

1 50 1 per school Printed

Manipulative 1 Ruler for math grade 3 N/A 1 per grade 3 student

Printed

e. Test Materials – The Contractor shall prepare and produce all Test Materials required for the Administration of the TCAP Assessments and provide samples to the State by the date set forth in the Annual Work Plan, so that the State will have sufficient opportunity to review all Test Materials for revision prior to each Assessment Administration.

(1) Student Test Booklets – The following specifications apply to all Student

Test Booklets unless otherwise indicated:

i. Test booklets must be individually sealed;

ii. One test booklet shall contain multiple Content Areas in grades 4-8, one test booklet per Content Area shall be developed for secondary Assessments;

iii. Grades 2-3 shall have a consumable test book for math and ELA;

iv. Each grade level or Content Area shall consist of customized forms

with a separate scan-able answer document per test booklet, with the exception of grade 2 and 3 consumable books;

v. A mechanism to individually identify each document is required and an

inventory of the same shall be provided to schools and districts;

vi. Test booklets shall contain an individual identification mechanism for

inventory management and test security and this information shall be included on inventory sheets provided to schools and districts;

vii. Modified format tests shall meet the following requirements:

1) Braille test booklets for each assessment shall be provided as

defined in Section A.8.c.(1) of this Contract; and

2) Large Print Test Books for each Assessment shall be provided as defined in Section A.8.c.(2) of this Contract.

viii. Test booklets shall be shrink-wrapped in units to be determined by the

number of versions, provided however, standard units in increments of 5 (e.g. 5 or 20) will be considered subject to the approval of the State;

ix. An appropriate distribution method shall be utilized by the Contractor,

including spiraling procedures, as directed by the State;

31

x. Booklets in units shall be numbered sequentially; xi. Approximately ten percent (10%) may be packaged in uncollated groups

for use with students with special Accommodations to have the test read aloud in small groups; and

xii. Assessments shall be accessible to students in public, State Special Schools, private schools, or home schools. Test booklets shall also be provided in sufficient numbers for each student to be assessed.

(2) Student Response Documents – The following specifications apply to all student information or Student Response Documents unless otherwise indicated by this Contract:

i. A machine scan-able answer document must be provided for grades 3-8 non-

consumable test booklets;

ii. For EOCs, a machine scan-able answer document should be provided for each students requiring a paper-based accommodation;

iii. Response documents shall be designed to accommodate a pre-gridded label

with student information or an overspray of student information;

iv. Assessments shall be accessible to students in public, State Special Schools, private schools, and home schools. Response documents shall be provided in sufficient numbers for each student to be assessed on paper versions through an ordering process approved by the State;

v. The ability to code student information on Student Response Documents or in a

CBT delivery system based on State and federal Reporting requirements, shall include without limitation:

1) First Name, Last Name, and Middle Initial;

2) Student ID Number (state assigned);

3) Birth Date;

4) Grade (2-12);

5) Class Period;

6) Modified Format (i.e. Braille, Large Print);

7) Accommodations as defined in this Contract or as directed by the State;

8) Not tested (e.g., absent, EL exclusion or medical exemption);

9) School schedule (e.g., traditional, modified block, block);

10) Test Date;

11) Test version or form;

12) Optional codes for State use; and

13) Report of irregularity status.

(3) Formula Reference Information – As specified by the Test Development Contractor,

the Contractor shall produce formula reference information for mathematics equations. Reference sheets shall be included in paper test booklets or as an individual document for the Content Area, provided electronically via PDF for posting

32

on the State website and provided for access via the CBT Platform. Reference sheets shall be provided in Braille and Large Print.

(4) Miscellaneous Manipulatives – The Contractor shall provide all manipulatives

necessary for the Assessment, as specified by the Test Development Contractor and approved by the State, including rulers and other consumable Items, during each Administration. All manipulatives shall be provided in accessible formats for all students including through an online platform and in Braille or Large Print if needed or deemed appropriate by the State. The Contractor will be notified of the specifications for manipulative materials required no less than three (3) months prior to the operational Assessment Administration. The historical provision of manipulative materials is shown in the table above.

f. Grouping of Answer Documents – The Contractor shall group answer documents to facilitate

processing, Quality Assurance, and Reporting. The procedure for grouping shall maximize ease of use and accuracy. The Contractor shall provide instructions for LEAs on the procedure as part of material returns.

g. Packaging and Distribution – The Contractor shall package and distribute paper Assessment

Materials directly to State, public, or private school systems as indicated by the State. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement

C.12 is incorporated in this Contract by reference as though set forth verbatim.

(1) Packaging Specifications. The Contractor shall, in collaboration with the State, prepare packaging specifications and include them in the Annual Work Plan specified in Section A.4.a.(1) of this Contract. The specifications should include at a minimum the following:

i. Procedures for packaging materials for shipment to LEAs, including but not

limited to:

1) Description of how materials are packed;

2) Examples of packaging and inventory lists;

3) Examples of bar coding or other document identification method; and

4) Description of inventory procedures.

ii. Procedures for packaging materials for return shipment to the Contractor, including but not limited to:

1) Detailed instructions for packaging materials;

2) Inventory procedures; and

3) Shipping information, including pre-paid labels.

(2) Packaging and Distribution Guidelines. The Contractor shall adhere to the following

guidelines for the packaging and distributing all paper Assessment Materials:

i. Provide a mechanism to uniquely identify each Student Test Booklet and response document. The Contractor shall provide a master file containing a record of the Student Identification Numbers sent to each school and district.

ii. In addition to quantities requested by the State necessary for

enrollments, provide an overage of all materials to allow for printing errors, shipment and site shortages.

iii. Packaging of books that are numbered sequentially within each package

33

within each school if they are spiraled sequentially.

iv. Shrink-wrapping of test booklets in units to be determined by the number of versions of each Assessment per Administration.

v. Pre-coded administrative materials shrink-wrapped by the Contractor, at

a minimum, by school and system.

vi. Each package shall have a label on the outside identifying the school and system.

vii. Shipping of printed products only after receiving prior written Approval by

the State after the State has verified the Assessment Materials were provided in accordance with this Contract.

viii. Packaging of Braille and Large Print materials that are labeled separately

and included in the shipment of Test Materials to LEAs.

ix. Providing Assessment Administration Materials and Assessment

Materials on site in each LEA according to the schedule established by the State or specified in the Annual Work Plan (as provided in Section A.4.a. (1) of this Contract), but in no event later than ten (10) business days prior to the first day of the Testing Window.

x. Providing shipping and distribution information for each Administration with samples of packaging labels, tags, and procedures to be included in materials for regional meetings defined in the table in Section A.4.j. of this Contract.

xi. Providing proper packaging by school of all Test Materials for shipment

to LEAs.

xii. Maintaining detailed records of the number of tests ordered, shipped, delivered and Scored for each school and LEA.

xiii. Utilizing transport companies that provide inside delivery and unload

large shipments onto loading docks.

xiv. Utilizing transport companies that provide inside pick-up, palletizing (if

necessary) and load large shipments from loading docks. xv. Utilizing methods of transport approved by the State as part of their

shipping method and using Tennessee-based companies whenever practicable.

xvi. Utilizing a shipping method that provides traceable delivery information.

If boxes that were used to initially ship materials are to be used again to return materials, this shall be made explicitly clear in all directions to LEA and school personnel and the boxes shall be of sufficient strength and construction to withstand reuse.

xvii. Designing, printing, and distributing detailed bill of lading/packaging lists for each school and LEA. The LEA and school packaging lists shall Itemize the materials that are being shipped to the LEA including but not limited to: contents of each box, quantities of Items shipped, inventory information, and destination.

xviii. Providing a description of check-in procedures used by the Contractor to

identify scoreable materials erroneously packaged with nonscorable materials. Separate procedures may be required for each Assessment.

34

xix. Providing a description of reconciliation/recovery procedures used by the Contractor to identify missing Test Materials from the LEAs.

xx. Providing procedures for providing appropriate materials for students

with IEP and Section 504 plans that require modified test formats such as Braille and Large Print. These materials shall be packaged and labeled separately and will be included in the same shipment to the LEA with other testing materials, if possible.

xxi. Providing inventory measures that verify by number all test booklets,

response documents, or other secure Assessment materials shipped to LEAs and returned after testing or used in a CBT delivery system as specified in Section A.9. of this Contract. These measures shall include a control process wherein all Assessment Materials being shipped to and from the Contractor’s facilities are counted and tracked. Missing shipments shall be traced, located, and redirected to the proper location, or replaced as needed.

xxii. Providing pre-paid shipping labels and materials for the return of all

Assessment Materials to the Contractor for Scoring at the Contractor’s expense.

xxiii. Tracking return shipments to ensure that LEAs return all Assessment

Materials to the Contractor for Scoring no later than five (5) business days after the close of the Testing Window for the Assessment.

xxiv. Notifying the State of those LEAs that do not return materials within the specified time frame.

xxv. Prior to testing, providing a process for LEAs to Report missing shipments

or Assessment Materials. The Contractor will ensure that shortages are shipped to LEAs immediately.

xxvi. After testing, preparing a missing Assessment Materials Report for Assessment documents based on the material verification process that is completed during Assessment Materials check-in. Reports will be prepared for each school with missing Assessment Materials listed. School Reports shall be delivered by the Contractor to system coordinators with a system-level summary and a State level summary to be delivered to the State. Any missing Assessment Materials returned by LEAs shall be recorded in the missing Assessment Material inventory maintained by the Contractor. The missing Assessment Materials Reports must be delivered to the State and systems daily as the check-in of secure Assessment Materials is processed. For each Administration, check-in and verification of secure materials must be completed and discrepancies resolved prior to the first shipment of Assessment results to Tennessee systems.

xxvii. Including a plan for handling straggler Assessment documents and

materials.

xxviii. Providing written Reports to the State documenting the check-in of all

secure Assessment Materials.

xxix. Cataloging, storing, and archiving completed student test response documents in a secure facility for twelve (12) months from the date of any Report.

xxx. Destroying archived hard copy student Assessment response materials in

a secure manner (i.e., shredding or incineration) at the completion of twelve (12) months after delivery of Score Reports as directed by the

35

State. At the end of the Term, remaining archived materials shall be shipped to the State, or a location designated by the State, for storage. The Contractor shall utilize electronic imaging and duplication technologies to archive all student responses in a secure data format (e.g., encrypted) on secure servers or by other electronic media. All archived materials shall be clearly labeled to include, at a minimum Assessment year, grade, Content Area, teacher name, school name, and system name. Archived materials shall be maintained by the Contractor for the duration of the Contract. A copy of archived materials shall be provided to the State annually via a secure electronic site on a timeline specified in the Annual Work Plan, which shall also provide a process for linking Score Reports to the teacher of record.

(3) Tracking and Managing Assessment Materials – The Contractor shall provide a Web-

Based Application that allows the State and LEAs the ability to track Assessment Materials. The Contractor agrees to collaborate with the State for the modification/revision/customization of the tracking application as needed to meet State criteria. The Web-Based Application shall include but not be limited to the following features:

i. Secure Assessment Materials (including but not limited to, test booklets and

response documents) that are individually identified;

ii. The generation of shipping labels to be placed on boxes for tracking that include bar coded information regarding the contents of each box. Return labels shall also be provided;

iii. The ability to confirm counts of Assessment Materials received prior to testing

and to notify the State or the Contractor of any discrepancies;

iv. The ability to Report counts of Assessment Materials being returned for processing and Scoring, including used, unused, and inactive Assessment Materials; and

v. The ability to produce pre-coded student information on response documents.

The Contractor shall provide a system that accepts uploaded student information from the student management system at the school, system, or State level or a State contractor for use on pre-id documents and online testing. The Contractor shall ensure that the Web-Based Application maintains compliance with the State’s student management system to enable the acquisition of student information. Uploaded data may also be used to pre-code teacher license information on response documents and teacher headers. Bar codes shall include all available information for each student as well as school and system identification information. Student Response Documents shall be pre-coded with all necessary information. The Contractor agrees to collaborate with the State for the modification, revision, or customization of the enrollment program as needed to meet State criteria.

A.9. Computer-Based Testing (“CBT”)

a. CBT Platform and System – The Contractor shall provide a CBT Platform and CBT

System for administering tests in grades and Content Areas as requested by the State that meets the technical specifications outlined in Section A.9.b. of this Contract. The CBT Platform and CBT System must be capable of demonstrating Comparability of Scores between the CBT Platform or CBT System in use prior to the Effective Date of this Contract and the CBT Platform or CBT System utilized by the Contractor. At the State’s request, the Contractor shall conduct a “real world” Field Test that contemplates the number of students the State anticipates participating in the Assessment for the estimated amount of time necessary to conduct the Assessment in order to confirm the Contractor’s readiness to provide CBT during the Testing Window. For purposes of this requirement, the Contractor shall maintain permanent capacity for

36

two hundred fifty thousand (250,000) concurrent CBT users and shall also maintain capacity for scaling to five hundred thousand (500,000) concurrent CBT users if directed by the State. In addition to the foregoing, the CBT Platform and CBT System shall meet the following requirements unless otherwise indicated in this Contract:

(1) CBT Platform Completeness – The Contractor shall provide a CBT Platform

that is adequate to support the CBT System and the Contractor’s obligations under this Contract. A complete CBT Platform includes, without limitation, all hardware, servers, routers, maintenance, support, or updates necessary to support the CBT System and Contractor’s obligations under this Contract, including functional and non-functional requirements.

(2) CBT System Completeness - The Contractor shall provide a CBT System that includes all software, schemas, configurations, server-side computing resources, licenses, processes and procedures, both human and machine-assisted, that support the entirety of functional and non-functional requirements set forth below, and that complies with the State’s requirements for Contractor Hosted Services Confidential Data, Audit, or Other Requirements as defined in Section E.8. of this Contract.

(3) Verification of Functional Requirements

i. The Contractor shall deliver a CBT System that is substantially defect free

and operates in accordance with the functional requirements required in this

Contract plus those advertised by the Contractor as a feature or capability of

their CBT System. The Contractor shall conduct extensive verification of the

CBT System using industry best practices for software Quality Assurance

prior to delivery of the CBT System to the State except where such delivery is

in support of a product preview, BETA, or other non-production usage.

ii. The Contractor shall support UAT by the State to verify the functional

requirements of this Contract. UAT shall take the form of scripted test cases

that regressively exercise each functional requirement of the CBT System on

each of the major device platforms and operating system versions in use in

Tennessee prior to each Testing Window. The Contractor shall not use UAT

as an alternative or means to supplement internal Quality Assurance

processes. UAT shall not reduce the Contractor’s obligation to deliver a CBT

System that is substantially free of defects.

iii. The UAT process shall include a formalized process for communicating and

tracking all defects, questions, or decisions arising from UAT and shall

include a process for prioritizing those defects that must be corrected and re-

verified prior to commencement of the Testing Window.

iv. The CBT System shall be made available for UAT a minimum of forty-five

(45) calendar days prior to commencement of the Testing Window and must

reflect the version of the code-base, test forms, and system configurations

that will be in use for the Testing Window.

v. The Contractor shall not promote the CBT System into a production ready

state without obtaining formal UAT sign-off from the State.

vi. The Contractor shall not make any change to any aspect of the CBT System, including without limitation, to software, infrastructure, configuration, or other components following formal UAT sign-off and prior to the end of the Testing Window without the prior written approval of the State.

(4) Evidence of Compliance with Non-Functional Requirements - The Contractor shall provide

the State with evidence in the form of documentation, test results, log files, or independent

37

third-party verification of compliance with all non-functional requirements, including

without limitation, the State’s requirements for Contractor Hosted Services Confidential

Data, Audit, and Other Requirements as defined in Section E.8. of this Contract, prior to

the first Testing Window following the Effective Date and annually thereafter.

b. CBT - Functional Requirements: Functional Requirements are defined as those

requirements that define the specific behaviors, workflows, and capabilities of the CBT System and shall contain the following:

(1) Ease of Use – The Contractor shall ensure that the user-interface of all student- facing

software components of the CBT System have been scientifically designed and verified through usability studies to be age-appropriate and consistent with the keyboarding and other computer skills typical of students in grades three through twelve.

(2) Predictability of Use - The Contractor shall ensure that all student-facing software

components of the CBT System implement user-interface paradigms, including; keyboard, mouse, and gesture actions that are consistent with the user interface paradigms established by the targeted operating system platform with emphasis on the unique paradigms established by the Windows, Chrome, MacOS and iOS operating systems.

(3) User Input - The Contractor shall ensure the CBT System supports keyboard, mouse,

touchpad, and touch screen user-input modalities as appropriate for the question type and computing device in use and that the input modalities available are consistently applied across Item types and device platforms.

(4) Accommodations and Assistive Technologies

i. The Contractor shall ensure the CBT System provides the following assistive

technologies and Accommodations for students with disabilities, including those

related to sight, hearing and mobility as follows:

1) Text to Speech, including pause, resume, “play from here” and “read highlighted text” capabilities;

2) Adjustable font size;

3) Screen, region, image, and text magnification;

4) Foreground and background color selection with high contrast pre-sets; and

5) Accessibility device support, including; headphones, microphones,

tablets, braille readers, and other input and content rendering

technologies.

ii. The Contractor shall ensure its CBT System supports any additional student

Accommodations and accessibility tools that may arise based on the needs of the student population at any given time. The State will notify the Contractor at least one year in advance of the requirement for new tools.

iii. The Contractor shall provide a viable mechanism to ensure that Accommodations

and accessibility tools can be configured to meet the specific needs of each student, can be defaulted to reduce repetitive entry for the same student, and are only available to the students for whom they have been configured.

(5) Online Testing Tools - The Contractor shall ensure the CBT System provides approved

tools and features to support completion of the Item types and test Items currently in use

in Tennessee and which have been developed for Tennessee in the future by test

33

content developers or commercial Item Banks, which currently include:

i. Calculator;

ii. Equation Editor;

iii. Ruler and Straight Edge;

iv. Formula or Vocabulary Reference Sheets;

v. Bookmark;

vi. Passage Highlighter;

vii. Pop-up foot notes; and

viii. Note pad.

ix. Test Item Type Compatibility - The Contractor shall leverage Item formats that take

advantage of the capabilities of computer technology whenever possible, provided such support does not introduce risk to the performance or device compatibility of the CBT System or introduce unacceptable incompatibilities with paper-based testing.

x. The State currently uses the following Item types:

1) Multiple choice;

2) Multiple select;

3) Extended text;

4) Line graph/interactive graph;

5) Matching;

6) Text entry;

7) Hot text; and

8) Composite.

xi. The State may request additional Item types or include additional Item types

supported by the Contractor in future test forms provided such Item types are supported by evidence of efficacy for summative testing purposes.

(6) Test Form Import – The Contractor shall ensure that the CBT System can seamlessly import

and export test forms, test Items and other content resources in the QTI format version and preserve the full fidelity of the content as authored and supported by QTI. The Contractor shall work with the State to support newer versions of the QTI specifications as those versions attain mainstream status in the market.

i. The Contractor shall receive and test for functionality the test content provided by the

Test Development Contractor. The Contractor shall work with the Test Development Contractor to carry out quality control testing of the content.

ii. The Contractor shall facilitate test content specifications, by collaborating with the Test

34

Development Contractor, sufficient to permit accurate and efficient transfer of test content and implementation of CBT.

iii. The Contractor shall design and carry out quality control testing in collaboration with

the Test Development Contractor prior to operational Administration to provide assurance that the Contractor’s CBT delivery system and associated software and services will deliver the test content accurately and efficiently (e.g., rendering, tools and supports, system latency, device compatibility, etc.).

(7) Test Security and Integrity

i. The Contractor shall provide a Secure Browser or other mechanism to ensure

students cannot access resources on the Internet, local area network, local machine, or peripheral device that may provide unauthorized support to a student. Features that shall be controlled by a Secure Browser during a test Administration and include:

1) Internet access via a browser, hyperlink, service, or installed application

other than the CBT System;

2) Access to local and peripheral hardware devices such as cameras,

microphones, speakers, and printers except where such access is approved to support an accommodation or accessibility requirement;

3) Access to messaging applications such as email, instant messaging,

chat and video conferencing;

4) Access to recording applications such as screen capture, screen print, video capture, and sound recording;

5) Access to unapproved reference applications, tools, or utilities such as dictionary, thesaurus, grammar, calculator, equation editor, or symbol editor;

6) Access to any locally or network installed application not directly related to

the CBT System; and

7) Access to operating system capabilities such as pop up messages,

application switching, and AI “assistants” such as Cortana or Siri.

ii. The Contractor shall provide documentation and training to districts on how to correctly establish and configure secure test environments that are optimized for the CBT System. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.15 is incorporated in this Contract by reference as though set forth verbatim.

(8) Consistency and Equivalency with Paper Based Testing

i. The Contractor shall ensure that the test forms, Items types and answer choices

loaded into the CBT System are accurate and consistent with the properties, parameters, and intended design of that form;

ii. The Contractor shall ensure that the presentation of the online form is

psychometrically equivalent to the form as implemented for PBT;

iii. The Contractor shall ensure that the functionality of the CBT System, including, tools, reference sheets, Accessibility features, and navigation remains consistent across test forms, Item types, and Assessment Administrations; and

35

iv. The Contractor shall support the State with access as needed to review all online test forms prior to each Assessment Administration and any revisions requested by the State shall be made by the Contractor in a timely manner.

(9) Device Equivalency – The Contractor shall ensure that psychometric and user

experience equivalency exists between all device types and operating systems approved for use in Tennessee for operational testing such that a user of any given device platform shall have no discernible advantage or disadvantage compared to a user utilizing a different device platform.

(10) Operational Testing Status - The Contractor shall ensure that the CBT System includes a

dashboard available to designated users at the State and district level that provides visibility into student test completion status at the district, school, class, course, and student level. The dashboard shall not be more than twenty-four (24) hours behind actual time and shall include search capabilities to allow users to rapidly narrow the scope of the status displayed.

(11) Recovery of Student Work - The Contractor shall ensure the CBT System includes a robust

and scalable mechanism for recovering all student work that was completed prior to an outage, system failure, or device malfunction of any kind that prevented submission of the student’s work under the normal process. Recoveries shall be completed within the Testing Window and the status of the recovery process shall be visible to the school district that requested the recovery through a Report or online mechanism. If a recovery is needed in the final three (3) calendar days of the Testing Window, the Contractor shall keep the Testing Window open as needed to allow the recovery to be completed within the Testing Window. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.20 is incorporated in this Contract by reference as though set forth verbatim.

(12) CBT System Availability - The Contractor shall make the CBT System available from 6:00

a.m. to 8:00 p.m. Central Time each day during the Testing Window to allow for unique testing situations, including homebound students. The Contractor shall ensure that adequate mechanisms are in place to control the specific timing of access to an Assessment Administration by a student to prevent early access or access after an Assessment Administration has closed. The Contractor shall ensure that appropriate measures are in place to prevent students from accessing test forms from unauthorized locations and to prevent unauthorized students from accessing a test.

(13) Contingency Planning – The Contractor shall implement a contingency plan prior to

each Testing Window that provides options for a student to complete a scheduled Assessment in the event of an online system failure, outage, defect, or other unexpected disruption that prevents a student from testing online. Contingency plans must be approved by the State in wr i t ing in advance and may include the staging of pre-printed Assessment Materials or the ability to print Assessment Materials on demand.

(14) Training Materials – The Contractor shall provide the following training to proctors and test administrators located in Tennessee school districts:

i. Downloadable user manuals and training materials for each user role;

ii. Print ready user manuals and training materials for test administrators and

proctors;

iii. On demand online (video) training materials for test administrators and proctors;

iv. Quick reference cards as required;

v. Regional, instructor-led training events; and

vi. On demand online webinars of recorded events.

36

(15) Training System – The Contractor shall provide a version of the CBT System for training purposes. The CBT training system shall be available to Tennessee users year-round and shall provide a user experience that is highly representative of the testing experience, functionality, tools, test forms, and test Items types that will be encountered during an Testing Window.

c. CBT - Non-Functional Requirements

Non-functional Requirements are defined as those requirements that define the operation, implementation, and technical aspects of the CBT. The Contractor’s attention is drawn to Section E.8. of this Contract, which defines the State’s requirements for Contractor Hosted Services Confidential Data, Audit, and Other Requirements. The following clauses are supplemental to that section:

(1) Client-Side Device Compatibility - The Contractor shall ensure that all client-side software

components, scripts, configurations, and protocols of the CBT are fully compatible with the matrix of approved computing devices, operating systems, versions, and specifications (“Client-Side Technology”) in use in Tennessee school districts, which in general include Windows, Chrome, iOS, and macOS devices. The Contractor and State shall review and determine the matrix of approved Client-Side Technology within thirty (30) days after the Effective Date of this Contract and each anniversary date during the Term taking into consideration the Client-Side Technology available in Tennessee school districts as determined through an annual survey, manufacturer support, or the proven compatibility of the CBT System as provided by the Contractor.

(2) Technical Field Support - The Contractor shall ensure that appropriately qualified technical

staff is staged in Tennessee during all Testing Windows such that a school district in Tennessee can be reached with in-person technical assistance within ninety (90) minutes of requesting such assistance. Technical staff shall be equipped with the necessary tools and resources to troubleshoot and remedy issues related to the CBT System of a technical nature, including, system defects, infrastructure malfunctions, connectivity, usability, and other issues impacting the student experience that district technical staff cannot reasonably diagnose or remedy without Contractor assistance. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C. Technical Requirement C.16 is incorporated in this Contract by reference as though set forth verbatim.

(3) Real World Verification – The Contractor shall conduct a minimum of one (1) field trial prior

to each Testing Window. The field trial shall utilize actual districts, district infrastructure, and students and be equivalent in scope and scale to that which is anticipated during the upcoming Testing Window. The Contractor shall provide all system components in order to capture test results that can be analyzed to determine in advance if the system has been sized and scoped appropriately for expected volume and load of any Assessment. The Contractor shall furnish the State with the results of all field trials, including those obtained from server-side infrastructure logs, performance counters, and other observations. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.17 is incorporated in this Contract by reference as though set forth verbatim.

(4) Secure Browser –

i. The Contractor may release one (1) Secure Browser per operating system per

school year and must do so a minimum of ninety (90) calendar days prior to the commencement of the Testing Window. The Contractor must seek written approval from the State to release more than one (1) Secure Browser each year or to release a Secure Browser within the ninety (90) calendar days prior to the commencement of the operational Testing Window.

ii. The Secure Browser must be compatible with and deployable using the mobile

37

device management (“MDM”) systems in use by Tennessee School districts, which include:

1) Mobile Guardian;

2) Lightspeed;

3) Airwatch;

4) Barracuda;

5) FileWave;

6) Jamf;

7) G-Suite Mobile management;

8) Intune; and

9) System Center Configuration Manager.

iii. The Contractor shall ensure the State is advised of any dependencies of the Secure Browser or other client-side software required by the CBT System, including software, drivers, or configurations that are not considered standard for the CBT operating system or operating system version targeted by the Secure Browser or other software.

iv. The Secure Browser for each platform must be compatible with the operating system

version and sub-version expected to be in use throughout the upcoming school year as determined by the approved matrix of Client-Side Technology and must include release documentation clearly describing any differences from the prior version, known issues, differences from other operating system platforms, known operating system version limitations, and other factors that may differentiate the student experience between operating system platforms.

v. The Secure Browser must implement test-security in accordance with the functional

requirements set forth in this Contract including detection of a student attempting to access an application outside of the testing platform, however, such detection must not include false-positives caused by legitimate background applications, such as; updates, anti-malware, or device management agents.

(5) General Fault Tolerance – The Contractor shall deliver a CBT System that is fault tolerant

by design and able to minimize negative student impact during unexpected events, such as; dropped network connections, broadband disruptions or other events typical in education environments across a diverse state. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Techn ica l Requirement C.19 is incorporated in this Contract by reference as though set forth verbatim.

(6) Root Cause Analysis - The Contractor shall provide a root-cause analysis of any technical or other error reported or observed during a Testing Window within two (2) business days of the occurrence.

(7) Dedicated System Resources – The Contractor shall dedicate server-side and other non-

public system resources to Tennessee such that events or actions of other customers or tenants of the same CBT System or hosting data center cannot impact a student’s testing experience in Tennessee. The Contractor shall provide evidence of this implementation in the form of an architectural diagram authenticated by a State approved third-party. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.21 is incorporated in this Contract by reference as though set forth verbatim.

38

(8) Cyber-Attack Prevention – The Contractor shall implement industry best practices for mitigating Denial of Service (“DOS”), Distributed Denial of Service (“DDOS”), Man in the Middle (“MITM”) or other similar events that can disrupt the student testing experience during an Testing Window, whether intentional, unintentional, internal, external, malicious, or benign in nature. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.22 is incorporated in this Contract by reference as though set forth verbatim.

(9) Access Control and User Logging – The Contractor shall implement industry best practices for access control to all public facing components of the CBT System to prevent unauthorized persons from accessing or viewing the test, administrator, or proctor components of the CBT System, including the impersonation of an authorized person. All system access shall be logged and include a timestamp, IP address, or other identifying marks commonly available for server-side through the HTTP request protocol.

(10) Capacity Planning – The Contractor shall conduct comprehensive capacity planning prior to each Testing Window to ensure that all system resources are appropriately sized and configured for the expected size and volume of that Testing Window.

Adequate system capacity shall be demonstrated through a real-world verification event as required by this Contract. The Contractor’s response to RFP ATTACHMENT

6.2. — SECTION C Techn ica l Requirement C.25 is incorporated in this Contract by reference as though set forth verbatim.

(11) Real Time Monitoring and Diagnostics – The Contractor shall implement a real time monitoring operation during all Testing Windows. Monitoring shall cover all components of the system used to deliver the CBT System including metrics related to volume, concurrency, security, bandwidth, resource utilization, and system performance. Monitoring shall be conducted by qualified personal located in a secure operations center and shall be supported by well-defined response and escalation protocols. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Techn ica l Requirement C.30 is incorporated in this Contract by reference as though set forth verbatim.

(12) Anti-Virus, Malware and End Point Protection – The Contractor shall implement or ensure implementation of industry best practices for anti-virus, anti-malware, and endpoint protection across all computing resources and vulnerable surface areas associated with the CBT System, whether managed by the Contractor, its subcontractors, or other service providers.

(13) Standard Operating Procedures and Change Controls

i. The Contractor shall define, implement and periodically verify standard operating procedures, protocols, and change control policies that cover all aspects of the people, processes, and technologies associated with the CBT System. Such policies shall include:

1) Software development lifecycle;

2) Quality Assurance;

3) Software deployment and environment management;

4) Production change controls;

5) Breech and incident response management;

6) Endpoint protection;

7) Physical security (building security);

8) HR policies;

39

9) Disaster recovery, backup, and business continuity;

10) Data retention and destruction;

11) Risk management; and

12) System maintenance and patching.

ii. The Contractor shall provide the State with copies of all standard operating procedures,

protocols, and change control policies related to the CBT System along with evidence of compliance in the form an approved audit and information on how these standard operating procedures, protocols, and change control policies are periodically verified. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirements C.27, C.31, C.32, and C.33 are incorporated in this Contract by reference as though set forth verbatim.

(14) Third- Party Organizations

i. The Contractor shall provide the State with a list of all third-party organizations, partners,

and service providers involved in the delivery of the CBT System, including the hosting and management of its computing resources and provision of human resources if applicable. The list shall include a brief overview of the organization, the physical location of human resources, technical facilities involved in the delivery of the CBT System, and an explanation of the role performed by that organization. Each third-party organizations, partners, or service providers utilized by the Contractor shall agree to the requirements of Section E.8. of this Contract. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.35 is incorporated in this Contract by reference as though set forth verbatim.

ii. The Contractor shall impose on all third-party organizations, partners, and service

providers involved in the delivery of the CBT System through legally enforceable vehicles the same obligations assumed by the Contractor under this Contract and obtain on behalf of the State evidence of compliance by those third- parties with those obligations.

iii. The Contractor shall be deemed the prime contractor for the purposes of this contract and shall assume all responsibility and liability to the State to the extent permitted under this Contract for the actions of third-party organizations, partners and service providers involved in the delivery of the CBT System.

(15) CBT System Developments, Customizations and Improvements

i. The Contractor shall not make any change to the CBT System design or

configuration during the Term of this Contract, including without limitation, software or infrastructure components, technologies, dependencies, or use of third-parties that might alter the performance, functionality, training, documentation, or user experience of the CBT System in any way without the written approval of the State.

ii. The Contractor shall ensure that any changes made to the CBT System for other

customers, whether those changes are to shared or non-shared components, cannot in any way impact or alter the performance, functionality, training, or documentation of user experience of the CBT System in Tennessee. The Contractor shall provide evidence of compliance with this requirement in the form of architectural documents, process documents, or other suitable means.

(16) Data Integration – The Contractor shall develop efficient and effective protocols for

the transference of data between the State and the Contractor for the purposes of registering students or communicating testing results. Data transference protocols shall include the ability to communicate additions, changes, or deletions to previously

40

transferred data. The Contractor shall provide tools to support verification of the completeness and accuracy of student data prior to the commencement of a Testing Window. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.36 is incorporated in this Contract by reference as though set forth verbatim.

(17) Maintenance and Planned Outages

i. The Contractor shall not plan any outage or schedule any maintenance at any

time including after-hours or on weekends during a Testing Window or one week prior to or following a Testing Window.

ii. The Contractor shall publish to the State a minimum of thirty (30) calendar days

in advance of any planned system outage or system maintenance. The publication shall include the purpose of the outage or maintenance, applicable release notes, and an overview of any risks and mitigations associated with the outage or maintenance.

(18) Notification – The Contractor shall notify the State immediately of any event, activity,

Service Anomaly, or other occurrence that may constitute a threat to the security or privacy of Tennessee State data, pose a risk to test security, or which has the potential to disrupt or invalidate any aspect of an Assessment or testing experience. Such notification shall occur immediately upon observation of the threat, risk, or potential risk regardless of the source of the observation and prior to any investigation or verification by the Contractor.

(19) State Agency Cooperation - The Contractor shall complete an internal controls questionnaire if requested by the State’s Office of the Comptroller.

(20) SOC 2 Audit - The Contractor shall ensure that a current SOC 2 audit is obtained for

all partner organizations or subcontractors utilized by the Contractor in the delivery of services to the State prior to the commencement of operational Testing Window for 2019-20.

A.10. Assessment Provisioning Activities

The Contractor shall provide Web-Based Applications as described below for use in platform and application testing and training, Assessment Administration, processing, or Reporting. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.41 is incorporated in this Contract by reference as though set forth verbatim. The Web-Based Applications shall provide the following functionality:

a. Authorized Use – The Contractor shall provide the means to assign and manage access (“Authorized Use”) to Web-Based Applications consistent with guidelines developed by the State as follows:

(1) Include a secure user hierarchy for use with all Web-Based Applications including,

but not limited to: State administrators, State users, State Reports only users, system level administrators, system level users, school level administrators, school level users, teachers, and students. The State shall have capacity to log on as an Authorized User to experience the CBT System the same as any other Authorized User. Access shall be provided to each level based upon guidelines provided by the State. Different Authorized Use rules may be applied to each Web-Based Application.

(2) Provide Integration capability and provision of user authentication/authorization/audit

services from the State identity platform. This is an identity brokerage, federation, and claims enrichment platform (using SAML 2.0) that the State operates to provide SSO (Single Sign On) and many other capabilities to end users, districts, and other State partners.

41

(3) Provide user management capabilities that allow State, System, and School administrators to manage all user accounts within their areas of authority.

b. Enroll Students – The Contractor shall provide the means to enroll students to participate in

the Assessment including a process for handling late additions or transfer students. The Contractor shall provide a user-friendly application to upload student information from schools, LEAs, the state student management system, or State contractors. Data may be used to pre-identify Student Response Documents or upload into the Contractor’s CBT delivery system as specified in Section A.9. of this Contract. The Contractor shall ensure that the application maintains compliance with the State’s student management system to enable the acquisition of student information from schools or systems. The Contractor agrees to collaborate with the State for the modification/revision/customization of the enrollment application as needed to meet State needs.

A.11. Assessment Processing Activities

The Contractor shall provide the State with processing activities for scanning, editing, Scoring, and Reporting of the TCAP Assessment results. The accurate, consistent and timely processing of Assessment results is a key component in maintaining the reliability, validity, and efficiency of the TCAP Assessment scores. The Contractor shall complete the processing activities for the TCAP Assessments according to timelines established in the Annual Work Plan required in Section A.4.a. (1) of this Contract.

The Contractor shall implement processing procedures that efficiently produce Scores that are reliable and valid for their intended interpretations and uses and in the time schedules specified in the Annual Work Plan. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technical Requirement C.42 is incorporated in this Contract by reference as though set forth verbatim. The Contractor shall provide procedures and support for all Assessments including without limitation the following:

a. Systems – The Contractor shall provide reliable systems to facilitate processing, Scoring, and

Reporting student results from each Assessment Administration. The systems shall include processes to verify the accuracy of data produced during each processing steps for each Administration, including the capability to disaggregate and aggregate the results.

b. Continuous Improvement – The Contractor shall collaborate with the State to evaluate test

processing and develop programs and services to improve test processing functions. The Contractor shall also recommend to the State how test processing procedures may be modified to improve productivity, efficiency, data turnaround times, or utility for Reporting and accountability.

c. Electronic Publications – The Contractor shall provide, at the request of the State, all contracted publications in such formats as are compatible with State systems or as otherwise directed by the State.

d. Scanning Specifications – For Quality Assurance and score verification purposes, a

comprehensive set of mock testing data that represents the full range of expected inputs must be generated and utilized in all aspects of processing, scanning, editing, Scoring, Reporting, and printing and must be completed and evaluated by the Contractor and the State at least six (6) weeks before the beginning of each Assessment Administration. The Contractor shall provide a written proposal for processing verification for review and Approval by the State at least twelve (12) weeks before Assessment Administration. The Quality Assurance process shall include a test deck for both computer and paper-based processing that has been through Quality Assurance and score verification for the State to use for comparison purposes. The Contractor shall provide programming to ensure that test deck information, including data from State-created test decks, is not included in operational data or Reports. A minimum of 150 mock response documents or cases per Content Area per Assessment will be coded to verify the following that:

42

(1) Each grid/selection area is properly scanned/received and recorded;

(2) Each response area is being scanned/received and mock response documents/cases for each version will be gridded to show each of the response choices is being scanned correctly;

(3) Student information including some from pre-identification files;

(4) All cases involving missing and incorrect information will be checked;

(5) All cases requiring editing are properly relayed to the State;

(6) Any other procedures to assure accuracy of data processing;

(7) Verification of absent, medical exempt, EL exempt, or nullified scores for processing;

(8) Scoring keys and Scoring programs are listed; and

(9) Records for students who were not tested are handled by the Contractor appropriately.

e. Data Files – The Contractor shall provide a data file for each Assessment Administration to the

State for production and quality control purposes. The data file shall meet the following criteria:

(1) Verify that Student Response Documents have been correctly Scored, and that all final Score values have been properly calculated (e.g., scale Scores and proficiency level);

(2) Monitor all aspects of scanning, editing, and Scoring throughout the time that the actual

response documents are being scanned, edited, Scored and processed until completion; and

(3) Develop and implement a data verification plan for each Administration that describes in

detail all steps to be implemented to ensure that the final Reports of results are accurate.

f. Quality - It is essential that each data file is provided accurately and timely, according to timelines as established and the Annual Work Plan as described in Section A.4.a.(1) of this Contract. If the Contractor fails to meet the deliverables as specified in the Annual Work Plan, it will be subject to the outcomes of the Score Card as outlined in Section A.14 of this Contract or the Liquidated Damages specified in Attachment B. of this Contract.

g. Data File Format – The Contractor shall submit scanned data and data generated through the

CBT System to the State or a State contractor in a file format as directed by the State that facilitates the generation of Reports required under this Contract.

h. Quality Assurance Data File – The Contractor shall support the State and its designees in

preliminary verification of Scoring and test equating using a Calibration sample of between 5,000-10,000 (not less than 20%) student records processed per Content Area per Assessment. The Contractor shall recommend the sampling specifications with a rationale subject to State approval of the final specifications. The Student Response Documents from these systems shall be Scored and all files and Reports identified will be generated by the Contractor. The Contractor shall verify the results after consultation with the State or the State’s designees.

i. Processing Plan – The Contractor shall provide to the State for review and written approval

a processing plan that includes the return of the Assessments to the Contractor for scanning, editing, Scoring, and Reporting. The Contractor shall implement the State approved processing plan. At a minimum, the processing plan shall provide details for CBT and PBT and shall include:

(1) Scoring methodology;

43

(2) Score Reporting;

(3) Generation and maintenance of Scoring data files;

(4) Quality Assurance procedures for each part of the Scoring process;

(5) The process for Scoring all student responses for multiple-choice;

(6) Details of the process of handling online responses for both CR and SR Items;

(7) The process for handling paper-based CR responses;

(8) The qualifications and training of key personnel involved in this process;

(9) A description of the software used to scan Student Response Documents;

(10) A verification of quality control procedures for each scanner;

(11) Quality reviews, to be monitored by the State, and logs/Reports regarding quality control procedures subject to the State’s review and approval; and

(12) Control procedures, subject to the State’s review and approval, for all Scoring

processes and products. These control procedures shall include, but not be limited to:

i. Steps to ensure that all Assessment Materials are correctly and reliably scored;

ii. Quality control review of all phases of production related to scanning Student

Response Documents;

iii. Verification of the Scoring program, editing, and resolution procedures for questionable response documents (for example, with multiple marks, poor erasures, poorly recorded oral responses or incomplete data); and

iv. Quality control review of all phases of production related to Reporting and

generating data files from the results.

A.12. Scoring Activities

The Contractor shall provide Scoring services for the TCAP Assessments. Accurate and consistent Scoring of responses is a key component in maintaining the reliability, validity, and consistency of TCAP Assessment scores. The Contractor shall maintain high levels of Scoring accuracy while meeting Scoring deadlines. The Contractor shall score the TCAP Assessments according to timelines established and the Annual Work Plan as described in Section A.4.a. (1) of this Contract. If the Contractor fails to meet the deliverables as specified in the Contract and the Annual Work Plan, the Contractor will be subject to the Liquidated Damages or fees at risk as outlined in the Score Card in Section A.14. of this Contract or as specified in Attachment B.

The Contractor shall implement Scoring processes that are reliable and valid as well as efficient in terms of time and expenditures. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C Technica l Requirement C.43 is incorporated in this Contract by reference as though set forth verbatim. The Contractor shall provide and implement Scoring procedures and support for all Assessments including but not limited to the following.

a. Scoring – The Contractor is responsible for Scoring all operational and field-tested Items in

a manner appropriate for the Item type and Administration mode. The Contractor shall implement Scoring procedures, quality control procedures and criteria, and Quality Assurance procedures and criteria that ensure the State receives Scores suitable for the State’s intended purposes.

44

b. Scoring of All Test Versions – The Contractor shall provide programming and support for Scoring of all test versions including computer- and paper-based, Large Print, and Braille.

c. Re-Scoring of Student Responses – The Contractor shall maintain sufficient capacity to facilitate Re-Scoring of response documents if necessary. Procedures for Re-Scoring Student Response Documents shall be developed in collaboration with and approved by the State.

d. Scoring Programs and Technical Information – The Contractor shall provide quality-

assured, production, and operational programs for Scoring services including operational test decks and response keys. The Contractor shall also provide quality-assured technical information required to provide and facilitate onsite and off-site test processing for Scoring. These services shall include, but not be limited to, the following:

(1) Scoring –

i. Using a methodology technically appropriate and approved by the State to produce

a single Scale Score for each student per grade level or Content Area tested.

ii. Providing Assessment data in various formats that will be used to determine various scores, data distributions, or summaries to support the Reports required under this Contract.

iii. Verifying that Student Response Documents are correctly Scored. A test deck as

defined in Section A.11.d. of this Contract shall be provided prior to each Administration for each Assessment that has been through Quality Assurance for the State to use for comparison purposes.

(2) Data Gather –

i. The Contractor shall collect all individual data through student data uploads into CBT

System or from the response document to include without limitation, Item responses and scores, including number of attempts, login information, length of time tested online, and all other fields. The Contractor shall provide a full description of all data captured by its CBT System.

ii. The Contractor shall gather information regarding test security procedures and

safeguards including but not limited to; serial numbered booklets, confidentiality agreement forms, erasure data, key/re-key information, etc.

iii. The Contractor shall provide a Comprehensive Data File (“CDF”) per Assessment and

provide it to the State through a secure mechanism. The CDF shall include all data from the most recent Administration. A final CDF for each Administration shall be provided to the State. This information shall also be provided to facilitate TVAAS information. The Contractor’s proposed format must be approved by the State and must meet State needs or systems, including the facilitation of required data disaggregation.

(3) Document Retention –

i. The Contractor’s Test Review Procedures shall include confirmation of a below

proficient Score and Re-Scoring under certain conditions.

ii. Test documents, Scoring keys, and student data in the possession of the Contractor shall be retained in a secure electronic format on secure servers for the duration of the Contract and for twelve (12) months after the Contract End Date.

e. Scoring Personnel, Methodology, Tools, and Procedures –The Contractor shall establish an

efficient and accurate Scoring methodology for CR Assessments for Prompt-based texts,

45

writing, mathematic tasks requiring explanation or modeling, or other activities that require students to provide written work. The methodology shall include Computer-Based responses or scanned paper-based responses scored using automated Scoring technology, human Scoring procedures (as detailed in the requirements below), or a hybrid model combining human and automated Scoring. The Contractor shall implement Scoring processes that are reliable and valid as well as efficient in terms of time and expenditures. The Contractor shall also provide Scoring procedures for all CR Assessments including but not limited to the following:

(1) Transition from human to automated Scoring during the Term of this Contract. The Contractor shall implement a plan, which includes benchmarks to be met in order for the State to proceed in adopting automated Scoring. The Contractor shall indicate the software to be used to support automated Scoring, evidence regarding the automated Scoring’s strengths and limitations, and evidence that the Contractor will be able to implement automated Scoring at operational scale with at least similar quality and efficiency as shown with human scorers.

(2) Develop and implement a scoring plan (the “Scoring Plan”). The Contractor shall

implement a Scoring Plan that accurately and efficiently Scores the student responses to CR Items. The Scoring Plan shall include Blind Scoring, how student responses will be assigned a score according to the Scoring guides, rubrics, and other materials provided by the Test Development Contractor and approved by the State. The Scoring Plan shall provide specific procedures and criteria for ensuring accurate scores, and for meeting State requirements for handling special cases (e.g., alert papers). The Scoring Plan shall provide examples of tools proposed to support Scoring, such as software for managing quality control measures and Reporting scorer performance. The Contractor shall incorporate the approved Scoring Plan in the Annual Work Plan.

(3) Assemble qualified personnel to oversee and conduct the Scoring. The Respondent shall provide to the State a list of Scoring personnel, including their responsibilities and qualifications. The State has the right to approve all key personnel proposed for Scoring including staff tasked with overseeing or managing the Scoring process.

(4) Establish materials to support accurate Scoring, including identifying and assembling anchor, qualifying, training sets, and validity papers as follows:

i. The Contractor shall provide the State with Anchor Papers (including Anchor

Text) for each Assessment. ii. The Contractor shall conduct range finding meetings for the purpose of

establishing anchor and qualifying sets to be used in Scoring the TCAP Assessments. The Contractor shall implement a process for conducting range finding, which shall include:

1) The State will provide Tennessee educators and expert consultants to

participate in the meetings. 2) The range finding meetings shall take place in a location mutually agreed to

by the State and Contractor. Dates for the meetings shall be established in the Annual Work Plan as described in Section A.4.a. (1). of this Contract.

3) The Contractor shall be responsible for all meeting expenses as outlined in

the table in Section A.4.j. of this Contract.

(5) The range finding meetings shall establish the anchor and qualifying and trainings sets to be used in Scoring the TCAP Assessments. Anchor, qualifying, and validity training sets shall not be changed or substituted without approval of the State. These sets are defined as:

1) Anchor set: a minimum of four student responses for each Score point at

46

each grade level/cluster. The anchor set shall consist of the best examples available for each Score point. There shall be one anchor set per Assessment.

2) Pre-qualifying Training set: one set consisting of at least 12 student

responses that cover the full range of Score points for each grade level/cluster, including “borderline” papers that represent common but difficult to score distinctions. The Pre-qualifying Training set of papers shall be used to train potential scorers.

3) Qualifying set: one set consisting of at least 12 student responses that cover

the full range of Score points for each grade level/cluster, including “borderline” papers that represent common but difficult to Score distinctions. The Qualifying sets of papers will be used in identifying persons who qualify to Score operationally.

4) Training set: one set consisting of at least 12 student responses that cover

the full range of Score points for each grade level/cluster. Training sets shall be developed by the Contractor and State staff as needed during live Scoring. Training sets shall be used for Calibration and Recalibration in working with readers during the Scoring process.

5) Validity set: one set consisting of at least 12 student responses that

cover the full range of Score points for each grade level/cluster, and which are designed to check whether scorers are Scoring accurately (e.g., if the most common student response from range-finding is a level 2, then the validity set would include several papers at the high 1, low 2, clearly 2, high 2, and low 3 levels). Validity papers are used to provide evidence that scorers are Scoring accurately consistently with State standards. The Contractor and State shall add to the validity set, during live Scoring if need be, such that the Contractor has enough papers to provide sufficient opportunities to check the accuracy of Scoring over the course of Scoring a particular task or Prompt.

6) The Contractor shall provide sufficient numbers of copies of the anchor, qualifying, trainings, or validity sets to be used in training, qualifying, or Scoring the TCAP Assessments.

(6) Conduct human scorer training using the established anchor, qualifying, and training

sets as defined in section A.12.e.(4). The Contractor shall implement a plan for conducting human scorer training, which shall include:

i. Training according to timelines established in Section A.4. of this Contract and the

Annual Work Plan required under Section A.4.a. (1) of this Contract. All costs associated with training shall be at the Contractor’s expense.

ii. The opportunity of State staff and expert consultants to be present during team

leader/human scorer training and the beginning of Scoring the TCAP Assessments using guidelines established collaboratively between the State and the Contractor.

(7) The Contractor shall implement a plan for managing and monitoring the effectiveness

of Scoring. This will be stipulated in the Annual Work Plan described in Section A.4.a.(1) of the Contract. In this regard, the Annual Work Plan shall include:

1) Definition of roles and responsibilities of personnel; 2) Flow of student essay papers/portfolios through the Scoring process; 3) Scoring methodology, use of human scorers, and Automated Scoring

technology;

47

4) Assignment of Scores to papers/portfolios; 5) Resolution of Scoring problems; 6) Retraining of human scorers; and 7) Monitoring and maintaining reliability of Scoring and quality control

throughout the process.

(8) The Contractor shall establish an efficient and accurate Scoring methodology acceptable to the State. The finally agreed upon Scoring method may include scanning student paper-based response documents and using Automated Scoring technology, provided the State approves of this Scoring method in writing. The Scoring method may include human Scoring as detailed in this section or a hybrid model which combines Automated Scoring with human scoring. Scores from the Automated Scoring technology shall be considered a scorer for monitoring and Reporting purposes.

(9) During TCAP Scoring, the Contractor shall produce daily and cumulative Scoring Reports to the State.

(10) The Contractor shall provide the State a copy of all student papers about which questions have been raised. This includes papers with anomalous Scores, papers that are incapable of being Scored, compositions that are considered “Alert” responses as specified in Section A.12.e.(2) of this Contract, and compositions that indicate the potential of administrator interference. Additional pages of student written work that accompany the answer document will not be Scored.

(11) In collaboration with the State, the Contractor shall develop a process of identifying responses that were neither Scored or capable of being Scored (e.g., non-English, off-topic, blank) and a system of coding these responses. In addition, the Contractor will develop a system for flagging alert status papers that indicate a serious or potentially serious problem for the writer (e.g., physical/sexual abuse, drug use, gang involvement, running away, suicide, severe anti-social tendencies, etc.). When such a paper is identified, the Contractor must transmit it immediately to the State, along with a means of identifying the student and system (e.g., LEA or school). The State will then notify the appropriate personnel. These papers, as well as all others, shall be treated as strictly confidential.

A.13. Assessment Reporting Activities

The Contractor shall accurately produce, prepare, develop, complete, post, and deliver all Assessment Reports in accordance to the terms of this Contract. The Contractor shall provide customized Reports as outlined in RFP Assessment Reporting Appendix 6.6.A. in paper, PDF, permanent storage device, and online format for each Assessment as directed by the State. Unless otherwise specified in this Contract or the Annual Work Plan, all Assessment Reports required by this Contract are due by 9 a.m. CT on the date outlined in RFP Assessment Reporting Appendix 6.6.A. If any due date falls on a non-business day (i.e., closed for State business, weekend, or state holiday), the Assessment Report is due on the following business day. The Contractor shall utilize a single, unique, and accurate Student ID Number and the student’s current school and LEA unique code identification number assigned to the student at the time of testing if required in an Assessment Report. Assessment Reports shall be separately designed and developed for each Assessment or each Test Year. The Contractor shall develop a new Report format and style that is satisfactory to the State. The Report format, including layout, data, and text, shall require final Approval by the State. The current reports are available at https://www.tn.gov/education/assessment/tnready/tnready-score-reports.html. The Contractor’s response to RFP ATTACHMENT 6.2. — SECTION C T echn ica l Requirement C.44 is incorporated in this Contract by reference as though set forth verbatim.

https://www.tn.gov/education/assessment/tnready/tnready-score-reports.html

48

At the State’s request, the Contractor shall implement a solution that provides flexible, dynamic Computer-Based Assessment Reporting capability at multiple levels. All Assessment Reports and Assessment Reporting resources shall be submitted to the State for review and approval prior to distribution.

a. The Contractor shall provide technical training, support, required programming, and software,

including installation and upgrading of software and programming, to the State necessary to furnish Assessment Reports. The software shall produce Assessment Reports in an easily updated electronic format. The Contractor shall provide written assurance of confidentiality and appropriate security measures to ensure that all student data contained in Assessment Reports is protected. All data security requirements as outlined in Sections A.5 and A.9.c. (1) and (2) of this Contract shall be met for all Assessment Reports.

b. The Contractor’s website shall include training and tools to ensure effective usage of the data provided by the system in providing instructional improvement. Training must include workshop level webinars or videos that promote data learning, create data friendly environments for educators and other users, and focus on all levels of users including teachers, administrators, policy makers and parents.

c. The Contractor shall Score and include in the applicable Assessment Report student Scores from late documents. These late documents, submitted after deadlines established for summary data aggregation, shall be processed by the Contractor. The Contractor shall provide final Assessment Reports in digital format available online and in paper format. Student data shall be included in ad hoc Assessment Reports but shall not be included in summary Assessment Reports.

d. The Contractor shall be responsible for shipping all printed Assessment Reports and labels with parent brochures to each LEA, State Special School, private school, and to the State. Assessment Reports for home-schooled students shall be sent to the State Home School Coordinator. Printed Assessment Reports shall be organized and packaged according to State specifications.

e. The Contractor shall be responsible for compliance with the confidentiality provisions of the

Family Education Rights and Privacy Act (FERPA), 20 USC 1232g, 34 CFR 99, Section E, the National School Lunch Act 42 USC 1758 (b) (2) (c), Chapter 905 of the Tennessee Public Acts of 2014 and any other State or federal laws, rules, or policies intended to protect individual privacy in all cases. The data protected by State and federal law shall only be included in data files delivered or made available to the State.

f. The Contractor shall provide a Web-Based Assessment Reporting and Assessment Report

delivery system. The system shall meet the same specifications and support requirements of other online products as defined in Sections A.9.c. (1) and (2) of this Contract. Assessment Reports shall be designed to meet State and federal Reporting requirements and may be revised as needed after each Administration.

g. The Web-Based Assessment Reporting and Assessment Report delivery system shall

include the following:

(1) An online help section that provides Authorized Users with a searchable database that can be used to answer most usability and technology questions related to the Assessment Reporting system. The customer support phone numbers and email addresses shall be posted in the help section. The help section shall include: comprehensive guide to test interpretation (“GTI”), tutorials, user’s guide, training materials, a quick reference guide, frequently asked questions, and links to the State and other reference websites;

(2) Access to Assessment Reports submitted to the State by the Contractor for the entire Contract term;

(3) An administrative section that will allow Authorized Users to manage all Authorized

49

User accounts within their areas of authority or jurisdiction. The administrative section shall provide security measures to include Authorized User hierarchy as defined in Section A.10.a.(1) of this Contract. The administrative section shall include Authorized User Reports to indicate when or if Authorized Users have logged on and what Assessment Reports they have viewed, downloaded, or created;

(4) Access to teacher Authorized Users to view appropriate Assessment Reports starting with the first Reporting window in February 2020;

(5) A downloads section that will allow Authorized Users to download specified data and Assessment Reports to local storage based upon Authorized User security level;

(6) A notification section that will allow the Contractor and the State to post information concerning the Assessment Reports for Authorized Users to view or print. The State may issue an embargo on certain data until its official release by the State. This notice and others may be posted on the State’s web-site or such other location as determined by the State;

(7) Compatibility with Windows and Macintosh applications. The Contractor shall provide the State with technical specifications required for any Web-Based Applications that comport with the State’s minimum system requirements in Section A.9.cof this Contract;

(8) As requested by the State, interactive ad hoc Assessment Reports customized to allow teacher and administrator users to select data based upon various criteria (including all student groups or demographic data described in this Contract) and create custom Assessment Reports for use in activities including but not limited to: re-rostering for future classes, student intervention, school improvement plans, and federal Reporting requirements. The Contractor shall implement solutions that feature best in class dynamic, Authorized User defined tables, and graphic displays. All data captured through any component of this Contract shall be made available for educators to ensure improved instruction including, but not limited to, use of accessibility features and accommodations, time on task, and click analysis. Each customized Report, as it is viewed on the screen, shall be capable of rendering in a PDF format through a print function. The customized Report may be printed in hard copy or downloaded to the desktop or permanent storage device;

(9) Each customized Assessment Report, as it is viewed on the screen, shall be capable of rendering in a PDF format through a print function. The customized Assessment Report must be capable of being printed in hard copy or downloaded locally;

(10) As requested by the State, a parent portal designed to provide parents with access to the individual student Reports including, but not limited to: student responses, essays, test Items with key information - including response annotations, scale score, performance information, growth data, previous test results, and any other resources or data currently available. The parent portal shall provide secure authentication for parent access in a standalone manner for LEA’s that do not have a separate parent portal and shall provide integration with the parent portal in the district Student Information System (“SIS”) where available, such that a parent who has authenticated to the SIS parent portal can access content in the Assessment parent portal without the need to reauthenticate. The State may request additional customization to the functionality and Report requirements available in the parent portal, as well as additional resources for users at no additional expense to the State. The following SIS are in use in Tennessee; Skyward, Aspen (Follett), PowerSchool, iNOW (Chalkable / Powerschool), Infinite Campus, and EduPoint.

(11) The Web-Based Assessment Reporting system shall maintain an archive of data available for access by all users from previous Administrations for the life of the Contract; and

(12) The Web-Based Assessment Reporting system archive shall remain available until

50

the end of the test year following the end of the Contract with limited technical support.

h. Assessment Report designs shall be reviewed by the State and the Contractor at a

minimum of once per test year. The Contractor shall make any required changes to Assessment Report designs to meet State needs or federal guidelines.

i. At the State’s request, the Contractor shall conduct focus group workshops with families, educators, and districts specific to Reporting needs and understanding at all levels of Reporting. The State shall participate in all workgroups and discussions regarding planning, reviewing. and obtaining stakeholder feedback to ensure Reports are clear and easily understood.

j. The Contractor shall provide customized student, teacher, class, school, LEA, or State level

Assessment Reports that adhere to the requirements in this Section or as otherwise required under the terms of this Contract.

k. The Contractor shall include certain common information on all Student and Score Reports

including but not limited to: Student or Score Report name/type, time of Assessment Administration (e.g., fall 2019) or Assessment date, Assessment name, Content Area, Assessment form, including modified version if applicable, teacher name, school name and system name.

l. The data from CBT or PBT test-takers shall be aggregated together at the class, teacher,

school, system, or state levels for operational online tests. The Contractor shall encrypt the data to ensure security of the Assessment Reports and all student information entered through the CBT System. The encryption key shall be shared with the State.

m. The Contractor shall provide a layout, mock-up, and sample for all files and Assessment

Reports for State approval prior to the posting or delivery of files or Assessment Reports to the districts. The timeline for these data files or Assessment Reports shall be established in the Annual Work Plan. Where applicable, the Contractor shall work with the State to define data elements and field lengths for all files or Assessment Reports. Prior to submission to the State, the Contractor is responsible for checking to ensure that all files or Assessment Reports are consistent and accurately reflect the data provided on any Assessment Reports required under this Contract.

n. The Contractor shall verify the accuracy and consistency of all data on data files and

Assessment Reports before submission to the State. Towards that end, the Contractor shall ensure that:

(1) All students tested are included; (2) All fields are accurate reflections of the test answer documents; (3) Braille and Large Print student records are coded correctly; (4) All data is included accurately in the printed Reports and data files; (5) All absentee, exempt (medically or EL), nullified, or refusal/did not attempt records are

included appropriately in the data; and the consistency and accuracy of the data files are independently verified. Any discrepancies are to be immediately adjusted by the Contractor, at the Contractor’s expense, to the State’s satisfaction.

o. Student Reports, the Parent Brochure, and Student Labels must be produced in paper and

electronically (e.g. pdf available for download online). All other Assessment Reports shall only be available in electronic format.

p. The Contractor shall produce a comprehensive guide to test interpretation (“GTI”) to describe

Assessment Report forms used for each Assessment, similar to the report found at https://www.tn.gov/content/dam/tn/education/testing/tnready/tnready_guide_test_interpretat

https://www.tn.gov/content/dam/tn/education/testing/tnready/tnready_guide_test_interpretation_eoc_2017-18.pdf

51

ion_eoc_2017-18.pdf, which is incorporated in this Contract by reference as though set forth verbatim, which is incorporated into this Contract by reference as though set forth verbatim. The GTI shall describe and provide examples of the various Report forms distributed to students, schools, and systems. It shall contain supportive information related to interpreting the test results, including but not limited to: Reporting categories assessed and definitions for technical Assessment terms. The comprehensive guide shall be developed for use by schools and systems and shall be posted on the State’s website. From the comprehensive GTI, a smaller Parent Brochure shall contain information pertinent to Student Reports. The Parent Brochure shall be translated into Spanish, Arabic, or any other language that the State deems necessary for distribution to parents of EL or LEP students. The Parent Brochure shall be distributed with the printed test Scores and posted on the State’s website. The GTI and the Parent Brochure shall: (1) be available in an electronic format that is accessible though the State’s website; (2) include images of Reports as directed by the State; and (3) be available in print format, one for each Student Report produced per Content area per Assessment.

q. The Contractor shall:

(1) Provide assistance to the State and LEAs on the interpretation

and use of summary test results for Assessment Program evaluation and accountability;

(2) Provide all thumbnails, graphics, and other content to the

State for use in training materials in original format;

(3) Post guides in the help section of the online Assessment Reporting system; (4) Provide comprehensive guides as defined in section A.13. that

are clear and easily understood by students, parents, teachers, and administrators;

(5) Develop, print, and distribute Parent Brochures for interpretation of individual Score Reports; and

(6) Work in consultation with other contractors engaged by the State as needed to ensure accurate Reporting.

A.14. Score Card

The State shall utilize a Score Card and Key Performance Indicators (“KPIs”) to measure the Contractor’s performance against expected outcomes. The Score Card and KPIs shall be utilized to evaluate the Contractor’s performance for the purpose of determining the percentage to be paid (payment rate) to the Contractor as set forth in Section C.3. of this Contract. The State reserves the right to exercise any other rights available to it under this Contract or at law or equity in lieu of reductions under this Section, provided that if the State reduces a payment percentage based on this Section A.14. for a delay, outage, error, failure to deliver, or other missed deliverable, the State will not also impose Liquidated Damages for the same delay, outage, error, failure to deliver, or other missed deliverable. Notwithstanding the foregoing it is the intent of the Parties that the decision whether to reduce Contractor fees under Section A.14. of this Contract or to impose Liquidated Damages under Attachment B of this Contract for those events involving the same event, facts, or circumstances is within the sole discretion of the State. The State shall give notice to the Contractor of the results of the Scorecard on or before the fifteenth (15

th) day following the

end of each calendar year during the term (e.g. if the calendar year ends June 30 the State will give notices to the Contractor by July 15

th, etc.) The

parties agree to an annual true-up of the compensation due to the Contractor in Section C.3. of this Contract within thirty (30) days of the notices of Scorecard results, whereby the Contractor’s compensation shall

https://www.tn.gov/content/dam/tn/education/testing/tnready/tnready_guide_test_interpretation_eoc_2017-18.pdf

52

be adjusted in accordance with the Scorecard results.

KPI Issue/Metric Payment

Rate Documentation

Method Scope

Reference

Technical Support Service

Average Wait Times for phone calls and live chat requests are less than two (2) minutes during Call Center hours, and

Maximum Wait Times for phone calls and live chat requests are less than five (5) minutes during Call center hours, and

100% of emails and voicemails sent to and left with the Call Center are responded to within one (1) working day of receipt outside of Testing Windows, and,

100% of emails and voicemails sent to and left with the Call Center are responded to within two (2) hours of receipt during a Testing Window, and,

95% of issues reported to the Call Center are resolved within three (3) business days of Receipt outside of Testing Windows, and,

95% of issues reported to the Call Center are resolved within twenty-four (24) hours during a Testing Window, and

95% customer satisfaction as measured by a response of “Agree” or “Strongly Agree” to positively worded statements/questions on the customer service survey provided by the Contractor.

1.0

Customer Service Logs

Vendor Summary

Reports

A.4.i

Average Wait Times for phone calls and live chat requests are less than five (5) minutes during Call Center hours, and

Maximum Wait Times for phone calls and live chat requests are less than ten (10) minutes during Call center hours, and

100% of emails and voicemails sent to and left with the Call Center are responded to within two (2) business days of receipt outside of Testing Windows, and

100% of emails and voicemails sent to and left with the Call Center are responded to within four (4) hours of receipt during a Testing Window, and

85% of issues reported to the Call Center are resolved within three (3) business days of receipt outside of Testing Windows, and,

85% of issues reported to the Call Center are resolved within twenty-four (24) hours during a Testing Window, and

80% customer satisfaction as measured by a response of “Agree” or “Strongly Agree” to positively worded statements/questions on the customer service survey provided by the Contractor.

0.5

Average Wait Times for phone calls and live chat requests are greater than or equal to

0.0

53


Rate Documentation

Method Scope

Reference

five (5) minutes during Call Center hours, or

Maximum Wait Times for phone calls and live chat requests are greater than or equal to ten (10) minutes during Call center hours, or

Not all emails and voicemails sent to and left with the Call Center are responded to within two (2) business days of receipt, or

Less than 85% of issues reported to the Call Center are resolved within three (3) business days of receipt.

Less than 80% customer satisfaction as measured by a response of “Agree” or “Strongly Agree” to positively worded statements/questions on the customer service survey provided by the Contractor

Quality of Computer-Based Testing (CBT) Experience

Zero (0) Service Anomalies 1.0


State Technology Directors’ Open Line Call Logs

State

Assessment Emails

A.9.d

No more than one Service Anomaly that lasts less than thirty (30) minutes 0.8

No more than two Service Anomalies that last less than thirty minutes each, or

No more than one (1) Service Anomaly that lasts greater than or equal to thirty (30) minutes but less than sixty (60) minutes.

0.6

More than two (2) Service Anomalies, or Any one (1) Service Anomaly that lasts

greater than or equal to sixty (60) minutes. 0.0

Quality of PBT Experience

All initial orders of paper Test Materials are delivered to all sites in accordance with the agreed-upon dates in the Annual Work Plan, and

All paper Test Materials are in good condition: more than 95% of printed Test Materials are free of errors and the shipped materials are not significantly damaged.

1.0

Annual Work Plan


State

Assessment Emails

A.4.a.(1) A.8.g

All initial orders of paper Test Materials are delivered to all sites within two (2) business days of the agreed-upon dates in the Annual Work Plan, and

All paper Test Materials are in good condition: more than 95% of printed Test Materials are free of errors and the shipped materials are not significantly damaged.

0.8

Some initial orders of paper Test Materials are delivered to all sites more than two (2) business days after the agreed-upon dates in the Annual Work Plan, or

Some paper Test Materials are in poor condition: more than 5% of printed Test Materials have printing errors or the shipped materials are

0.4

54


Rate Documentation

Method Scope

Reference

significantly damaged.

Timeliness

95% or more of the Contractor’s deliverables in the Annual Work Plan are completed on or before the agreed-upon dates.

1.0 Annual Work Plan

Online Tracking

Logs

State Assessment

Emails

A.4.a.(1) and (2)

Less than 95% but 85% or more of the Contractor’s deliverables in the Annual Work Plan are completed on or before the agreed-upon dates.

0.8

Less than 85% of the Contractor’s deliverables in the Annual Work Plan are completed on or before the agreed-upon dates.

0.4

Reports

All Reports are delivered on or before the agreed-upon dates in the Contract, and

All Reports contain accurate Scale Scores and performance level classifications in the student records such that there is no need to recalculate or republish test results that have been publicly reported.

1.0

Contract

Cumulative Data Files (CDFs)

provided by the Contractor

Feedback from

Districts

A.13.f and j

At least 90% of the Reports are delivered on or before the agreed-upon dates in the Contract, and

At least 90% of Reports contain accurate Scale Scores and performance level classifications in the student records such that there is no need to recalculate or republish test results that have been publicly reported.

0.8

Less than 90% of the Reports are delivered on or before the agreed-upon dates in the Contract, and

Less than 90% of Reports contain accurate Scale Scores and performance level classifications in the student records such that there is no need to recalculate or republish test results that have been publicly reported.

0.6

A.15. Warranty. Contractor represents and warrants that the term of the warranty (“Warranty Period”) shall be the greater of the Term of this Contract or any other warranty generally offered by Contractor, its suppliers, or manufacturers to customers of its goods, services, or deliverables. The goods, services, or deliverables provided under this Contract shall conform to the terms and conditions of this Contract throughout the Warranty Period. Any nonconformance of the goods, services, or deliverables to the terms and conditions of this Contract shall constitute a “Defect” and shall be considered “Defective.” If Contractor receives notice of a Defect during the Warranty Period, then Contractor shall correct the Defect, at no additional charge.

Contractor represents and warrants that the State is authorized to

55

possess and use all equipment, materials, software, and deliverables provided under this Contract.

Contractor represents and warrants that all goods, services, or deliverables provided under this Contract shall be provided in a timely and professional manner, by qualified and skilled individuals, and in conformity with standards generally accepted in Contractor’s industry.

If Contractor fails to provide the goods, services, or deliverables as warranted, then Contractor will re-provide the goods, services, or deliverables at no additional charge. If Contractor is unable or unwilling to re-provide the goods, services, or deliverables as warranted, then the State shall be entitled to recover the fees paid to Contractor for the Defective goods, services, or deliverables. Any exercise of the State’s rights under this Section shall not prejudice the State’s rights to seek any other remedies available under this Contract or applicable law.

A.16. Inspection and Acceptance. The State shall have the right to inspect all

goods, services, or deliverables provided by Contractor under this Contract.

If, upon inspection, the State determines that the goods, services, or

deliverables are Defective, the State shall notify Contractor, and Contractor

shall re-deliver the goods or provide the services at no additional cost to the

State. If after a period of thirty (30) calendar days following delivery of goods

or performance of services the State does not provide a notice of any Defects,

the goods, services, or deliverables shall be deemed to have been accepted

by the State.

B. TERM OF CONTRACT:

B.1. Term of Contract. This Contract shall be effective on DATE (“Effective

Date”) and ending on DATE (“Term”). The State shall have no obligation for goods, services, or deliverables provided by the Contractor prior to the Effective Date.

B.2. Renewal Options. This Contract may be renewed upon satisfactory

completion of the Term. The State reserves the right to execute up to three (3) renewal options under the same terms and conditions for a period not to exceed twelve (12) months each by the State, at the State's sole option. In no event, however, shall the maximum Term, including all renewals or extensions, exceed a total of sixty (60) months.

B.3. Term Extension. The State may extend the Term an additional period of

time, not to exceed one hundred-eighty (180) calendar days beyond the expiration date of this Contract, under the same terms and conditions, at the State’s sole option. In no event, however, shall the maximum Term, including all renewals or extensions, exceed a total of sixty (60) months.

C. PAYMENT TERMS AND CONDITIONS:

C.1. Maximum Liability. In no event shall the maximum liability of the State under this Contract

exceed Written Dollar Amount ($Number) (“Maximum Liability”). This maximum liability amount is subject to A.14. of this Contract. This Contract does not grant the Contractor any exclusive rights. The State does not guarantee that it will buy any minimum quantity of goods, services, or deliverables under this Contract. Subject to the terms and conditions of this Contract, the Contractor will only be paid for goods, services, or deliverables provided under this Contract after a purchase order is issued to Contractor by the State or as otherwise specified by this Contract.

C.2. Compensation Firm. The payment methodology in Section C.3. of this Contract shall constitute

56

the entire compensation due the Contractor for all goods, services, or deliverables provided under this Contract regardless of the difficulty, materials or equipment required. The payment methodology includes all applicable taxes, fees, overhead, and all other direct and indirect costs incurred or to be incurred by the Contractor.

C.3. Payment Methodology. The Contractor shall be compensated based on the payment

methodology for goods, services, or deliverables authorized by the State in a total amount as set forth in Section C.1.

a. The Contractor’s compensation shall be contingent upon the satisfactory provision

of goods, services, or deliverables as set forth in Section A.

b. The Contractor shall be compensated based upon the following payment methodology:

Goods, services, or deliverables Description Amount

(per compensable increment)

TBD

C.4. Travel Compensation. The Contractor shall not be compensated or reimbursed for travel

time, travel expenses, meals, or lodging.

C.5. Invoice Requirements. The Contractor shall invoice the State only for goods delivered and accepted by the State or services satisfactorily provided at the amounts stipulated in Section C.3., above. Contractor shall submit invoices and necessary supporting documentation, no more frequently than once a month and no later than thirty (30) calendar days after goods or services have been provided to the following address:

Tennessee Department of Education Division of Data and Research Andrew Johnson Tower – 10

th Floor

710 James Robertson Parkway Nashville, TN 37243 615.741.0720

a. Each invoice, on Contractor’s letterhead, shall clearly and accurately detail all of the following information (calculations must be extended and totaled correctly):

(1) Invoice number (assigned by the Contractor); (2) Invoice date; (3) Contract number (assigned by the State); (4) Customer account name: Tennessee Department of Education, Division of

Assessment, Accountability, and Data Governance

(5) Customer account number (assigned by the Contractor to the above-referenced Customer);

(6) Contractor name; (7) Contractor Tennessee Edison registration ID number; (8) Contractor contact for invoice questions (name, phone, or email); (9) Contractor remittance address; (10) Description of delivered goods, services, or deliverables provided and invoiced,

including identifying information as applicable; (11) Number of delivered or completed units, increments, hours, or days as applicable, of

each good or service invoiced; (12) Applicable payment methodology (as stipulated in Section C.3.) of each good or

service invoiced; (13) Amount due for each compensable unit of good or service; and

57

(14) Total amount due for the invoice period.

b. Contractor’s invoices shall:

(1) Only include charges for goods delivered or services provided as described in

Section A and in accordance with payment terms and conditions set forth in Section C;

(2) Only be submitted for goods delivered or services completed and shall not include any charge for future goods to be delivered or services to be performed;

(3) Not include Contractor’s taxes, which includes without limitation Contractor’s sales and use tax, excise taxes, franchise taxes, real or personal property taxes, or income taxes; and

(4) Include shipping or delivery charges only as authorized in this Contract.

c. The timeframe for payment (or any discounts) begins only when the State is in

receipt of an invoice that meets the minimum requirements of this Section C.5. C.6. Payment of Invoice. A payment by the State shall not prejudice the State's right to object to or

question any payment, invoice, or other matter. A payment by the State shall not be construed as acceptance of goods delivered, any part of the services provided, or as approval of any amount invoiced.

C.7. Invoice Reductions. The Contractor's invoice shall be subject to reduction for amounts included

in any invoice or payment that is determined by the State, on the basis of audits conducted in accordance with the terms of this Contract, to not constitute proper compensation for goods delivered or services provided.

C.8. Deductions. The State reserves the right to deduct from amounts, which are or shall become due

and payable to the Contractor under this or any contract between the Contractor and the State of Tennessee, any amounts that are or shall become due and payable to the State of Tennessee by the Contractor.

C.9. Prerequisite Documentation. The Contractor shall not invoice the State under this Contract until

the State has received the following, properly completed documentation.

a. The Contractor shall complete, sign, and present to the State the "Authorization Agreement for Automatic Deposit Form" provided by the State. By doing so, the Contractor acknowledges and agrees that, once this form is received by the State, payments to the Contractor, under this or any other contract the Contractor has with the State of Tennessee, may be made by ACH; and

b. The Contractor shall complete, sign, and return to the State the State-provided W-9 form.

The taxpayer identification number on the W -9 form must be the same as the Contractor's Federal Employer Identification Number or Social Security Number referenced in the Contractor’s Edison registration information.

D. MANDATORY TERMS AND CONDITIONS:

D.1. Required Approvals. The State is not bound by this Contract until it is duly approved by the

Parties and all appropriate State officials in accordance with applicable Tennessee laws and regulations. Depending upon the specifics of this Contract, this may include approvals by the Commissioner of Finance and Administration, the Commissioner of Human Resources, the Comptroller of the Treasury, and the Chief Procurement Officer. Approvals shall be evidenced by a signature or electronic approval.

D.2. Communications and Contacts. All instructions, notices, consents, demands, or other

communications required or contemplated by this Contract shall be in writing and shall be made by certified, first class mail, return receipt requested and postage prepaid, by overnight courier service with an asset tracking system, or by email or facsimile transmission with recipient confirmation. All communications, regardless of method of transmission, shall be addressed to

58

the respective Party at the appropriate mailing address, facsimile number, or email address as stated below or any other address provided in writing by a Party.

The State:

Mary Batiwalla Tennessee Department of Education

Division of Assessment, Accountability, and Data Governance 10

th Floor, Andrew Johnson Tower

710 James Robertson Pkwy Nashville, TN 37243 [email protected]

The Contractor:

Contractor Contact Name & Title Contractor Name Address Email Address Telephone # Number FAX # Number

All instructions, notices, consents, demands, or other communications shall be considered effective upon receipt or recipient confirmation as may be required.

D.3. Modification and Amendment. This Contract may be modified only by a written amendment

signed by all Parties and approved by all applicable State officials.

D.4. Subject to Funds Availability. The Contract is subject to the appropriation and availability of State or federal funds. In the event that the funds are not appropriated or are otherwise unavailable, the State reserves the right to terminate this Contract upon written notice to the Contractor. The State’s exercise of its right to terminate this Contract shall not constitute a breach of Contract by the State. Upon receipt of the written notice, the Contractor shall cease all work associated with the Contract. If the State terminates this Contract due to lack of funds availability, the Contractor shall be entitled to compensation for all conforming goods requested and accepted by the State and for all satisfactory and authorized services completed as of the termination date. Should the State exercise its right to terminate this Contract due to unavailability of funds, the Contractor shall have no right to recover from the State any actual, general, special, incidental, consequential, or any other damages of any description or amount.

D.5. Termination for Convenience. The State may terminate this Contract for convenience without

cause and for any reason. The State shall give the Contractor at least thirty (30) calendar days written notice before the termination date. The Contractor shall be entitled to compensation for all conforming goods delivered and accepted by the State or for satisfactory, authorized services completed as of the termination date. In no event shall the State be liable to the Contractor for compensation for any goods neither requested nor accepted by the State or for any services neither requested by the State nor satisfactorily performed by the Contractor. In no event shall the State’s exercise of its right to terminate this Contract for convenience relieve the Contractor of any liability to the State for any damages or claims arising under this Contract.

D.6. Termination for Cause. If the Contractor fails to properly perform its obligations under this

Contract in a timely or proper manner, or if the Contractor materially violates any terms of this Contract (“Breach Condition”), the State shall have the right to immediately terminate the Contract and withhold payments in excess of compensation for completed services or provided goods. Notwithstanding the above, the Contractor shall not be relieved of liability to the State for damages sustained by virtue of any Breach Condition and the State may seek other remedies allowed at law or in equity for breach of this Contract.

D.7. Assignment and Subcontracting. The Contractor shall not assign this Contract or enter into a

subcontract for any of the goods, services, or deliverables provided under this Contract without the prior written approval of the State. Notwithstanding any use of the approved subcontractors,



59

the Contractor shall be the prime contractor and responsible for compliance with all terms and conditions of this Contract. The State reserves the right to request additional information or impose additional terms and conditions before approving an assignment of this Contract in whole or in part or the use of subcontractors in fulfilling the Contractor’s obligations under this Contract. This includes the right to request the resumes or vitae, performance history, and track record with contracts of a similar size and nature.

D.8. Conflicts of Interest. The Contractor warrants that no part of the Contractor’s compensation shall

be paid directly or indirectly to an employee or official of the State of Tennessee as wages, compensation, or gifts in exchange for acting as an officer, agent, employee, subcontractor, or consultant to the Contractor in connection with any work contemplated or performed under this Contract.

The Contractor acknowledges, understands, and agrees that this Contract shall be null and void if the Contractor is, or within the past six (6) months has been, an employee of the State of Tennessee or if the Contractor is an entity in which a controlling interest is held by an individual who is, or within the past six (6) months has been, an employee of the State of Tennessee.

D.9. Nondiscrimination. The Contractor hereby agrees, warrants, and assures that no person shall be

excluded from participation in, be denied benefits of, or be otherwise subjected to discrimination in the performance of this Contract or in the employment practices of the Contractor on the grounds of handicap or disability, age, race, creed, color, religion, sex, national origin, or any other classification protected by federal or state law. The Contractor shall, upon request, show proof of nondiscrimination and shall post in conspicuous places, available to all employees and applicants, notices of nondiscrimination.

D.10. Prohibition of Illegal Immigrants. The requirements of Tenn. Code Ann. § 12-3-309 addressing

the use of illegal immigrants in the performance of any contract to supply goods, services, or deliverables to the state of Tennessee, shall be a material provision of this Contract, a breach of which shall be grounds for monetary and other penalties, up to and including termination of this Contract.

a. The Contractor agrees that the Contractor shall not knowingly utilize the services of an

illegal immigrant in the performance of this Contract and shall not knowingly utilize the services of any subcontractor who will utilize the services of an illegal immigrant in the performance of this Contract. The Contractor shall reaffirm this attestation, in writing, by submitting to the State a completed and signed copy of the document at Attachment C, semi-annually during the Term. If the Contractor is a party to more than one contract with the State, the Contractor may submit one attestation that applies to all contracts with the State. All Contractor attestations shall be maintained by the Contractor and made available to State officials upon request.

b. Prior to the use of any subcontractor in the performance of this Contract, and semi-

annually thereafter, during the Term, the Contractor shall obtain and retain a current, written attestation that the subcontractor shall not knowingly utilize the services of an illegal immigrant to perform work under this Contract and shall not knowingly utilize the services of any subcontractor who will utilize the services of an illegal immigrant to perform work under this Contract. Attestations obtained from subcontractors shall be maintained by the Contractor and made available to State officials upon request.

c. The Contractor shall maintain records for all personnel used in the performance of this

Contract. Contractor’s records shall be subject to review and random inspection at any reasonable time upon reasonable notice by the State.

d. The Contractor understands and agrees that failure to comply with this section will be

subject to the sanctions of Tenn. Code Ann. § 12-3-309 for acts or omissions occurring after its effective date.

e. For purposes of this Contract, "illegal immigrant" shall be defined as any person who is

not: (i) a United States citizen; (ii) a Lawful Permanent Resident; (iii) a person whose physical presence in the United States is authorized; (iv) allowed by the federal

60

Department of Homeland Security and who, under federal immigration laws or regulations, is authorized to be employed in the U.S.; or (v) is otherwise authorized to provide services under the Contract.

D.11. Records. The Contractor shall maintain documentation for all charges under this Contract. The

books, records, and documents of the Contractor, for work performed or money received under this Contract, shall be maintained for a period of five (5) full years from the date of the final payment and shall be subject to audit at any reasonable time and upon reasonable notice by the State, the Comptroller of the Treasury, or their duly appointed representatives. The financial statements shall be prepared in accordance with generally accepted accounting principles.

D.12. Monitoring. The Contractor’s activities conducted and records maintained pursuant to this

Contract shall be subject to monitoring and evaluation by the State, the Comptroller of the Treasury, or their duly appointed representatives.

D.13. Progress Reports. The Contractor shall submit brief, periodic, progress Reports to the State as

requested. D.14. Strict Performance. Failure by any Party to this Contract to require, in any one or more cases,

the strict performance of any of the terms, covenants, conditions, or provisions of this Contract shall not be construed as a waiver or relinquishment of any term, covenant, condition, or provision. No term or condition of this Contract shall be held to be waived, modified, or deleted except by a written amendment signed by the Parties.

D.15. Independent Contractor. The Parties shall not act as employees, partners, joint venturers, or

associates of one another. The Parties are independent contracting entities. Nothing in this Contract shall be construed to create an employer/employee relationship or to allow either Party to exercise control or direction over the manner or method by which the other transacts its business affairs or provides its usual services. The employees or agents of one Party are not employees or agents of the other Party.

D.16 Patient Protection and Affordable Care Act. The Contractor agrees that it will be responsible for

compliance with the Patient Protection and Affordable Care Act (“PPACA”) with respect to itself and its employees, including any obligation to report health insurance coverage, provide health insurance coverage, or pay any financial assessment, tax, or penalty for not providing health insurance. The Contractor shall indemnify the State and hold it harmless from any costs to the State arising from Contractor’s failure to fulfill its PPACA responsibilities for itself or its employees.

D.17. Limitation of State’s Liability. The State shall have no liability except as specifically provided in this Contract. In no event will the State be liable to the Contractor or any other party for any lost revenues, lost profits, loss of business, decrease in the value of any securities or cash position, time, goodwill, or any indirect, special, incidental, punitive, exemplary or consequential damages of any nature, whether based on warranty, contract, statute, regulation, tort (including but not limited to negligence), or any other legal theory that may arise under this Contract or otherwise. The State’s total liability under this Contract (including any exhibits, schedules, amendments or other attachments to the Contract) or otherwise shall under no circumstances exceed the Maximum Liability. This limitation of liability is cumulative and not per incident.

D.18. Limitation of Contractor’s Liability. In accordance with Tenn. Code Ann. § 12-3-701, the

Contractor’s liability for all claims arising under this Contract shall be limited to an amount equal to two (2) times the Maximum Liability amount detailed in Section C.1. and as may be amended, PROVIDED THAT in no event shall this Section limit the liability of the Contractor for: (i) intellectual property or any Contractor indemnity obligations for infringement for third-party intellectual property rights; (ii) any claims covered by any specific provision in the Contract providing for Liquidated Damages; or (iii) any claims for intentional torts, criminal acts, fraudulent conduct, or acts or omissions that result in personal injuries or death.

D.19. Hold Harmless. The Contractor agrees to indemnify and hold harmless the State of Tennessee

as well as its officers, agents, and employees from and against any and all claims, liabilities, losses, and causes of action which may arise, accrue, or result to any person, firm, corporation,

61

or other entity which may be injured or damaged as a result of acts, omissions, or negligence on the part of the Contractor, its employees, or any person acting for or on its or their behalf relating to this Contract. The Contractor further agrees it shall be liable for the reasonable cost of attorneys’ fees, court costs, expert witness fees, and other litigation expenses for the State to enforce the terms of this Contract.

In the event of any suit or claim, the Parties shall give each other immediate notice and provide all necessary assistance to respond. The failure of the State to give notice shall only relieve the Contractor of its obligations under this Section to the extent that the Contractor can demonstrate actual prejudice arising from the failure to give notice. This Section shall not grant the Contractor, through its attorneys, the right to represent the State in any legal matter, as the right to represent the State is governed by Tenn. Code Ann. § 8-6-106.

D.20. HIPAA Compliance. The State and Contractor shall comply with obligations under the Health

Insurance Portability and Accountability Act of 1996 (“HIPAA”), Health Information Technology for Economic and Clinical Health (“HITECH”) Act and any other relevant laws and regulations regarding privacy (collectively the “Privacy Rules”). The obligations set forth in this Section shall survive the termination of this Contract.

a. Contractor warrants to the State that it is familiar with the requirements of the Privacy

Rules, and will comply with all applicable requirements in the course of this Contract. b. Contractor warrants that it will cooperate with the State, including cooperation and

coordination with State privacy officials and other compliance officers required by the Privacy Rules, in the course of performance of the Contract so that both parties will be in compliance with the Privacy Rules.

c. The State and the Contractor will sign documents, including but not limited to business

associate agreements, as required by the Privacy Rules and that are reasonably necessary to keep the State and Contractor in compliance with the Privacy Rules. This provision shall not apply if information received or delivered by the parties under this Contract is NOT “protected health information” as defined by the Privacy Rules, or if the Privacy Rules permit the parties to receive or deliver the information without entering into a business associate agreement or signing another document.

d. The Contractor will indemnify the State and hold it harmless for any violation by the

Contractor or its subcontractors of the Privacy Rules. This includes the costs of responding to a breach of protected health information, the costs of responding to a government enforcement action related to the breach, and any fines, penalties, or damages paid by the State because of the violation.

D.21. Tennessee Consolidated Retirement System. Subject to statutory exceptions contained in Tenn. Code Ann. §§ 8-36-801, et seq., the law governing the Tennessee Consolidated Retirement System (“TCRS”), provides that if a retired member of TCRS, or of any superseded system administered by TCRS, or of any local retirement fund established under Tenn. Code Ann. §§ 8- 35-101, et seq., accepts State employment, the member's retirement allowance is suspended during the period of the employment. Accordingly and notwithstanding any provision of this Contract to the contrary, the Contractor agrees that if it is later determined that the true nature of the working relationship between the Contractor and the State under this Contract is that of “employee/employer” and not that of an independent contractor, the Contractor, if a retired member of TCRS, may be required to repay to TCRS the amount of retirement benefits the Contractor received from TCRS during the Term.

D.22. Tennessee Department of Revenue Registration. The Contractor shall comply with all applicable

registration requirements contained in Tenn. Code Ann. §§ 67-6-601 – 608. Compliance with applicable registration requirements is a material requirement of this Contract.

D.23. Debarment and Suspension. The Contractor certifies, to the best of its knowledge and belief, that

it, its current and future principals, its current and future subcontractors and their principals:

62

a. are not presently debarred, suspended, proposed for debarment, declared ineligible, or voluntarily excluded from covered transactions by any federal or state department or agency;

b. have not within a three (3) year period preceding this Contract been convicted of, or had

a civil judgment rendered against them from commission of fraud, or a criminal offense in connection with obtaining, attempting to obtain, or performing a public (federal, state, or local) transaction or grant under a public transaction; violation of federal or state antitrust statutes or commission of embezzlement, theft, forgery, bribery, falsification, or destruction of records, making false statements, or receiving stolen property;

c. are not presently indicted or otherwise criminally or civilly charged by a government entity (federal, state, or local) with commission of any of the offenses detailed in section b. of this certification; and

d. have not within a three (3) year period preceding this Contract had one or more public

transactions (federal, state, or local) terminated for cause or default.

The Contractor shall provide immediate written notice to the State if at any time it learns that there was an earlier failure to disclose information or that due to changed circumstances, its principals or the principals of its subcontractors are excluded, disqualified, or presently fall under any of the prohibitions of sections a-d.

D.24. Force Majeure. “Force Majeure Event” means fire, flood, earthquake, elements of nature or acts of God, wars, riots, civil disorders, rebellions or revolutions, acts of terrorism or any other similar cause beyond the reasonable control of the Party except to the extent that the non-performing Party is at fault in failing to prevent or causing the default or delay, and provided that the default or delay cannot reasonably be circumvented by the non-performing Party through the use of alternate sources, workaround plans or other means. A strike, lockout or labor dispute shall not excuse either Party from its obligations under this Contract. Except as set forth in this Section, any failure or delay by a Party in the performance of its obligations under this Contract arising from a Force Majeure Event is not a default under this Contract or grounds for termination. The non-performing Party will be excused from performing those obligations directly affected by the Force Majeure Event, and only for as long as the Force Majeure Event continues, provided that the Party continues to use diligent, good faith efforts to resume performance without delay. The occurrence of a Force Majeure Event affecting Contractor’s representatives, suppliers, subcontractors, customers or business apart from this Contract is not a Force Majeure Event under this Contract. Contractor will promptly notify the State of any delay caused by a Force Majeure Event (to be confirmed in a written notice to the State within one (1) day of the inception of the delay) that a Force Majeure Event has occurred, and will describe in reasonable detail the nature of the Force Majeure Event. If any Force Majeure Event results in a delay in Contractor’s performance longer than forty-eight (48) hours, the State may, upon notice to Contractor: (a) cease payment of the fees until Contractor resumes performance of the affected obligations; or (b) immediately terminate this Contract or any purchase order, in whole or in part, without further payment except for fees then due and payable. Contractor will not increase its charges under this Contract or charge the State any fees other than those provided for in this Contract as the result of a Force Majeure Event.

D.25. State and Federal Compliance. The Contractor shall comply with all applicable state and

federal laws and regulations in the performance of this Contract.

D.26. Governing Law. This Contract shall be governed by and construed in accordance with the laws of the State of Tennessee, without regard to its conflict or choice of law rules. The Tennessee Claims Commission or the state or federal courts in Tennessee shall be the venue for all claims, disputes, or disagreements arising under this Contract. The Contractor acknowledges and agrees that any rights, claims, or remedies against the State of Tennessee or its employees arising under this Contract shall be subject to and limited to those rights and remedies available under Tenn. Code Ann. §§ 9-8-101 - 408.

D.27. Entire Agreement. This Contract is complete and contains the entire understanding between

the Parties relating to its subject matter, including all the terms and conditions of the Parties’

63

agreement. This Contract supersedes any and all prior understandings, representations, negotiations, and agreements between the Parties, whether written or oral.

D.28. Severability. If any terms and conditions of this Contract are held to be invalid or

unenforceable as a matter of law, the other terms and conditions of this Contract shall not be affected and shall remain in full force and effect. The terms and conditions of this Contract are severable.

D.29. Headings. Section headings of this Contract are for reference purposes only and shall not

be construed as part of this Contract.

D.30. Incorporation of Additional Documents. Each of the following documents is included as a part of this Contract by reference. In the event of a discrepancy or ambiguity regarding the Contractor’s duties, responsibilities, and performance under this Contract, these Items shall govern in order of precedence below:

a. any amendment to this Contract, with the latter in time controlling over any earlier

amendments;

b. this Contract with any attachments or exhibits (excluding the items listed at subsections c. through f. below), which includes Attachments A, B, and C;

c. Any URLs incorporated by reference in this Contract;

d. the incorporation of URLs includes the URL as it exists as of the Effective Date and any changes to the URL that may occur from time-to-time during the Term;

e. any clarifications of or addenda to the Contractor’s proposal seeking this Contract;

f. the State solicitation, as may be amended, requesting responses in competition for this Contract;

g. any technical specifications provided to proposers during the procurement process to award this Contract; and

h. the Contractor’s response seeking this Contract.

D.31. Iran Divestment Act. The requirements of Tenn. Code Ann. § 12-12-101, et seq., addressing contracting with persons as defined at Tenn. Code Ann. §12-12-103(5) that engage in investment activities in Iran, shall be a material provision of this Contract. The Contractor certifies, under penalty of perjury, that to the best of its knowledge and belief that it is not on the list created pursuant to Tenn. Code Ann. § 12-12-106.

D.32. Insurance. Contractor shall maintain insurance coverage as specified in this Section. The State reserves the right to amend or require additional insurance coverage, coverage amounts, and endorsements required under this Contract. Contractor’s failure to maintain or submit evidence of insurance coverage, as required, is a material breach of this Contract. If Contractor loses insurance coverage, fails to renew coverage, or for any reason becomes uninsured during the Term, Contractor shall immediately notify the State. All insurance companies providing coverage must be: (a) acceptable to the State; (b) authorized by the Tennessee Department of Commerce and Insurance (“TDCI”); and (c) rated A- / VII or better by A.M. Best. All coverage must be on a primary basis and noncontributory with any other insurance or self-insurance carried by the State. Contractor agrees to name the State as an additional insured on any insurance policy with the exception of workers’ compensation (employer liability) and professional liability (errors and omissions) insurance. All policies must contain an endorsement for a waiver of subrogation in favor of the State. Any deductible or self-insured retention (“SIR”) over fifty thousand dollars ($50,000) must be approved by the State. The deductible or SIR and any premiums are the Contractor’s sole responsibility. The Contractor agrees that the insurance requirements specified in this Section do not reduce any liability the Contractor has assumed under this Contract including any indemnification or hold harmless requirements.

64

To achieve the required coverage amounts, a combination of an otherwise deficient specific policy and an umbrella policy with an aggregate meeting or exceeding the required coverage amounts is acceptable. For example: If the required policy limit under this Contract is for two million dollars ($2,000,000) in coverage, acceptable coverage would include a specific policy covering one million dollars ($1,000,000) combined with an umbrella policy for an additional one million dollars ($1,000,000). If the deficient underlying policy is for a coverage area without aggregate limits (generally Automobile Liability and Employers’ Liability Accident), Contractor shall provide a copy of the umbrella insurance policy documents to ensure that no aggregate limit applies to the umbrella policy for that coverage area. In the event that an umbrella policy is being provided to achieve any required coverage amounts, the umbrella policy shall be accompanied by an endorsement at least as broad as the Insurance Services Office, Inc. (also known as “ISO”) “Noncontributory—Other Insurance Condition” endorsement or shall be written on a policy form that addresses both the primary and noncontributory basis of the umbrella policy if the State is otherwise named as an additional insured.

Contractor shall provide the State a certificate of insurance (“COI”) evidencing the coverages and amounts specified in this Section. The COI must be on a form approved by the TDCI (standard ACORD form preferred). The COI must list each insurer’s National Association of Insurance Commissioners (NAIC) number and be signed by an authorized representative of the insurer. The rd

65

COI must list the State of Tennessee – CPO Risk Manager, 312 Rosa L. Parks Ave., 3 floor Central Procurement Office, Nashville, TN 37243 as the certificate holder. Contractor shall provide the COI ten (10) business days prior to the Effective Date and again thirty (30) calendar days before renewal or replacement of coverage. Contractor shall provide the State evidence that all subcontractors maintain the required insurance or that subcontractors are included under the Contractor’s policy. At any time, the State may require Contractor to provide a valid COI. The Parties agree that failure to provide evidence of insurance coverage as required is a material breach of this Contract. If Contractor self-insures, then a COI will not be required to prove coverage. Instead Contractor shall provide a certificate of self-insurance or a letter, on Contractor’s letterhead, detailing its coverage, policy amounts, and proof of funds to reasonably cover such expenses. The State reserves the right to require complete, certified copies of all required insurance policies, including endorsements required by these specifications, at any time.

The State agrees that it shall give written notice to the Contractor as soon as practicable after the State becomes aware of any claim asserted or made against the State, but in no event later than thirty (30) calendar days after the State becomes aware of such claim. The failure of the State to give notice shall only relieve the Contractor of its obligations under this Section to the extent that the Contractor can demonstrate actual prejudice arising from the failure to give notice. This Section shall not grant the Contractor or its insurer, through its attorneys, the right to represent the State in any legal matter, as the right to represent the State is governed by Tenn. Code Ann. § 8-6-106.

The insurance obligations under this Contract shall be: (1)—all the insurance coverage and policy limits carried by the Contractor; or (2)—the minimum insurance coverage requirements and policy limits shown in this Contract; whichever is greater. Any insurance proceeds in excess of or broader than the minimum required coverage and minimum required policy limits, which are applicable to a given loss, shall be available to the State. No representation is made that the minimum insurance requirements of the Contract are sufficient to cover the obligations of the Contractor arising under this Contract. The Contractor shall obtain and maintain, at a minimum, the following insurance coverages and policy limits.

a. Commercial General Liability (“CGL”) Insurance

1) The Contractor shall maintain CGL, which shall be written on an ISO Form CG

00 01 occurrence form (or a substitute form providing equivalent coverage) and shall cover liability arising from property damage, premises and operations products and completed operations, bodily injury, personal and advertising injury, and liability assumed under an insured contract (including the tort liability of another assumed in a business contract).

The Contractor shall maintain single limits not less than one million dollars ($1,000,000) per occurrence. If a general aggregate limit applies, either the general aggregate limit shall apply separately to this policy or location of occurrence or the general aggregate limit shall be twice the required occurrence limit.

b. Workers’ Compensation and Employer Liability Insurance

1) For Contractors statutorily required to carry workers’ compensation and employer

liability insurance, the Contractor shall maintain:

i. Workers’ compensation in an amount not less than one million dollars

($1,000,000) including employer liability of one million dollars ($1,000,000) per accident for bodily injury by accident, one million dollars ($1,000,000) policy limit by disease, and one million dollars ($1,000,000) per employee for bodily injury by disease.

2) If the Contractor certifies that it is exempt from the requirements of Tenn. Code

Ann. §§ 50-6-101 – 103, then the Contractor shall furnish written proof of such

66

exemption for one or more of the following reasons:

i. The Contractor employs fewer than five (5) employees;

ii. The Contractor is a sole proprietor;

iii. The Contractor is in the construction business or trades with no

employees;

iv. The Contractor is in the coal mining industry with no employees;

v. The Contractor is a state or local government; or

vi. The Contractor self-insures its workers’ compensation and is in

compliance with the TDCI rules and Tenn. Code Ann. § 50-6-405.

c. Automobile Liability Insurance

1) The Contractor shall maintain automobile liability insurance which shall cover

liability arising out of any automobile (including owned, leased, hired, and non- owned automobiles).

2) The Contractor shall maintain bodily injury/property damage with a limit not less

than one million dollars ($1,000,000) per occurrence or combined single limit.

d. Technology Professional Liability (Errors & Omissions)/Cyber Liability Insurance

1) The Contractor shall maintain technology professional liability (errors &

omissions)/cyber liability insurance appropriate to the Contractor’s profession in an amount not less than twenty million dollars ($20,000,000) per occurrence or claim and twenty million dollars ($20,000,000) annual aggregate, covering all acts, claims, errors, omissions, negligence, infringement of intellectual property (including copyright, patent and trade secret); network security and privacy risks, including but not limited to unauthorized access, failure of security, information theft, damage to destruction of or alteration of electronic information, breach of privacy perils, wrongful disclosure and release of private information, collection, or other negligence in the handling of confidential information, and including coverage for related regulatory fines, defenses, and penalties.

2) Such coverage shall include data breach response expenses, in an amount not

less than twenty million dollars ($20,000,000) and payable whether incurred by the State or Contractor, including but not limited to consumer notification, whether or not required by law, computer forensic investigations, public relations and crisis management firm fees, credit file or identity monitoring or remediation services and expenses in the performance of services for the State or on behalf of the State hereunder.

e. Crime Insurance

1) The Contractor shall maintain crime insurance, which shall be written on a “loss

sustained form” or “loss discovered form” providing coverage for third party fidelity, including cyber theft and extortion. The policy must allow for reporting of circumstances or incidents that may give rise to future claims, include an extended reporting period of no less than two (2) years with respect to events which occurred but were not reported during the term of the policy, and not contain a condition requiring an arrest or conviction.

Any crime insurance policy shall have a limit not less than one million dollars ($1,000,000) per claim and one million dollars ($1,000,000) in the aggregate. Any crime insurance policy shall contain a Social Engineering Fraud Endorsement with a limit of not less than two hundred and

67

fifty thousand dollars ($250,000). This insurance may be written on a claims-made basis, but in the event that coverage is cancelled or non-renewed, the Contractor shall purchase an extended reporting or “tail coverage” of at least two (2) years after the Term.

D.33. Major Procurement Contract Sales and Use Tax. Pursuant to Tenn. Code Ann. § 4-39-102 and

to the extent applicable, the Contractor and the Contractor’s subcontractors shall remit sales and use taxes on the sales of goods, services, or deliverables that are made by the Contractor or the Contractor’s subcontractors and that are subject to tax.

E. SPECIAL TERMS AND CONDITIONS:

E.1. Conflicting Terms and Conditions. Should any of these special terms and conditions conflict with

any other terms and conditions of this Contract, the special terms and conditions shall be subordinate to the Contract’s other terms and conditions.

E.2. Confidentiality of Records. Strict standards of confidentiality of records and information shall be maintained in accordance with applicable state and federal law. All material and information, regardless of form, medium or method of communication, provided to the Contractor by the State or acquired by the Contractor on behalf of the State that is regarded as confidential under state or federal law shall be regarded as “Confidential Information.” Nothing in this Section shall permit Contractor to disclose any Confidential Information, regardless of whether it has been disclosed or made available to the Contractor due to intentional or negligent actions or inactions of agents of the State or third parties. Confidential Information shall not be disclosed except as required or permitted under state or federal law. Contractor shall take all necessary steps to safeguard the confidentiality of such material or information in conformance with applicable state and federal law.

The obligations set forth in this Section shall survive the termination of this Contract.

E.3. Additional Lines, Items, or Options. At its sole discretion, the State may make written requests to

the Contractor to add lines, Items, or options that are needed and within the Scope but were not included in the original Contract. Such lines, Items, or options will be added to the Contract through a Memorandum of Understanding (“MOU”), not an amendment.

a. After the Contractor receives a written request to add lines, Items, or options, the Contractor shall have ten (10) business days to respond with a written proposal. The Contractor’s written proposal shall include:

(1) The effect, if any, of adding the lines, Items, or options on the other goods, services, or deliverables required under the Contract;

(2) Any pricing related to the new lines, Items, or options;

(3) The expected effective date for the availability of the new lines, Items, or options; and

(4) Any additional information requested by the State.

b. The State may negotiate the terms of the Contractor’s proposal by requesting revisions to the proposal.

c. To indicate acceptance of a proposal, the State will sign it. The signed proposal shall constitute a MOU between the Parties, and the lines, Items, or options shall be incorporated into the Contract as if set forth verbatim.

d. Only after a MOU has been executed shall the Contractor perform or deliver the new lines, Items, or options.

68

E.4. Intellectual Property Indemnity. The Contractor agrees to indemnify and hold harmless the State

of Tennessee as well as its officers, agents, and employees from and against any and all claims or suits which may be brought against the State concerning or arising out of any claim of an alleged patent, copyright, trade secret or other intellectual property infringement. In any such claim or action brought against the State, the Contractor shall satisfy and indemnify the State for the amount of any settlement or final judgment, and the Contractor shall be responsible for all legal or other fees or expenses incurred by the State arising from any such claim. The State shall give the Contractor notice of any such claim or suit, however, the failure of the State to give such notice shall only relieve Contractor of its obligations under this Section to the extent Contractor can demonstrate actual prejudice arising from the State’s failure to give notice. This Section shall not grant the Contractor, through its attorneys, the right to represent the State of Tennessee in any legal matter, as provided in Tenn. Code Ann. § 8-6-106.

E.5. Software License Warranty. Contractor grants a license to the State to use all software

provided under this Contract in the course of the State’s business and purposes.

E.6. Software Support and Maintenance Warranty. Contractor shall provide to the State all

software upgrades, modifications, bug fixes, or other improvements in its software that it makes generally available to its customers.

E.7. Extraneous Terms and Conditions. Contractor shall fill all orders submitted by the State under this

Contract. No purchase order, invoice, or other documents associated with any sales, orders, or supply of any good or service under this Contract shall contain any terms or conditions other than as set forth in the Contract. Any such extraneous terms and conditions shall be void, invalid and unenforceable against the State. Any refusal by Contractor to supply any goods, services, or deliverables under this Contract conditioned upon the State submitting to any extraneous terms and conditions shall be a material breach of the Contract and constitute an act of bad faith by Contractor.

E.8. Contractor Hosted Services Confidential Data, Audit, and Other Requirements

a. “Confidential State Data” is defined as data deemed confidential by State or Federal

statute or regulation. The Contractor shall protect Confidential State Data as follows:

(1) The Contractor shall ensure that all Confidential State Data is housed in the

continental United States, inclusive of backup data.

(2) The Contractor shall encrypt Confidential State Data at rest and in transit using the current version of Federal Information Processing Standard (“FIPS”) 140-2 validated encryption technologies.

(3) The Contractor and the Contractor’s processing environment containing

Confidential State Data shall either (1) be in accordance with at least one of the following security standards: (i) International Standards Organization (“ISO”) 27001; (ii) Federal Risk and Authorization Management Program (“FedRAMP”); or (2) be subject to an annual engagement by a CPA firm in accordance with the standards of the American Institute of Certified Public Accountants (“AICPA”) for a System and Organization Controls for service organizations (“SOC”) Type II audit. The State shall approve the SOC audit control objectives. The Contractor shall provide proof of current ISO certification or

FedRAMP authorization for the Contractor and Subcontractor(s), or provide the State with the Contractor’s and Subcontractor’s annual SOC Type II audit Report within 30 calendar days from when the CPA firm provides the audit Report to the Contractor or Subcontractor. The Contractor shall submit corrective action plans to the State for any issues included in the audit Report within 30 calendar days after the CPA firm provides the audit Report to the Contractor or Subcontractor.

If the scope of the most recent SOC audit Report does not include all of the

69

current State Fiscal Year, upon request from the State, the Contractor shall provide to the State a letter from the Contractor or Subcontractor stating whether the Contractor or Subcontractor made any material changes to their control environment since the prior audit and, if so, whether the changes, in the opinion of the Contractor or Subcontractor, would negatively affect the auditor’s opinion in the most recent audit Report.

No additional funding shall be allocated for these certifications, authorizations, or audits as these are included in the Maximum Liability of this Contract.

(4) The Contractor shall annually perform Penetration Tests and Vulnerability

Assessments against its Processing Environment. “Processing Environment” shall mean the combination of software and hardware on which the Application runs. “Application” shall mean the computer code that supports and accomplishes the State’s requirements as set forth in this Contract. “Penetration Tests” shall be in the form of attacks on the Contractor’s computer system, with the purpose of discovering security weaknesses which have the potential to gain access to the Processing Environment’s features and data. The “Vulnerability Assessment” shall be designed and executed to define, identify, and classify the security holes (vulnerabilities) in the Processing Environment. The Contractor shall allow the State, at its option, to perform Penetration Tests and Vulnerability Assessments on the Processing Environment.

(5) Upon State request, the Contractor shall provide a copy of all Confidential State Data it holds. The Contractor shall provide such data on media and in a format determined by the State.

(6) Upon termination of this Contract and in consultation with the State, the Contractor shall destroy all Confidential State Data it holds (including any copies such as backups) in accordance with the current version of National Institute of Standards and Technology (“NIST”) Special Publication 800-88. The Contractor shall provide a written confirmation of destruction to the State within ten (10) business days after destruction.

b. Minimum Requirements

(1) The Contractor and all data centers used by the Contractor to host State data,

including those of all Subcontractors, must comply with the State’s Enterprise Information Security Policies as amended periodically. The State’s Enterprise Information Security Policies document is found at the following URL: https://www.tn.gov/finance/strategic-technology-solutions/strategic-technology- solutions/sts-security-policies.htm l.

(2) The Contractor agrees to maintain the Application so that it will run on a current, manufacturer-supported Operating System. “Operating System” shall mean the software that supports a computer's basic functions, such as scheduling tasks, executing applications, and controlling peripherals.

(3) If the Application requires middleware or database software, Contractor shall

maintain middleware and database software versions that are at all times fully compatible with current versions of the Operating System and Application to ensure that security vulnerabilities are not introduced.

c. Comptroller Audit Requirements

Upon reasonable notice and at any reasonable time, the Contractor and Subcontractor(s) agree to allow the State, the Comptroller of the Treasury, or their duly appointed representatives to perform information technology control audits of the Contractor and all Subcontractors used by the Contractor.



70

Contractor will maintain and cause its Subcontractors to maintain a complete audit trail of all transactions and activities in connection with this Contract. Contractor will provide to the State, the Comptroller of the Treasury, or their duly appointed representatives, access to Contractor or subcontractor personnel for the purpose of performing the information technology control audit.

The information technology control audit may include a review of general controls and application controls. General controls are the policies and procedures that apply to all or a large segment of the Contractor’s or Subcontractor’s information systems and applications and include controls over security management, access controls, configuration management, segregation of duties, and contingency planning. Application controls are directly related to the application and help ensure that transactions are complete, accurate, valid, confidential, and available. The audit shall include the Contractor’s and Subcontractor’s compliance with the State’s Enterprise Information Security Policies and all applicable requirements, laws, regulations or policies.

The audit may include interviews with technical and management personnel, physical inspection of controls, and review of paper or electronic documentation.

For any audit issues identified, the Contractor and Subcontractor(s) shall provide a corrective action plan to the State within thirty (30) calendar days from the Contractor or Subcontractor receiving the audit Report.

Each party shall bear its own expenses incurred while conducting the information technology controls audit.

d. Business Continuity Requirements. The Contractor shall maintain sets of

documents, instructions, and procedures which enable the Contractor to respond to accidents, disasters, emergencies, or threats without any stoppage or hindrance in its key operations (“Business Continuity Requirements”). Business Continuity Requirements shall include:

(1) “Disaster Recovery Capabilities” refer to the actions the Contractor takes to

meet the Recovery Point and Recovery Time Objectives defined below. Disaster Recovery Capabilities shall meet the following objectives:

i. Recovery Point Objective (“RPO”). The RPO is defined as the maximum

targeted period in which data might be lost from an IT service due to a major incident six(6) hours (except where redefined in the scorecard in Section A.14. of the Contract)

ii. Recovery Time Objective (“RTO”). The RTO is defined as the targeted

duration of time and a service level within which a business process must be restored after a disaster (or disruption) in order to avoid unacceptable consequences associated with a break in business continuity: 6 hours (except where redefined in the scorecard in Section A.14. of the Contract)

(2) The Contractor or its subcontractors shall perform at least one Disaster

Recovery Test every three hundred sixty-five (365) calendar days. A “Disaster Recovery Test” shall mean the process of verifying the success of the restoration procedures that are executed after a critical IT failure or disruption occurs. The Disaster Recovery Test shall use actual State Data Sets that mirror production data, and success shall be defined as the Contractor verifying that the Contractor can meet the State’s RPO and RTO requirements. A “Data Set” is defined as a collection of related sets of information that is composed of separate elements but can be manipulated as a unit by a computer. The Contractor shall provide written confirmation to the State after each Disaster Recover Test that its Disaster Recovery Capabilities meet the RPO and RTO requirements.

71

E.9. Family Educational Rights and Privacy Act & Tennessee Data Accessibility, Transparency and

Accountability Act. The Contractor shall comply with the Family Educational Rights and Privacy Act of 1974 (20 U.S.C. 1232(g)) and its accompanying regulations (34 C.F.R. § 99) (“FERPA”). The Contractor warrants that the Contractor is familiar with FERPA requirements and that it will comply with these requirements in the performance of its duties under this Contract. The Contractor agrees to cooperate with the State, as required by FERPA, in the performance of its duties under this Contract. The Contractor agrees to maintain the confidentiality of all education records and student information. The Contractor shall only use such records and information for the exclusive purpose of performing its duties under this Contract.

The Contractor shall also comply with Tenn. Code Ann. § 49-1-701, et seq., known as the “Data Accessibility, Transparency and Accountability Act,” and any accompanying administrative rules or regulations (collectively “DATAA”). The Contractor agrees to maintain the confidentiality of all records containing student and de-identified data, as this term is defined in DATAA, in any databases, to which the State has granted the Contractor access, and to only use such data for the exclusive purpose of performing its duties under this Contract.

Any instances of unauthorized disclosure of data containing personally identifiable information in violation of FERPA or DATAA that come to the attention of the Contractor shall be reported to the State within twenty-four (24) hours. Contractor shall indemnify and hold harmless State, its employees, agents and representatives, from and against any and all claims, liabilities, losses, or causes of action that may arise, accrue, or result to any person or entity that is injured or damaged as a result of Contractor’s failure to comply with this section.

E.10. Lobbying. The Contractor certifies, to the best of its knowledge and belief, that:

a. No federally appropriated funds have been paid or will be paid, by or on behalf of the Contractor, to any person for influencing or attempting to influence an officer or employee of an agency, a member of Congress, an officer or employee of Congress, or an employee of a member of Congress in connection with the awarding of any federal contract, the making of any federal grant, the making of any federal loan, the entering into of any cooperative agreement, and the extension, continuation, renewal, amendment, or modification of any federal contract, grant, loan, or cooperative agreement.

b. If any funds other than federally appropriated funds have been paid or will be paid to any

person for influencing or attempting to influence an officer or employee of any agency, a member of Congress, an officer or employee of Congress, or an employee of a member of Congress in connection with any contract, grant, loan, or cooperative agreement, the Contractor shall complete and submit Standard Form-LLL, “Disclosure Form to Report Lobbying,'' in accordance with its instructions.

c. The Contractor shall require that the language of this certification be included in the

award documents for all sub-awards at all tiers (including subcontracts, sub-grants, and contracts under grants, loans, and cooperative agreements) and that all subrecipients shall certify and disclose accordingly.

This certification is a material representation of fact upon which reliance was placed when this transaction was made or entered into and is a prerequisite for making or entering into this transaction imposed by 31 U.S.C. § 1352.

E.11. Contractor Commitment to Diversity. The Contractor shall comply with and make reasonable

business efforts to exceed the commitment to diversity represented by the Contractor’s Response to 33111-03819, B.15. and resulting in this Contract.

The Contractor shall assist the State in monitoring the Contractor’s performance of this commitment by providing, as requested, a monthly Report of participation in the performance of this Contract by small business enterprises and businesses owned by minorities, women, service-

72

disabled veterans, and persons with disabilities. Such Reports shall be provided to the State of Tennessee Governor's Office of Diversity Business Enterprise in the TN Diversity Software available online at: https://tn.diversitysoftware.com/FrontEnd/StartCertification.asp?TN=tn&XID=9810.

E.12. Liquidated Damages. If failure or partial performance of any term or provision of the Contract

occurs, (“Liquidated Damages Event”), the State may assess damages on Contractor (“Liquidated Damages”). The State shall notify the Contractor of amounts to be assessed as Liquidated Damages. The Parties agree that due to the complicated nature of the Contractor’s obligations under this Contract it would be difficult to specifically designate a monetary amount for Contractor’s failure to fulfill its obligations regarding the Liquidated Damages Event as these amounts are likely to be uncertain and not easily proven. Contractor has carefully reviewed the Liquidated Damages contained in Attachment B and agrees that these amounts represent a reasonable relationship between the amount and what might reasonably be expected in the event of a Liquidated Damages Event, and are a reasonable estimate of the damages that would occur from a Liquidated Damages Event. The Parties agree that the Liquidated Damages represent solely the damages and injuries sustained by the State in losing the benefit of the bargain with Contractor and do not include any injury or damage sustained by a third party. The Contractor agrees that the Liquidated Damages are in addition to any amounts Contractor may owe the State pursuant to the indemnity provision or any other sections of this Contract.

The State is not obligated to assess Liquidated Damages before availing itself of any other remedy. The State may choose to discontinue Liquidated Damages and avail itself of any other remedy available under this Contract or at law or equity.

E.13. Partial Takeover of Contract. The State may, at its convenience and without cause, exercise a partial takeover of any service that the Contractor is obligated to perform under this Contract, including any service which is the subject of a subcontract between Contractor and a third party (a “Partial Takeover”). A Partial Takeover of this Contract by the State shall not be deemed a breach of contract. The Contractor shall be given at least thirty (30) calendar days prior written notice of a Partial Takeover. The notice shall specify the areas of service the State will assume and the date the State will be assuming. The State’s exercise of a Partial Takeover shall not alter the Contractor’s other duties and responsibilities under this Contract. The State reserves the right to withhold from the Contractor any amounts the Contractor would have been paid but for the State’s exercise of a Partial Takeover. The amounts shall be withheld effective as of the date the State exercises its right to a Partial Takeover. The State’s exercise of its right to a Partial Takeover of this Contract shall not entitle the Contractor to any actual, general, special, incidental, consequential, or any other damages irrespective of any description or amount.

E.14. Personally Identifiable Information. While performing its obligations under this Contract,

Contractor may have access to Personally Identifiable Information held by the State (“PII”). For the purposes of this Contract, “PII” includes “Nonpublic Personal Information” as that term is defined in Title V of the Gramm-Leach-Bliley Act of 1999 or any successor federal statute, and the rules and regulations thereunder, all as may be amended or supplemented from time to time (“GLBA”) and personally identifiable information and other data protected under any other applicable laws, rule or regulation of any jurisdiction relating to disclosure or use of personal information (“Privacy Laws”). Contractor agrees it shall not do or omit to do anything which would cause the State to be in breach of any Privacy Laws. Contractor shall, and shall cause its employees, agents and representatives to: (i) keep PII confidential and may use and disclose PII only as necessary to carry out those specific aspects of the purpose for which the PII was disclosed to Contractor and in accordance with this Contract, GLBA and Privacy Laws; and (ii) implement and maintain appropriate technical and organizational measures regarding information security to: (A) ensure the security and confidentiality of PII; (B) protect against any threats or hazards to the security or integrity of PII; and (C) prevent unauthorized access to or use of PII. Contractor shall immediately notify State: (1) of any disclosure or use of any PII by Contractor or any of its employees, agents and representatives in breach of this Contract; and (2) of any disclosure of any PII to Contractor or its employees, agents and representatives where the purpose of such disclosure is not known to Contractor or its employees, agents and representatives. The State reserves the right to review Contractor's policies and procedures

73

used to maintain the security and confidentiality of PII and Contractor shall, and cause its employees, agents and representatives to, comply with all reasonable requests or directions from the State to enable the State to verify or ensure that Contractor is in full compliance with its obligations under this Contract in relation to PII. Upon termination or expiration of the Contract or at the State’s direction at any time in its sole discretion, whichever is earlier, Contractor shall immediately return to the State any and all PII which it has received under this Contract and shall destroy all records of such PII.

The Contractor shall report to the State any instances of unauthorized access to or potential disclosure of PII in the custody or control of Contractor (“Unauthorized Disclosure”) that come to the Contractor’s attention. Any such Report shall be made by the Contractor within twenty-four (24) hours after the Unauthorized Disclosure has come to the attention of the Contractor. Contractor shall take all necessary measures to halt any further Unauthorized Disclosures. The Contractor, at the sole discretion of the State, shall provide no cost credit monitoring services for individuals whose PII was affected by the Unauthorized Disclosure. The Contractor shall bear the cost of notification to all individuals affected by the Unauthorized Disclosure, including individual letters and public notice. The remedies set forth in this Section are not exclusive and are in addition to any claims or remedies available to this State under this Contract or otherwise available at law.

E.15. Federal Funding Accountability and Transparency Act (FFATA). This Contract requires the Contractor to provide supplies or services that are funded in whole or in part by federal funds that are subject to FFATA. The Contractor is responsible for ensuring that all applicable requirements, including but not limited to those set forth herein, of FFATA are met and that the Contractor provides information to the State as required.

The Contractor shall comply with the following:

a. Reporting of Total Compensation of the Contractor’s Executives.

(1) The Contractor shall report the names and total compensation of each of its five

most highly compensated executives for the Contractor’s preceding completed Fiscal Year, if in the Contractor’s preceding Fiscal Year it received:

i. 80 percent or more of the Contractor’s annual gross revenues from federal

procurement contracts and federal financial assistance subject to the Transparency Act, as defined at 2 CFR 170.320 (and subawards); and

ii. $25,000,000 or more in annual gross revenues from federal procurement

contracts (and subcontracts), and federal financial assistance subject to the Transparency Act (and subawards); and

iii. The public does not have access to information about the compensation of

the executives through periodic reports filed under section 13(a) or 15(d) of the Securities Exchange Act of 1934 (15 U.S.C. 78m(a), 78o(d)) or section 6104 of the Internal Revenue Code of 1986. (To determine if the public has access to the compensation information, see the U.S. Securities and Exchange Commission total compensation filings at http: //www.sec.gov/answers/execomp.htm.).

As defined in 2 C.F.R. § 170.315, “Executive” means officers, managing partners, or any other employees in management positions.

(2) Total compensation means the cash and noncash dollar value earned by the

executive during the Contractor’s preceding Fiscal Year and includes the following (for more information see 17 C.F.R. § 229.402(c)(2)):

i. Salary and bonus.

http://www.sec.gov/answers/execomp.htm.)

74

ii. Awards of stock, stock options, and stock appreciation rights. Use the dollar amount recognized for financial statement Reporting purposes with respect to the Fiscal Year in accordance with the Statement of Financial Accounting Standards No. 123 (Revised 2004) (FAS 123R), Shared Based Payments.

iii. Earnings for services under non-equity incentive plans. This does not include

group life, health, hospitalization or medical reimbursement plans that do not discriminate in favor of executives, and are available generally to all salaried employees.

iv. Change in pension value. This is the change in present value of defined

benefit and actuarial pension plans.

v. Above-market earnings on deferred compensation which is not tax qualified.

vi. Other compensation, if the aggregate value of all such other

compensation (e.g. severance, termination payments, value of life insurance paid on behalf of the employee, perquisites or property) for the executive exceeds $10,000.

b. The Contractor must report executive total compensation described above to the State by the

end of the month during which this Contract is awarded.

c. If this Contract is amended to extend the Term, the Contractor must submit an executive total compensation report to the State by the end of the month in which the term extension becomes effective.

d. The Contractor will obtain a Data Universal Numbering System (DUNS) number and maintain

its DUNS number for the term of this Contract. More information about obtaining a DUNS Number can be found at: http://fedgov.dnb.com/webform/

The Contractor’s failure to comply with the above requirements is a material breach of this Contract for which the State may terminate this Contract for cause. The State will not be obligated to pay any outstanding invoice received from the Contractor unless and until the Contractor is in full compliance with the above requirements.

E.16. Survival. The terms, provisions, representations, and warranties contained in this Contract which

by their sense and context are intended to survive the performance and termination of this Contract, shall so survive the completion of performance and termination of this Contract.

http://fedgov.dnb.com/webform/

75

IN WITNESS WHEREOF,

CONTRACTOR LEGAL ENTITY NAME:

CONTRACTOR SIGNATURE DATE

PRINTED NAME AND TITLE OF CONTRACTOR SIGNATORY (above)

TN DEPARTMENT OF EDUCATION:

PENNY SCHWINN, COMMISSIONER DATE

76

ATTACHMENT A. KEY TABLES

TCAP Assessment Program Context

English Language Math Science Social Studies Grade 2 Subpart 1: 40 minutes Subpart 1: 40 minutes (no

Subpart 2: 40 minutes calculator) Subpart 3: 40 minutes Subpart 2: 42 minutes (no Subpart 4: 42 minutes calculator) Total: 162 minutes Total: 82 minutes

Grade 3 Subpart 1: 80 minutes Subpart 1: 45 minutes (no *Field Test* (writing) calculator) Subpart 1: 50 Subpart 2: 35 minutes Subpart 2: 30 minutes Total: 50 minutes Subpart 3: 35 minutes Subpart 3: 40 minutes Subpart 4: 30 minutes Total: 115 minutes Total: 180 minutes



Grade 6 Subpart 1: 85 minutes Subpart 1: 45 minutes (no *Field Test* Subpart 1: 50 (writing) calculator) Subpart 1: 50 Subpart 2: 50 Subpart 2: 50 minutes Subpart 2: 30 minutes Total: 75 minutes Total: 100 Subpart 3: 50 minutes Subpart 3: 50 minutes minutes Subpart 4: 45 minutes Total: 125 minutes Total: 230 minutes



End of Course (EOC)

English I/II Subpart 1: 85 minutes (writing) Subpart 2: 50 minutes Subpart 3: 95 minutes Total: 230 minutes

Integrated I/II/III, Algebra I/II, Geometry Subpart 1: 35 minutes (no calculator)

Biology *Field Test* Subpart 1: 50 minutes Total: 75 minutes

US History Subpart 1: 50 minutes (writing) Subpart 2: 45 minutes Subpart 3: 45

TCAP Assessment Administration Context

Content Area

Test Administration Mode Approx. # Students

Social Studies

Grades 3, 4, 5, 6, 7, & 8

1 x yearly (spring)

Paper-based (2018-19) – possible move to Computer- Based for grades 5-8 (please reflect CBT or PBT

74,000/grade

Social Studies

Alternate Grades 3, 4, 5, 6, 7, 8 & High

1 x yearly (spring)

Paper-based 7000

Science Grades 3 & 4 1 x yearly Paper-based 74,000/grade

77

Content Area

Test Administration Mode Approx. # Students

Science Grades 5, 6, 7 &

1 x yearly (spring)

Computer-Based 74,000/grade

Science Alternate Grades 3, 4, 5, 6, 7, 8 & High

1 x yearly (spring)

Paper-based 7000

ELA Alternate Grade 2

1 x yearly (spring)

Paper-based 500

Math Alternate Grade 2

1 x yearly (spring)

Paper-based 500

ELA Grades 2, 3, 4, 5, 6, 7, & 8

1 x yearly (spring)

Paper-based (2018-19) – possible move to Computer- Based for grades 5-8 in future years (please reflect CBT or PBT option in costs). Possible move to early Administration of subpart 1

74,000/grade

Math Grades 2, 3, 4, 5, 6, 7, & 8

1 x yearly (spring)

Paper-based (2018-19) – possible move to Computer- Based for grades 5-8 in

74,000/grade

Science Biology EOC 2 x yearly (fall & spring)

Computer-Based 15,000 (fall), 59,000 (spring)

Math Algebra 1 EOC, Algebra 2 EOC, Geometry EOC, Integrated Math 1 EOC, Integrated Math 2 EOC, Integrated Math 3 EOC

2 x yearly (fall & spring)

Computer-Based Alg 1: 6,600 (fall), 56,000 (spring) Alg 2: 17,000 (fall) 46,000 (spring) Geo: 14,000 (fall) 47,000 (spring) IM 1: 650 (fall), 14,000 (spring) IM 2: 1100 (fall), 13,000 (spring) IM 3: 900 (fall), 8,200 (spring)

ELA English 1 EOC, English 2 EOC


Computer-Based Possible move to early Administration of subpart 1 (essay portion) for future years.

Eng 1: 13,000 (fall) 62,000 (spring) Eng 2: 15,000 (fall) 58,000 (spring)

Social Studies

US History EOC


Computer-Based Possible move to early Administration of subpart 1 (essay portion) for future years.

USH: 14,000 (fall) 44,000 (spring)

Note: Alternate assessment for ELA and mathematics grades 3-8 and EOC is MSAA, and is not within the scope of this RFP.

TCAP Assessment Forms Specifications

Grades 2–8 (Spring Administration Only) Content

Area

Grade Level

Estimated Number of

Forms

Total Number of Items

Number of FT Items

Number of OP Items

Estimated Number of

pages

Color/B& W

Print

Math 2 3 55 6 49 50 - 75 Color

3 14 58 8 50 50 - 75 B&W

78

Content Area

Grade Level

Estimated Number of

Forms

Total

Number of Items

Number of FT Items

Number of OP Items

Estimated Number of

pages

Color/B& W

Print

? 4 14 58 8 50 50 - 75 B&W

5 14 60 8 52 50 - 75 B&W

6 14 62 11 51 50 - 75 B&W

7 14 62 11 51 50 - 75 B&W

8 14 61 11 50 50 - 75 B&W

ELA

2 3 107 21 86 100 - 120 Color

3 14 41 11 30 75 - 100 B&W

4 14 43 11 32 75 - 100 B&W

6 14 47 11 36 75 - 100 B&W

7 14 47 11 36 75 - 100 B&W

8 14 46 11 35 75 - 100 B&W

Science

3 4 30 4 26 30 - 40 B&W

4 4 30 4 26 30 - 40 B&W

5 3 54 9 45 60 - 80 B&W

6 3 54 9 45 60 - 80 B&W

7 3 54 9 45 60 - 80 B&W

8 3 54 9 45 60 - 80 B&W

Social Studies *

3 15 30 4 26 40 - 50 B&W

4 15 30 4 26 40 - 50 B&W

5 15 50 10 40 60 - 80 B&W

6 13 50 10 40 60 - 80 B&W

7 13 50 10 40 60 - 80 B&W

8 15 50 10 40 60 - 80 B&W

* Social Studies Grades 3-5 provided in case the grade levels are reinstated during the Contract

EOC – Fall Administration Course Estimated

Number of Forms

Total

Number of Items

Number of FT Items

Number of OP Items

Estimated Number of

pages

Color/B& W Print

Algebra 1 1 62 11 51

60-75*

B&W

Algebra 2 1 63 11 52 B&W

Geometry 1 61 11 50 B&W

Int Math 1 1 61 11 50 B&W

Int Math 2 1 59 11 48 B&W

Int Math 3 1 60 10 50 B&W

English I 1 47 11 36 B&W

English II 1 47 11 36 Color

U.S. History 1 60 10 50 B&W

Biology 1 53 8 45 B&W

* TCAP/EOC Assessments are primarily Computer-Based tests; paper-based forms are available as an accommodation and therefore only require a very small number.

EOC – Spring Administration

79

Course Estimated Number of

Forms

Total

Number of Items

Number of FT Items

Number of OP Items

Estimated Number of

pages

Color/B&W Print

?

Algebra 1 11 62 11 51

60-75*

B&W

Algebra 2 11 63 11 52 B&W

Geometry 11 61 11 50 B&W

Int Math 1 3 61 11 50 B&W

Int Math 2 3 59 11 48 B&W

Int Math 3 2 60 10 50 B&W

English I 14 47 11 36 B&W

English II 14 47 11 36 Color

U.S. History 15 60 10 50 B&W

Biology 4 53 8 45 B&WWW W

* TCAP/EOC Assessments are primarily CBT; PBT is available as an accommodation and therefore only requires a very small number.

Alternate* (Spring Administration Only)

Content

Are

Grade Level

Estimated Number of

Forms

Total

Number of Items

Number of FT Items

Number of OP Items

Estimated Number of

pages

Color/B& W

Print

Grade 2

Math 2 38 6 32

50-60 for

Test Book

60-70 for Directions for Test

Administrators (DTA)

Color

ELA 2 36 8 28 Color

Science

3 2 30 4 26 B&W

4 2 24 4 20 B&W

5 2 24 4 20 B&W

6 2 24 4 20 B&W

7 2 24 4 20 B&W

8 2 24 4 20 B&W

Biology 2 24 4 20 B&W

Social Studies* *

3 2 24 4 20 B&W

4 2 24 4 20 B&W

5 2 24 4 20 B&W

6 2 24 4 20 B&W

7 2 24 4 20 B&W

8 2 24 4 20 B&W

* Alternate Assessments for ELA and mathematics grades 3-8 and EOC are MSAA and are therefore not within the scope of this RFP.

** Social Studies Grades 3-5 provided in case the grade levels are reinstated during the Contract

80

LIQUIDATED DAMAGE EVENT (i.e., the Contractor’s failure to meet a requirement under this Contract; all liquidated damage amounts are based on a cost per State business day that continue to accrue until such

time as the Liquidated Damage Event ends)

Cost per State business day

ATTACHMENT B

LIQUIDATED DAMAGES

LIQUIDATED DAMAGE EVENT (i.e., the Contractor’s failure to meet a requirement under this Contract; all

liquidated damage amounts are based on a cost per State business day that continue to accrue until such time as the

Liquidated Damage Event ends)


A.4. Program Management

A.4.a.(1) Annual Work Plan – the Contractor’s failure to deliver the Annual Work Plan by the dates required by this Contract shall subject the Contractor to Liquidated Damages for each business day until delivery of the Annual Work Plan to the State.

$25,000.00

A.7. Psychometric and Research Activities

A.7.c. Field Test Item Data Files - the Contractor’s failure to provide Field Test Data Files as required by this Contract shall subject the Contractor to Liquidated Damages for each business day after the Contractor receives a notice of non-compliance from the State until the non-compliance is remedied by the Contractor.

$25,000.00

A.7.h. Technical Report - the Contractor’s failure to timely develop, publish, and deliver the Technical Report to the State by the dates required by this Contract shall subject the Contractor to Liquidated Damages for each business day until development, publication, and delivery of the Technical Report.

$10,000.00

A.8. Assessment Materials

A.8.b.(1) Student Demo - the Contractor’s failure to timely complete a Student Demo by the dates required by this Contract shall subject the Contractor to Liquidated Damages for each business day until completion of the Student Demo.

$15,000.00

A.8.d.(1) User's Guide CBT System - the Contractor’s failure to timely complete and deliver the User’s Guide for the CBT System by the dates required by this Contract shall subject the Contractor to Liquidated Damages for each business day until completion and delivery of the User’s Guide for the CBT System to the State or delivery to such persons, schools, or LEAs as directed by the State.

$30,000.00

81





Any of the following: A.8.c.(2) Large Print Test Book - the Contractor’s failure to timely provide samples of the Large Print Test Book by the dates required by this Contract or the Contractor’s failure to provide the same to the State, or such persons, schools or LEAs as directed by the State by the times required by the Contract, shall subject the Contractor to Liquidated Damages for each business day until a sample Large Print Test Books is so provided. A.8.d.(2) Assessment Administration Manual (“TAM”) Development (include web optimized pdf version) - the Contractor’s failure to timely provide samples of the TAM by the dates required by this Contract or the Contractor’s failure to provide the same to the State, or such persons, schools or LEAs as directed by the State by the times required by the Contract, shall subject the Contractor to Liquidated Damages for each business day until a sample TAM is so provided. A.8.d.(2) Test Administrator Proctor Scripts (“TAPS”) Development (include web optimized pdf version) - the Contractor’s failure to timely deliver samples of the TAPS by the dates required by this Contract or the Contractor’s failure to deliver final completed TAPS to the State, or such persons, schools or LEAs as directed by the State, by the times required by the Contract, shall subject the Contractor to Liquidated Damages for each business day until the TAPS samples or final completed TAPS are so delivered. A.8.e.(1) Student Test Booklets – Two Content Areas - the Contractor’s failure to timely provide Student Test Booklets – Two Content Areas – to the State, or to the persons, schools, or LEAs as directed by the State, by the dates required by this Contract, shall subject the Contractor to Liquidated Damages for each business day until Student Test Booklets – Two Content Areas – are provided in accordance with this Contract. A.8.e.(1) Student Test Booklets – One Content Area - the Contractor’s failure to timely provide Student Test Booklets – One Content Areas – to the State , or to the persons, schools, or LEAs as directed by the State, by the dates required by this Contract, shall subject the Contractor to Liquidated Damages for each business day until Student Test Booklets – One Content Areas – are provided in accordance with this Contract. A.8.e.(2) Student Response Documents – Two Content Areas - the Contractor’s failure to timely provide Student Response Documents – Two Content Areas – to the State, or to the persons, schools, or LEAs as directed by the State, by the dates required by this Contract, shall subject the Contractor to Liquidated Damages for each business day until Student Response Documents – Two Content Areas – are provided in accordance with this Contract.

A.8.e.(2) Student Response Documents – One Content Area – Two Content Areas - the Contractor’s failure to timely provide Student Response Documents – One Content Areas – to the State, or to the persons, schools, or LEAs as directed by the State, by the dates required by this Contract, shall subject the Contractor to Liquidated Damages for each business day until Student Response Documents – One Content Areas – are provided in accordance with this Contract.

$25,000.00

82





A.9. Computer-Based Testing Delivery System

A.9.a. CBT Platform (includes application, maintenance, customizations, and updates) - the Contractor’s failure to meet the requirements of Section A.9.a. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by the Contractor.

$65,000.00

A.9.a. CBT System Specifications (includes technical; requirements, website hosting) - the Contractor’s failure to meet the requirements of Section A.9.a. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by the Contractor.

$75,000.00

A.6. Quality Assurance (includes any breach of a technical requirement) - the Contractor’s failure to meet the requirements of Section A.6. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by the Contractor.

$50,000.00

A.10. Assessment Provisioning Activities

A.10.a. Authorized Use - the Contractor’s failure to meet the requirements of Section A.10.a. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by the Contractor. or A.10.b. Enroll Students - the Contractor’s failure to meet the requirements of Section A.10.b. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by the Contractor.

$25,000.00

A.12. Scoring Activities

83





A.12.d. Comprehensive Data Files (CDF) - the Contractor’s failure to meet the requirements of Section A.12.d. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by

the Contractor.

$50,000.00

A.13. Reporting Activities

The Contractor’s failure to meet the requirements of Section A.13. of this Contract shall subject the Contractor to Liquidated Damages for each business day from the time the Contractor receives a notice of non-compliance (specifying the reasons for non-compliance) from the State until the non-compliance has been remedied by the Contractor. Events giving rise to a notice of non-compliance with Section A.13. and subjecting the Contractor to Liquidated Damages includes, without limitation, any of the following: A.13 Student Reports grade 3-8 A.13 EOC Student Reports A.13 Student Labels grades 3-8 A.13 EOC Student Labels A.13 Raw Score Report A.13 Standard Analysis Report A.13 Summary Report A.13 Roster Report A.13.Parent Brochure

$25,000.00

ATTACHMENT C

AT TES T AT IO N RE PERSON NEL USE D IN CO NT R AC T P ERFO RM AN C E

If the attestation applies to more than one contract, modify this row accordingly.

SUBJECT CONTRACT NUMBER:

CONTRACTOR LEGAL ENTITY NAME:

EDISON VENDOR IDENTIFICATION NUMBER:

If the attestation applies to more than one contract, modify the following paragraph accordingly.

The Contractor, identified above, does hereby attest, certify, warrant, and assure that the Contractor shall not knowingly utilize the services of an illegal immigrant in the performance of this Contract and shall not knowingly utilize the services of any subcontractor who will utilize the services of an illegal immigrant in the performance of this Contract.

CONTRACTOR SIGNATURE

NOTICE: This attestation MUST be signed by an individual empowered to contractually bind the Contractor. Attach evidence documenting the individual’s authority to contractually bind the Contractor, unless the signatory is the Contractor’s chief executive or president.

PRINTED NAME AND TITLE OF SIGNATORY

DATE OF ATTESTATION