request for proposal€¦ · web viewdevelopment of. methods and training for. assessing...

REQUEST FOR PROPOSALS

DEVELOPMENT OF

METHODS and TRAINING forASSESSING PERSONAL-EXPERIENCE OUTCOMES

for Adults with Developmental or Physical Disabilities and Frail Eldersin Wisconsin’s Medicaid-funded HCBS and

Managed Long-Term Care Programs

RFP # 1571-DDES-SMProposals due November 9, 2006

September 27, 2006

Wisconsin Department of Health and Family Services

Request for Proposal: Development of Methods and Trainingfor Assessing Personal-experience Outcomes

Table of ContentsSECTION I. GENERAL INFORMATION..........................................................................1I-1 Purpose of this Solicitation............................................................................................................................1

I-2 Procuring and Contracting Agency..............................................................................................................1

I-3 Funding and Duration of Contract...............................................................................................................2

I-4 Who May Submit a Proposal.........................................................................................................................2

I-5 Project Background........................................................................................................................................3

SECTION II. SCOPE OF THE PROJECT........................................................................8II-1 Deliverables.....................................................................................................................................................8

11-2 Approval of Deliverables.............................................................................................................................13

II-3 Project Description.......................................................................................................................................14

SECTION III. PREPARING AND SUBMITTING A PROPOSAL...................................18III-1 Notice of Intent to Submit a Proposal.........................................................................................................18

III-2 Timeline.........................................................................................................................................................18

III-3 General Instructions.....................................................................................................................................18

III-4 Incurring Costs.............................................................................................................................................19

III-5 VendorNet.....................................................................................................................................................19

III-6 Right to Reject Proposals and Negotiate Contract Terms........................................................................19

III-7 Proposer’s Conference.................................................................................................................................19

III-8 Presentations and Interviews.......................................................................................................................20

III-9 Clarification and/or Revisions to the Specifications and/or Contract Requirements............................20

III-10 Executed Contract to Constitute Entire Agreement.................................................................................21

III-11 Reasonable Accommodations......................................................................................................................21

III-12 Submitting the Proposal...............................................................................................................................21

III-13 Standard Terms and Conditions.................................................................................................................22

III-14 Multiple Proposals........................................................................................................................................22

III-15 Withdrawal of Proposals.............................................................................................................................22

SECTION IV. PROJECT PROPOSAL...........................................................................24IV-1 Proposal Format and Organization............................................................................................................24

IV-2 Request for Proposal Cover Sheet (DOA – 3261)......................................................................................25

IV-3 Part I - Executive Summary – 3 pages........................................................................................................25

IV-4 Part II – Proposal.........................................................................................................................................25

IV-5 Part III – Attachments.................................................................................................................................33

IV-5 Part IV – Required forms............................................................................................................................34

Wisconsin Department of Health and Family Services September 2006

i

Request for Proposal: Development of Methods and Trainingfor Assessing Personal-experience Outcomes

SECTION V. SELECTION AND AWARD PROCESS...................................................35V-1 Review and Evaluation of Proposals...........................................................................................................35

V-2 Awarding of Contract..................................................................................................................................39

V-3 Termination of Contract..............................................................................................................................40

SECTION VI. FORMS AND ATTACHMENTS...............................................................41VI-1 Notice of Intent to Submit............................................................................................................................41

VI-2 Reference List of Clients..............................................................................................................................41

VI-3 Organizational Chart...................................................................................................................................41

VI-5 Required Procurement Forms.....................................................................................................................41

VI-6 Submission Check List – one page..............................................................................................................42

APPENDIX A: DEFINITIONS OF TERMS USED IN THIS RFP....................................43APPENDIX B: PERSONAL-EXPERIENCE OUTCOMES FOR LONG-TERM CARE...45APPENDIX C: RESUME FORMAT...............................................................................48APPENDIX D: REFERENCES AND RESOURCES......................................................49APPENDIX E: FIFTH AND SIXTH DELIVERABLES....................................................50APPENDIX F: RFP EVALUATION CRITERIA..............................................................53


ii

Request for Proposal: Development of Methods and Training for Assessing Personal-experience Outcomes

Section I. General Information

I-1 Purpose of this Solicitation

The State of Wisconsin, as represented by the Department of Health and Family Services, Division of Disability and Elder Services, Bureau of Long-term Support, intends to use the results of this solicitation to award a contract for the development of a tool and related tasks which will assess personal-experience outcomes with regard to quality of life for certain individuals1 and other activities related to the implementation and use of that assessment tool.

This document provides interested parties with information to enable them to prepare and submit a proposal for a project that includes developing:

1) An interview tool for eliciting personal preferences and priorities for each of the 12 outcomes, with unique but related versions for each of four target groups;

2) Instructions for the conducting, recording, and scoring of interviews;3) A training program and materials for care managers and for quality

reviewers, and4) A method for assessing interviewers’ reliability in administering the

outcomes interview.

Concurrently with the release of this RFP, the Department is separately issuing an RFI for two additional deliverables, in addition to the four for which this RFP is seeking a contractor. Work on the first four deliverables must be planned and proceed with the awareness that the Department’s intention is eventually to complete the additional deliverables, which are described in Appendix E.

I-2 Procuring and Contracting Agency

This Request for Proposal (RFP) is issued by the Wisconsin Department of Health and Family Services, which is the sole point of contact for the State of Wisconsin during the selection process. The person responsible for managing the procurement process is:

Susan McKercherPurchasing ManagerDepartment of Health and Family ServicesDivision of Management and Technology1 West Wilson Street, Room 750Madison, WI 53702

1 Adults with developmental or physical disabilities and frail elders, who are receiving long-term care services through the Wisconsin Medical Assistance Program.


-- 1 --


(608) [email protected] (608) 264-9874

The Program Manager/Contact is:

Sharon RyanDepartment of Health and Family ServicesDivision of Disability and Elder Services1 West Wilson Street, Room 518Madison, WI 53702(608) [email protected]

The contract resulting from this RFP will be administered by the Wisconsin Department of Health and Family Services. The Contract Administrator is Karen McKim, Quality and Research Manager, Division of Disability and Elder Services.

The Department of Health and Family Services is the Medicaid agency for the State of Wisconsin. The Department administers the programs that provide home- and community-based Medicaid waiver services.

I-3 Funding and Duration of Contract

Funding for the four deliverables is $300,000 and is available through December 31, 2007. With this RFP, the Department will contract for the four deliverables described beginning on page 8.

The Department is issuing a separate RFI for two additional deliverables, which are described in Appendix E. That RFI seeks information and estimates from interested parties regarding how the additional two deliverables might be accomplished, over what period of time, and how much such a project might cost. Proposers responding to this RFP are not required to respond to the RFI, and responses to the RFI will have no bearing on the award of the contract resulting from this RFP.

I-4 Who May Submit a Proposal

Any profit or non-profit organization or entity, government agency, or educational institution may apply, and partnerships among any organizations, agencies and institutions that are formed for the specific purpose of this project may also apply.

The proposal must describe the extent of the proposer’s intended physical presence in Wisconsin during the project. The successful proposer will be required to maintain extensive contact and involvement with consumers and also with


-- 2 --

mailto:[email protected]


Department and other staff in Wisconsin over the course of the contract. Project staff will need frequent physical presence in Wisconsin, sometimes at short notice.

I-5 Project Background

A. Why does the presence of personal-experience outcomes need to be assessed?

A strong focus on quality-of-life outcomes has long been a hallmark of Wisconsin’s Medicaid-funded long-term care programs that provide home and community-based long-term care. These programs include Community Options Program (COP) and Community Options Program Waiver (COP-W), the Community Integration Program and Brain Injury Waiver Program (CIP and BIW), and more recently the managed-care waivers of Family Care (FC), and the Wisconsin Partnership Program (WPP). The current expansion of managed long-term care will continue to incorporate the concepts of personal-experience outcomes as a part of their efforts in quality assurance and program rationale.

Objectively verifiable health, safety, and functional status outcomes are and always will be among these programs’ intended results. Assessment methods for these outcomes are relatively well established and are used in both long-term and acute medical care.

However, unlike time-limited medical care, long-term care continues over the course of a person’s lifetime and as a result has a greater potential for either enhancing or impairing aspects of a person’s quality of life, including housing, employment, community participation, and even the extent to which he or she can develop friendships and maintain relationships with family members. Poor-quality long-term care limits and constrains a person’s options in these areas; high-quality long-term care supports a person’s ability to live his or her life consistent with his or her desires to the greatest extent possible.

As a result, long-term care must attend to a broader set of quality-of-life results than does acute care, results that are beyond and in addition to clinical and functional outcomes. These outcomes must be woven into the operations of the programs, by:

Assessing each individual’s desired personal-experience outcomes as part of his or her assessment and care-planning process (that is, determining what personal-experience outcomes are desired as part of his or her quality of life and to what extent those outcomes are present in the person’s life). The long-term care program standards and contract language requires that a person centered assessment , and that the subsequent service plan development include a person centered planning process where personal-experience outcomes are identified;


-- 3 --


Ensuring that quality-management efforts discover the extent to which person-specific personal-experience outcomes are identified and achieved, which will be the cornerstone of overall quality monitoring and continuous quality improvement efforts of both the Department of Health and Family Services (DHFS) and the local care-management agencies; and

Reporting the program’s performance in achieving personal-experience outcomes to the program’s stakeholders.

Although assessing quality-of-life outcomes may be more difficult and less objective or empirical than, for example, assessing physical health status, personally experienced quality of life can be assessed in a rigorous and reasonably reliable way. In fact, if long-term care professionals are to devise or assist consumers of long-term care in devising plans of care that truly support and do not interfere with consumers’ desired personal experience of their own quality of life, ways to reliably assess personal-experience outcomes must be identified. It is through this identification that quality of long-term care programs can be monitored and improved.

B. What are personal-experience outcomes?

First, they focus on personal experience. Anyone can ascertain or even dictate the recreational activities of any other individual, but only the one who participates in the recreation can judge whether those activities enhance his or her quality of life. Similarly, others might ascertain or decide when and what an individual eats, but only the person who is eating can judge his/her personal experience of the meal. A dream job for one person might be insufferable for another; it is the personal experience of the job that determines whether it contributes to quality of life. A situation that one person experiences as restful privacy might give another person an experience of frightening isolation.

Second, they are outcomes. They are conditions or circumstances that are of value to the individual in and of themselves, rather than as means to ends. They do not describe the quality of, or satisfaction with services; they describe the quality of life.

As such, personal-experience outcomes sometimes include circumstances that cannot be attained through the provision of even the highest-quality long-term care services. For example, the death of a beloved life-partner will unavoidably impair the survivor’s sense of continuity and security in his or her life. However, providers of high-quality long-term care can continue to support continuity and security in other ways and possibly help to prevent additional avoidable negative consequences.

The Department is in the final stages of working with consultants and advisory groups of long-term care consumers and local-agency staff on a project, known as


-- 4 --


Quality Close to Home, that reviewed our current methods of assessing personal-experience outcomes and that adopted a set of 12 high-level statements that represent the personal-experience outcomes that our long-term care programs aspire to support for their members, which are included as Appendix B. These 12 outcomes cover the following areas:

Residential situation; Choice of services and supports; Daily routine; Relationships with family and friends; Meaningful activities; Community involvement; Continuity and stability; Respect and fair treatment; Privacy; Health; Perception of safety; and Freedom from abuse and neglect.

C. How have personal-experience outcomes been used in Wisconsin?

Personal-experience outcomes (though not articulated precisely as this set of 12 outcomes) have been used for many years in Wisconsin’s Home and Community-Based Services (HCBS) long-term care programs, for three purposes:

Assessing desired results for individuals in the process of care planning; Assessing achievement of outcomes for quality-management purposes;

and Reporting aggregate results to stakeholders.

Personal-experience outcomes in care planning: The CIP and BIW, COP-W, FC and WPP have all placed significant emphasis on person-centered care planning, which requires identification of personal-experience outcomes. The CIP and BIW, COP-W programs require each assessment to include information about the individual’s desired personal-experience outcomes. In the Medical Assistance (MA) Waiver Manual the following language appears with regard to requirements to identify and then arrange support and service surrounding personal-experience outcomes: The service/support plan shall be arranged in such a manner as to “...enable the applicant to meet or maintain their individual personal-experience outcomes...”. In the managed-care organizations (FC and WPP) one of the contract elements is as follows with regard to the MCO responsibilities in the area of quality management: “(The MCO) demonstrates improvement in the support provided to consumers in achieving their desired outcomes”.

In addition, in 2004 a special project surrounding persons diagnosed with Alzheimer’s disease developed a very specific set of assessment method for


-- 5 --


individual outcomes, to be used by service providers. The Developmental Disabilities unit has provided person-centered training using tool such as Essential Lifestyle Planning (ELP) which is grounded in personal-experience outcomes thinking for many years. Other training efforts of the Department have provided general training in the principles of outcomes-based care planning. A recent web-based training aims to provide the student with a basic understanding of personal-experience outcomes care planning and will soon be available.

However, beyond the efforts mentioned above, no program has adopted a comprehensive interview tool or method for care managers to use when assessing and creating care plans surrounding personal-experience outcomes. The assessment of desired outcomes is not universally or reliably incorporated into assessment tools/forms. In addition, care managers in all programs express a desire for additional guidance and training in the assessment of personal-experience outcomes.

Personal-experience outcomes in quality management: While we know of no local agency that has assessed personal-experience outcomes as part of its own quality-management program, the CIP and BIW, COP and COP-W, FC and WPP have each used measurement of personal-experience outcomes in their state-level quality management efforts. These efforts have had varied success; the quality reviewers have provided care managers with useful feedback in individual cases, whereas there is limited experience with significant quality improvement resulting from aggregate results. Within all of the waiver programs, state-level and other quality management staff believes that outcomes measurement has much more potential value for quality management than has been realized.

Personal-experience outcomes in stakeholder reporting: The Department conducted personal-experience outcomes interviews in both the WPP and FC programs and was able to report the results in an aggregate way for the purposes of stakeholder reporting, with those reports being favorably received. The reports helped to establish support of members’ personal-experience outcomes as a primary part of the programs’ mission, among both internal and external stakeholders.

Quality Close-to-Home Project: As a part of the soon-to-be complete three-year Quality Close to Home CMS Grant awarded to the Department, Wisconsin DHFS has determined that it will include measuring the achievement of personal-experience outcomes as a part of a comprehensive quality management approach. A review of existing tools and methods used both in Wisconsin and also nationally occurred early in the grant period.

In order to be most effective, any measurement tool must be standardized. A review of the Participant Experience Survey (PES) determined that tool did not meet DHFS’ expectations in the area of personal-experience outcomes measurement.


-- 6 --


There is a strong demand within the state for a single uniform method of assessing personal-experience outcomes that can be used by both care managers and by other quality management staff. For collaboration between care managers and others involved in quality management to have value, this interaction depends upon their ability to communicate with a shared language and concepts.

The successful proposal will integrate the existing efforts within the Department into the new project’s deliverables. In short, this project must build on our experiences to date and create a new, fully useful method to measure personal-experience outcomes.

Managed long-term care expansion: The Department is engaged in transitioning the current long-term care system care into a managed-care system. This will require changes in both the current managed-care pilot organizations (FC and WPP) along with the local agencies that are now managing fee-for-service waiver programs.

The selected entity can expect to see the emergence of managed-care organizations through the duration of this contract, and the deliverables of this contract will need to be well suited to the reorganization taking place. A tool which is useful to care managers properly positions the concept of personal-experience outcomes within our long-term care service delivery system. With the renewed sense of quality management occurring as managed care emerges, we believe it is the right time to introduce this refinement and innovation.

In addition, even for the counties that transition later to managed care, the CIP, COP and BIW Waivers have recently added a new requirement to the Individual Service Plan form which requires that the care manager identify and list personal-experience outcomes alongside service planning for each waiver participant.


-- 7 --


Section II. Scope of the Project

II-1 Deliverables

All deliverables will be in the public domain; the contractor will retain no exclusive rights related to their reproduction, distribution, use, or revision.

With this RFP, the Department is seeking to contract with a consultant who can lead a project to assist Wisconsin’s long-term care officials, providers, and consumers in developing the following four deliverables.

A. An interview tool for eliciting personal preferences and priorities for each of the 12 outcomes, with unique but related versions for:

Elders who are frail who also possess cognitive and language-based communications abilities,

Adults with physical disabilities who also possess cognitive and language-based communications abilities;

Adults who are developmentally disabled who also possess cognitive and language-based communications abilities; and

Adults who have severe cognitive limitations and who are without language-based communication abilities. (Typically these persons communicate both through alternatives to language means and with the assistance of a designated proxy).

To ensure that the outcomes identified truly reflect those which are individualized and personal-experience based, we expect that the interview tool will need to take the form of a discussion guide rather than a set of required or standardized questions.

Existing interview tools with elements of this type that are currently used in Wisconsin are the Community Options Program (COP), the Developmental Disabilities Section’s tool (PROACT) and the Recovery-oriented Systems Assessment (ROSA) tool2, outcome-assessment tool used by quality reviewers in the community mental-health programs. In the past, the other waiver programs have used similar tools and also those based in person-centered planning on occasion.

The need for a discussion-type interview rather than a set of standard questions stems from several causes. First, few individuals understand what an ‘outcome’ is as the term is used in long-term care. Care managers have found that asking consumers straightforward questions about what they hope to get out of long-term care tends to yield requests for specific services or goods. More directed

2 http://dhfs.wisconsin.gov/dsl_info/NumberedMemos/DDES/CY_2005/2005-21appendixB.pdf


-- 8 --

http://dhfs.wisconsin.gov/dsl_info/NumberedMemos/DDES/CY_2005/2005-21appendixB.pdf


questioning is needed to draw a discussion of the things that truly constitute quality of life for each person individually.

Second, each of the 12 outcomes is a high-level statement that contains more specific elements (see Appendix B), so that conversations are necessary to explore the underlying content. The individual’s unique preferences for each outcome must be solicited to determine whether the high-level outcome is present for that person (e.g., what is it about this living situation that makes it desirable to the individual? What does the individual want to do in the community, if anything?) This need to ascertain preferences specific to each individual rules out the use of standardized survey-type questions.

Third, care managers need conversational interviews for the additional purpose of assessing the relative importance each consumer places on the various desired outcomes. The individual’s strength of preference must be solicited if the care manager is going to be able to set appropriate priorities in the care planning process. Rarely does a care plan need to support all 12 outcomes actively, and some sequencing is often necessary to represent the individual’s priorities. For example, an unstable health issue might need to be resolved before community participation can be at the level the person desires. Likewise, the care manager needs to understand the person’s preferences for outcomes that currently seem unimportant. These outcomes might not be important to the consumer either because the consumer genuinely does not have a preference, or because the consumer has a preference that is currently achieved. The effective care manager understands priority ranking of personal-experience outcomes from the perspective of the individual consumer; it is essential for the process to remain driven by the individual’s desires and preferences.

While being conversational, the interviews must also have a basic consistency of content with regard to the underlying elements of the high-level outcomes. Consistency is typically promoted by increasing the amount of structure in the measurement and reducing the amount of interviewer judgment required. However, we are seeking a discussion-based interview tool appropriate to the infinite variety of individual preferences for these outcomes. The successful proposer will have the ability to draw upon or devise other ways to promote consistency among interviews. Most of the outcome interview tools that have been used by DHFS HCBS programs include suggested interview questions to elicit consistent, comparable information from the individual, and a second set of standardized questions to guide the interviewer’s determination as to whether the outcome is present.

Another challenge to reliability is the subjective nature of the information the interview is intended to discover—the interview seeks to discover how each person feels about things such as his or her job, safety, roommate and health. Moods and temporary setbacks can affect feelings, but the interview tool will need to help the


-- 9 --


interviewers discover a general ongoing level of satisfaction with the various components of quality-of-life.

Although this general set of outcomes is intended to describe quality of life for all target groups and all individuals, separate versions of the tool are needed for the different target groups. For some outcomes, the desired results are likely to be different enough to merit at least a slightly different discussion guide. For example, although meaningful activities are a part of every individual’s quality of life, interviews with adults with physical disabilities are more likely to focus on employment issues in this area than are interviews with elders who are frail. Similarly, although physical health is a part of every individual’s quality of life, discussions about physical health with elders who are frail are likely to have a different focus than interviews with adults with development disabilities who may be in early adulthood.

To ensure its validity with all members who are served by these long-term care programs, the interview tool will also need specific questions and interviewing techniques for persons with serious cognition difficulty and/or language based communication difficulty. Clearly articulated and explained methods for working with this target group are a key to the ultimate acceptance and credibility of personal-experience outcomes measurement, both as a basis for care planning and also quality management.

B. Instructions for the conducting, recording, and scoring of interviews

The interview tool will need a manual—a set of clearly written instructions—for its administration, recording, and scoring.

The first purpose of clearly written instructions is to support consistent administration of the assessment method among interviewers and over time. While classroom training and consultation with experts are necessary, experience in Wisconsin has shown that much of the care managers’ on-going skills-building is gained on the job and through collaboration with colleagues. Care managers must be able to learn from, and later refer to, accessible, normative written instructions if they are to develop their own skills and consult with their co-workers.

The second purpose of clearly written instructions is to support use of the interviews’ results by quality managers and stakeholders. Wisconsin’s experience has shown that it is important for those who will be relying upon the results of the interviews, particularly quality managers and external stakeholders, to be able to understand how the information was collected and what it represents. An assessment method that is a ‘black box’ presents a barrier to those who seek to understand and to act upon the information it provides. Instructions for the interviews must be well-written, clear, and readily available to all interviewers and stakeholders.


-- 10 --


The interview will need to lead to the creation of nominal variables (e.g., the outcome is present/is not present, or is being supported/does not now need support/is not being supported) that can be aggregated among interviews or compared over time (e.g., last year, the consumer had eight outcomes present; this year, he has ten.) Therefore, the interview tool will need to have clear methods and instructions for transforming the information gathered in interviews into nominal variables that are clear enough to support reasonable consistency between co-workers and also those in different agencies. Instructions need to differ somewhat depending on whether the tool is used for assessment and care planning or quality review.

While we expect only one tool, there will be several versions. As such, instructions on tool use will likely need to differ somewhat for assessment and care-planning uses and overall quality management use.

Care managers will be administering the interview, in most cases, to people with whom they have ongoing relationships, while quality reviewers will usually be administering the interview to people they have met through that interview.

Care managers will be using the interview tool as just one part of a broader assessment process, and so should be able to administer the interview in either a single sitting or over several meetings, depending upon the member’s needs, preferences, or schedule. Quality reviewers, on the other hand, will usually need to administer the interview within a single episode, extending it to two meetings only when necessary.

Care managers need to record individual preferences in more detail than quality reviewers, in order to provide quality assessments and service planning; quality reviewers will need to record information that is useful for feedback to care managers as a part of the larger scheme of quality improvement.

Local care-management agencies have expressed significant interest in the possible development of an automated method of reporting the results of personal-experience outcomes interviews, possibly associated with the Department’s web-based Long-Term Care Functional Screen3 (LTC FS). While the eligibility and level-of-care-determination function of the Long-Term Care Functional Screen must remain inviolate, we are interested in a method of recording and reporting the results of the personal-experience outcome interviews that could be automated, perhaps in a discrete application closely associated with the Long-Term Care Functional Screen, but clearly separate from the LTC FS.

C. Training program and materials for care managers and for quality reviewers

3 http://dhfs.wisconsin.gov/LTCare Wisconsin Department of Health and Family Services September 2006

-- 11 --

http://dhfs.wisconsin.gov/LTCare


Experience in Wisconsin has demonstrated the need for strong and effective training in outcomes-assessment, and the Department is developing training in the basic concepts and skills needed to identify personal-experience outcomes and incorporate them in care planning.

Assessing personal-experience outcomes is a highly complex skill, and training must continue over time, with the interviewers having the opportunity to gain information, practice, receive feedback, and improve. Interviewers’ skills will tend to ‘drift’ over time as they forget or unconsciously re-interpret things they learned in training, and interviewers will continually be encountering novel situations. Therefore, the training program needs to include not just curriculum materials, but a recommended sequence of learning, practicing, feedback, and consultation. It is also necessary that the training program provide managers and supervisors of the interviewers a certain level of understanding, so that they can support the interviewers in their task.

The training program will need to differ somewhat for care managers and for quality reviewers, who will be using the interview tools for slightly different purposes. For example, care managers will need to integrate the results of the outcomes interviews into assessments and service planning, while quality reviewers will need a level of reliability which is able to be established in perhaps just one interview which will provide feedback to overall quality management considerations. Skills training specific to the role of the quality reviewers other than the care managers should be a part of the training.

It is important to remember that care managers are trained by the local agencies to perform assessments and complete long-term care planning, so the outcomes-assessment training for care managers will be, at a minimum, a responsibility shared by the Department and the local care-management agencies.

This contract will design, create, test, assess, and revise a training program and materials that will, at the end of the contract, leave the Department and local long-term care management agencies with the ability to carry on that training. This contract will not include the delivery of training beyond the life of this contract. We are aware that the planned schedule for this contract, ending in December 2007, is quite limited in relation to the time usually needed for developing, testing, revising, and retesting a training program. The successful proposer will describe the most that can be done in the period of this contract, and how the proposer at the end of the contract will leave the Department with a clear plan for additional necessary activities.

D. A method for assessing interviewers’ reliability in administering the outcomes interview

In the past, the Department has used outcomes-assessment methods with varying degrees of rigor in the steps taken to ensure that interviewers reliably administer the


-- 12 --


interview over time or to different individuals. Among the techniques that could be used, separately or in combination, to assess the competence or reliability of interviewers are statistical analysis of interviewers’ findings, observations of interviews by expert interviewers, or prompt re-interviewing of a small number of interviews by an expert.

It is likely we will want the reliability-testing for care-managers and for quality-reviewers to differ somewhat. Because the administration of these interviews by care managers is intended to be an ongoing part of every initial and ongoing assessment, the process used for assessing interviewer reliability will, in practice, be one part of the quality management system for assuring the quality of the care manager assessments and service-planning. Quality reviewers, who will fulfill a role of validating results of some of the care managers’ interviews, will likely need a higher standard of reliability.

Finally, the measurement of interviewers’ reliability will need to be an ongoing endeavor, and we are expecting that development, testing, and refinement of reliability-testing methods cannot be carried out to the full extent desirable during the contract period. Therefore, the successful proposer will describe the most that can be done in the period of this contract, and how the proposer at the end of the contract will leave the Department with instructions and guidance regarding the ongoing efforts that will be needed to establish and improve inter-rater reliability over time.

11-2 Approval of Deliverables

A deliverable item is tangible evidence of work completed or work product. All deliverables and collected data, work papers, source materials and backup documentation relating to the deliverables will become the property of the State of Wisconsin.

The Contract Administrator, Karen McKim, will provide review and approval of deliverables submitted by the Contractor. Deliverables must receive a sign-off accepting the deliverable as final from the Contract Administrator before the deliverable is considered complete.

If the Contract Administrator identifies a deficiency in a deliverable, the Contractor shall resubmit the deliverable in a corrected form.

After a deliverable is approved, five copies are to be submitted to the Contract Administrator. A draft of a deliverable may be requested prior to the due date of the deliverable.

The Department reserves the right to renegotiate due dates of deliverables with the contractor to ensure that the reports correspond to Department internal deadlines or to Terms and Conditions of the federal governments.


-- 13 --


The contractor is responsible for the accuracy and completeness of the information contained in all technical documents and reports submitted as a deliverable.

II-3 Project Description

The Department does not have a well-developed or specific plan that we intend to impose on the process of developing and producing this contract’s deliverables. The successful proposer will consider the following ideas and then use the proposer’s expertise to develop and propose a more detailed workplan that will achieve the intended goals for the contract.

A. Take advantage of experience

Wisconsin’s long experience and experimentation with personal-experience outcomes has created a body of knowledge and expertise that can and should be incorporated at the start and all through the contract. For the sake of the contract’s efficiency and to ensure that contract results are recognizable to and accepted by Wisconsin’s long-term care community, the contract will need to draw upon the following:

Many consumers in Wisconsin understand the concept of personal-experience outcomes and some have been interviewed about their personal-experience outcomes multiple times. The contractor will establish good working relationships with consumers and consumer groups representing each of the four target groups and obtain substantive input from them, particularly on the content validity of the outcomes-measurement method and the interview burden it places on interviewees.

Department staff and managers have worked with the results of outcomes-assessment interviews, and have well-developed opinions on features that are more useful and less useful. The contractor will need to obtain substantive input from them regarding their experience with, and needs for, results from the various outcomes-assessment methods.

Staff of local care-management agencies, most specifically care managers, have attempted to use different outcomes-assessment methods and have attempted to use the feedback provided by different methods, and are aware of their needs and the various strengths and shortcomings of the methods used to date. The contractor will need to establish good working relationships with the local care-management agencies that are most interested in this project, and will obtain substantive input from them regarding their needs.

Quality-management contractors in Wisconsin retain interviewers who have used different outcomes-assessment methods and who can explain the attributes


-- 14 --


of a method that would be most useful. The contractor will need to obtain substantive input from them regarding their experience with various outcomes-assessment methods.

Outcomes-assessment training programs have been created, conducted, and taken; the trainers and trainees have well-founded opinions and insight about what is useful and effective. The contractor will need to obtain substantive input from the trainers and trainees regarding the elements and attributes of an effective outcomes-assessment training program.

Other stakeholders in the long-term care programs administered by the Bureau of Long-Term Support are familiar with reports of outcomes assessments, and will be able to articulate what types of information they find useful. The contractor will need to establish good working relationships with the major stakeholders (The Wisconsin Council on Long-Term Care Reform, and leading advocacy groups) and will obtain substantive input from them regarding their experience with, and needs for, reports containing information about the long-term care programs’ outcomes.

The contractor will seek to understand and make extensive use of this expertise and insight, to support the efficiency with which this contract proceeds, to increase the likelihood that the deliverables will meet expectations and fulfill needs, and to build support and acceptance for the final product.

The contractor will need to understand that, while these parties’ extensive experience with various outcomes-assessment methods provides a wealth of insight and ideas that can be used to advance the project toward its goals, it has also left the various parties with some (not an unworkable amount of) contradictory opinions, interests, and priorities. The successful proposer will be prepared to be able to identify these contradictions and work effectively to resolve or work with them.

Appendix C includes a list of online materials and links to these materials that provide relevant background and explanation related to how Wisconsin’s long-term care programs have conceptualized, measured, and employed the measurement of personal-experience outcomes in the past. These materials will be useful to proposers as resources, references, and starting points for this contract.

B. Take advantage of interest and enthusiasm

Commitment to personal-experience outcomes as part of the central mission of long-term care services is widespread and well-established among consumers and consumer advocates and among state and local staff of the programs administered by the Bureau of Long-term Support. State-level stakeholders and the Department are very interested in incorporating reliable assessment of outcomes in the expanding managed long-term care programs, and some local care-management


-- 15 --


agencies have a strong, active desire for the successful production of the deliverables anticipated for this contract.

If the contractor establishes good working relationships with local care-management agencies, we believe that these local agencies will be willing to participate in field-testing of interim products at various stages of development, and that this involvement has the potential of providing the contractor with invaluable guidance, feedback and insight.

C. Keep stakeholders informed

Maintaining interest and enthusiasm for this project and preparing for the ready adoption of the contract’s deliverables will require the contractor to provide frequent progress updates and to accept guidance and recommendations from several stakeholder groups through the life of the contract. Among the groups who will be strongly interested in this project are:

The soon-to-be-created Quality Management Council, which is convened by the Department and comprised of local care management agencies’ quality management staff;

The Wisconsin Council on Long-Term Care (http://www.wcltc.state.wi.us); Professional groups, such as county human-services agencies committees and

providers’ associations; The Department’s Cross-unit functional teams for Quality Management and for

Training; and The Wisconsin Council on Developmental Disabilities (http://www.wcdd.org),

Council on Physical Disabilities, (http://www.pdcouncil.state.wi.us), and the Board on Aging and Long-term Care (http://longtermcare.state.wi.us/home).

D. Rapid production of first version

It is likely that review of existing outcomes-assessment methods used in Wisconsin, and familiarity with the various interested parties’ ideas and desires will provide the basis for rapid production of a first draft of an interview tool for each of the target groups, which can then be shared with various professional and stakeholder individuals or groups for comment and further development. Their input, along with the contractor’s professional expertise, could provide the basis for production of a field-testable first version.

Once the first version of the interview tools, scoring methods, and instructions are complete, the first version of the training program could be developed. Soon after, willing local agencies could field-test the products as part of their ongoing care-planning processes, and state program administrators could put it into immediate field-test in the various programs’ ongoing quality-review efforts. Clear leadership from the contractor could enable these parties to provide valuable feedback and suggestions for further refinement.


-- 16 --

http://longtermcare.state.wi.us/home

http://www.pdcouncil.state.wi.us/

http://www.wcdd.org/

http://www.wcltc.state.wi.us/


At some early stage of development, either before or during the field testing of the first version, strong and substantial input from consumers and their guardians must be obtained regarding the content-validity of the tool. The Department will be open to suggestions for revision of any of the twelve outcomes statements or their definitions, if limitations or flaws in these statements become apparent as consumers or their representatives provide insight into the validity of the outcomes-measurement method.

Specific concerted efforts will need to be made to bring together the contractor’s own expertise with that of guardians, advocates, and direct-care providers of people with significant cognitive impairments to develop credible, valid methods of assessing personal-experience outcomes for those individuals.

E. Plan ahead to provide the basis for additional work.

Concurrently with the release of this RFP, the Department is separately issuing an RFI for two additional deliverables, in addition to the four for which this RFP is seeking a contractor. While there is no requirement that any proposer who responds to this RFP will also respond to the RFI, work on the first four deliverables must be planned and proceed with the awareness that the Department’s intention is eventually to contract for two additional deliverables related to the outcomes-assessment method:

Testing and possible revision to establish the method’s reliability and validity in assessing the presence of personal-experience outcomes and guidance on how the method should continue to be assessed over time, and

Development of methods for compiling the results of the interviews and for using those results in effective quality-management activities.

More information on these later stages of our intended work on outcomes measurement is included in Appendix E.


-- 17 --


Section III. Preparing and Submitting a Proposal

III-1 Notice of Intent to Submit a Proposal

Proposers are requested to submit a Notice of Intent to Submit a Proposal. (Refer to Section VI. Forms and Other Attachments). The Notice of Intent to Submit does not commit an organization to submitting a proposal. Any supplemental written information related to this RFP developed by the Division will be provided only to those agencies who have filed a Notice of Intent, or to agencies who request such information.

The Notice of Intent is to be submitted by e-mail to Sharon Ryan at [email protected].

The Notice should be sent by October 9, 2006.

III-2 Timeline

September 27, 2006.......................... Release of RFPOctober 9, 2006................................ Due date for Notice of Intent to SubmitOctober 9, 2006................................ Due date for e-mails with questionsOctober 16, 2006.............................. Proposer’s ConferenceOctober 20, 2006.............................. Release of answers to e-mail questions and

questions at Proposer’s ConferenceNovember 9, 2006............................ Due date for proposalsNovember 20, 2006.......................... Oral Interviews, if necessaryNovember 30, 2006.......................... Announcement of Intent to AwardDecember 15, 2006........................... Award of Contract

III-3 General Instructions

The evaluation and selection of a contractor and the contract will be based on the information submitted in the proposal plus references and any presentations or interviews which may be required if clarification is needed. Failure to respond to each of the requirements in the RFP may be the basis for rejecting a response.

Elaborate proposals (e.g., expensive artwork, multimedia presentations), beyond that sufficient to present a complete and effective proposal, are not necessary or desired.


-- 18 --



III-4 Incurring Costs

The State of Wisconsin is not liable for any cost incurred by proposers in replying to this RFP.

III-5 VendorNet

VendorNet is the State’s vendor registration system through which the State guarantees that it will notify vendors of opportunities to bid or to submit proposals. The VendorNet purchasing information and vendor notification service is available to all businesses and organizations that want to sell to the state.

Anyone may access VendorNet on the Internet at http://vendornet.state.wi.us to get information on state purchasing practices and policies, goods and services that the state buys, and tips on selling to the state. Vendors may use the same Web site address for inclusion on the bidders list for goods and services that the organization wants to sell to the state.

A subscription with notification guarantees the organization will receive an e-mail message each time a state agency, including any campus of the University of Wisconsin System, posts a request for bid or a request for proposal in their designated commodity/service area(s) with an estimated value over $25,000. Organizations without Internet access receive paper copies in the mail. Increasingly, state agencies also are using VendorNet to post simplified bids valued at $25,000 or less. Vendors also may receive email notices of these simplified bid opportunities.

III-6 Right to Reject Proposals and Negotiate Contract Terms

The State reserves the right to reject any and all proposals. The State may negotiate the terms of the contract, including the award amount, with the selected proposer prior to entering into a contract. If contract negotiations cannot be concluded successfully with the highest scoring proposer, the agency may negotiate a contract with the next highest scoring proposer.

III-7 Proposer’s Conference

A Proposer’s Conference, if needed, will be held on October 16, 2006 at the offices of the Department of Health and Family Services, One West Wilson Street, Madison, Wisconsin in Room 950A. At this time, staff will respond to questions received by e-mail and any additional questions from proposers.

If no questions are received prior to the Proposer’s Conference, the State reserves the right to cancel the meeting. It is in the best interest of all proposers who intend


-- 19 --

http://vendornet.state.wi.us/


to respond to the ‘Methods and Training for Assessing Personal-Experience Outcomes’ RFP to attend the Proposer’s Conference.

III-8 Presentations and Interviews

Proposers may be required to make a presentation regarding their proposal. The presentation will be scored by the evaluation committee. Presentations should be made by key project personnel, and include any subcontractor personnel designated with the direct responsibility for the areas they are presenting. Presentations will be limited to one and one-half hour. An additional thirty minutes will be set aside immediately following the oral presentation for evaluation committee members to ask follow-up questions.

The purpose of the presentation is for proposers to demonstrate understanding of the requirements of the prospective contract and demonstrate their capability, including staffing, to meet the contract requirements.

III-9 Clarification and/or Revisions to the Specifications and/or Contract Requirements

Questions concerning this RFP are to be submitted via e-mail to Sharon Ryan at [email protected].

Answers to questions posed by e-mail by October 16, 2006 will be answered at the Proposer’s Conference. Any questions answered orally at the conference are not binding. All answers to questions raised and answered at the conference or posed by e-mail will be followed up with written questions and answers via e-mail by October 20, 2006. Answers will be e-mailed to all agencies that submit a Notice of Intent. To the extent practicable, proposer’s questions will remain as received. However, the Department may consolidate and paraphrase questions for purposes of clarity and nonrepetition.

The questions and answers will be considered official addenda to the RFP.

Proposers are expected to raise any questions, exceptions, or additions they have concerning the RFP before or at the time of the Proposer’s Conference.

Telephone questions are discouraged. Oral responses, information, dates, and/or advice (including telephonic responses, information and/or advice, and any oral responses given during the Proposer’s Conference) received by a prospective proposer from the Department or Department staff shall not, in any manner whatsoever and whether before or after the release of the RFP, be binding on the State of Wisconsin, unless followed-up and explicitly confirmed and stated in writing by the Contract Administrator.


-- 20 --



If a proposer discovers any significant ambiguity, error, conflict, discrepancy, omission, or other deficiency in this RFP, the proposer should notify immediately the Procurement Manager of such error and request modification or clarification of the RFP.

In the event that it becomes necessary to provide additional clarifying data or information, or to revise any part of the RFP, revisions/amendments and/or supplements will be provided through VendorNet and to all proposers who have requested to be informed of revisions, amendments, or supplements.

Each proposal shall stipulate that it is predicated upon the requirements, terms, and conditions of this RFP and any supplements or revisions thereof.

Any contact with State employees concerning this RFP are prohibited, except as authorized by Sharon Ryan, the RFP Program Manager/Contact, during the period from the date of release of the RFP until the release of the notice of intent to award.

III-10 Executed Contract to Constitute Entire Agreement

In the event of contract award, the contents of this RFP (including all attachments), RFP addenda and revisions, and the proposal of the successful proposer, and additional terms agreed to, in writing, by the Department and the Contractor, shall become part of the contract. Failure of the successful proposer to accept these as a contractual agreement may result in cancellation of the award.

The following priority for contract documents will be used if there are any conflicts or disputes: Official Purchase Orders State RFP Proposal to the RFP Standard Terms and Conditions Supplemental Standard Terms and Conditions

III-11 Reasonable Accommodations

The Department will provide reasonable accommodations, including the provision of informational material in an alternative format, for qualified individuals with disabilities upon request. If you anticipate the need for accommodations at the Proposer’s Conference, contact Sharon Ryan at (608) 267-7378 or [email protected].

III-12 Submitting the Proposal

The complete proposal must be received by 5:00 PM CDT on November 9, 2006. Late proposals will be rejected. There will be no exceptions


-- 21 --



The only page where signatures are required is the Request for Proposal Cover Sheet (DOA-3621). For further instructions on the format of the proposal, see “Proposal Format and Organization,” beginning one page 24.

The proposals must be received by the date and time noted above by:

Mr. Aaron SchmittDepartment of Health and Family ServicesOne West Wilson Street, Room 518Madison, Wisconsin 53707

Mr. Schmitt will document the date and time of arrival on the proposal package. Late proposals will not be accepted.

Note: If a proposal is sent by mail, receipt of a proposal by the state mail system does not constitute receipt of a proposal for purposes of this RFP.

To ensure confidentiality of the document, the hard copy of the proposal should be packaged, sealed and show the following information on the outside of the package: Proposer’s name and address Title of RFP and RFP Reference Number Proposal due date

III-13 Standard Terms and Conditions

The State of Wisconsin reserves the right to incorporate standard State contract provisions into any contract negotiated with any proposal submitted responding to this RFP (Standard Terms and Conditions, DOA-3054 and Supplemental Standard Terms and Conditions for Procurements for Services, DOA-3681). Failure of the successful proposer to accept these obligations in a contractual agreement may result in cancellation of the award.

III-14 Multiple Proposals

Multiple proposals from a single proposer are permitted. Each such proposal must be complete in itself and must be submitted separately.

III-15 Withdrawal of Proposals

Proposals shall be irrevocable until contract award unless the proposal is withdrawn. Proposers may withdraw a proposal in writing at any time up to the proposal closing date and time. To accomplish this, the written request must be signed by an authorized representative of the proposer and submitted to the RFP


-- 22 --


project manager. If a previously submitted proposal is withdrawn before the proposal due date and time, the proposer may submit another proposal at any time up to the proposal closing date and time.


-- 23 --


Section IV. Project Proposal

IV-1 Proposal Format and Organization

A. Format and Signatures

Please submit in paper: six copies of the proposal and its attachments, and on CD: an electronic copy of the Request for Proposal Cover Sheet (DOA-3621); and a searchable electronic copy (Microsoft Word or Excel) of Parts II (Executive

Summary) and III (Project Proposal) of the proposal.

The paper copy of the proposal will be considered the true and final copy of the proposal and should have original signatures. The electronic version of the ‘Request for Proposal Cover Sheet’ is not required to have signatures. Both submission formats—paper and CD—must meet the due date and time noted in Section III-12 on page 21.

Each paper copy of the proposal should consist of two volumes—the first containing the proposal narrative and the second containing the attachments. Six copies should be submitted. (That is, there should be 12 volumes total – six containing a copy of the proposal narrative, and six each containing one set of the attachments.)

The proposal should be on letter-size paper (8 ½ x11) and be formatted in a readable font no smaller than 11 point; text portions of the proposal should contain no more than six lines per vertical inch.

The paper copy of the proposal will be considered the true and final copy of the proposal and should have original signatures. The only page where signatures are required is the Request for Proposal Cover Sheet (DOA-3621) form.

Documents in addition to an electronic copy of form DOA-3621 form and Parts II and III of the proposal may be included on CD, if the electronic documents are clearly titled on the CD.

B. Organization

Proposals should be organized in the order and with the headings and subheadings that are shown in the submission checklist on page 42. The following is an elaboration of each part of the proposal.


-- 24 --


IV-2 Request for Proposal Cover Sheet (DOA – 3261)

The first page of the proposal is the completed ‘Request for Proposal Cover Sheet,’ form # DOA-3261 (R08/2003), available through VendorNet at the Proposal Page for RFP 1571-DDES-SM along with the RFP and other required forms

IV-3 Part I - Executive Summary – 3 pages

The purpose of the Executive Summary is to condense and highlight the contents of the proposal in a manner that provides the readers with a sense of the proposer’s approach to the contracted activities.

IV-4 Part II – Proposal

Proposers shall provide clarity and conciseness in all sections.

A. Presence in Wisconsin and Involvement with Wisconsin-based People The proposal must describe the extent of the proposer’s intended physical presence in Wisconsin during the contract. For the reasons described in Section 1.3.2 of this RFP, which have to do with taking advantage of available experience of professionals and consumer in Wisconsin, taking advantage of the interest and enthusiasm of all the long-term care stakeholders, keeping stakeholders informed, and building ‘buy-in’ to the project, the successful proposer will be required to maintain extensive contact and involvement with consumers and staff in Wisconsin over the course of the project. We believe that this will require frequent physical presence of project staff in Wisconsin, sometimes at short notice.

B. Understanding of the Mission The successful proposer will demonstrate, through answers to the following questions, an accurate understanding of the intended purpose and desired results of this project. Please provide a response (fewer than 1,200 words for each question) to each of the following questions:

B-1. The conceptual definition of ‘outcome’‘Outcomes’ is a word that means many things to many people, and is often used as a synonym for ‘result,’ or ‘goal.’ Also, in addition to the 12 outcomes identified in Appendix B, circumstances or conditions that have been characterized as outcomes include ‘I have a good relationship with my care manager,’ ‘I am not required to wait to see my doctor,’ and ‘I am receiving help with my alcohol problem.’

How do you define ‘outcome’ with regard to long-term care and how do you distinguish outcomes from the other circumstances, conditions, or attributes that are important aspects of quality care?


-- 25 --


B-2. The value of outcomes assessment to care planningCreating a plan of care for a person with disabilities is a complex process involving many different components, only one of which is identifying the consumer’s desired personal-experience outcomes and determining whether they are present is only one. In fact, attention to desired personal-experience outcomes is, in some programs, not even a required part of the assessment and care-planning process.

Describe the value of personal-experience outcomes assessment to the whole process of creating a long-term care plan. In other words, in what ways do care plans based on assessment of personal-experience outcomes differ from care plans that are not?

B-3. The nature of outcomesRegardless of ability or disability, the 12 outcomes identified in Appendix B are not likely to be present continuously in any human’s life. On some days, the sun shines and the wind is at our back; on others, it’s as if a black cloud follows us around and rains on our every action. Arthritis pain can come and go; our roommate can be pleasant or obstreperous. This natural variability creates a challenge for assessing quality-of-life.

In addition, even when we take action to correct an unacceptable circumstance, it might take some time for even the best efforts to achieve the outcome we seek. This fact of life (that bad situations cannot always be corrected overnight even with the best intervention) creates a different sort of challenge for assessing quality-of-life at any given point in time.

Describe the ways in which an outcomes-measurement method might take these challenges into account to produce a reasonably reliable and valid measurement of quality of life that long-term care programs can use for care planning and for quality management.

B-4. Use of outcomes by care managers and by quality reviewers It is important that care managers (those who create the care plans that are intended to support achievement of outcomes and who monitor the consumers’ quality of life) and quality reviewers (those who are charged with objective assessment of the performance of the program) measure the same things. That is, it’s important that quality reviewers look for the same results that care managers are trying to achieve, and vice-versa. Therefore, the interview tool they use to assess outcomes should be the same.

However, care managers and quality reviewers will be collecting the information for somewhat different uses and will be administering the interviews under somewhat different circumstances. Describe some of these differences, and how you think an assessment tool might take them into account.


-- 26 --


B-5. Variety of intervieweesThe infinite variety of humanity extends—no surprise—into the long-term care population. Different languages, different cultures, different levels of trust, and different communication styles will all come into play. Some consumers will be reticent with their care managers but open up to the friendly-stranger quality reviewer, while other consumers will open up to their care managers but be reticent with the stranger. Some consumers will want to share their accounts of years of frustration; others will not want to ‘complain’ or be ‘any trouble for anyone.’ Some will want to talk for hours, others may tire after two questions.

Describe some of these challenges that you have encountered in your previous work, and how interviewers (whether they are care managers or quality reviewers) can overcome their effects on the interviews, or take them into account, and attain a reasonably reliable understanding of personal-experience outcomes in spite of these challenges.

B-6. Role of guardians and caregivers People with severe disabilities are very often highly dependent on significant others in their lives, most notably guardians and primary caregivers (not always the same people). These people may have particularly acute insight into the desires and preferences of the consumer—or they may seem to lack any ability or willingness to separate the consumer’s desires and preferences from their own.

Please discuss the roles of guardians and caregivers in the assessment of consumers’ personal-experience outcomes. How can they be helpful in the process? How can they be difficult? How, if at all, or under what circumstances can or should they be involved in the assessment of a consumer’s desired personal-experience outcomes?

C. Organizational Capabilities Any profit or non-profit organization, government agency, or educational institution may apply, and partnerships among any organizations, agencies and institutions that are formed for the specific purpose of this project may apply.

The successful proposer will demonstrate, through answers to the following questions, organizational experience and qualification that will enable the proposer to fulfill the intended purpose and desired results of this project. Please provide a response to each of the following questions. Each response should contain fewer than 1,200 words, and we are mindful that clear, concise writing can produce even shorter responses and still contain the information we need.


-- 27 --


C-1. Experience with interviewing and developing interview tools/methods for these target groups

Describe the proposer’s experience with developing interview instruments or methods that elicit information or opinions from frail elders or adults with physical or developmental disabilities with cognitive abilities and language-based communication abilities, and from adults without cognitive or language-based communication abilities. Include in this description information pertaining to: The type of output or results of the interview method or tool (that is,

was the interview used to provide information for individual care plans or activities, to create aggregated information to be used in research or evaluation, or information for supervision or management purposes, or for another purpose?), and

For each project, the amount and type of development work performed by the proposer, and the amount and type of work of work performed, or direction given by, the sponsor or partner in the development.

Proposers may submit examples of completed work that will help us evaluate your work (e.g., the interview tools, instructions, results from administration of the interviews.) If included as attachments to the proposal, these examples will not count toward the 1,200-word limit.

Describe the proposer’s experience with interviewing adults in these target groups. This experience might include interviewing conducted while testing the interview tools and methods the proposer has developed, or interviewing conducted for other purposes.

C-2. Experience with developing measurable constructs and quantitative indicators of qualitative conditions

(See the Definition section for an explanation of how the terms ‘measurable construct’ and ‘indicator’ are used in this RFP.) Describe the proposer’s experience with developing measurable constructs for conditions or circumstances similar to personal-experience outcomes. Include information pertaining to: the considerations that shaped the development of the measurable

construct; The proposer’s approach to developing the construct; The considerations that shaped the development of the indicator; and The proposer’s approach or process of developing the indicators.

C-3. Experience developing and evaluating training programs Describe the proposer’s experience with developing training programs for skills similar to those that will be required to administer the personal-experience outcomes measurement method anticipated by this proposal, or for those who train them. Include in this description information pertaining to:


-- 28 --


How the proposer assessed the training needs and existing skill levels of the intended recipients of the training;

How the proposer considered and responded to learning habits and patterns common among professionally educated adults;

The steps the proposer took to ensure that curriculum materials were readable, effective, and useful;

Any steps the proposer took to ensure that the training program developed skills, as distinct from imparting knowledge;

Any steps the proposer took or recommended to ensure that the new skills were supported or reinforced in the workplace after the training;

How the proposer assessed the effectiveness of the training.

C-4. Experience assessing competence or consistency of interviewers Describe the proposer’s experience with developing methods for assessing the reliability of interviewers or surveyors charged with administering a complex measurement method such as the one proposed by this project. Include in this description information pertaining to: How the proposer addressed the question of what level of consistency,

reliability, or competence was acceptable for the measurement method being tested;

The conceptual design of the reliability-testing method(s), and some of the considerations that shaped it; and

How results of the tests were conveyed to the interviewers or the organizations that were responsible for their performance.

C-5. Stability of organization Describe the proposer’s general experience in areas of business related to this project. Include information pertaining to: A short history of the company, The company’s size and resources; The company’s ownership (public company, partnership, university

affiliation, etc.).

D. Staff Qualifications and ExperienceDescribe the qualifications and experience of the person(s) who will be assigned to this project. For each person, in no more than three pages per person, and using the form provided in Appendix C, describe his or her anticipated responsibilities and estimated time commitment to this project, and describe his or her relevant qualifications and job experience. If the experience of any of these people was described in previous sections related to organizational experience, resumes do not need to be duplicated in this section; they can be incorporated by reference.

E. Project Methodology – 30 pages The Methodology Section (sometimes referred to as the Technical Approach Section) is to include a statement demonstrating the proposer's understanding of the scope and purpose of the project and a cogent description of the proposer's


-- 29 --


approach to creating or producing the four deliverables described in Section II-1 (page 8). The deliverables are:

1) An interview tool for eliciting personal preferences and priorities for each of the 12 outcomes, with unique but related versions for: Frail elders with cognitive and language-based communications

abilities, Adults with physical disabilities and cognitive and language-based

communications abilities; Adults with development disabilities and cognitive and language-

based communications abilities; and Adults with severe cognitive limitations or without language-

based communication abilities. (Typically these persons communicate both through alternatives to language means and with the assistance of a designated proxy)

2) Instructions for the conduct, recording, and scoring of interviews for using that tool in care-planning assessments and in quality-review applications;

3) Training program and materials for care managers and for quality reviewers, and

4) A method for assessing interviewers’ competence or reliability in administering the outcomes interview in the field.

As noted, this contract will be awarded for the completion of only the first four deliverables. However, because it will be useful for work on the first four deliverables to be undertaken with awareness of the intended later steps, the methodology section should look ahead to the entire project. The fifth and sixth deliverables are listed here for your information and a greater description of these two deliverables I in Appendix E.

5) Testing and possible revision to establish the method’s reliability and validity in assessing the presence of personal-experience outcomes and guidance on how the method should continue to be assessed over time, and

6) Development of methods for compiling the results of the interviews and for using those results in effective quality-management activities.

Explanations should include descriptions of how and why things are to be done, in addition to stating what is to be done. The descriptions should include a discussion of anticipated problems or concerns, and discussion of ways to handle these problems. A review of section II-2 (beginning on page 14) may be helpful in describing your methodology.

F. Work Plan The work plan should relate clearly to the first four deliverables in the proposed methodology, should specify the major activities, specific tasks, lead person,


-- 30 --


timeline, and products for each of the four deliverables delineated in the methodology. The workplan should include regularly scheduled meetings with Department staff involved in project management. The work plan should cover only the first four deliverables and only the work anticipated for completion by December 31, 2007.

The work plan submitted with the proposal will serve as an initial plan that may be further negotiated following announcement of award of contract.

G. Budget and Budget Narrative

BudgetThe budget should be in Excel format and include a one-page line-item budget, a staff worksheet and, if applicable, a worksheet with a subcontractor line-item budget.

Line-Item Budget. Proposers are encouraged to use the format below for the one-page line-item budget worksheet. Some line items may not be applicable to all proposers.

Note: The Department will not cover staff training, airfare or equipment.

Staff. The second worksheet should show the calculations for personnel. It should show the name (if known), position, base salary, % time, and fringe benefits of all agency project personnel. Fringe benefits should be calculated utilizing the proposer’s current rate.

Subcontracts. If the proposer plans to subcontract, to the degree that the proposer has the necessary information, a third worksheet should be included that shows the subcontractor budget. It should follow the general guidelines of the sample line-item budget below.

Line-Item Budget Format

Budget Period: (Beginning Date and End Date)

I. PERSONNEL (show specifics on staff worksheet)Name, Position, Base Salary, % Time _____Fringe Benefits _____Personnel Subtotal _____

II. OTHER COSTSa. Office Operations _____b. Leased Space _____c. Travel _____d. Meetings _____e. Other costs (identify) _____


-- 31 --


III. SUBCONTRACT(S) _____IV. TOTAL I – IV _____

Budget Narrative

Format Requirements: Although information to justify a line item (such as job descriptions for personnel) might be contained within the proposal, a separate and complete justification for each line item must be provided in the budget narrative. The budget narrative should include the following information for each line item:

Description of the specific item (What is it?) Description of how the specific item relates to the proposal (Why is this

item needed to fulfill the proposal objectives?)

PERSONNEL

In addition to the information required on the line-item budget (base salary, percentage of time, etc.), describe the activities of each person as they relate to the proposal. Fringe benefits should be calculated utilizing your agency’s current rate.

OTHER COSTS

a. Office Operations: Specify the projected expense for office items and materials such as telephone, printing, office furniture, etc. Do not pad this line item.

b. Leased Space: Calculate at agency current rate for staff newly hired for the project.

c. Staff Travel: Describe the basis for calculation (i.e., number of people traveling, destination(s), cost of lodging, estimated number of miles traveled, mileage reimbursement rate.) Airfare is not covered and only in-state travel is covered, based on state maximums allowed.

d. Meetings: Briefly indicate the purpose of the meetings and describe the basis for calculation—i.e, the number or people, use of the funds (parking, meals, facilities, etc) and the rates expected to be paid.

e. Other costs: The following information should be provided: The nature of the planned expenditure and the purpose of the cost; and If indirect costs (administrative and operational expenses not related to

the aforementioned categories) are included, the amount should not exceed 9% of the sum of personnel and other direct costs.


-- 32 --


SUBCONTRACTS

For each proposed subcontract, the following information should be provided:

The scope of services to be provided; A description of the intended scope of services for the actual contract

and an indication of what the subcontractor’s work will contribute to project objectives; and

The basis for calculating the requested amount.

Note: The prime contractor is responsible for contract performance when subcontractors are used. However, when subcontractors are used, they must abide by all terms and conditions of the contract.

The Department should not be named as a party to a subcontract. The contractor maintains fiscal responsibility for its contracts, which includes reporting expenses associated with the subcontract to the Department.

Finalized subcontracts must be approved by the Department prior to the Department issuing payment against them.

The State of Wisconsin is committed to the promotion of minority business in the state's purchasing program and a goal of placing 5% of its total purchasing dollars with certified minority businesses. Authority for this program is found in § 15.107(2), 16.75(4), 16.75(5) and 560.036(2), Wisconsin Statutes.

The State of Wisconsin policy provides that minority-owned business enterprises certified by the Wisconsin Department of Commerce, Bureau of Minority Business Development should have the maximum opportunity to participate in the performance of its contracts. The supplier/contractor is strongly urged to use due diligence to further this policy by awarding subcontracts to minority-owned business enterprises or by using such enterprises to provide goods and services incidental to this agreement, with a goal of awarding at least 5% of the contract price to such enterprises.

A listing of certified minority businesses, as well as the services they provide, is available from the Department of Administration, Office of the Minority Business Program, 608/267-7806. The listing is published on the Internet at: http://www.doa.state.wi.us/dsas/mbe/index.asp.

IV-5 Part III – Attachments

This part should include the following attachments:A. Reference List of ClientsB. Organizational ChartC – W. Other Attachments (information not considered a required submission)


-- 33 --

http://www.doa.state.wi.us/dsas/mbe/index.asp


IV-5 Part IV – Required forms

This part should include the following attachments: DOA-3477, Vendor Information Form; DOA-3027, Designation of Confidential and Proprietary Information; Submission Check List

See RFP Section VI. Forms and Attachments, page 41, for format and content guidelines for the forms and other attachments.


-- 34 --


Section V. Selection and Award Process

V-1 Review and Evaluation of Proposals

V-1.1 Presentations and Interviews

Based on an evaluation of the written proposal, top-scoring proposers may be asked to make a presentation or participate in an interview to support and clarify their proposal.

Failure of a proposer to make a presentation or participate in an interview may result in rejection of the proposer’s proposal.

V-1.2 Proposal Scoring

Initially, the Contract Administrator and RFP Program Manager/Contact will review each proposal against the Submission Checklist to make certain all parts of the proposal were submitted. Missing parts may be cause for rejection of the proposal.

Accepted proposals will be reviewed by an evaluation committee and scored against stated criteria.

A proposer may not contact any member of an evaluation committee with any matter relating to this solicitation except at the Department’s direction. The committee may review references, request interviews, and/or request presentations and use the results in scoring the proposals.

The evaluation committee’s scoring will be tabulated and proposals ranked based on the numerical scores received. Proposals from certified Minority Business Enterprises may have points weighted by a factor of 1.05 to provide up to a 5% preference to these businesses (Wis. Stats. 16.75(3m)).

V-1.3 Evaluation Criteria

Proposals will be scored as follows:

A. Presence in Wisconsin and Involvement with Wisconsin-Based People – 10% of the total points

Points will be awarded based on the extent to which reviewers believe that information provided in Section A of the Proposal Narrative (described on page 25 of this RFP) indicates that the proposer will come to understand Wisconsin’s perspective and values with regard to personal-experience outcomes in long-term


-- 35 --


care planning and quality management; will make good use of expertise and experience in this state; and will develop a strong understanding of the systems in which the products of this project will be used. Due to the complexity of the deliverables of this project, reviewers will also look for a willingness of the proposer to be available for no-cost clarification and reasonable consultation for a period of at least one year after completion of the contract.

B. Understanding the Mission – 16% of the total points

Points will be awarded based on the extent to which reviewers believe that answers to the questions posed in ‘Understanding of the Mission’ (beginning page 25 of this RFP) reflect a well-developed understanding of the concepts of personal-experience outcomes as they are conceived in this project; of how the outcome tool and the interview results are to be used in long-term care planning, assessment and quality management; and of the challenges facing this project.

B-1. Evaluation criteria: Reviewers will look for: 1) evidence that the proposer understands the difference between outcomes, processes, and outputs; and between desired point-in-time milestones and valued quality-of-life conditions; and 2) evidence that the proposer can articulate these concepts in an understandable manner.

B-2. Evaluation criteria: Reviewers will look for: evidence that the proposer understands the care-planning process in which this outcome-assessment tool will be used, that is, that the proposer is aware of some of the other components (assessing functional abilities and clinical conditions, informal supports, etc.) and how assessment of outcomes is a different and distinct, but related, task. Reviewers will also look for evidence that the proposer is familiar with the concept of a member-centered or person-centered care plan, and understands how that differs from service plan.

B-3. Evaluation criteria: Reviewers will look for evidence that the proposer has sufficient comprehension of the reliability challenges facing a measurement endeavor such as this one, and that the proposer has constructive, sensible thought about the challenges to be able to produce a credible deliverable. The Department is not settled in any preconceived idea of the best way to establish or even conceptualize reliability of a quality-of-life measurement tool; we want to work with someone who can constructively take up this challenge. Reviewers will also look for evidence that the proposer is able to distinguish between services and supports, and cognizant of the fact that an outcome that is being supported but is not yet achieved is different from an outcome that is currently present.

B-4. Evaluation criteria: Reviewers will look for evidence that the proposer understands both the care-planning process and the quality-review process, the different types of information that care managers and quality


-- 36 --


reviewers need to carry out their responsibilities; and the different circumstances under which care managers and quality reviewers will administer the interviews.

B-5. Evaluation criteria: Reviewers will look for evidence of relevant skills and/or knowledge of methods of eliciting complex, subjective information from people who present challenges to interviewers (e.g., tiring quickly, short attention spans, providing inconsistent answers, etc.) and evidence of constructive ideas for training and/or remediation to help interviewers attain the skills they need to assess personal-experience outcomes.

B-6. Evaluation criteria: Reviewers will look for evidence of an understanding of: 1) the unique challenges of assessing the choices and preferences of a person through a guardian or proxy and of assuring that the voice of the consumer is heard, and that they understand what the role of the guardian does and does not entail, and 2) the importance of, and methods for, identifying when the guardian’s wishes may be coming to the forefront and adapting the interview to take this into account.

C. Organizational Capabilities – 9% of the total points

Points will be awarded based on the extent to which reviewers believe that information provided in Section C of the Proposal Narrative (described on page 27 of this RFP) indicates that the proposer’s experience is relevant to the tasks required in this project in content and nature; that the proposer understands the ways in which their experience is and is not relevant; that the experience produced high quality results; and that the experience is of sufficient volume or extent; and that the proposer ensures continuity of staff between the referred experience cited and the proposal submitted.

C-1. Evaluation criteria: Reviewers will look for: 1) the number of interview tools and methods the proposer has developed (not the length of their experience); 2) the similarity between the tools/methods the proposer has developed and the one this project will develop (discussion methods are more similar than multiple-choice-type interviews; interviews about opinions or personal experience are more similar than interviews that gather empirical or objective information; 3) the extent of the proposer’s experience with the target groups included in this project.

C-2. Evaluation criteria: Reviewers will look for: 1) the extent of appreciation of the challenges and inherent constraints of turning complex, real-world circumstances into discrete measures; 2) the extent to which the proposer’s previous work was relevant to the task at hand; and 3) the quality of the cited previous work.


-- 37 --


C-3. Evaluation criteria: Reviewers will look for: 1) the extent of the proposer’s demonstrated appreciation of the special requirements for training in a complex skill that cannot be developed in a classroom setting but must be developed over time with practice and individual feedback; 2) a record of rigorously evaluating the effectiveness of a training program, and 3) the relevance of the proposer’s experience to the type of training envisioned by this project.

C-4. Evaluation criteria: Reviewers will look for: 1) the relevance of the proposer’s experience; 2) evidence of flexible, creative thought regarding methods of determining inter-rater reliability, along with an awareness of constraints such as not burdening too many consumers with back-to-back interviews, etc., and 3) the extent to which the proposer’s work exhibits an ability to report results in a way that they can be used to improve reliability.

C-5. Evaluation criteria: Reviewers will look for evidence that the proposer has sufficient, well-established resources to complete this project successfully.

D. Staff Qualifications – 10% of the total points

Points will be awarded based on the extent to which reviewers believe that information provided in Section D of the Proposal Narrative (described on page 29 of this RFP) indicates that the proposer will assign qualified and experienced individuals to project’s tasks, and that staff assigned to tasks involving interaction with stakeholders will be likely to work productively with those stakeholders.

E. Project Methodology – 35% of the total pointsPoints will be awarded based on the extent to which reviewers believe that information provided in Section E of the Proposal Narrative (described on page 29 of this RFP) demonstrates a clear and realistic formulation of the project goals, contract components, and performance requirements.

Evaluation criteria: Points will be awarded to the proposal based on the extent to which: 1) the extent to which the methodology can be expected to complete the four promised deliverables before December 31, 2007 and how much, if any, will remain to be completed after the close of this project; and 2) the extent to which this methodology will produce a product that will be credible to the users and stakeholders. Reviewers will not consider or evaluate any methodology discussion pertaining to the fifth and sixth deliverables in Appendix E.


-- 38 --


F. Work Plan – 16% of the total points

Points will be awarded based on the extent to which reviewers believe that information provided in Section F of the Proposal Narrative (described on page 30 of this RFP) clearly delineates the intended tasks, procedures, and staff responsibilities, and the extent to which reviewers believe that the proposed workplan can realistically be expected to produce the anticipated deliverables.

G. Budget and Narrative – 4% of the total points

Points will be awarded based on the extent to which reviewers believe that information provided in Section G of the Proposal Narrative (described on page 31 of this RFP) indicates expenditures congruent with the work plan and clear identification of useful purposes of expenditures.

V-1.4 Right to Reject Proposals and Negotiate Contract Terms

The Department reserves the right to reject any and all proposals. The Department may negotiate the terms of the contract, including the award amount, with the selected proposer prior to entering into a contract. If contract negotiations cannot be concluded successfully with the highest scoring proposer, the Department may negotiate a contract with the next highest scoring proposer.

V-2 Awarding of Contract

V-2.1 Award and Final Offers

The Department will compile the final score for each proposal. The award will be granted in one of two ways. The award may be granted to the highest scoring proposer. Alternatively, the highest scoring proposer or proposers may be requested to submit final and best offers.

If final and best offers are requested by the Department and submitted by the proposer, they will be evaluated against the stated criteria, and scored and ranked by the evaluation committee. The award then will be granted to the highest scoring proposer. However, a proposer should not expect that the Department will request a best and final offer.

V-2.2 Notification of Intent to Award

All proposers who respond to this RFP will be notified in writing of the State’s intent to award the contract(s) as a result of this RFP.

After notification of the intent to award is made, and under the supervision of agency staff, copies of proposals will be available for public inspection from 8:00 a.m. to 3:30 p.m. at the Department of Health and Family Services, Room 518, One


-- 39 --


West Wilson Street, Madison, Wisconsin. Proposers may schedule reviews with Mr. Aaron Schmitt at 608/266-2718.

V-2.3 Appeals Process

Notices of Intent to Protest and protests must be made in writing to the Secretary of the Department of Health and Family Services. Protestors should make their protests as specific as possible and should identify statutes and Wisconsin Administrative Code provisions that are alleged to have been violated.

Any written Notice of Intent to Protest the intent to award a contract must be filed with:

Secretary Helene NelsonDepartment of Health and Family ServicesOne West Wilson Street, Room 650Madison, Wisconsin 53702

It must be received in his office no later than five (5) working days after the Notices of Intent to Award are issued.

Any written protest must be received within ten (10) working days after the Notice of Intent to Award is issued.

The decision of the Secretary may be appealed to the Secretary of the Department of Administration within five (5) working days of issuance, with a copy of such appeal filed with the Department of Health and Family Services. The appeal must allege a violation of a Wisconsin statute or a section of the Wisconsin Administrative Code.

V-3 Termination of Contract

The Department may terminate the contract at any time at its sole discretion by delivering 30 (thirty) days written notice to the contractor. Upon termination, the agency's liability will be limited to the pro rata cost of the services performed as of the date of termination plus expenses incurred with the prior written approval of the agency. In the event that the contractor terminates the contract, for any reason whatsoever, it will refund to the agency within 30 (thirty) days of said termination, all payments made hereunder by the agency to the contractor for work not completed or not accepted by the agency. Such termination will require written notice to that effect to be delivered by the contractor to the agency not less than 30 (thirty) days prior to said termination.


-- 40 --


Section VI. Forms and Attachments

VI-1 Notice of Intent to Submit

The Notice of Intent to Submit is not required, but will be helpful in assisting us in notifying potential proposers of any developments, changes in plans, or additional information. Notices may be submitted by e-mail to the RFP Program Manager/Contact, Sharon Ryan, at [email protected].

Due date for submission of the Notice of Intent is October 9, 2006.

The notice should include: A statement of intent to submit a proposal Proposer name and address Contact person name, telephone number, and e-mail address

VI-2 Reference List of Clients

Please submit the following information for two clients for whom the proposer has provided services related to the RFP in recent years, who may be contacted as references for information on the proposal:

Company name and contact information; Title, duration, scope, outcomes, and budget total of each project Short (4-5 sentence) description of the project.

VI-3 Organizational Chart

The organizational chart should indicate names and lines of authority between all key project personnel. The chart is considered a complement to information provided for the Project Organization and Staffing part of the proposer.

VI-5 Required Procurement FormsDOA-3477, Vendor Information FormDOA-3027, Designation of Confidential and Proprietary Information


-- 41 --



VI-6 Submission Check List – one page

The following submission check list should be copied, filled out, and submitted as the last page of the paper copy of the proposal.

Proposer Name____________________________________________________

General Submission Requirements1. _____ Proposal was submitted by the specified time and date.2. _____ Proposal complies with page and word limits.

The proposal is organized into the following sections:

Volume One: Proposal

_____ Completed Request for Proposal cover sheet (form DOA-3261)

_____ Part I - Executive Summary

Part II - Proposal______ A. Presence in Wisconsin______ B. Understanding of the Mission______ C. Organizational Capabilities______ D. Staff Qualifications and Experience______ E. Project Methodology______ F. Workplan______ G. Budget and Budget Narrative

Volume Two: Attachments and Required Forms

Part III - Attachments_____ Reference List (See page 41.)_____ Organizational Chart _____ Any examples of completed work submitted at vendor’s discretion (See page

28.)

Part IV – Required Forms_____ DOA-3477, Vendor Information Form_____ DOA-3027, Designation of Confidential and Proprietary Information_____ DOA-3261 DOA RFP Cover Sheet_____ Submission Check List


-- 42 --


Appendix A: Definitions of Terms Used in this RFP

The following definitions are used through the RFP.

Care-management agency means a county agency operating an HCBS waiver program providing long-term care to adults with physical or developmental disabilities or frail elders, a Family Care CMO, a Partnership or PACE site, or any future local agencies providing managed long-term care under Wisconsin’s managed-care expansion.

Care manager means a nurse, social worker, or other professional with responsibility for assessing a long-term care consumer’s health, functional abilities, well-being, desired outcomes, and needs for support and who then uses that information to work with the consumer to devise a plan of care, put services in place, and monitor the results.

Clinical outcome means a condition or circumstance in a consumer’s life that relates to health, safety, or well-being. Unlike personal-experience outcomes, clinical outcomes are objectively measurable by someone other than the consumer, and their presence or absence can be determined without knowing the consumer’s individual preferences or concerns.

Consumer means an adult with a disability or a frail elder who has a need for long-term care services;

Department means the Wisconsin Department of Health and Family Services.

Functional outcome means an ability that the consumer has or does not have to perform certain functions, tasks, or activities. Unlike personal-experience outcomes, functional outcomes are objectively measurable by someone other than the consumer, and their presence or absence can be determined without knowing the consumer’s individual preferences or concerns.

Indicator means a compilation or tabulation of discrete, individual ratings or assessments (such as individuals’ outcomes) that indicates (not necessarily measures precisely) the extent or frequency of the condition being rated or assessed among a group of individuals. Indicators are most valuable when accompanied by some point or standard of comparison, such as change over time, benchmarks, or comparisons among similarly situated groups.

Individual outcome means a single consumer’s unique, valued outcome related to one or more of the personal-experience outcomes defined in Appendix B. For example, “I want to play outside with my grandchildren when they visit on Saturdays” could be an individual outcome associated with ‘I have the best possible health,’ ‘I have relationships with family and friends I care about,’ and ‘I decide how I spend my day.’

Measurable construct means an operational definition of an intangible circumstance or condition, which will allow it to be measured or quantified. For example, the measurable construct for ‘I choose where and with whom I live’ could be something like: “Affirmative answers to all three of the questions, ‘Is this where you want to live, or would you rather be living somewhere else?,’ ‘Could you live somewhere else or with someone else if you wanted to?’ and ‘Do you like living with these people/person, or would you rather be living with someone else or alone?’”


-- 43 --


Member means an adult with a disability or a frail elder who is receiving long-term care services through one of Wisconsin’s Medicaid-funded programs that include home and community-based long-term care services.

Personal-experience outcome means one of the twelve outcomes identified and defined in Appendix B.

Proposer means a firm submitting a proposal in response to this RFP.

Quality reviewer means a person, usually a social-service professional, who has responsibility to discover evidence of the quality of care provided to consumers and the effectiveness of care management teams, providers, care-management agencies, or programs, and to provide that evidence to others in a way that supports their ability to improve quality.

State means State of Wisconsin.

Contractor means the proposer awarded the contract.


-- 44 --


Appendix B: Personal-experience outcomes for long-term care

Assisting people to achieve their desired individual quality-of-life outcomes is one of the primary goals of our long-term care system. The following statements and definitions demonstrate the areas of life that people in long-term care programs have identified as being important to their quality of life. They are stated in the first person to emphasize the importance of the personal voice and experience of the individual. These statements provide a framework for learning about and understanding the individual’s needs, values, preferences, and priorities in the assessment and care planning process and in monitoring the quality of our long-term care programs.

CHOICEWhen people participate in human service systems, they often feel a loss of control over their lives as professionals or others in authority get involved. In our long-term care system we strive to empower the individuals who receive services (participants, members, or consumers) to have choices—to have a "voice" or say about things that affect their quality of life and to make decisions as they are able. People with cognitive disabilities are supported to actively participate in the ways they are able, and their decision-makers (guardians or POA) keep their perspectives in mind for making decisions. The following statements reflect some of the ways in which the system can help support people to maintain control over their lives.

I decide where and with whom I live.One of the most important and personally meaningful choices I can make is deciding where and with whom to live. This decision must acknowledge and support my individual needs and preferred lifestyle. My home environment has a significant effect on how I feel about myself and my sense of comfort and security.

I make decisions regarding my supports and services. Services and supports are provided to assist me in my daily life. Addressing my needs and preferences in regard to who is providing the services or supports and how and when they are delivered allows me to maintain dignity and control. To the extent that I desire and am able, I am informed and involved in the decision-making process about the services and supports I receive. I am aware that I have options and can make informed choices.

I decide how I spend my day. Making choices about activities of daily life, such as sleeping, eating, bathing, and recreation enhances my sense of personal control, regardless of where I live. Within the boundaries of the other choices I have made (such as employment or living with other people), I am able to decide when and how to do these daily activities. It gives me a sense of comfort and stability knowing what to expect in my daily routine. It is important to me that my preferences for when certain activities occur are respected and honored to the extent possible.


-- 45 --


PERSONAL EXPERIENCEA person's day-to-day experience should meet his or her expectations of a high quality life. People who participate in a long-term care programs need to feel they are ‘citizens’, not parts of a ‘program’ and that they are treated with respect. The focus of supports and services is to assist people in their daily lives, not to take them over or get in the way of the experience.

I have relationships with family and friends I care about.People for whom I feel love, friendship, and intimacy are involved in my life. These relationships allow me to share my life with others in meaningful ways and helps affirm my identity. To the extent that I desire, people who care about me and my well-being provide on-going support and watch out for my best interests. I do things that are important to me.My days include activities such as employment or volunteer opportunities, education, religious activities, involvement with my friends and family, hobbies, or other personal interests. I find these activities enjoyable, rewarding, and they give me a sense of purpose.

I am involved in my community. Engaging in the community in ways that I enjoy provides me with a sense of belonging and connection to others. Having a presence in my community enhances my reputation as a contributing member. Being able to participate in community activities gives me opportunities for socialization and recreation.

My life is stable.My life is not disrupted by unexpected changes for which I am not prepared. The amount of turnover among the people who help me (paid and unpaid) is not too much for me. My home life is stable, and I am able to live within my means. I do not worry about changes that may occur in the future because I think I am reasonably well prepared.

I am respected and treated fairly.I feel that those who play a continuing role in my life respect me. I am treated fairly as a person, program participant, and citizen. This is important to me because it can affect how I view myself in relation to others and my sense of self-worth.

I have privacy.Privacy means that I have time and space to be by myself or with others I choose. I am able to communicate with others in private as needed. Personal information about me is shared to the extent that I am comfortable. Privacy allows me to be free from intrusion by others and gives me a sense of dignity.


-- 46 --


HEALTH and SAFETYHealth and safety is an essential and critical part of life that can affect many other areas of a person's life. The following outcome statements represent the person's right to determine what is important to him or her in these areas, and what risks he or she is comfortable with. It's about what the person feels he or she needs to meet personal priorities. It is not an assessment of whether or not the person’s circumstances meet others’ standards for good health, risk, or safety.

I have the best possible health.I am comfortable with (or accepting of) my current physical, mental, and emotional health situation. My health concerns are addressed to the extent I desire. I feel I have enough information available to me to make informed decisions about my health.

I feel safe.I feel comfortable with the level of safety and security that I experience where I live, work, and in my community. I am informed and have the opportunity to judge for myself what is safe. People understand what I consider to be an acceptable level of risk and respect my decisions. If I am unable to judge risk for myself due to my level of functioning, I have access to those that can support me in making those determinations.

I am free from abuse and neglect.I am not experiencing abuse or neglect of my person, property, or finances. I do not feel threatened or mistreated. Any past occurrences have been adequately dealt with or are being addressed.


-- 47 --


APPENDIX C: Resume Format

Qualifications and Relevant ExperienceStaff Assigned to Outcomes-Measurement Development Project

Name: Title:

Organization: Proposed role in this project:

EDUCATION (Add lines as necessary.)

Institution Degree Year Field of study

PROFESSIONAL EXPERIENCE For each relevant professional experience, identify the project/activity and the organization(s) with/for which the project/activity was performed; the approximate date of the experience; and a short summary of the individual’s role or accomplishments on the project. Add lines as necessary; please limit entire summary for each individual to no more than three pages.

Project/Activity and Organization:

Role / accomplishments:

Date(s)



Date(s)



Date(s)


-- 48 --


APPENDIX D: References and Resources

The following documents, all available online, can provide potential proposers with additional insight and understanding into how the Wisconsin HCBS programs conceptualize personal-experience, or quality-of-life, outcomes and how we have tried to employ them in care planning and in quality management.------------------------

Identifying Individual Outcomes: An Introduction to Outcome-based Care Planning: Long-term care professionals in the Department and at the University of Wisconsin-Madison developed this online training course for social workers, care managers, service/support brokers, nurses, supervisors, and other professionals who work with consumers in Wisconsin’s long-term care programs. (Note: a $15 registration is required for this online course.) http://mynursingce.son.wisc.edu/index.pl?id=460363

Family Care Quality CMO Member Outcomes: Four outcomes assessments were done between November 2000 and June 2004. The 14 outcomes were developed by a group of consumers, providers, advocates, and DHFS staff. One valuable feature of this measurement method is that it considers each individual’s preferences when determining whether an outcome or the support for that outcome is present. The four summary reports can be accessed at the following website. http://dhfs.wisconsin.gov/LTCare/ResearchReports/CMOMemberOutcomes.htm

Partnership Member Outcomes: The Initial Assessment & Program for All-Inclusive Care for the Elderly (PACE) Member Outcomes: The Initial Assessment: Both summary reports can be accessed at the following web-site.http://www.dhfs.state.wi.us/WIpartnership/ProPublications.htm

Quality of Life Outcomes for People with Alzheimer’s Disease and Related Dementia: This document presents a Care Planning Tool for Providers and describes outcomes, indicators, measures and related good practice patterns.http://dhfs.wisconsin.gov/aging/dementia/outcomes.htm

Wisconsin’s Consumer Outcomes Survey: This CMS document summarizes the Member Outcomes Assessment process, lessons learned, and how the outcomes’ information is incorporated into the overall quality management system.http://dhfs.wisconsin.gov/LTCare/ResearchReports/Index.htm

Outcome and Assessment Information Set (OASIS) Resources: This website has OASIS resources and teaches people how to read, interpret and use the reports for quality improvement.http://dhfs.wisconsin.gov/rl_DSL/Training/OASISaudio1204.htm


-- 49 --

http://dhfs.wisconsin.gov/rl_DSL/Training/OASISaudio1204.htm

http://dhfs.wisconsin.gov/LTCare/ResearchReports/Index.htm

http://dhfs.wisconsin.gov/aging/dementia/outcomes.htm

http://dhfs.wisconsin.gov/LTCare/ResearchReports/CMOMemberOutcomes.htm

http://mynursingce.son.wisc.edu/index.pl?id=460363


APPENDIX E: FIFTH AND SIXTH DELIVERABLES

E. Completed tests establishing the tool’s validity and reliability in assessing the presence of desired personal-experience outcomes, and guidance on how the tool should continue to be assessed over time

‘Validity’ is the extent to which the interview method actually measures that which it is intended to measure. The outcomes-assessment tools used in Wisconsin up to this point have all had reasonable levels of face and content validity. That is, the labels applied to the outcomes, such as ‘People have privacy,’ were generally agreed to include all the important elements and to be closely related to the questions asked of interviewees. Construct validity however, which is established by comparing the results of one measure to those of another measure that is recognized as valid, has not been established with these tools, and presents challenges, as no other measurement methods exist.

‘Reliability’ of the measurement method is the degree to which the tool produces the same result when situations are the same. Again, the measurement of personal-experience outcomes presents challenges for reliability testing. There is a limited extent to which any one of us expresses satisfaction with the quality of our life, as our perception of the quality of our lives can vary within a day even in addition to normal day-to-day variation. The quality of a person’s life in many of these personal-experience outcomes can also change rapidly.

The successful proposer will understand that the objective of this project is to establish reasonable levels of validity and reliability, considering the intended uses of the interview results. The Department is not, in the near future, planning to use personal-experience outcomes as the sole basis for any decisions, such as pay-for-performance incentive payments. The uses will be in care planning, in which the most important objective is enabling the care manager to understand the member’s concerns, preferences, and priorities for his or her life, and in quality-management, in which the most important objectives are providing useful feedback to care managers and in compiling aggregate performance indicators that are useful guides for further quality-management investigation, rather than as conclusive, stand-alone measurements. In Wisconsin’s experience, striving for levels of validity and reliability that are beyond the levels needed for the purposes at hand can impede progress to objectives that are both more achievable and more useful.

To enable use of the method in quality management (which requires consensual acceptance of the results as reasonably true), this project will need to devise and carry out tests and demonstrations of the method’s validity and reliability for each of the four target groups. At a minimum, the face and content validity of the assessment method should be demonstrated in a way that can be explained to stakeholders. Additional tests and demonstrations of other types of validity will be helpful.


-- 50 --


F. Methods of using the results of outcome interviews in quality management.

The level of effort needed to conduct an outcomes-assessment interview limits the number of these interviews that can be conducted by state-level quality reviewers. Conducting interviews of a statistically valid sample of members in each local care-management organization in each target group—specifically for quality-management purposes—would place an unacceptable burden on the local agencies, the consumers, and the Department’s quality-management budget. The only reasonable way to conduct that many outcomes-assessment interviews is to have each care manager conduct the interviews—which care managers are, in fact, expected to do as part of person-centered care planning.

However, relying on that large a number of interviewers, some of whom can be expected to allow their desire to discover successful achievement of outcomes affect their assessments, presents serious challenges to reliability of the reported results. If DHFS is to use outcomes-assessment information submitted by the local agencies in our statewide quality management (QM) activities, we will need to conduct some sort of ‘validation,’ or look-behind, to verify that the information is being collected in the prescribed manner (so that it can be aggregated) and that it is being reported accurately (so that it can be trusted). Quality Review staff at the Department, in the CMO or any federally qualified external quality review organization with which the Department might contract will be familiar with the concept and processes of validation, but the challenges of validating data collected through oral interviews is daunting. On the other hand, reviewing the work of the care managers to ensure that they can perform this particular task well has value beyond simply validating submitted data; the feedback that such a validation process would generate for the care managers would help them in performing their core function better. We expect that the CMO will, as a part of its local quality assurance and quality improvement efforts will rely on data and other information with respect to implementing this outcomes tool.

We believe that there is a solution to this conundrum that involves making use of both care managers’ outcomes assessments and a reasonable number of strategically-selected outcomes assessments conducted by both internal and external reviewers of quality. This project will examine the various challenges to the collection of information useful for quality-management purposes, and create specific recommendations regarding how best they can be overcome.

This project will need to provide the Department and the local care-management agencies with methods by which the results of the outcome interviews can be used in quality management. We are interested in learning or developing ways that:

State-level quality reviewers can select and interview samples of members in a statewide program to the best advantage for both state and local quality-management efforts. State-level quality-management efforts will be enhanced


-- 51 --


by generalize-able and comparable information over time and among local agencies; local quality-management efforts will be enhanced by more textured information about why outcomes were or were not found to be present.

High level specifications for a system of data reporting which enables the local to report outcome results to the Department including technical “opportunities” including the cross referencing of existing data sources (e.g., LTC FS).

Care managers can report the nominal results of the interviews (e.g., ‘present/not present) in ways that can be aggregated and analyzed;

The Department can validate reported results to ensure that the reported results are accurate; and

Results reported by either care managers or quality reviewers, or both, can be compiled and reported to stakeholders and others.


-- 52 --


APPENDIX F: RFP Evaluation Criteria

A. Presence in Wisconsin and Involvement with Wisconsin-Based People (10%)

A.1 Consider the availability of contractor staff to Wisconsin project staff through the duration of the contract and a one-yaer follow-up period. (5%)

A.2 Consider the ability of the proposer to come to understand Wisconsin’s perspective and values with regard to personal-experience outcomes in long-term care planning and quality management; make good use of expertise and experience in this state; and develop a strong understanding of the systems in which the products of this project will be used (5%)

B. Understanding the Mission (16%)

B.1 Look for evidence that the proposer understands and can articulate the difference between outcomes, processes, and outputs; and between desired point-in-time milestones and valued quality-of-life conditions. (4%)

B.2 Look for evidence that the proposer understands the care-planning process and how the outcomes can be incorporated into a plan of care. Assess whether the proposer understands how other components—assessing functional abilities, clinical conditions, informal supports—are related but different from outcomes assessment. Evaluate the proposer’s concept of member-centered or person-centered care plan and how it differs from a service plan. (2%)

B.3 Look for evidence that the proposer has sufficient comprehension of the reliability challenges with a tool that measures outcomes. Evaluate whether the proposer has constructive, sensible thought about how to produce a credible deliverable. (2%)

B.4 Look for evidence that the proposer understands both the care-planning function and the quality-review function; the different types of information that care managers and quality reviewers need to carry out their responsibilities; and the different circumstances in which care managers and quality reviewers will administer the interviews. (2%)

B.5 Look for evidence that the proposer has relevant skills, experience and/or knowledge of methods of eliciting complex, subjective information from people who present challenges to interviewers, and evidence of constructive ideas for training and/or remediation to help interviewers attain the skills they need to assess personal-experience outcomes. (4%)


-- 53 --


B.6 Look for evidence that the proposer has an understanding and/or experience in assessing the choices and preferences of a person through a guardian or proxy and of assuring the voice of the consumer is heard, and that they understand what the role of the guardian does and does not entail, and that they can distinguish when the guardian’s wishes have come to the forefront. (2%)

C. Organizational Capabilities (9%)

C.1 Look for: 1) the number of interview tools and methods the proposer has developed (not the length of their experience); 2) the similarity between the tools/methods the proposer has developed and the one this project will develop (discussion methods are more similar than multiple-choice-type interviews; interviews about opinions or personal experience are more similar than interviews that gather empirical or objective information; 3) the extent of the proposer’s experience with the target groups included in this project. (2%)

C.2 Look for: 1) the extent of appreciation of the challenges and inherent constraints of turning complex, real-world circumstances into discrete measures; 2) the extent to which the proposer’s previous work was relevant to the task at hand; and 3) the quality of the cited previous work. (2%)

C.3 Look for: 1) the extent of the proposer’s demonstrated appreciation of the special requirements for training in a complex skill that cannot be developed in a classroom setting but must be developed over time with practice and individual feedback; 2) a record of rigorously evaluating the effectiveness of a training program, and 3) the relevance of the proposer’s experience to the type of training envisioned by this project. (2%)

C-4. Look for: 1) the relevance of the proposer’s experience; 2) evidence of flexible, creative thought regarding methods of determining inter-rater reliability, along with an awareness of constraints such as not burdening too many consumers with back-to-back interviews, etc., and 3) the extent to which the proposer’s work exhibits an ability to report results in a way that they can be used to improve reliability. (2%)

C-5. Look for evidence that the proposer has sufficient, well-established resources to complete this project successfully. (1%)

D. Staff Qualifications (10%)

D.1 Look at the extent to which the proposer will assign sufficient staff with the right skill set for the intended tasks and procedures. (6%)


-- 54 --


D.2 Look at the extent to which staff assigned to work with stakeholders have appropriate skills and experience for productive interactions. (4%)

E. Project Methodology (35%)

E.1 Consider the extent to which each of the four promised deliverables will be completed, or the proportion of each planned for completion by December 31, 2007. (15%)

E.2 Assess the extent that the methodology will produce a product that will be credible to the users and stakeholders. (20%)

F. Work Plan (16%)

F.1 Clarity of the work plan and timetables in meeting specifications with proposer staff resources (8%)

F.2 Look at the extent to which reviewers believe that the proposed workplan can realistically be expected to produce the anticipated deliverables. (8%)

G. Budget and Narrative (4%)

G.1 Look for congruence between the indicated expenditures and the work plan, and clear identification of useful purposes of the expenditures. (4%)

Items listed above will provide up to a 100% score; the total points of proposers may be weighted by 105% to allow for a 5% preference to a certified minority business enterprise under § 16.75(3m), Wis. Stats.

This is information appropriate for providing to prospective proposers. Evaluator should have more detail or benchmarks on which to base their point awards.


-- 55 --