Draft_RFP_081413.docx

Draft RFP_081413

SECTION B – SUPPLIES OR SERVICES AND PRICES/COSTS

B.1 DESCRIPTION OF SERVICES

The purpose is this contract is to develop a Research, Measurement, Assessment, Design, and Analysis (RMADA) IDIQ that will provide analytic support and technical assistance for models and demonstration programs that are derived under the Patient Protection and Affordable Care Act (ACA). This support will include the design, implementation and evaluation of research and/or payment and service delivery models to test their potential for reducing expenditures for Medicare, Medicaid, Children’s Health Insurance Program (CHIP), and uninsured beneficiaries while maintaining or improving the quality of care.

B.2 TYPE OF CONTRACT (IDIQ)

This is an indefinite-quantity/indefinite-delivery type contract. The Government shall place orders for individual requirements. The Contractor shall be required to furnish at least a stated minimum quantity of services. In addition, if ordered, the contractor must furnish any additional quantities, not to exceed the stated maximum.

B. 3 MINIMUM AND MAXIMUM QUANTITIES

During the contract period, the Government shall place orders totaling a minimum of $1,000.00. This reflects the contract minimum for the entire period of performance, including any options. The maximum IDIQ amount (i.e., the maximum ceiling across all RMADA IDIQ’s) shall not exceed $7,000,000,000 (inclusive of all costs and fee) over the life of the contract.

B. 4 WITHHOLDING OF CONTRACT PAYMENTS (To be determined on a Task Order Level)

Notwithstanding any other payment provisions of this contract, failure of the Contractor to submit required reports when due or failure to perform or deliver required work, supplies, or services, will result in the withholding of payments under this contract unless such failure arises out of causes beyond the control, and without the fault or negligence of the Contractor as defined by the clause entitled "Excusable Delays" or "Default", as applicable. The Government shall promptly notify the Contractor of its intention to withhold payment of any invoice or voucher submitted.

B.5 TYPE OF ORDERS

Task orders issued under this IDIQ contract include, but are not limited to, firm fixed price (FFP); cost-plus-fixed fee (CPFF); time and materials (T&M); and/or labor hours (LH).

B.6 FIXED FEE (COST REIMBURSEMENT TASK ORDERS ONLY)

The fixed fee will be negotiated on a task order by task order basis.

The fixed fee set forth in each individual CPFF task order under this contract will be paid in installments at the time of each monthly provisional payment. Unless otherwise indicated, the amount of each such fixed fee installment shall be paid at the same ratio as the total fixed fee is to total estimated costs. NOTE: The payment of fixed fee under Task Orders is subject to the withholding provisions of FAR 52.216-8, entitled “Fixed Fee.” Any balance of fixed fee shall be paid to the contractor, or any overpayment of fixed fee shall be repaid by the contractor or otherwise credited to the Government, when specified in FAR 52.216-8.

B.7 TRAVEL/OTHER DIRECT COSTS/ MATERIAL (Cost Reimbursement Task Orders including T&M)

Travel, Material and Other Direct Costs shall be reimbursed at actual costs incurred in accordance with 52.216-7, Allowable Cost and Payment, and the Federal Travel Regulation as applicable. These items will not be separately priced in this contract, but will be set forth, when applicable, on each Task Order.

B.8 INDIRECT CEILING RATES

The government is allowing a ceiling on the indirect rates provided in Section G of this contract to address reasonable business fluctuations across contract years. The indirect rate ceiling for this contract are as follows and apply to all contract years as exercised:

These items will be set forth, when applicable, on each task order.

SECTION C – DESCRIPTION/SPECIFICATIONS/WORK STATEMENT

C.1 Background

Under this Research, Measurement, Assessment, Design and Analysis (RMADA) Indefinite Delivery Indefinite Quantity (IDIQ) umbrella contract, the Centers for Medicare & Medicaid Services (CMS) will award task orders (TOs) for a wide range of analytic support and technical assistance activities that support models and demonstration programs created or derived under the auspices of the Patient Protection & Affordable Care Act (ACA), and future health reform legislation where new delivery and payment reform models are enacted. The demands of new reforms created under ACA have redefined the way CMS approaches and conducts research activities and demonstrations affecting Medicare, Medicaid, CHIP, and uninsured populations. The role of state and private sector payers is also redefined as many of the new models include multiple payers working in collaboration with CMS to reform the care delivery system. The RMADA will provide CMS with a robust tool to meet those challenges. Some of the major activities this umbrella contract will address include the following: designing, maintaining and refining model/demonstration design and operations; monitoring model site implementations; designing and carrying out surveys and other data collection activities; obtaining and analyzing secondary data sources including Medicare, Medicaid and Children’s Health Insurance Program (CHIP), and private payer sources that support model design and evaluations. Some other evaluation activities envisioned under the RMADA include reporting on formative and summative analyses, providing rapid cycle quarterly evaluation feedback to all model participants and CMS, and the creation of summative annual and final program findings.

CMS is the Federal agency responsible for the operation and oversight of the Medicare/Medicaid programs and the Federal participant in the State-operated Medicaid and CHIP. CMS’s mission is to assure health care security for the general public through the management of programs that include Medicare, Medicaid, State Children Health Insurance Program(s), and provides leadership in the broader health care marketplace to improve the health of all citizens. As an effective steward of public funds, CMS is committed to strengthening and modernizing the nation’s health care system to provide access to high quality care and improved health at lower costs. To help accomplish this mission, CMS was given broader authorities through ACA for the planning, analysis, implementation, and rapid cycle evaluation of innovation and demonstration activities to determine effectiveness and feasibility for broader dissemination, scale and sustainability. This includes creating and testing innovative payment and service delivery models, building collaborative learning networks to facilitate the collection and analysis of innovation, the implementation of effective practices, developing necessary technology to support this activity, conducting rapid cycle analysis of the programs and participants, and close to real time reporting of interim findings and summative findings as feasible. The use of this umbrella contract is expected to assist CMS in carrying out its mission by facilitating validation and disseminating information about new care models and payment approaches to serve Medicare and Medicaid beneficiaries seeking to enhance the quality of health and health care and reducing cost through improvement.

C.2 Technical Considerations

The work awarded under the RMADA will involve the design, implementation and evaluation of a broad range of research and/or payment and service delivery models to test their potential for reducing expenditures for Medicare, Medicaid, CHIP, and uninsured beneficiaries while maintaining or improving quality of care.

New models/demonstrations and research afforded under ACA are different from past CMS demonstrations and evaluations, and have changed the way we approach and conduct these activities. The scope of activities of these new models can include all areas of health care research: costs, access, quality, service delivery models, and financing and payment approaches. They tend to be larger, more complex, and can be scaled without Congressional approval if programs can demonstrate cost savings while maintaining or improving quality. These new models/demonstrations can often be characterized as being highly visible and tightly integrated with other programs.

Today’s models/demonstrations afforded under ACA often interact with one another or with national CMS programs in terms of overlapping provider participation, regulatory structures, and use of the same information technology tools and services. Such interactions must be carefully anticipated and managed to account for complex legal, operational, and policy priorities. Today’s models/demonstrations also have more program integrity considerations and involvement from enforcement agencies. Unlike past demonstrations, there is a tremendous focus on shared learning activities among participants because lessons learned from other industries demonstrate that diffusion of knowledge is a key factor in the expansion of models. Furthermore, today’s models/demonstrations typically move from design to announcement and implementation in less than a year, allowing little room for missed deadlines or coordination lapses. Most draw intense interest and scrutiny from market stakeholders and the press as well as Congress and other policymakers. Health care costs continue to rise and there are high expectations for CMS to both rapidly assess these models/demonstrations, and scale those most promising nationally without any delay.

Expectations for support of today’s models/demonstrations have also evolved. Unlike past demonstrations, greater emphasis is being placed on supporting one or more phases of the model lifecycle. This requires close coordination with multiple CMS and HHS entities, and the ability to respond quickly to evolving situations. This work may require the expertise of multi-disciplinary staff including economists, physicians and other health professionals, pharmacists, social scientists, actuaries, statisticians, health policy analysts, and systems analysts and programmers. Contractor staff must also have a wide range of health care and health policy expertise, and the ability to adequately staff multiple large scale CMS models at the same time. Contractor staff should have the analytic and management ability to accurately assess program risks, design and, if necessary, implement multiple contingency plans for successful model implementation. Because CMS engages model participants in a “high touch” manner, support staff must be able to provide intensive technical assistance in an accessible, client-oriented manner and at scale to all model participants. -

The need for analyses based on real time claims and utilization data is a unique factor that distinguishes today’s evaluation of models as opposed to prior demonstrations. Some other considerations include the need for quarterly reports for all models rather than just those at risk, continuous feedback to providers as part of improvement efforts, regression based approaches as opposed to actuarial approaches, and significantly more focus on the use of qualitative methods such as case studies. These methods provide insight into factors associated with process analyses and help determine the success or failure of specific aspects of the relevant programs. Furthermore, because Innovation Center models often include collaboration among multiple payers and other entities, the current evaluation approaches will need to account for the need to gather, coordinate, and analyze private payer and other private data sources. In addition, evaluations involving other entities, such as payers, should plan to examine the role of CMS as a convener and how the model is received by both participating and non-participating affected parties.

C.3 Requirements

Tasks that the Contractor may be required to perform under the RMADA, include but are not limited to:

A. Model Design:

Identify problem and opportunity to address (business case)

CMS is building a diverse portfolio of innovative service delivery and payment models. Typically a model conducted by the Center for Medicare and Medicaid Innovation (CMMI), begins with the selection of a concept or opportunity it wants to explore. It is then defined at a high level in a condensed, written business plan referred to as an Innovation Center Investment Plan or ICIP Light. For CMMI, the objective of the ICIP is to define the problem being addressed, the target audience, framing the basic model concept and identifying potential evidence to support the proposed approach.

In defining the problem, opportunity, and/or concept, the Contractor shall consider CMS’ written business plan for the proposed model and have the capacity to perform the following tasks:

1. Conduct Literature review, evidence assessment, environmental scan, and market research;

a. Exploratory analyses: Conducting an environmental scan and literature review for evidence and examples to support the problem/concept definition, preliminary model/demonstration design and strength of the opportunity; analysis involving the collection of primary and/or secondary data and using quantitative or qualitative analytical methods.

b. Preparation of written materials and participation in briefings: Prepare summary report(s) that integrates findings from the literature review and data analyses, identifies current gaps in knowledge; and presents refined study questions, options to further address policy issue(s); relevant parameters, and recommendations for further action; drafting and/or editing the written business case, including support in definition of problem and opportunity, target populations, available evidence, documenting risks and assumptions in the proposed model; preparing written briefing materials and conducting oral presentations to senior-level CMS and Departmental officials.

c. Need for waivers: Identify legal, regulatory, and policy issues and requirements, including if waivers are necessary.

2. Acquire and analyze data;

a. Obtain CMS, private health care sector and/or State data: The Contractor shall make arrangements to obtain all CMS data necessary for defining the problem and/or opportunity for improvement, including CMS claims and non-claims data, enrollment/eligibility information, and State data. As part of the process of obtaining CMS data, the Contractor shall be responsible for obtaining any necessary Data Use Agreements (DUAs) and for adhering to CMS data security policies and procedures.

3. Meet with subject matter experts;

a. Stakeholder engagement: Consulting with experts in and outside of CMS, in the public and private sector; having preliminary discussions with federal and/or State officials or other experts to refine the policy issues to be examined and identify additional data sources and reports to be examined.

b. Facilitate Stakeholder Opportunities for Input: As pertinent to a given TO, the Contractor may be instructed to develop and carry out a detailed plan for generating participant, stakeholder and/or subject matter expertise input in any relevant phases of an evaluation or research TO including the design phase, the creation of data collection tools, the analysis phase, and the report writing phases. As part of this activity, the contractor may be tasked with facilitating the collection of stakeholder input, compiling, analyzing and assessing the input received and in incorporation of the input into the evaluation and research activities as appropriate.

4. List of intervention choices: Identify specific intervention strategies, benefits, financing options, etc.

a. Actuarial analyses: Performing actuarial analyses to evaluate likely impacts and payment options and methodologies, and simulate benchmark methodologies; provide a detailed financial impact model in the form of either a waiver cost estimate or business case model that evaluates return on investment/savings.

b. Quantitative and qualitative statistical analyses: using standardized statistical methods perform high level analyses linking outcomes to improved health, healthcare and lower costs.

c. Rapid Cycle Evaluation: The evaluation approach with CMMI is rapid cycle with close to real time production of findings.

5. Assess the evaluability of the options.

a. Using statistical tools and justifiable assumptions about likely behavioral responses under the model/demonstration, determine whether the proposed options will yield sufficient sample sizes to answer key questions.

b. Assess whether it will be possible to construct adequate comparison groups and whether the necessary data will be available.

c. Assess what other competing programs in the marketplace might dilute the impact of the intervention or contaminate the comparison group.

6. Analyze Return on Investment (ROI) for different options.

The contractor will develop estimates of ROI for various program design options in order to assist CMMI in selecting the option that is most likely to meet the ROI objectives while also meeting other objectives such as high quality health service delivery.

Develop Innovation Center Concept Brief

After the Problem and Opportunity have been defined, the Contractor shall perform tasks related to supporting CMS in designing a model/demonstration to test an innovative payment and/or service delivery model. For models/demonstrations being conducted under CMS authority, CMS leadership approves further definition of the concept into a fully formed Innovation Center Investment Plan (ICIP).

1. Develop Specific Model Design & Operational Features and clearly define what is being test, by whom, where, how long, and the specific purpose of the tests.

1. Identify beneficiary population and target geographic locations and options

1. Identify provider populations

1. Identify specific intervention strategies, benefits, and/or financing changes

1. Identify payment operational requirements and strategies

1. Identify data operational requirements and collection strategies

1. Develop preliminary evaluation approach Based on a general project design, the contractor will prepare a preliminary design detailing the relevant research questions of interest and how the questions of interest will be addressed.

The design approach should, at a minimum, specify the items listed below:

· Statement of the research questions and hypotheses to be addressed by the evaluation.

· At a high level, a description of the data and analytic approach to addressing the research questions and hypotheses.

· Design and perform simulations (repeated);

B. Solicit

1. Provide support to CMS in all aspects of administering the solicitation/application/requests for proposal (used interchangeably) process, including developing business requirements and implementing a web-based application process, providing support/help.

2. Assist CMS with the review process, including developing a scoring rubric/methodology to assign scores to proposals, identifying, training, and convening reviewers/technical expert panels, collecting their scores, and providing them with technical assistance.

3. Conduct readiness review and vet potential participants for program integrity related issues

4. Assist CMS in developing briefing materials and decision memos regarding recommendations for selection for participation in models/demonstrations

5. If needed, revise waiver cost estimates or financial models to include in clearance package.

C. Build, Run & Evaluate

Model/demonstration designs are reviewed and approved by the CMS Administrator, the Department of Health and Human Services, and the Office of Management and Budget. The Contractor may be requested to assist in the development of some non-sensitive materials describing the model/demonstration, which may be used during CMS’ decision-making process. Once approved, CMS may initiate implementation and operation of the model/demonstration.

The Contractor will design and carry out an appropriate monitoring approach to identify potential unintended consequences of the program, detect whether participants make progress and quality improvements, and assess whether programmatic requirements are being adequately followed. The monitoring activity will serve as a primary source of information by tracking the operation of participants within the initiative. This includes general operational progress such as improvements in services furnished by participating providers, and performance progress such as improvements in care quality, care experience, and other measureable indications.

In supporting model/demonstration implementation and operation, the Contractor shall have the capacity to perform the following tasks

Build

1. Orientation/ongoing technical assistance to model/demonstration participants

a. Orientation/ongoing technical assistance to participants: Providing support to model/demonstration participants selected to test the payment and/or service delivery intervention(s) related to a variety of operational activities including, but not limited to the use of data, interpretation of findings, payment, learning and diffusion, evaluation, data collection, quality/performance measurement, etc.; troubleshooting for IT issues such as electronic file transfers to/from participant organizations, establishing an inquiry system/process and/or help desk to respond to inquiries; proactively address participants’ information needs and use a variety of media to provide training, including webinars, conference calls, face-to-face meetings, and written materials to educate them about protocols

b. Provide Technical Assistance for Self-Administered Design Plans: Where relevant, the Contractor shall provide technical assistance for the design of Awardees self-evaluations. The goal of this task is to ensure that the design and execution of the self-evaluation plan is feasible and informative. Further, this technical assistance should, to the extent possible, correct deficiencies in self-evaluations that would significantly impact CMS’ ability to independently evaluate the proposed projects. However, this task should not impede progress on completing other tasks. Further, we hope that the self-evaluation skills gained through this task will be sustainable beyond the life of the projects.

2. Develop and Prepare Initiative-Specific Standard Operating Procedures & Programmatic Protocols (i.e. payment administration, marketing, grievances and appeals, stakeholder input)

3. Develop operational requirements and systems for enrollment/attribution, including developing outreach strategy/materials, claims algorithms, etc.; maintain master enrollment list of beneficiaries and/or providers (comprehensive and point in time)

a. Providing support to CMS and model/demonstration participants in whatever method is used for enrollment. Models/demonstrations may enroll providers and/or beneficiaries, and use a variety of methods including active (recruitment) or passive (attribution) methodologies. The Contractor shall provide support to CMS in running attribution methodologies, preparing and maintaining current lists of model participants, taking into account potential overlaps with other programs and entering them, as needed, into the CMS Master Database Management system (MDM); conduct appropriate information exchanges with the CMS Center for Program Integrity (CPI) and the Division of Technical Payment Policy, the Office of Inspector General (OIG), the Department of Justice, and other enforcement bodies, as directed by CMS.

4. Develop operational requirements and systems for sharing data, including processes for administering data use agreements

a. Support Data Sharing: Producing appropriate data files, establishing mechanism for transfer of data files to participants, ensuring timely transfer of data files to participants and complying with all CMS data use requirements, including administering data use agreements with all model/demonstration participants, as needed. The contractor shall ensure accurate documentation of participation/data sharing status for beneficiaries/patients and providers.

b. Distribute Data Files: The Contractor may be instructed to distribute data to CMS and to participants. The rate of such distributions may be as frequent as quarterly to all participants, and therefore may require extensive resources and IT capabilities from the Contractor. For all data distributed by the Contractor, appropriate documentation and evidence of compliance with DUAs, including the return of original datasets or documentation/evidence of their destruction or archival, shall be provided

c. Analytic Files: The Contractor shall distribute limited and focused analytic files based on claims or other evaluation data to designated persons within the Awardee sites as needed as part of the process of analysis, the contractor will be responsible for the creation of analytic data files. The creation of analytic data files to be used to answer the evaluation questions will include the development and documentation of the necessary codes, outputs, quality assurance structures and controls to guarantee the analysis results. In the approach for creating the analytic files, the Contractor must accommodate the need for rapid-cycle analysis. The Contractor shall have an approach to the pulling of data and the analysis thereof that allows for the reporting of findings on a monthly or quarterly basis. All CDs, DVDs, data tapes, and other files shall be cleaned, edited, and ready for use for use in administering the program while it is still operational.

5. Develop operations for calculating and administering various types of payments, including care management fees, incentive payments , shared savings

a. Assisting CMS in calculating the appropriate payment rate for the selected model/demonstration participants, including using risk adjustment methodologies, maintaining payment information history for use in monitoring and evaluation, responding to participants’ payment questions, and providing general technical support to CMS for ongoing updates and adjustments to initially agreed upon payment rates.

6. Develop process for responding to beneficiary/provider inquiries

a. Supporting CMS in interface with 1-800-Medicare, State Medicaid agencies, or other organizations and/or conduct other approaches for monitoring beneficiary concerns related to the administration of the model/demonstration.

7. Create/maintain systems for collecting performance data:

a. Obtain appropriate claims, survey, and clinical data for measurement, conduct necessary trainings for participants, establish new or use existing IT platforms and software as appropriate. The Contractor may be required to determine performance benchmarks, calculate quality scores, perform site reviews and audits where required, and create performance reports for participants.

8. Develop operational requirements for monitoring

a. The Contractor will determine the need for and obtain the necessary data and conduct all relevant analyses to assist CMS in providing ongoing oversight. Such data collection and analysis activities will be based on data from participants, CMS and other secondary data, routine surveys and other primary data, ad hoc data. The frequency at which the Contractor must obtain secondary data may be as frequent as monthly and may be as frequent as quarterly collection for primary data. Quantitative and qualitative analysis of the various types of data should be rapid, methodologically sound, robust, comprehensive, and appropriate to the respective research questions and the target audiences. Upon identification of potential issues of concern, the Contractor will notify CMS or a CMS Contractor with auditing responsibilities for further investigation of any suspicious patterns and trends.

9. Maintain critical systems of record

10. Administrative support

a. Provide administrative support to CMS workgroups that include CMS and HHS staff. Support may include scheduling meetings, preparing materials for meetings, developing meeting minutes and action items

Run

1. Administer beneficiary and/or provider enrollment/attribution process and maintain master enrollment list of beneficiaries (comprehensive and point in time)

a. The Contractor shall maintain records on participation status, data on attributes of the participants, data on performance scores, and produce ad hoc operational reports upon request by CMS.

2. Calculate and process various types of payments (disbursed or received).

3. Create and share data, as applicable, with model participants

4. Respond to beneficiary/provider inquiries

5. Provide technical assistance

a. The contractor shall, as required, provide technical assistance to the operating sites to improve project implementation, such as in soliciting participants, developing referrals, working with community partners, and designing processes to operate the project design."

6. Refine the model/demonstration

a. Updating any aspect of model/demonstration design and operations, as needed, to improve its success in improving health and care, and reducing costs.

7. Conduct routine and ad hoc program monitoring activities

a. The contractor shall monitor participants for non-compliance with program rules and terms and conditions of the model/demonstration and report to CMS any findings

8. Create and produce monitoring reports

a. The Contractor will design and prepare reports in the frequency and format appropriate to the data type, target audience, and evolving nature of the programs. The intended audiences of the reports are CMS and model participants. For reports to CMS, the content should be appropriate for addressing the needs for program oversight, evaluation, and learning and diffusion. For reports to participants, the content should include information that would be helpful for participants to track their progress among peers and to gauge areas for improvement. Reports to participants may include varying levels of grouping (i.e., at the state, hospital system, and hospital level or other levels) as appropriate to the research questions at hand and the program or demonstration being examined.

b. The contractor shall provide technical assistance to report recipients needed to understand and interpret the reports.

c. The formats of the reports should address all of the needs of the audiences, and may include, but not be limited to, trend reports, dashboards, customizable data tables and charts, summary statistics, and summary findings. The reports must be produced at a rapid manner, and may range from immediate notification to annual reports, depending on the content and urgency of the information to be delivered. At a minimum, the Contractor should have the capacity to generate a large volume of monitoring reports on a quarterly basis.

Evaluate

Demonstration/Model evaluation and/or research activities include the steps needed to plan, develop, and carry out evaluation of CMS-sponsored models or conduct CMS sponsored research activities, quantitative and qualitative analyses to address the research questions that are the focus of the model/research, and extensive report writing. It is anticipated that these evaluation activities may on occasion include the development of Public Use Data Files from the model/research evaluation data. At the conclusion of the contract, the Contractor shall provide CMS with a copy of all analytic data files generated during the study. Following are examples of activities/functions that normally would be expected to be included in many evaluation task orders:

1. Preliminary Evaluation Design

Development of a framework, including specification of testable research hypotheses, and programmatic and policy process and outcome measures of interest

a. The Design Plan shall provide a detailed and comprehensive overview of the methodological and analytical steps to be taken to address all of the purposes and research questions pertinent to the specific task order. For the evaluation, the design approach must be sufficiently detailed to comprehensively describe the intervention being tested and how the effects of the intervention will be assessed. The evaluation/research design plan so developed must address all relevant research questions for a given task order and be implementable in a robust yet cost effective manner. As part of this effort the Contractor shall design and carry out process analyses (qualitative analyses to assess project processes) and impact analysis.

b. The design approach should, at a minimum, specify the items listed below:

· Statement of the research questions and hypotheses to be addressed by the evaluation.

· How each question or hypothesis will be answered or tested by the research design.

· Addressing what data will be necessary in order to test the relevant hypotheses. For data elements necessary for the carrying out relevant analyses, the contractor shall make a critical assessment of the availability, cost to obtain, if any, and content and quality of existing data sources. The evaluation design shall further propose additional data collection activities that may be desirable and specify sampling designs and survey instruments as required.

· An analysis plan that specifies which statistical treatments of the data will be utilized.

· An analysis plan that incorporates, where appropriate, standardized metrics and analytic approaches that are coordinated within participants of each initiative or model and potentially across other Innovation Center or CMS initiatives.

· Identify quality performance strategies and metrics

· Identify awardee base data reporting requirements

· Identify preliminary learning and diffusion strategy and drivers

· Identify preliminary actuarial and scaling strategies

· Design and perform simulations (repeated)

2. Development of detailed and comprehensive overview of the methodological and analytical steps, including identification of methodological and statistical analytic techniques to be used

3. Determination of data needs and approach to obtaining data

a. If the approved Design Plan calls for primary data collection in the form of interviews, focus groups, or site visits or other approaches, the Contractor shall be responsible for developing, for each separate data collection activity, the associated materials. Where relevant, the contractor is responsible for obtaining any necessary approvals and clearances for the use of the data collection instruments created. This may include but is not necessarily limited to receiving approval for use under the Paperwork Reduction Act.

b. Create and design the materials associated with data collection. The Contractor shall:

· Incorporate existing measures and constructs, where appropriate. The existing items should be ones that have undergone validity testing, preferably in a sample representative of the population in which the instrument will be administered.

· If an existing data collection instrument is not available or is not suitable for addressing the needs of the evaluation, the Contractor shall develop its own data collection instrument with all necessary associated materials. To the extent that pilot testing is necessary or relevant to these processes, the contractor shall carry it out in accordance with any OMB-PRA restrictions

4. Develop methodologies and execute calculations for financial and payment simulations in support of the model design and evaluation

5. Conduct evaluations:

a. Policy analyses: Conducting policy analyses to assess emerging issues and changing policy directions in health care using a variety of research techniques and developing a high level financial impact model for the proposed concept. The contractor shall reviewing Medicare, Medicaid, CHIP and other Federal and State laws, regulations, policies, and programs to understand how they contribute to the problem or the opportunity. The contractor analyze the limits of any activity that CMS may undertake to address a problem, and identify potential solutions; identifying and exploring the types and sources of information available on a selected issue and assess, through limited review, the potential relevance and value of each.

6. Creation of Primary Data Collection Materials

a. Conduct Primary Data Collection Activities: The Contractor is responsible for obtaining any necessary primary data in order to carry out the agreed upon approaches in Design Plan. All activities associated with collecting, processing, quality assurance activities and editing the data shall be included. Within primary data collection activities, the following are common sources

b. Participant Submitted data: The Contractor must design the approach to collect and obtain any participant reported data. The Contractor must be able to carry out the data collection process with minimal provider burden. This may entail IT capabilities that allow the Contractor to abstract data from existing provider data systems where appropriate, and to have the capacity to receive and track large volumes of data transmitted from participants to CMS and the Contractor.

c. Case Studies: As appropriate, the Contractor shall conduct case studies to collect in-depth information that can be used primarily for three purposes: 1) to inform feedback provided to the participating sites or to CMS, 2) to inform CMS oversight of the program, and 3) to derive information that can be used to address the evaluation and monitoring questions. The Contractor must have sufficient resources to conduct large numbers of case studies in a manner that is both methodologically sound and appropriate for the scope of the initiatives.

d. Surveys: As appropriate the Contractor shall conduct surveys of Beneficiary and/or Beneficiary Proxies, Organizations, Providers, Stakeholders, and other persons or groups as relevant to the specific TO.

e. Interviews: As appropriate the Contractor shall conduct interviews with specific subject matter experts, participant staff members, Beneficiaries and/or Beneficiary Proxies, Organizations, Providers, Stakeholders, and other persons or groups as relevant.

f. Learning Network Observation: As appropriate the Contractor shall observe and document Learning Network activities necessary to address the research questions.

g. Other: As appropriate, the Contractor shall collect any necessary data as agreed upon in the design report.

7. Obtain CMS and/or State data

a. Obtain CMS, private health care sector and/or State data: The Contractor shall make arrangements to obtain all data necessary for supporting the design of the model/demonstration, including CMS claims and non-claims data, enrollment/eligibility information, and State and private health care sector data as required for the task. As part of the process of obtaining CMS data, the Contractor shall be responsible for obtaining any necessary Data Use Agreements (DUAs) and for adhering to CMS data security policies and procedures

b. CMS Claims and Eligibility Data: Eligibility and claims data are available through a variety of approaches that differ in ways including the timeliness of the creation of data pulls relative to the services being examined and the relative ease of access. Under current CMS data policies, there are a number of data retrieval mechanisms for Medicare, Medicaid, and CHIP administrative data and claims. In addition, for specific evaluation projects, the contractor may need to obtain Medicaid and/or non-Medicaid data directly from States. As warranted by the need for rapid cycle results, the Contractor must develop data flow processes to ensure the Contractor receives data in a timely fashion with an awareness of detailed nuances for each program’s data systems and strengths and weaknesses. CMS is committed to facilitating the Contractor’s access to these claims data to the extent possible; however, it is ultimately the Contractor’s responsibility to ensure they are able to effectively access and use their chosen data retrieval mechanism.

c. Non-Claims Data: The Contractor shall arrange for the timely access to non-claims data sources as relevant to the analysis needs for both participants and relevant comparison group members. The Contractor shall assess the need for, and if necessary obtain data from other data sources.

8. Conduct Primary Data Collection Activities, including collection of participant submitted data, site visits, focus groups, surveys, and interviews

a. The data collection portion of the design plan shall specify in detail what data collection activities will be undertaken with respect to the collection of both primary and secondary data. It should, at a minimum data collection plans should contain the purpose of the data collection, the source of the data, draft data collection instruments, discussion of samples sizes and sampling technique, detailed time tables and budget estimates, and cost of obtaining the data.

b. The Contractor would also be required to prepare any required Office of Management and Budget reports clearance packages for the implementation of the data collection plan.

9. Carry out all qualitative and quantitative analyses of primary and secondary data necessary to address the research and/or evaluation questions in accordance with the approved design plan

a. The Contractor shall carry out all qualitative and quantitative analyses of primary and secondary data necessary to address the research and/or evaluation questions in accordance with the approved design plan. The qualitative analyses should be viewed as the context within which the quantitative analyses are described. That is, the qualitative and quantitative analyses should not be approached as distinct and separate activities. One should inform the other in a very explicit and deliberate manner. The Contractor shall inform the COR of any significant or irreconcilable problems encountered with the data analysis and of any deviations from the analysis plan.

b. The contractor shall be prepared to perform analysis by type of intervention or other relevant categories. To the extent possible, the Contractor will evaluate the project using data at the individual level to answer all applicable research questions. In some cases, we anticipate the need for or benefit of grouping similar participants together to improve the generalizability and statistical precision of results, as well as benefitting from economies of scale.

10. Create and produce quarterly or monthly reports for CMS

a. The evaluation approach within CMS is rapid cycle with close to real time production of findings as feasible. It is expected that the Contractor has the capabilities to carry out rigorous regression based approaches for quarterly reports.

b. Findings must be available to CMS and all of the participants throughout the life of the project. Specifically, CMS will expect to receive timely information on findings that can be used by policy makers to make decisions about programmatic changes and to gauge progress. CMS expects to provide model participants with feedback reports to support their continued improvement, which may lead to modifications of interventions as participants learn and become more sophisticated. In accordance with Federal Law, all reports shall be required to be 508 compliant. Contractors are expected to be sufficiently versed in and staffed for 508 activities so as to not appreciably delay the distribution of reports after receipt of the approval of the final version of the document.

c. Other Reports -- The Contractor shall be responsible for all other reports that may need to be generated at a frequency ranging from monthly to semi-annual to an ad hoc basis. One example of such reports include quarterly program compliance reports for use by CMS for the purposes of overseeing the administration of the program in terms of adherence to the rules set forth in the participant agreements.

11. Create and produce quarterly feedback reports for model participants

a. The Contractor shall prepare quarterly or monthly reports appropriate for distribution to model participants or other designated parties involved in the administration of a model or initiative. The reports should describe the results of monitoring activities and interim evaluation findings at a granular level that allows for an examination of results at and within a participant.

12. Create and produce annual reports for CMS

a. The Contractor shall provide annual reports reflecting evaluation and/or research activities occurring in each of the contract years. Each Annual Report should be reflective of the work described in the Design Plan and the research questions.

b. The annual reports are expected to address findings of interest among significant subpopulations of interest. The annual report shall contain the key take away messages and executive summaries. Each annual report should be cumulative and summative, and should address trends and changes over time, and should be able to stand as a final evaluation product. The annual report at the end of the final option period (or the last exercised option) will be of special emphasis, and will represent the final report of the contract.

13. Create and produce Final report for CMS

0. The Contractor shall submit the one (1) original and two (2) copies of the approved Final Report to CMS upon completion of the period of performance set forth in the task order. The Final Report shall present the final results of the evaluations of each participants of the initiative and the overall assessment of the model. The final results shall address all of the research questions outlined in the Evaluation Design Report and describe the methods used to obtain these findings. The Final Report should draw from the Evaluation Design Report and build on the previous rapid cycle and annual reports.

14. Conduct evaluative support for operations, L&D and scaling

Model Learning and Diffusion

The learning and diffusion activities undertaken by the contractor will support the aims of raising awareness and engagement, achieving results, and disseminating new and promising ideas.

The learning and diffusion activities function as a vehicle to design, test, implement, and, ultimately, spread new models of care and payment delivery being developed. By serving multiple projects of varying sizes and in varying stages of readiness, contractors will coordinate across projects to ensure successful learning and implementation throughout program models.

In support of this effort, the contractor shall:

1. Manage and coordinate the learning systems and improvement networks for CMS program models, including:

· Development and execution of a results-oriented learning system including development of aims, key drivers, change strategies and measures;

· Leadership, guidance, and project management for learning communities;

· Content (curriculum) development necessary to achieve desired results, using expert faculty in the topic areas of need;

· Training in basic to advanced content of quality improvement methods and tools and operations research, including the Model for Improvement and use of the PDSA cycle. (API, Langley, et al.).

· Baseline and ongoing assessment of organizations participating in program models, supporting planning and content execution of learning community

· Faculty recruitment, management, and deployment;

· In-person meeting planning, logistics, and facilitation;

· Web-based meeting planning, logistics, and facilitation;

· Change package development and revision for program models; capturing and packaging key financial, business, clinical care and patient experience elements to facilitate structured learning and spread; and

· Development and execution of system-level measurement for each model program with creation of a view of progress toward program aims and learning system improvement.

2. Manage and coordinate discovery and dissemination activities to support and accelerate the ongoing learning of organizations involved in CMS program models, including:

· Site visits with structured qualitative interviews to capture ongoing learning and emerging results;

· Support for peer-to-peer learning from site visits to organizations with early indications of success;

· Development of brief narrative reports to highlight emerging learning and successes (story banking);

· Identification of promising practices and high performing sites as faculty presenters at in-person and virtual meetings;

· Monitoring and reporting the participation of organizations and participants in learning activities;

· Development and maintenance of meaningful views (e.g. maps and graphs) demonstrating engagement of organizations in CMS program models, spread of specific activities, and emerging results demonstrating progress in achieving the three measures of better health, better care and lower costs through improvement for each program model; and

· Development and maintenance of a knowledge management system for each model programs, including both documentation structure and organization in a web portal environment and a systematic method to capture lessons learned.

3. Maintain and continuously improve a web-based platform for knowledge sharing, collaboration, networking and reporting with the capacity to support multiple, parallel learning communities to support learning and diffusion activities of the CMS, including:

· Proactively identify, develop and implement new functionality based on industry best practice, monitoring of use and user input; and

· Support program models using the web platform by monitoring site activity and assist staff and participants in utilizing functionality optimally.

4. Develop regionally distributed networks capable of providing support to organizations participating in program models.

· These organizations will be identified in close coordination with CMS staff as required to support specific models.

· The contractor will provide the financial,…