RFP_AHRQ-15-10006_-_Amendment_2.pdf

Request for Proposal Amendment 2 v. May-14 AHRQ-15-10006 Coordinating Center for Comparative Health System Performance Page 1 of 84

AHRQ-15-10006 Amendment No. 2

Coordinating Center for Comparative Health System Performance

Table of Changes

No. Description of Change Section Page No.

1. Add Note on Conflict of interest Section A 2

2. Clarification of the term “methodological guide” Section C 14

3. Add Conflict of interest contract clause Section H.14 46

4. Add/Updated/Removed Contract clauses Section I 47-52

5. Add Budget assumptions Section J 53

6. Updated Annual Representations and Certifications Section K.1 54

7. Add Conflict of interest solicitation provision Section L 63

8. Add Solicitation Provisions Selection L.1 63

9. Clarification on Past Performance Section L.8 77

10. Revision/Clarification on business proposal Section L 72-81

11.

Removed the Small Disadvantaged Business Participation Plan requirement in its entirety

Section L and M 63-84

Note: Amendments are indicated by highlight throughout the document as indicated in the table above.

If a revision to this solicitation based on an individual industry question or comment was not included in this amendment please assume that either (1) the Government considers the original solicitation clearly articulated (offerors are cautioned to read the RFP carefully), (2) the response would be dependent on the offerors proposed approach, (3) the government did not understand what question was being asked, and/or (4) the Government does not have any additional information to provide at this time.

Coordinating Center for Comparative Health System Performance Page 2 of 84

Request for Proposal – Amendment #2 No. AHRQ-15-10006

PART I - THE SCHEDULE

SECTION A - SOLICITATION FORM

Date Issued: March 27, 2015, Amendment No. 2 Date Issued: March 4, 2015 (Original RFP) Date Questions Due: March 16, 2015, 11:59 PM, EST Please Note: All questions and inquiries will be processed on a non-attribution basis.

Date Notice of Intent Due: April 14, 2015 Date Proposals Due: April 21, 2015, 11:00 AM, EST

You are invited to submit a proposal to the Agency for Healthcare Research and Quality (AHRQ) for Request for Proposal (RFP) No. AHRQ-15-10006 entitled “Coordinating Center for Comparative Health System Performance”. Your proposal must be developed and submitted in accordance with the requirements and instructions of this RFP.

Please note: Conflict of Interest: A significant organizational conflict of interest has been identified for this solicitation. See section L herein for additional information.

Solicitation Information: Please review the table of contents and refer to the appropriate section of this RFP for requirements, proposal instructions and evaluation information. The Government intends to award this contract without negotiations, so it is imperative that questions are addressed during the solicitation phase, and that all proposals submitted include the required information and accept the terms and conditions as stated. Failure to comply with proposal instructions or submit required information may result in the removal of a proposal from consideration for award. Please review Section L carefully.

Proposal Intent: If you intend to submit a proposal in response to this solicitation, please inform the Contracting Officer of your intent by completing the Proposal Intent Response Form (See section L.12 and Attachment A) and send it to the Contracting Officer no later than March 31, 2015. You may mail it to the address below; fax it to 301-427-1740, Attention: Michelle Beatty, Contract Specialist/Contracting Officer or email it to Michelle.Beatty@ahrq.hhs.gov.

Questions: All Questions regarding this solicitation shall be received in this office no later than the date shown above. All questions shall be submitted electronically by e-mail to Michelle Beatty, Contract Specialist, at the following email address: Michelle.Beatty@ahrq.hhs.gov.

Subject line shall read: Proposal Questions RFP No. AHRQ-15-10006. Answers to questions and all solicitation amendments will be posted to the Federal Business Opportunities web page:

www.fbo.gov. It is your responsibility to monitor the web site.

Coordinating Center for Comparative Health System Performance Page 3 of 84

Additional Information:

A cost plus fixed fee contract is contemplated for a period of one year, with four one-year options.

The Government anticipates awarding one contract from this solicitation on or about

November 2, 2015.

Proposals submitted in response to this solicitation must state that they are valid for a period of at least 120 days (reference FAR clause 52.215-1 in Section L of this document.

The NAICS code for this requirement is 541990 with a size standard of $14 million.

Proposal Submission: The proposal shall be signed by an authorized official to bind your organization and must be received in our Contracts Office by the due date specified above.

Your proposal must be mailed to the following address:

Agency for Healthcare Research and Quality Division of Contracts Management 540 Gaither Road Rockville, Maryland 20850 Attn: Michelle Beatty, (301) 427-1781

Hand carried proposals may be dropped off at the above location. However, please allow ample time as proposals cannot be accepted until they have gone through security. We will not be held responsible for any delays that may be incurred getting your proposal through security.

NOTE: The U.S. Postal Service’s “Express Mail” does not deliver to our Rockville, Maryland address. Packages delivered via this service will be held at a local post office for pick-up. The Government will not be responsible for picking up any mail at a local post office. If a proposal is not received at the place, date, and time specified herein, it will be considered a “late proposal.”

System for Award Management: In accordance with Federal Acquisition Regulation, all contractors must be registered in the System for Award Management (SAM) database in order to conduct business with the government [See Section I - FAR clause 52.204-7 System for Award Management]. Access the System for Award Management (SAM) at http://www.sam.gov.

Reminder: The Procurement Integrity Act prohibits the unauthorized disclosure of, inquiry about, or acceptance of procurement information. Communications with any other individual outside the AHRQ contracting office may result in rejection of an offeror’s proposal. All requests for any information concerning this RFP should be referred to Michelle Beatty, (301) 427-1781 or e-mail: Michelle.Beatty@ahrq.hhs.gov. Please note e-mail requests should state subject as

RFP AHRQ-15-10006.

Coordinating Center for Comparative Health System Performance Page 4 of 84

Table of Contents

PART I ‐ THE SCHEDULE

SECTION A ‐ SOLICITATION FORM

SECTION B ‐ SUPPLIES OR SERVICES AND PRICES/COSTS

SECTION C ‐ DESCRIPTION/SPECIFICATION/WORK STATEMENT

SECTION D ‐ PACKAGING AND MARKING

SECTION E ‐ INSPECTION AND ACCEPTANCE

SECTION F ‐ DELIVERIES OR PERFORMANCE

SECTION G ‐ CONTRACT ADMINISTRATION DATA

SECTION H ‐ SPECIAL CONTRACT REQUIREMENTS

PART II – CONTRACT CLAUSES

SECTION I ‐ CONTRACT CLAUSES

PART III‐ LIST OF DOCUMENTS, EXHIBITS AND ATTACHMENTS

SECTION J – SOLICITATION LIST OF ATTACHMENTS

PART IV– REPRESENTATIONS AND INSTRUCTIONS

SECTION K ‐ REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS OF OFFERORS

SECTION L ‐ INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

SECTION M‐ EVALUATION FACTORS FOR AWARD

Attachments

Attachments listed in Section J are appended at the end of this document.

Coordinating Center for Comparative Health System Performance Page 5 of 84

SECTION B - SUPPLIES OR SERVICES AND PRICES/COSTS

B.1 BRIEF DESCRIPTION OF SUPPLIES OR SERVICES

“Coordinating Center for Comparative Health System Performance”. See Section C for a complete description.

B.2 ESTIMATED COST AND FIXED FEE

a. The Estimated Cost (exclusive of fees) for performance of the work under the base period of this contract, including direct and indirect costs is $ To Be Determined (TBD).

b. The Fixed Fee for the base period of this contract is $TBD. The fixed fee shall be paid in installments based on the percentage of completion of work, as determined by the Contracting Officer. Payment shall be subject to the withholding provisions of the Clause ALLOWABLE COST AND PAYMENT and FIXED FEE incorporated herein.

c. The total contract value, represented by the sum of the estimated cost plus the fixed fee for the base period is $TBD.

Period of Performance

Services Estimated Cost Fixed Fee Total

Base Period Severable $ $ $

B.3 OPTION COSTS

If the Government exercises its option(s) pursuant to the option article in Section H of this contract, the Government’s total contract amount represented by the sum of the estimated cost and fixed fee will be revised as follows:

Period of

Services Estimated Cost Fixed Fee Total

Option Period 1 Severable $ $ $ Option Period 2 Severable $ $ $ Option Period 3 Severable $ $ $ Option Period 4 Severable $ $ $

The base period and each option are separate and distinct from one another for accumulation of costs and billing purposes.

Coordinating Center for Comparative Health System Performance Page 6 of 84

OPTION TASKS

SOW Task Services Estimated Cost Fixed Fee Total

Task 5 Base Period Severable $ $ $ Task 6 Base Period Severable $ $ $ Task 7 Base Period Severable $ $ $

Task 5 Option Period 1 Severable $ $ $ Task 6 Option Period 1 Severable $ $ $ Task 7 Option Period 1 Severable $ $ $

Task 5 Option Period 2 Severable $ $ $ Task 6 Option Period 2 Severable $ $ $ Task 7 Option Period 2 Severable $ $ $

Task 5 Option Period 3 Severable $ $ $ Task 6 Option Period 3 Severable $ $ $ Task 7 Option Period 3 Severable $ $ $

Task 5 Option Period 4 Severable $ $ $ Task 6 Option Period 4 Severable $ $ $ Task 7 Option Period 4 Severable $ $ $ Task 8 Option Period 4 Severable $ $ $

B. 4 PROVISIONS APPLICABLE TO DIRECT COSTS

a. Items Unallowable Unless Otherwise Provided

Notwithstanding the clauses, ALLOWABLE COST AND PAYMENT, and FIXED FEE, incorporated into this contract, unless authorized in writing by the Contracting Officer, the costs of the following items or activities shall be unallowable as direct costs:

(1) Acquisition, by purchase or lease, of any interest in real property;

(2) Rearrangement or alteration of facilities;

Coordinating Center for Comparative Health System Performance Page 7 of 84

(3) Purchase or lease of any item of general purpose-office furniture or office equipment regardless of dollar value. (General purpose equipment is defined as any items of personal property which are usable for purposes other than research, such as office equipment and furnishings, pocket calculators, etc.);

(4) Accountable Government property;

(5) Travel to attend general scientific meetings;

(6) Foreign Travel;

(7) Any costs incurred prior to the contract's effective date;

(8) Rental of meeting rooms not otherwise expressly paid for by the contract;

(9) Any formal subcontract arrangements not otherwise expressly provided for in the contract;

(10) Consultant fees;

(11) Information Technology hardware or software;

(12) Food and Beverages; and

(13) No funding or expenditures are authorized under this contract for travel or meeting expenses of any person that is not an employee of the contractor or approved subcontractor in accordance with the terms of the contract. See 31 USC § 1345 for appropriations restrictions.

Note: Food and Beverages are unallowable costs. Contracting Officer Authorization for Food and Beverages will be only made as specifically authorized by statute, which is extremely rare. The contractor shall not provide food or beverages of any type. The contractor shall not accept food or beverages, even if included in, or provided ‘free’ as part of a meeting/conference ‘package’.

b. Travel Costs

This contract is subject to the provisions of Public Law (P.L.) 99-234 which amends the

Office of Federal Procurement Policy Act to provide that contractor costs for travel, including lodging, other subsistence, and incidental expenses, shall be allowable only to the extent that they do not exceed the amount allowed for Federal employees. The Contractor, therefore, shall invoice and be reimbursed for all travel costs in accordance with Federal Acquisition Regulations (FAR) 31.205-46.

Coordinating Center for Comparative Health System Performance Page 8 of 84

The following applies to Educational Institutions: The Contractor shall invoice and be reimbursed for all travel costs in accordance with OMB Circular A-21 - "Cost Principles for Educational Institutions."

B.5 CONTRACTING OFFICER AUTHORIZATION

Other provisions of this contract notwithstanding, approval of the following items within the limits set forth is hereby granted without further authorization from the Contracting Officer.

To Be Determined

Coordinating Center for Comparative Health System Performance Page 9 of 84

SECTION C - DESCRIPTION/SPECIFICATION/WORK STATEMENT

Independently and not as an agent of the Government, the Contractor shall be required to furnish all the necessary services, qualified personnel, material, equipment, and facilities, not otherwise provided by the Government, as needed to perform the Statement of Work as follows:

Statement of Work

Unless otherwise noted under a specific task, this Statement of Work applies to the base period and all options (if exercised).

1. INTRODUCTION/BACKGROUND

A major AHRQ priority is producing evidence that will increase the affordability, efficiency and quality of health care. AHRQ seeks to develop and spread evidence to measure and enhance the efficiency of health systems. Delivery system reform has created new models of service delivery intended to facilitate the spread of patient-centered outcomes research (PCOR). Increasingly, physicians and other providers work in new types of complex arrangements with ownership structures and incentives that are not well understood (Casalino, 2014). In particular, evidence on how to effectively target PCOR-related dissemination efforts to physicians or to delivery systems has been limited (Casalino, 2014).

AHRQ seeks to improve the efficiency and value of U.S. health care by understanding how high performing health care systems quickly diffuse PCOR evidence into practice to improve outcomes and reduce costs. AHRQ’s goal is to identify, characterize, and classify types of delivery systems and to compare the performance (quality, resource use, cost) of these systems with each other.

The purpose of this contract is to establish a single Comparative Health System Performance Coordinating Center to facilitate and support activities of the Centers of Excellence to be funded under FOA HS-14-011, Comparative Health System Performance in Accelerating PCOR Dissemination. The Systems Performance Coordinating Center is intended to support the Centers of Excellence on Comparative Health System Performance activities by leveraging expertise, infrastructure, and resources across three Centers of Excellence (CoEs). The Coordinating Center shall conduct activities such as data analyses;

convening stakeholders and technical experts; and facilitating collaboration across CoEs to harmonize data, methods, and measures where possible and to foster cross-CoE collaborations. The Coordinating Center shall serve as the focal point for synthesizing research and evidence from the COEs on the characteristics of high-performing health systems and practices to accelerate PCOR dissemination to improve health system performance. The Coordinating Center shall also serve as a resource to promote broad dissemination of research on the characteristics of high-performing health systems and practices to accelerate PCOR dissemination to improve health system performance.

Casalino, Lawrence P. Identifying Key Areas for Delivery System Research. Rockville, MD:

Agency for Healthcare Research and Quality; 2014. AHRQ Publication No. 14-0024EF.

Coordinating Center for Comparative Health System Performance Page 10 of 84

2. REFERENCES

For the “Comparative Health System Performance in Accelerating PCOR Dissemination” (RFA-HS-14-011), see http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-14-011.html

3. SCOPE OF WORK

Task 1: Convene Technical Experts and Stakeholders The Coordinating Center shall include a Technical Expert Panel (TEP) which shall consist of relevant and interested stakeholders representing diverse perspectives on health system organization, PCOR dissemination and implementation, and other forms of systems performance. In addition to providing regular consultation and advice to the Coordinating Center on all aspects of this contract, TEP members shall serve as presenters or discussants at the annual in person meetings (see Subtasks 2.2 and 2.3).

Subtask 1.1: Establish Technical Expert Panel (TEP) Within eight (8) weeks from the effective date of the award, the Contractor shall establish a standing TEP that consists of a group of twelve (12) to fifteen (15) individuals who are recognized experts in the field of measuring and evaluating systems performance. The panel shall consist of technical experts and relevant stakeholders representing delivery systems, academics, professional societies (e.g., American Medical Association), associations (e.g., American Hospital Association), and Federal agencies (e.g., Center for Medicare and Medicaid Innovation). In particular, the Contractor shall:

Define proposed criteria to nominate TEP members and a description of TEP member role and responsibilities, and submit these criteria for approval by the COR.

Submit proposed TEP member names and curriculum vitae to the COR for review and approval.

Invite TEP members.

Ensure that TEP members do not have any conflicts of interest, including financial and contractual relationships with grantees. This may involve having TEP members sign appropriate Disclosure Statements.

Subtask 1.2: Develop Plan and Timeline for TEP Consultation The Contractor shall facilitate TEP consultation to the grantees and AHRQ. TEP members shall serve as an advisory panel to AHRQ.

TEP consultation shall include:

Discussion of data and measures being collected and planned research activities.

Advice on how to harmonize measures across the Centers of Excellence.

Feedback on the type of evidence that will be most useful to stakeholders.

Type of dissemination vehicles to reach stakeholders most quickly and usefully.

Periodic feedback on grantees’ research analyses and Contractor dissemination products.

Identification of research gaps to shape future research initiatives by the Agency.

Coordinating Center for Comparative Health System Performance Page 11 of 84

1.2.a: Draft Plan and Timeline for TEP Consultation

Within two (2) weeks after establishing the TEP, the Contractor shall submit for the COR’s approval a Draft TEP Consultation Plan and Timeline, which shall be a revision of the Preliminary TEP Consultation approach submitted with the proposal.

o The TEP Consultation Plan shall specify organizational and logistical support for all TEP meetings, including preparing materials, and scheduling and coordinating meetings.

Within four (4) weeks after establishing the TEP, the Contractor shall finalize the schedule for convening the TEP.

o The TEP shall be convened annually in person for the one day Annual TEP Meeting at AHRQ offices in Washington DC. The meeting shall be timed to overlap with the Annual Grantee Workshop, so that there is an opportunity for grantees and the TEP to interact.

o In addition to the annual in person meeting, TEP meetings shall be held quarterly during the course of the contract and shall be conducted either via conference call or virtually (e.g., Webex).

1.2.b: Final Plan and Timeline for TEP Consultation Within two (2) weeks after receipt of comments, the Contractor shall incorporate the comments and feedback into the Final TEP Consultation Plan for providing organizational and logistical support for all TEP meetings. The Final TEP Consultation Plan shall be submitted for the COR’s approval.

Subtask 1.3: Convene TEP The Contractor shall convene the TEP in accordance with Subtask 1.2.a.

No later than ten (10) days prior to the meeting date the contractor shall submit a draft agenda for each TEP meeting to the COR for review and approval by the COR.

No later than one (1) week after the TEP meeting, the contractor shall submit a draft report on the deliberations of the meeting to the COR for review and approval.

Within one (1) week after receipt of comments, the contractor shall submit a final report incorporating the COR’s comments to the COR for approval.

NOTE: This shall be in accordance with the offeror’s proposal of which elements including the approach to this task will be incorporated before award.

Task 2: Coordinate and Facilitate Cross-Grantee Collaboration The Contractor shall serve in the role of facilitator and work with the grantees and appropriate AHRQ staff (to be determined by the COR) to create infrastructure to support cross-grantee alignment of concepts, terms, and measures for comparing health system characteristics and performance.

Subtask 2.1: Create Grantee Work Groups on Systems Performance To facilitate collaborations and synthesis of findings, the Contractor shall create three (3) to four (4) grantee work groups consisting of grantees who received funding through the

Coordinating Center for Comparative Health System Performance Page 12 of 84

“Comparative Health System Performance” announcement (RFA-HS-14-011) to discuss research design, data issues, and findings across grantee projects.

Within eight (8) weeks of the effective date of award, the Contractor shall work with the grantees and the COR to create a list of grantee work groups and members. The proposed grantee work groups and membership shall be submitted to the COR for review and approval.

Subtask 2.2: Convene Grantee Work Groups The Contractor shall host and manage all arrangements for the grantee work group meetings.

The grantee work groups shall be convened annually in person for the one day Annual Grantee Workshop at AHRQ offices in Washington DC. The meeting shall be timed to overlap with the Annual TEP Meeting, so that there is an opportunity for grantees and the TEP to interact.

In addition to the annual in person meeting, grantee work group meetings shall be held quarterly during the course of the contract and shall be conducted either via conference call or virtually (e.g., Webex).

No later than ten (10) days prior to the grantee work group meeting, webinar and/or conference call, the Contractor shall develop an agenda and provide the necessary materials to be submitted to the COR for review and approval.

Within one (1) week after the grantee work group meeting, the contractor shall submit a draft report on the deliberations of the work group to the COR for review and approval.

Subtask 2.3: Annual Grantee Workshop on Synthesizing Research & Evidence on Systems Performance The Annual Grantee Workshop shall be convened in person in Washington DC, annually.

2.3.a: Draft Plan for Convening Annual Grantee Workshop on Synthesizing Research & Evidence on Systems Performance Within six (6) months of the effective date of award, the Contractor shall develop and submit to the COR a Draft Workshop Plan, which shall be a revision of the Preliminary Workshop approach submitted with the proposal. The Workshop Plan shall include the following elements:

o Meeting invitations;

o List of participants, subject to the COR’s approval and including appropriate AHRQ staff to be determined by the COR;

o Logistics and staffing, including a skilled meeting facilitator;

o Agenda for meeting;

o List of topics for discussion including coordination and synthesis of research and possible areas of cross-fertilization of projects;

o Approach for achieving meeting objectives;

o Approach for summarizing the proceedings and disseminating its products to key stakeholders;

o List of materials that will be distributed to participants before, during, and after the meeting.

Coordinating Center for Comparative Health System Performance Page 13 of 84

Up to ten (10) current or past AHRQ grantees not funded through the Systems FOA but working on related issues may be invited to participate as presenters or discussants at the Annual Grantee Meeting.

2.3.b: Final Plan for Convening Annual Grantee Workshop on Synthesizing Research & Evidence on Systems Performance Within two (2) weeks after receipt of comments, the Contractor shall incorporate the feedback into a Final Workshop Plan. The Contractor shall revise the Final Workshop Plan based on feedback and submit to the COR for approval.

2.3.c: Convene Annual Grantee Workshop on Synthesizing Research & Evidence on Systems Performance The Contractor shall convene the Annual Grantee Workshop in accordance with the provisions of Subtask 2.3.a.

No later than one (1) week after the meeting, the contractor shall submit a draft report on the deliberations of the workshop to the COR for review and approval.

Within one (1) week after receipt of comments, the contractor shall submit a final report incorporating the COR’s comments shall be submitted to the COR.

NOTE: This shall be in accordance with the offerors proposal of which elements including the approach to this task will be incorporated before award.

Subtask 2.4: Facilitate Measure Development The Contractor shall identify and compare measures of systems characteristics and performance across the Centers of Excellence. The Contractor shall work with the grantees to identify commonalities in their measures of a) health system characteristics and b) performance (e.g. quality, cost, resource use). The Contractor shall provide methodological expertise and technical assistance to grantees in the development of new measures, assessment of existing measures, and/or data collection instruments that can be used to collect common data elements across multiple funded projects.

2.4.a: Draft Plan to Harmonize Measures across Grantee Research Projects

Within six (6) months of the effective date of award, the Contractor shall develop and submit to the COR a Draft Measurement Plan, which shall be a revision of the Preliminary Measurement approach submitted with the proposal. The Measurement Plan shall include the following elements:

o Activities to facilitate harmonization of measures across grantee data cores and research projects.

o Areas where common or harmonized measures could be useful to the grantees and AHRQ.

o Relevant sources and methods to draw upon in harmonizing measures of health system organization and performance.

o Challenges anticipated in harmonizing measures and strategies for addressing these challenges.

The Contractor shall recommend to the COR for approval a list of common data elements to be collected by the grantees in a uniform and reliable manner.

Coordinating Center for Comparative Health System Performance Page 14 of 84

2.4.b: Final Plan to Harmonize Measures across Grantee Research Projects Within two (2) weeks after receipt of comments, the Contractor shall incorporate the comments and feedback into a Final Measurement Plan. The Contractor shall revise the Final Measurement Plan based on feedback and submit to the COR for approval.

2.4.c: Implement Plan to Harmonize Measures across Grantee Research Projects The Contractor shall implement the provisions of Subtask 2.4.a. via the approved Final

Measurement Plan to harmonize measures across grantee research projects.

Within twelve (12) months of effective date of award, the contractor shall submit a draft report on implementing the Final Measurement Plan to the COR for review and approval.

Within one (1) week after receipt of comments, the contractor shall submit a final report incorporating the COR's comments to the COR.

NOTE: This shall be in accordance with the offeror’s proposal of which elements

Subtask 2.5: Coordinate Data Sharing Among Grantees For planning purposes, relevant data might include but need not be limited to grantee surveys of health care organizations; American Hospital Association survey databases; SK&A Information Services databases; Medicare fee-for-service claims; state all-payer claims databases; and electronic health record data from delivery system organizations.

The Contractor shall coordinate data sharing among grantees and the development of data use agreements (DUA).

The Contractor shall implement up to fifteen (15) DUAs and maintain records. The Contractor shall submit a copy of DUAs to the COR for review and approval.

Task 3: Synthesize and Share Research & Evidence on Systems Performance with Key Audiences The Contractor shall facilitate the synthesis of research findings and methodological reports, creation of products for health systems and other key stakeholders, and a plan to facilitate adoption and use of the evidence and products by health systems, policymakers and other stakeholders.

Subtask 3.1: Develop Products on Systems Performance Based on the work of the grantees and related sources, the Contractor shall review and synthesize the evidence being produced and develop relevant products described in Subtask 3.1.c on topics related to the characteristics of systems and their performance.

3.1.a: Draft Plan for Products on Systems Performance Within six (6) months of the effective date of award, the Contractor shall develop and submit to the COR a Draft Products Plan, which shall be a revision of the Preliminary Products approach submitted with the proposal. The Products Plan shall identify anticipated major themes and synergies across grantee projects that shall be highlighted in the products that the Contractor will develop. In preparing the Products Plan, the Contractor shall consult with AHRQ staff and the TEP about the potential major themes and types of products on systems characteristics and performance described in Subtask 3.1.c.

Coordinating Center for Comparative Health System Performance Page 15 of 84

NOTE: This shall be in accordance with the offeror’s proposal of which elements

3.1.b: Final Plan for Products on Systems Performance Within two (2) weeks after receipt of comments, the Contractor shall incorporate comments and feedback from the COR and appropriate AHRQ staff with regard to the Final Products Plan. The Contractor shall revise the Final Products Plan based on feedback and submit to the COR for approval.

3.1.c: Implement the Plan for Products on Systems Performance The Contractor shall implement the provisions of Subtask 3.1 via the approved Final Plan for products on systems performance. Implementing the plan shall entail the development of the following 508-compliant products during the designated year:

Year 1 (Base Year), 2015:

Two (2) to three (3) reports (about twenty (20) pages in length) and one (1) peer-reviewed article shall be submitted to the COR within twelve (12) months of effective date of award that might include:

o Evidence syntheses that synthesize past research and evidence on comparative health system performance.

o Research agendas to identify high priority gaps in the understanding of current state of health system organization and performance.

o Related topics or issues recommended by the TEP and approved by the COR.

Year 2 (Option Year 1), 2016:

Two (2) to three (3) reports (about twenty (20) pages in length) shall be submitted to the COR within twelve (12) months of the option exercise that might include white papers and/or methodological reports focusing on issues in measure development and harmonization.

Three (3) to four (4) briefs that shall include, Data Briefs that present statistical findings from grantee data cores or other relevant sources and/or Policy Briefs that synthesize evidence from grantee research projects and related evidence shall be submitted to the COR within twelve (12) months of the option exercise.

Year 3 (Option Year 2), 2017:

Two (2) to three (3) reports (about twenty (20) pages in length) that shall include white papers and/or methodological reports focusing on issues in measure development and harmonization shall be submitted to the COR within twelve (12) months of the option exercise.

Three (3) to four (4) briefs that might include Data Briefs that present statistical findings from grantee data cores or other relevant sources and/or Policy Briefs that synthesize evidence from grantee research projects and related evidence shall be submitted to the COR within twelve (12) months of the option exercise.

Coordinating Center for Comparative Health System Performance Page 16 of 84

Year 4 (Option Year 3), 2018:

One (1) to two (2) reports (about twenty (20) pages in length) and one (1) peer-reviewed article that shall include white papers and/or methodological reports focusing on issues in measure development and harmonization shall be submitted to the COR within twelve (12) months of the beginning of contract year

Three (3) to four (4) briefs that might include Data Briefs that present statistical findings from grantee data cores or other relevant sources and/or Policy Briefs that synthesize evidence from grantee research projects and related evidence shall be submitted to the COR within twelve (12) months of the option exercise.

Year 5 (Option Year 4), 2019:

A summative report synthesizing major grantee findings and one (1) peer-reviewed journal article shall be submitted to the COR within twelve (12) months of the beginning of contract year.

Three (3) to four (4) briefs that include Data Briefs that present statistical findings from grantee data cores or other relevant sources and/or Policy Briefs that synthesize evidence from grantee research projects and related evidence shall be submitted to the COR within twelve (12) months of the option exercise.

Subtask 3.2: Provide a Strategy to Share Research & Evidence on Systems Performance with Key Audiences The Contractor shall collaborate with AHRQ and the grantees to develop and execute a strategy and plan to share research and evidence with key audiences, including delivery systems, policymakers, researchers, and the public.

3.2.a: Draft Plan to Share Research & Evidence on Systems Performance with Key Audiences (BASE PERIOD ONLY) Within twelve (12) months of the effective date of the award, the Contractor shall develop and submit to the COR a Draft Plan for dissemination of products, which shall be a revision of the Preliminary Dissemination approach submitted with the proposal. The Draft Dissemination Plan shall include, at a minimum:

Describing the Dissemination Plan for each product in Task 3.1, including the nature and extent of distribution.

Arranging for panels and sessions in at least two (2) conferences, meetings, and/or other venues per contract year.

Preparing materials from products or other materials derived from this contract at conferences, meetings, and other venues.

Developing and implementing a thorough and systematic approach to identify the best venues and approaches to address key audiences.

Working with AHRQ’s Office of Communication and Knowledge Transfer (OCKT) to arrange press conferences, notices of grantee publications, and postings on appropriate AHRQ web sites. Facilitating dissemination of findings through existing systems performance networks, delivery systems, and related organizations.

Coordinating Center for Comparative Health System Performance Page 17 of 84

3.2.b: Final Plan to Share Research & Evidence on Systems Performance with Key Audiences Within two (2) weeks of receiving COR’s comments, the Contractor shall incorporate comments and feedback from the COR and appropriate AHRQ staff into a Final Dissemination Plan. The Contractor shall revise the Final Dissemination Plan based on feedback and submit to the COR for approval.

3.2.c: Implement Plan to Share Research & Evidence on Systems Performance with Key Audiences

The Contractor shall implement the provisions of Subtask 3.1 via the approved Final Dissemination Plan to share research & evidence on systems performance.

Within twelve (1) months of effective date of award, the contractor shall submit a draft progress report on implementing the Final Dissemination Plan to the COR for review and approval.

Within one (1) week after receipt of comments, the contractor shall submit a final progress report incorporating the COR's comments to the COR.

NOTE: This shall be in accordance with the offeror’s proposal of which elements including the approach to this task will be incorporated before award.

Task 4: Program Management The Contractor shall provide general planning, administration, and communication support for the conduct of all contract activities.

Subtask 4.1: Organize and Host Kick-Off Meeting

Within ten (10) days of the effective date of award, the Contractor shall participate in person for a Kick-off meeting at AHRQ with the COR and appropriate AHRQ staff. The purpose of the Kick-off meeting is to discuss the overall objectives and specific elements of work to be performed, to review the scope of work and delivery schedule, to delineate further roles and responsibilities, and to establish communication protocols.

No later than five (5) days prior to Kick-off meeting, the Contractor shall submit a draft agenda, and any draft materials to be discussed at the meeting, to the COR for review and approval.

Within one (1) week after the Kick-off meeting, the Contractor shall provide an electronic summary of the discussion to all participants of the meeting.

Subtask 4.2: Develop Work Plan 4.2.a: Draft Work Plan Within two (2) weeks of the effective date of award, the Contractor shall prepare and submit to the COR a comprehensive Draft Work Plan, which shall be a revision of the Preliminary Work approach submitted with the proposal. The Work Plan shall describe how the major components of the specific tasks outlined in this Statement of Work (SOW) will be carried out. Revisions will reflect additional details discussed during the Kick-off meeting.

4.2.b: Final Work Plan

Within one (1) week after receipt of comments, the Contractor shall submit a Final Work Plan to the COR for approval. The plan must include all information necessary

Coordinating Center for Comparative Health System Performance Page 18 of 84 to establish the “baseline plan” for this project and an appendix highlighting any changes from the Draft Work Plan.

Annually, the Contractor shall provide revisions to the work plan as necessary to the COR in electronic format.

Subtask 4.3: Plan and Conduct Bi-Weekly Meetings, Conference Calls, and Other Communications

The Contractor shall schedule bi-weekly meetings with the COR and appropriate AHRQ staff, unless the COR stipulates that a meeting is unnecessary. During these meetings, the Contractor shall review and summarize current activities and any issues that may have arisen.

No later than two (2) business days prior to the meeting, webinar and/or call, the Contractor shall develop an agenda and submit the necessary materials to the COR for review and approval.

Within one (1) week after the meeting, the Contractor shall provide an electronic summary of the discussion to all participants of the meeting.

In the event that less formal communication is required between the Contractor and COR, the Contractor shall use email communications and phone calls as appropriate.

Subtask 4.4: Organize and Manage Project-related Documents In consultation with the COR, the Contractor shall track, maintain, and archive all project-related documentation for access by AHRQ staff.

The Contractor shall maintain a consistent style, format, and quality in all reports and written materials.

The Contractor shall develop and implement a reliable back-up system to prevent permanent, accidental loss of any research and reports generated throughout the project. Back-up shall be in media and formats readily accessible to the COR.

The Contractor, with approval of the COR, shall archive and place older files in an alternative storage site.

The Contractor shall convert hard copy documents to electronic versions.

Subtask 4.5: Provide Monthly Status Reports Within fifteen (15) days after the end of the month being reported, the Contractor shall submit to the COR a monthly progress report. The report will be filed via ARRS or another data system specified by AHRQ.

The monthly status report shall state in concise form: 1) a brief description of what was actually accomplished for each task during the reporting period, in comparison to planned milestones; 2) problems or delays experienced and actions that are proposed to alleviate the problem(s); 3) current and proposed expenditures relative to original schedule and budget; 4) adjustments or changes to correct problems; and 5) anticipated activities for the next reporting period, including anticipated staffing requirements, level of effort and costs.

Monthly status reports shall be organized according to Tasks/Subtasks and include activities related to:

Coordinating Center for Comparative Health System Performance Page 19 of 84 o Convening of grantee work groups o Implementation of the plan for products on systems performance o Implementation of the plan to share research & evidence on systems performance.

Subtask 4.6: Provide Annual Report The Contractor shall develop an Annual Report that describes the background and approach for the project and summarizes activities under the contract for the contract year.

The Annual Report shall be drafted with the intent that it will be distributed for review and comment to the COR and appropriate AHRQ staff.

4.6.a: Draft Table of Contents for Annual Report No later than two (2) months prior to the end of the contract year, the Contractor shall submit to the COR a proposed Table of Contents outlining the content of the Annual Report to the COR for review and feedback.

4.6.b: Draft Annual Report No later than one (1) month prior to the end of the contract year and after incorporating comments on the Table of Contents, the Contractor shall submit electronically a Draft Annual Report to the COR and appropriate AHRQ staff for review and feedback.

4.6.c: Submit Annual Report Within one (1) week after receipt of comments and no later than the end of the contract year, the Contractor shall develop and submit the Annual Report.

OPTIONAL Task 5: Procure and Maintain Data on Systems Characteristics to Produce Compendium on Systems Performance.

Under this task, the Contractor will procure annual data on system characteristics not already available to the project through the grants.

Subtask 5.1: Procure Data on Systems Characteristics to Produce Compendium on

The Contractor shall procure annual data on health system characteristics from SK&A Information Services and the American Hospital Association (AHA), for use by both AHRQ and the Contractor.

SK&A data shall include information on the census of office-based physicians with National Provider Identifier (NPI) and linkages to medical group, hospital and health system ownership, and accountable care organization participation.

AHA Survey of Care Systems and Payment data shall include information on the census of acute care hospitals with CMS Certification Number (CCN) and details on hospital organization and systems-related practices.

Coordinating Center for Comparative Health System Performance Page 20 of 84

Subtask 5.2: Maintain Data on Systems Characteristics to Produce Compendium on

The Contractor shall document and maintain procured data for inclusion in the Compendium and provide AHRQ with access to data, results, and documentation.

NOTE: This optional task may be exercised one time in the base and once each option period.

OPTIONAL Task 6: Provide Statistical Data Analysis and Programming Services to Produce Compendium on Systems Performance The Contractor shall conduct statistical data analyses and programming services to produce the compendium on systems performance. Programming activities include the creation of data files and variables on systems characteristics and systems performance from procured data and other available secondary data (i.e., Medicare fee-for-service claims); aggregation of variables by organization and geography; and linkage with appropriate grantee-provided data. Statistical data analysis activities include descriptive statistics, cross-tabulations, and data visualizations (i.e., charts, maps) of systems performance by organization, geography, and systems characteristics.

Statistical data analysis and programming services will address specific research questions as directed by the COR. The Contractor shall provide services for at least four (4) analytic requests per contract year as directed by the COR. For example, one analytic request might be to examine the association between practice characteristics and Accountable Care Organization participation and rates of preventable hospitalizations. Using procured data on systems characteristics (e.g., SK&A Office-based Physicians Database) and available data (e.g., Healthcare Cost and Utilization Project State Inpatient Databases) on systems performance, the Contractor shall create analytical files by applying Prevention Quality Indicator (PQI) algorithms to hospital claims; creating relevant health system variables; running statistical analyses; and documenting output.

Subtask 6.1: Provide Data Files and Documentation

Using appropriate grantee-provided data, procured data on systems characteristics and other secondary data, the contractor shall create new data files and variables with documentation.

The Contractor shall ensure data files include identifiers for potential linkage of systems information across grantees and with procured data.

The Contractor shall design, test, and run programs to meet analytic requests identified in collaboration with AHRQ. The task includes providing specified results in tabular formats and providing full documentation of files created and work formed on all analytic tasks upon their completion.

The Contractor shall provide AHRQ with copies of data files and documentation for in-house programming and analysis, as requested by the COR.

Subtask 6.2: Provide Statistical Analysis and Programming Services The Contractor shall be required to work with econometric and statistical techniques used with large administrative datasets, including surveys and claims data. This task shall include designing, testing and running statistical analyses, including descriptive

Coordinating Center for Comparative Health System Performance Page 21 of 84 statistics, cross-tabulations and data visualizations, using programs such as SAS and Stata.

The Contractor shall produce ten (10) special data runs and documentation, given on one (1) week’s notice, in response to requests from other government agencies and private organizations, as directed by the COR. For example, one special run might be to report the characteristics of Accountable Care Organization entities using procured AHA data. Data runs and documentation shall be submitted to the COR for approval.

NOTE: This optional task may be exercised one time in the base and once each option period.

OPTIONAL Task 7: Produce Compendium on Systems Performance The Contractor shall work in collaboration with grantees to produce a single compendium on systems performance. The goal of the compendium is to link and/or combine grantee data (ie, surveys, claims) and procured data to make it possible to create a cross-project aggregate picture of systems characteristics and outcomes. The compendium shall contain information on the types and characteristics of healthcare delivery systems and systems performance (quality, resource use, and costs) using combined grantee data and procured data. The purpose of the compendium is to provide information that can be used to inform PCOR dissemination strategies so that delivery systems can more quickly adopt high-performance strategies and improve patient outcomes.

Subtask 7.1: Draft Plan to Produce Compendium on Systems Performance Within six (6) months of the beginning of the contract year, the Contractor shall develop and submit to the COR a Draft Compendium Plan for producing a compendium on systems performance based on grantee data cores and associated research projects.

The Contractor shall discuss contents and format of the compendium with grantees and the TEP.

As described in subtask 2.5, the Contractor shall work with grantees to develop appropriate data use agreements to share the names of the health systems they studied as well as relevant identifiers (e.g., Medicare hospital identifiers and physician National Provider Identifiers), along with outcome measures and systems’ performance on these measures.

The Contractor shall use identifiers to link information about particular systems across grantees and to link with procured data on systems characteristics and other secondary data.

Within twelve (12) months of the beginning of the contract year, the contractor shall create up to four (4) analytic data files for the compendium with documentation using appropriate grantee-provided data, procured data on systems characteristics and other secondary data to be submitted to the COR for approval.

The compendium shall be in the form of a 508-compliant National Report on Comparative Health System Performance (about 200 pages in length). The compendium shall include statistics on the prevalence of systems and their characteristics, including breakdowns by system type, geography, and other

Coordinating Center for Comparative Health System Performance Page 22 of 84 characteristics. The compendium may contain separate components derived from the grantee data cores.

Subtask 7.2: Final Plan to Produce Compendium on Systems Performance Within two (2) weeks after receipt of comments, the Contractor shall incorporate the comments and feedback into the Final Compendium Plan. The Contractor shall revise the Final Compendium Plan based on feedback and submit to the COR for approval.

Subtask 7.3: Implement Plan to Produce Compendium on Systems Performance

The Contractor shall implement the plan to produce the compendium and submit the draft Compendium to the COR for review and approval.

Within twelve (12) months of the beginning of…