AHRQ-19-10003_Final.pdf

Request for Proposal v. 9/5/18; FAC 100 AHRQ-19-10003 CAPSTONE Page 1 of 103

Request for Proposal No. AHRQ-19-10003

PART I - THE SCHEDULE

SECTION A - SOLICITATION FORM

Date Issued: November 15, 2018 Date Questions Due: November 29, 2018 02:00 PM, EST Please Note: All questions and inquires will be processed on a non-attribution basis.

Date Notice of Intent Due: December 17, 2018 Date Proposals Due: January 3, 11:00 AM, EST

You are invited to submit a proposal to the Agency for Healthcare Research and Quality (AHRQ) for Request for Proposal (RFP) No. AHRQ-19-10003 entitled “CAPSTONE - Outcome Measure Harmonization and Data Infrastructure for Patient Centered Outcomes Research in Depression”. Your proposal must be developed and submitted in accordance with the requirements and instructions of this RFP.

Please Note: This RFP is a total small business set aside. See FAR 52.219-6 and 52.219-14, herein.

Solicitation Information: Please review the table of contents and refer to the appropriate section of this RFP for requirements, proposal instructions and evaluation information. The Government intends to award this contract without negotiations, so it is imperative that all questions are addressed during the solicitation phase, and that all proposals submitted include the required information and accept the terms and conditions as stated. Failure to comply with proposal instructions or submit required information may result in the removal of a proposal from consideration for award. Please review Section L carefully.

Proposal Intent: If you intend to submit a proposal in response to this solicitation, please inform the Contracting Officer of your intent by completing the Proposal Intent Response Form (See section L.11 and Attachment A) and send it to the Contracting Officer no later than December 17, 2018. You may mail it to the address below; fax it to 301-427-1740, Attention: Tara Bertolini, Contract Specialist/Contracting Officer or email it to tara.bertolini@ahrq.hhs.gov.

Questions: All Questions regarding this solicitation shall be received in this office no later than the date shown above. All questions shall be submitted electronically by e-mail to Tara Bertolini, Contract Specialist/Contracting Officer, at the following email address:

tara.bertolini@ahrq.hhs.gov. Subject line shall read: Proposal Questions RFP No. AHRQ-19- 10003. Answers to questions and all solicitation amendments will be posted to the Federal

Business Opportunities web page: www.fbo.gov. It is your responsibility to monitor the web site.

CAPSTONE Page 2 of 103

Additional Information:

a. A time and materials contract is contemplated for a four (4) month base period and one

(1), twenty (20) month option period.

b. The Government anticipates awarding one contract from this solicitation on or about May

15, 2019.

c. Proposals submitted in response to this solicitation must state that they are valid for a period of at least 120 days (reference FAR clause 52.215-1 in Section L of this document.)

d. The NAICS code for this requirement is 541690 with a size standard of $15M.

Proposal Submission: The proposal shall be signed by an authorized official to bind your organization and must be received by the due date specified above. The proposal must be submitted to the following address:

Agency for Healthcare Research and Quality (AHRQ)* Division of Contracts Management* 5600 Fishers Lane* Mail Stop 06N34B* Rockville, Maryland 20857* Attn: Tara Bertolini, (301) 427-1705*

*All boxes must be clearly labeled with this information.

Please read this section carefully. Some mail and package delivery procedures have changed due to security controls. All mail and packages are screened at a local offsite facility before being delivered to 5600 Fishers Lane.

The U.S. Postal Service’s “Express Mail” does not deliver to our Rockville, Maryland address.

Please do not use this means of delivery for proposal submission.

Proposals delivered by commercial package delivery company (example: Fedex, UPS, DHL, etc.):

Submit and label as instructed above. These companies are familiar with the delivery procedures at 5600 Fishers Lane.

Hand carried or hand carry courier service: The 5600 Fishers Lane building security does not allow packages to be delivered directly to the building. Hand carried or courier delivered proposals should be labeled as instructed above, however they must be delivered directly to the offsite screening facility at 12225 Wilkins Avenue Rockville MD 20852. The operating hours at the Wilkins Avenue facility are 7:00 AM to 3:30 PM Monday through Friday. Please note that anyone who arrives at the Fishers Lane location with a delivery will be directed to the screening facility, however this will cause delay. Packages will be accepted and date/time stamped at the screening facility. Please allow ample time as proposals cannot be date/time stamped until they have gone through security.

CAPSTONE Page 3 of 103

System for Award Management: In accordance with Federal Acquisition Regulation, all contractors must be registered in the System for Award Management (SAM) database in order to conduct business with the government [See Section I - FAR clause 52.204-7 System for Award Management]. Access the System for Award Management (SAM) at http://www.sam.gov.

Reminder: The Procurement Integrity Act prohibits the unauthorized disclosure of, inquiry about, or acceptance of procurement information. Communications with any other individual outside the AHRQ contracting office may result in rejection of an offeror’s proposal. All requests for any information concerning this RFP should be referred to Tara Bertolini, (301) 427-1705 or e-mail:

tara.bertolini@ahrq.hhs.gov. Please note e-mail requests should state subject as RFP AHRQ-19- 10003.

CAPSTONE Page 4 of 103

Table of Contents

PART I ‐ THE SCHEDULE

SECTION A ‐ SOLICITATION FORM

SECTION B ‐ SUPPLIES OR SERVICES AND PRICES/COSTS

SECTION C ‐ DESCRIPTION/SPECIFICATION/WORK STATEMENT

SECTION D ‐ PACKAGING AND MARKING

SECTION E ‐ INSPECTION AND ACCEPTANCE

SECTION F ‐ DELIVERIES OR PERFORMANCE

SECTION G ‐ CONTRACT ADMINISTRATION DATA

SECTION H ‐ SPECIAL CONTRACT REQUIREMENTS

PART II – CONTRACT CLAUSES

SECTION I CONTRACT CLAUSES

PART III‐ LIST OF DOCUMENTS, EXHIBITS AND ATTACHMENTS

SECTION J – SOLICITATION LIST OF ATTACHMENTS

PART IV– REPRESENTATIONS AND INSTRUCTIONS

SECTION K ‐ REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS OF OFFERORS

SECTION L ‐ INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

SECTION M‐ EVALUATION FACTORS FOR AWARD

CAPSTONE Page 5 of 103

SECTION B - SUPPLIES OR SERVICES AND PRICES/COSTS

B.1 BRIEF DESCRIPTION OF SUPPLIES OR SERVICES

“CAPSTONE - Outcome Measure Harmonization and Data Infrastructure for Patient Centered Outcomes Research in Depression”. See Section C for a complete description.

B.2 TIME & MATERIALS

All work shall be performed in accordance with the Statement of Work. No hours shall be billed for labor categories or labor rates not included as part of this contract as detailed below:

Labor Category Hourly Rate

Type of Material Not to Exceed Amount

TOTAL CEILING PRICE BASE PERIOD/Phase I (Labor and Materials) $

B.3 OPTION PRICES

If the Government exercises its option(s) pursuant to the option article in Section H of this contract, the following applies:

Option Period 1/Phase II

Labor Category Hourly Rate

CAPSTONE Page 6 of 103

Type of Material Not to Exceed Amount

TOTAL CEILING PRICE OPTION PERIOD 1/Phase II (Labor and Materials) $

B.4 PROVISIONS APPLICABLE TO DIRECT COSTS

a. Items Unallowable Unless Otherwise Provided

Notwithstanding the clauses, ALLOWABLE COST AND PAYMENT incorporated into this contract, unless authorized in writing by the Contracting Officer, the costs of the following items or activities shall be unallowable as direct costs:

(1) Acquisition, by purchase or lease, of any interest in real property;

(2) Rearrangement or alteration of facilities;

(3) Purchase or lease of any item of general purpose-office furniture or office equipment regardless of dollar value. (General purpose equipment is defined as any items of personal property which are usable for purposes other than research, such as office equipment and furnishings, pocket calculators, etc.);

(4) Accountable Government property;

(5) Travel to attend general scientific meetings;

(6) Foreign Travel;

(7) Any costs incurred prior to the contract's effective date;

(8) Rental of meeting rooms not otherwise expressly paid for by the contract;

(9) Any formal subcontract arrangements not otherwise expressly provided for in the contract;

(10) Consultant fees;

(11) Information Technology hardware or software;

(12) Food and Beverages; and

CAPSTONE Page 7 of 103

Note: Food and Beverages are unallowable costs. Contracting Officer Authorization for Food and Beverages will be only made as specifically authorized by statute, which is extremely rare. The contractor shall not provide food or beverages of any type. The contractor shall not accept food or beverages, even if included in, or provided ‘free’ as part of a meeting/conference ‘package’.

b. Travel Costs

This contract is subject to the provisions of Public Law (P.L.) 99-234 which amends the

Office of Federal Procurement Policy Act to provide that contractor costs for travel, including lodging, other subsistence, and incidental expenses, shall be allowable only to the extent that they do not exceed the amount allowed for Federal employees. The Contractor, therefore, shall invoice and be reimbursed for all travel costs in accordance with Federal Acquisition Regulations (FAR) 31.205-46.

The following applies to Educational Institutions: The Contractor shall invoice and be reimbursed for all travel costs in accordance with OMB Circular A-21 - "Cost Principles for Educational Institutions."

No funding or expenditures will be authorized under this contract for travel or meeting expenses of any person that is not an employee of the contractor or approved subcontractor in accordance with the terms of the contract. See 31 USC § 1345 for appropriations restrictions.

B.5 CONTRACTING OFFICER AUTHORIZATION

Other provisions of this contract notwithstanding, approval of the following items within the limits set forth is hereby granted without further authorization from the Contracting Officer.

To Be Determined

CAPSTONE Page 8 of 103

SECTION C - DESCRIPTION/SPECIFICATION/WORK STATEMENT

Independently and not as an agent of the Government, the Contractor shall be required to furnish all the necessary services, qualified personnel, material, equipment, and facilities, not otherwise provided by the Government, as needed to perform the Statement of Work as follows:

Statement of Work

Background As of April 2018, the AHRQ Registry of Patient Registries (RoPR) (see https://patientregistry.ahrq.gov/) contains records for almost 4000 registries for many different reasons including research, quality improvement, public reporting, and post market surveillance.

These registries represent a huge investment in data infrastructure and a potential wealth of data that could be reused for patient centered outcomes research. However, the lack of standardization of definitions in the registries makes analysis across registries impossible.

To address this issue, AHRQ has developed a core set of harmonized definitions of patient centered outcome measures in five different clinical areas (atrial fibrillation, asthma, depression, lung cancer and lumbar spondylolisthesis) in an existing Patient Centered Outcomes Research Trust Fund (PCORTF) funded project (see https://effectivehealthcare.ahrq.gov/topics/registry-of-patient-registries/outcome-measures-framework) . Despite the recognized value of these harmonized core outcome measures, stakeholders are reluctant to put resources into implementing them because of three main perceived barriers:

1. Burden on patients and clinicians to collect patient reported outcomes from patients;

2. Disruptions to clinical care when collecting structured data from clinicians or challenges in extracting high quality information from unstructured data already in patient records;

and

3. Difficulty working with electronic health records.

This project will address the barriers cited by registries and demonstrate feasibility and value by incorporating data collection in a way that fits into the workflow of busy practices and facilitates communication between specialists and primary care. This project will demonstrate how incorporation of harmonized outcome measures to clinical systems could improve clinical care, and improve compliance and sustainability for registry and clinical system developers. For example, this project will allow streamlined reporting of practices to CMS Qualified Clinical Data Registries (QCDRs) which will lead to less burden and higher payments for the practices under the Medicare Incentive Payment System (MIPS) and thus be more sustainable. Further, by collecting higher quality data into registries, the registries may be able to become platforms for pre and post market studies of medical products (for example, by enabling nesting of clinical trials in registry infrastructure) leading to revenue for the clinical sites while simultaneously lowering the cost of clinical research (see https://www.ctti-clinicaltrials.org/projects/registry-trials).

This project will specifically address data infrastructure for treatment of depression. Depression is a common condition managed by both primary care and specialty care providers.

Antidepressants are one of the most frequently prescribed drug classes during physician office visits (https://www.cdc.gov/nchs/fastats/drug-use-therapeutic.htm) yet depression remains

CAPSTONE Page 9 of 103 poorly recognized and inadequately treated in clinical care in the United States (https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2546155). Second line therapies include antipsychotics and implantable medical devices, but there is little evidence on the effectiveness of these treatments and which patients would benefit from these therapies.

Clinical care has important uncertainties that could be addressed by data in registries and patient care encounters.

The final list of harmonized outcome measures for depression are:

OMF

CATEGORY

OUTCOME

MEASURE

DEFINITION

MEASUREMENT

NOTES

Survival All-cause mortality

Death from any cause, reported in 12-month intervals.

Survival Death from suicide

Patient with a diagnosis of major depression or dysthymia who died from suicide, reported in 12-month intervals.

This should be captured where feasible; however, it should be noted that this information may not be recorded accurately or available to all providers.

CAPSTONE Page 10 of 103

OUTCOME

MEASURE

DEFINITION

MEASUREMENT

NOTES

Clinical Response

Improvement in Depressive Symptoms – Remission

Patient age 18 or older with a diagnosis of major depression or dysthymia and an initial PHQ-9* score > 9 who demonstrates remission defined as a PHQ-9 score less than 5.

*The PHQ-9 or another brief, publicly available, validated patient-reported instrument with empirically derived cutpoints equivalent to the PHQ-9 cutpoints for remission and response and for which an evidence-based crosswalk to the PHQ-9 exists should be used to measure clinical response. Other measures may be used in addition for research or other purposes.

Timeframe for measurement:

6 months (+/- 60 days)

12 months (+/- 60 days)

In some implementations, it would beneficial to capture earlier responses and remissions and to obtain higher degrees of follow-up.

Additional measurements outside of the windows listed above are recommended as supplemental measures.

CAPSTONE Page 11 of 103

OUTCOME

MEASURE

DEFINITION

MEASUREMENT

NOTES

Clinical Response

Improvement in Depressive

Response dysthymia and an initial PHQ-9* score > 9 who demonstrates a response to treatment defined as a PHQ-9 score that is reduced by 50% or greater from the initial PHQ-9 score.

reported instrument with empirically derived cutpoints equivalent to the PHQ-9 cutpoints for remission and response and for which an evidence-based crosswalk to the PHQ-9 exists should be used to measure clinical response. Other responses and remissions and to obtain higher degrees of follow-up.

Additional measurements outside of the windows listed above are

CAPSTONE Page 12 of 103

Clinical Response

Worsening in Depressive

Recurrence dysthymia and an initial PHQ-9* > 9 who demonstrates remission (defined as a PHQ-9 score < 5) of at least two months’ duration and subsequently experiences a recurrence of a depressive episode, defined as a 50% increase in PHQ-9 score or defined as a PHQ-9 score > 9 OR hospitalization for depression or suicidality.** reported instrument with empirically derived cutpoints equivalent to the PHQ-9 cutpoints for remission and response and for which an evidence-based crosswalk to the PHQ-9 exists should be used to measure clinical response. Other

**This definition was proposed by the workgroup. Data accruing from ongoing registries are needed to assess the feasibility of using this definition to capture recurrence.

responses and remissions and to obtain higher degrees of follow-up.

Additional measurements outside of the windows listed above are

Events of Interest

Adverse Events Depression treatment-related adverse events. Use of a brief, CAPSTONE Page 13 of 103

OUTCOME

MEASURE

DEFINITION

MEASUREMENT

NOTES

publicly available, validated measurement tool to capture adverse events is recommended.

Reported in 12-month intervals.

Events of Interest

Suicide Ideation & Behavior

Selection of ‘several days’, ‘more than half the days’ or ‘nearly every day’ option on PHQ-9 item 9 (“Thoughts that you would be better off dead or of hurting yourself in some way”).

Supplemental assessments of suicide ideation and behavior should be completed for patients who screen positive for suicide ideation on the PHQ-9 or when a clinician has concerns about suicidality. Supplemental assessments should be completed using an appropriate, brief, validated instrument, such as the Concise Health Risk Tracking (CHRT) scale. Includes nonfatal suicide attempts/suicide attempt behaviors, planning/preparatory acts, active suicidal ideation.

Reported in 12-month intervals (in conjunction with the PHQ-9 suicide item).

Patient Reported

Depression-specific Quality of Life

Depression-specific quality of life should be measured using a brief, validated, publicly available instrument that is appropriate for the population of interest, such as the Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES- Q). Reported in 12-month intervals.

CAPSTONE Page 14 of 103

OUTCOME

MEASURE

DEFINITION

MEASUREMENT

NOTES

Resource Utilization

Depression-related Resource Utilization

All resource utilization (as measured by cost) related to treatment or management of depression, including medications, psychotherapy, office visits, urgent care center visits, ED visits, and hospitalizations. Reported in 12-month intervals.

Resource Utilization

Work Productivity

Work productivity loss (overall work impairment/absenteeism plus presenteeism), as measured by the Work Productivity and Activity Impairment Questionnaire (WPAI), reported in 12-month intervals.

The AHRQ Confidentiality Statute, 42 U.S.C. 299c-3(c) and (d), applies to the activities conducted under the contract. The statute protects both identifiable individuals and identifiable establishments and provides, in part, that data may not be used for any purpose other than the purpose for which it was supplied absent the consent of the individual or establishment supplying the data. Neither compliance with HIPAA nor IRB approval necessarily ensures that the requirements of the AHRQ Confidentiality Statute are met.

References:

AHRQ RoPR system: https://patientregistry.ahrq.gov/

Information about the RoPR system and Outcome Measure Repository including the library of harmonized outcome measures, design documents:

https://effectivehealthcare.ahrq.gov/topics/registry-of-patient-registries/abstract/

AHRQ Applications and Systems Development Requirements. Appendix 2A in AHRQ Publishing and Communications Guidelines. AHRQ Publication No. 08(-09)-M049, April 2009. Agency for Healthcare Research and Quality, Rockville, MD.

https://www.ahrq.gov/research/publications/pubcomguide/index.html

AHRQ Web Site Guidelines in AHRQ Publishing and Communications Guidelines. AHRQ Publication No. 08(-09)-M049, April 2009. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/research/publications/pubcomguide/pcguide2.htmlAHRQ Information Security and Privacy Policy. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/policy/electronic/privacy/infosecurity.html

CAPSTONE Page 15 of 103

Overall Objectives/Goals of the Contract

The overall objectives of this contract are to facilitate and advance the use of registries and registry research.

Specifically, the objectives of this contract are:

(1) Collect electronic health data from existing structured and unstructured information in Electronic Health Records (EHRs) and directly form patients at multiple clinical sites at fixed intervals for patient outcomes associated with treatment for depression

(2) Transfer the collected data to electronic patient registries

(3) Develop governance and data use agreement templates with the registries so that data may be made available to external researchers.

(4) Develop infrastructure and tools to allow replication for other clinical registries and clinical systems. This project will examine whether this approach to data collection and transfer enhances the ability to use registries for research on patient outcomes.

The anticipated outcomes of this project are:

Phase I:

Risk analysis of number and ability of EHRs to send/exchange data to the registries

Phase II:

1. Tools for clinicians and patients to facilitate integration of the harmonized depression outcome measures into electronic health records (EHRs) and registries so that these data will be available for clinical research, patient-centered outcomes research, quality improvement and implementation research.

2. Proof of concept for a standards-based approach (an approach that uses technical standards for data extraction/collection/exchange) for collecting and reporting patient outcomes information to clinicians within their workflow and simultaneously transmitting the data to registries to make it available for research.

3. Tools to make it easier for researchers and registry developers to integrate registries with clinical systems.

4. Governance and data use agreement templates that can be used by registries to make data available to external researchers.

AHRQ is only funding the feasibility of creating a connection between the registries and the EHRs. AHRQ will neither own the connection nor will it fund the ongoing operations and maintenance cost for the connection.

Phase I: Risk Analysis (May 15, 2019 – Sept 23, 2019)

Task 1: Project management and Reports

CAPSTONE Page 16 of 103

Subtask 1.1: Kick-Off Call

At the beginning of the project, the contractor shall participate in an extended initial conference call with the COR and appropriate AHRQ staff to discuss the overall objectives and specific elements of work to be performed. During this call, any changes that are needed to the protocol and timeline that was submitted with the proposal shall be discussed.

Contact information for participants and signed conflict of interest statement for any new project team members who have been added since the original proposal was submitted to AHRQ will be provided from the contractor to the COR. The contractor shall schedule the call within 3 days of the Effective Date of the Contract (EDOC). Within 3 days of the call, the contractor shall send by email to the COR a summary of the conference call.

Deliverable: Call summary and a signed conflict of interest statement for any new members of the project team members who have been added since the original proposal was submitted to AHRQ.

Subtask 1.2: Conference Calls / Status Reports

Beginning one month after the Kick-Off Call, the contractor shall participate in monthly conference calls with the COR and other appropriate AHRQ staff to discuss the report and progress to completing the work.

Within 3 days of the call, the contractor shall send by email to the COR a summary of the conference call along with a concise progress report. The progress report shall document progress in completing tasks and any problems encountered and how they were addressed, and any foreseeable problems.

Deliverables: Call summaries and progress reports

Task 2: Convene a Stakeholder Panel

The contractor shall in consultation with AHRQ and ASPE, organize a stakeholder panel of no more than 20 members representing consumers of outcomes information, such as patient organizations, provider organizations, payers, quality measurement groups, life sciences organizations, research funders, and federal agencies. The contractor shall prepare the stakeholder panel roster and meetings agendas and other meeting materials at least one week before the meetings and submit to the COR. The panel will provide initial feedback on the protocol through a web-based meeting. This initial feedback will include how the project can contribute to overcoming barriers for clinical sites (especially primary care) to participate in data collection infrastructure that can be used for multiple purposes, including quality improvement and research. The feedback will also include how to incorporate elements into the project that will enhance sustainability and adoption of lessons learned at additional sites after the end of this project to build data capacity. The contractor shall incorporate suggestions from project partners, AHRQ and the stakeholder panel review into the modified protocol (task 3). The contractor shall prepare meeting notes and submit to the COR no later than one week after the stakeholder meetings.

CAPSTONE Page 17 of 103

Deliverables:

Stakeholder panel roster, meeting agendas and other meeting materials, meeting notes.

Task 3: Recruitment and Configuration

The contractor shall recruit at least 10 sites from each of 1) a primary care focused registry (e.g., PRIME Registry), 2) a specialty-focused registry (e.g., APA registry) (registries will have been identified during the proposal), and 3) an integrated healthcare delivery system. The contractor shall prepare all documents necessary for Institutional Review Boards (IRB) approval and submit to the COR and the IRBs as required for each site. The contractor shall also begin to prepare information needed for documents for OMB Paperwork Reduction Act clearance.

The contractor shall evaluate the registries against the requirements for registry-readiness to receive/exchange data that was submitted with the proposal and modify the protocol to meet these requirements. The contractor shall also evaluate the specific EHRs at each clinical site for readiness to send/exchange data to the registries. The contractor shall submit the modified protocol to the COR together with a risk analysis of the whether the objectives of the contract can be met in Phase II.

Deliverables: IRB documentation, modified protocol and risk analysis.

Note: Dependent on findings of the risk analysis, AHRQ and ASPE will make determination on whether to proceed to Phase II. AHRQ and ASPE may consult experts at the Office of the National Coordinator of Health Information Technology (ONCHIT) to review the risk analysis to determine whether technical feasibility is appropriately assessed.

Phase II: Tool development. (Sept 24, 2019 to May 15, 2021)

Task 1: Project Management and Dissemination

Subtask 1.1: Project Management and Reports

The contractor shall participate in monthly conference calls with the COR and other appropriate AHRQ staff to discuss the report and progress to completing the work.

Within 3 days of the call, the contractor shall send by email to the COR, a summary of the conference call along with a concise progress report. The progress report shall document progress in completing tasks and any problems encountered and how they were addressed, any foreseeable problems.

Deliverables: Call summaries and progress reports

Subtask 1.2: Dissemination Plan

The contractor shall finalize the preliminary dissemination plan that was submitted with the contractor proposal and submit to the COR. The dissemination plan shall

CAPSTONE Page 18 of 103 address the objectives, benefits, and activities of this project, and how the tools may be used by other registries and clinical sites. The plan should also include a list of professional meetings (for example, ISPOR, DIA, Health Datapalooza) where information about this project may be presented.

The contractor shall submit abstracts and present the project at at least five professional meetings over the period of performance.

Deliverables: Final dissemination plan

Subtask 1.3:

The contractor shall work with AHRQ to obtain OMB approval for data collection under the Paperwork Reduction Act (PRA). The contractor shall submit all required documents and responses to OMB questions to the COR and other AHRQ personnel as needed.

Deliverables: OMB PRA documents.

Task 2: Convene Stakeholder Panel

The contractor shall update the stakeholder panel on project progress four times per year through web based meetings. The contractor shall prepare the stakeholder panel roster and meetings agendas and other meeting materials at least one week before the meetings and submit to the COR. The contractor shall prepare meeting notes and submit to the COR no later than one week after the stakeholder meetings. In the final quarter of the contract, the stakeholder panel will review the final reports and outputs of the project, prioritize research questions on depression care that can be answered using data from harmonized depression outcome measures, and provide feedback on the deliverables.

Deliverables:

Stakeholder panel roster, meeting agendas and other meeting materials, meeting notes.

Task 3: Configuring registries and Final IRB approval

The contractor shall work with the two registries to assure their clinical registries are configured to support the receipt, exchange and delivery of clinical and patient reported outcomes specific to AHRQ’s harmonized depression measures (receipt from EHRs and a patient mobile application (task 5.2); exchange between the registries and delivery back to the clinical environment). The contractor shall test whether the two registries are ready for the next steps to using test data. The contractor shall develop and submit to the COR a report on registry configuration and challenge using test data.

The contractor shall obtain final IRB approval for data collection and exchange and submit documentation of IRB approval at all sites to the COR.

Deliverables: Report on registry configuration and challenge using test data Documentation of final IRB approval

CAPSTONE Page 19 of 103

Task 4: Develop products to facilitate the adoption of the AHRQ core set of depression outcome measures in EHRs

In order to facilitate the adoption of the AHRQ core set of depression outcome measures in EHRs, the contractor shall develop standardized implementation models, Fast Healthcare Interoperability Resources (FHIR, a standard for exchanging healthcare information electronically) profiles as appropriate, and an implementation guide following all of the procedures specified for the HL7 collaboration (https://www.hl7.org/fhir/implementationguide.html). The contractor shall deliver these documents to COR and work with Health Level 7 (HL7) to make the documents freely available on the HL7 website and provide information needed to implement the outcome measure definitions in EHRs.

Deliverables: Standardized implementation models, FHIR and/or similar profiles and implementation guide.

Task 5: Data extraction and exchange

The contractor shall maintain engagement with at least 10 clinical sites from each of two registries and an integrated health system and work with the registries on ongoing issues that may arise with registry configuration. The contractor shall pilot collection of the harmonized outcome measures as described in the modified protocol (task 3).

Subtask 5.1: Structured data collection

The contractor shall pilot collection of outcome measures linked to existing codes and value sets and extract the information from EHRs.

Subtask 5.2: Patient Reported Outcome Measures

The collection of the Patient Reported Outcomes (PROs) at fixed intervals will be particularly important to enable comparisons of outcome measures across provider types and care settings; currently, patients are seen and PROs collected at a wide variety of time points depending on their provider type, access to care/insurance status, and depression severity. Some of the sites have some capacity to collect PRO data at their sites when the patient is seen in an office visit. The contractor shall augment this capacity through mobile/web technologies to achieve the collection at fixed intervals independent of visits (i.e. mobile, web). If possible, the contractor shall leverage the FY17 OS- PCORTF PRO project ‘Advancing the Collection and Use of Patient-Reported Outcomes (PROs) through Health Information Technology,’ outputs (in particular the specifications and standards for integration into the EHR and allow collection at fixed intervals independent of patient visit to the provider). In addition, the contractor shall develop a function for automated reminders to patients to fill out the forms at fixed time intervals.

The contractor shall conduct electronic data collection of patient-reported outcomes (including the Patient Health Questionnaire (PHQ)-9) at baseline and every six months for one year, and generate outcome measurements that are presented back to the clinicians through standards-based integration with EHRs. The contractor shall combine the

CAPSTONE Page 20 of 103 electronic data collection from patients with the clinical data extracted from the EHR, and provide comparison reporting based on the OMH measures.

Subtask 5.3: Use of Natural Language Processing (NLP) for Unstructured Text Data

Much of the information in EHRs is collected as unstructured text data in “notes” sections.

Natural language processing is increasingly used to extract data from EHRs and several commercial and open source options for implementing NLP exist. Where direct collection of structured data from EHRs, clinicians and patients is not feasible, the contractor shall investigate and, where feasible, collect relevant unstructured data from the EHRs. The contractor shall process the data with existing tools using natural language processing and machine learning to identify specific data elements from the text information for specific characteristics or outcomes, such as adverse events and death by suicide, that could not be collected using the OMH standards directly. If such natural language processing and machine learning models are used for processing textual information, the contractor shall validate the models for performance by sampling patient records and having physician expert review compared to the output of the algorithms.

Subtask 5.4: Data Quality

A premise of this work is that high quality data can be captured in the EHR and sent to registries for use in research. To show that the predecessor steps yield data of high quality, the contractor shall validate data quality using an AHRQ-approved method.

Examples of techniques include source system loop back verification, ongoing source-to-source verification, data-issue tracking, and sampling data from EHRs and data sent to registries comparing results of outcome measure computation.

Subtask 5.5: Reporting to registries and back to clinicians

The contractor shall collect and report outcomes information from and to clinicians within their workflow through their EHR. These outcomes data will be submitted to registries that use the same data elements. Providing outcomes information within the clinician workflow through the EHR will enable clinicians to use that information in clinical decision-making. The selection of the standards-based approach will be determined during the contract period, with consideration of the EHRs used at the participating sites and whether those EHRs have fully implemented FHIR resources.

Task 6: Pilot Data Analysis and Prioritizing Topics for Patient-Centered Outcome Research

There are many patient-centered outcomes research questions that could be answered using real-world, robust data (i.e., EHR data sent to registries). The contractor together with the stakeholder panel shall design a pilot where the baseline registry data from the 2 selected registries, including those extracted from clinical records, can be obtained and used in analysis to answer a sample research question. This pilot analysis will serve as proof of concept that

CAPSTONE Page 21 of 103 clinical data harmonized on depression outcome measures sent to registries can be robust enough to answer research questions.

The contractor shall work with the registries to produce governance and data use agreement templates. The contractor together with the stakeholder panel will create and prioritize a list of research topics related to depression care that can be answered through the use of depression outcome measures harmonized across data sources. The contractor shall prepare and submit to the COR a report detailing pilot data analysis and high priority topics for patient-centered outcome research on depression care using real world, robust data and the approach used to determine prioritization.

Deliverable: report detailing pilot data analysis and high priority topics for patient-centered outcome research on depression care using real world, robust data and the approach used to determine prioritization

Task 7: Final report and artifacts

The contractor shall develop a draft and final report including methods used for data extraction, submitting data to the registries and back to clinicians including barriers encountered (both technical and institutional) and solutions to those barriers and assessment of lessons learned applicable to implementation of harmonized outcome measures in other clinical areas. Specifically, the final report and artifacts shall include:

Tools for clinicians and patients to facilitate integration of the harmonized depression outcome measures into electronic health records (EHRs) and registries so that these data will be available for clinical research, patient-centered outcomes research, quality improvement and implementation research.

Proof of concept for a standards-based approach (an approach that uses technical standards for data extraction/collection/exchange) for collecting and reporting patient outcomes information to clinicians within their workflow and simultaneously transmitting the data to registries to make it available for research.

Tools to make it easier for researchers and registry developers to integrate registries with clinical systems.

Governance and data use agreement templates that can be used by registries to make data available to external researchers.

The draft report will be posted on the AHRQ website by AHRQ for public review and also reviewed by invited peer reviewers, project partners AHRQ and the stakeholder panel. The contractor shall prepare a final report and written disposition of comments received and submit to the COR.

The contractor shall also prepare all artifacts from the project that may be reused in other clinical sites and registries to AHRQ in a format that can be made accessible to all interested stakeholders.

CAPSTONE Page 22 of 103

All reports and artifacts shall be consistent with AHRQ guidelines (see references in background section) as applicable, including 508 compliance.

In addition, the contractor shall develop drafts of at least two manuscripts suitable for publishing in peer reviewed journals based on the project final report and deliver to the

COR.

Deliverables: Draft report, Final report and disposition of all comments, Drafts of two manuscripts, Project artifacts

CAPSTONE Page 23 of 103

SECTION D - PACKAGING AND MARKING

All deliverables required under this contract shall be packaged, marked and shipped in accordance with Government specifications. At a minimum, all deliverables shall be marked with the Contract Number, Contractor Name and Item Number. The Contractor shall guarantee that all required materials shall be delivered in immediate usable and acceptable condition. Note as appropriate on the face page of each deliverable, or on the binding, (1) one volume only, or (2) volume 1 of 2, etc.

CAPSTONE Page 24 of 103

SECTION E - INSPECTION AND ACCEPTANCE

E.1 INSPECTION AND ACCEPTANCE

a. The Contracting Officer or the duly authorized representative will perform inspection and acceptance of materials and services to be provided.

b. For the purpose of this SECTION the Contracting Officer’s Representative is the authorized technical representative of the Contracting Officer.

c. Inspection and acceptance will be performed at:

Agency for Healthcare Research and Quality 5600 Fishers Lane Rockville, Maryland 20857

E.2 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)

This contract incorporates the following clause by reference, with the same force and effect as if it were given in full text.

52.246-6 Inspection Time-and Material and

Labor Hour (May 2001)

CAPSTONE Page 25 of 103

SECTION F - DELIVERIES or PERFORMANCE

F.1 PERIOD OF PERFORMANCE

a. The base period of performance (Phase I) of this contract shall be from May 15, 2019 through September 23, 2019.

b. If the Government exercises any Options pursuant to the OPTION PROVISION Article in Section H of this contract, the period of performance will be revised as follows:

Option Period Option Period Option Period 1/Phase II September 24, 2019 – May 15, 2021

F.2 DELIVERABLE SCHEDULE

The items specified for delivery below are subject to the review and approval of the Contracting Officer’s Representative (COR) before final acceptance. The contractor shall be required to make revisions deemed necessary by the COR. The deliverable schedule may only be changed by a contract modification signed by an AHRQ Contracting Officer. Days mean calendar days.

The contractor shall submit the following items electronically to the COR:

Task Deliverables Due Date Recipient Applicable Section 508 Standards

Phase I: Risk Analysis (May 15, 2019 – Sept 23, 2019) 1.1

Call summary, COI forms

6/1/19

AHRQ COR NA

1.2

Call summaries and progress reports monthly AHRQ COR NA

2 Stakeholder panel roster, meeting agenda and other meeting materials, 1 week prior to the meeting date

AHRQ COR,

Stakeholder panel

302 Functional Performance Criteria

602 Support Documentation

603 Support Services

All WCAG A & AA Success Criteria except:

2.4.1 Bypass

Blocks

2.4.5 Multiple

Ways

CAPSTONE Page 26 of 103

Consistent Navigation

3.2.4

Identification meeting notes 1 week after the meeting date; No later than: 8/1/19

AHRQ COR,

Stakeholder panel

302 Functional Performance Criteria

602 Support Documentation

603 Support Services

All WCAG A & AA Success Criteria except:

2.4.1 Bypass

Blocks

2.4.5 Multiple

Ways

Consistent Navigation

3.2.4

Identification

IRB Documentation No later than:

8/15/19 IRB submission 302 Functional

Performance Criteria

602 Support Documentation

603 Support Services

All WCAG A & AA Success Criteria

3 Modified Protocol No later than:

8/15/19

Public 302 Functional Performance Criteria

602 Support Documentation

603 Support Services

CAPSTONE Page 27 of 103

All WCAG A & AA Success Criteria

3 Risk Analysis No later than:

8/15/19

Phase II – Tool development (Sept 24, 2019 to May 15, 2021) 1.1

Call summaries and progress reports monthly AHRQ COR NA

1.2 Dissemination Plan 12/1/2019 AHRQ COR NA

1.3 OMB documents Draft

documents by Sept 30, Modifications as needed.

Stakeholder panel roster, meeting agendas and other meeting materials, meeting notes

Every 3 months

AHRQ COR NA

3 Report on registry configuration and challenge using test data

3/15/20 AHRQ COR NA

3 Documentation of Final IRB approval

3/15/20 AHRQ COR NA

Standardized implementation models, FHIR and/or similar profiles and implementation guide

3/15/21 Public 302 Functional Performance Criteria

602 Support Documentation

603 Support Services

All WCAG A & AA Success Criteria

6 Report on pilot data analysis and high priority topics for PCOR research on depression

5/15/21 Public 302 Functional Performance Criteria

602 Support Documentation

CAPSTONE Page 28 of 103

603 Support Services

All WCAG A & AA

Draft report Final report, disposition of comments, all artifacts and manuscripts

3/15/21 5/15/21

Public 302 Functional Performance Criteria

602 Support Documentation

603 Support Services

All WCAG A & AA

Copies of any deliverables should be sent to the Contracting Officer upon request.

Agency for Healthcare Research and Quality ATTN: Contracting Officer Division of Contracts Management, OMS 5600 Fishers Lane Mail Stop 06N34B Rockville, Maryland 20857

F.3 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)

This contract incorporates the following clause by reference, with the same force and effect as if they were given in full text. Upon request, the Contracting Officer will make their full text available.

FEDERAL ACQUISITION REGULATION (FAR) (48 CFR CHAPTER 1) CLAUSE:

52.242-15, Stop Work Order (AUG 1989) with Alternate I (APRIL 1984).

CAPSTONE Page 29 of 103

SECTION G - CONTRACT ADMINISTRATION DATA

G.1 KEY PERSONNEL, HHSAR 352.237-75 (DEC 2015)

Pursuant to the Key Personnel clause incorporated in Section I of this contract, the following individual(s) is/are considered to be essential to the work being performed hereunder:

NAME TITLE

(To Be Determined)

The clause cited above contains a requirement for review and approval by the Contracting Officer of written requests for a change of Key Personnel reasonably in advance of diverting any of these individuals from this contract. Receipt of written requests at least 30 days prior to a proposed change is considered reasonable.

G.2 CONTRACTING OFFICER’S REPRESENTATIVE

The following Contracting Officer’s Representative will represent the Government for the purpose of this contract:

(TO BE COMPLETED AT TIME OF CONTRACT AWARD)

The Contracting Officer’s Representative is responsible for: (1) monitoring the contractor's technical progress, including the surveillance and assessment of performance and recommending to the contracting officer changes in requirements; (2) interpreting the statement of work and any other technical performance requirements; (3) performing technical evaluation as required; (4) performing technical inspections and acceptances required by this contract; and

(5) assisting in the resolution of technical problems encountered during performance.

The Government may unilaterally change its Contracting Officer’s Representative designation.

G.3 CONTRACTING OFFICER

The Contracting Officer is the only person with authority to act as an agent of the Government under this contract. Only the Contracting Officer has authority to: (1) direct or negotiate any changes in the statement of work; (2) modify or extend the period of performance; (3) change the delivery schedule; (4) authorize reimbursement to the contractor of any costs incurred during the performance of this contract; or (5) otherwise change any terms and conditions of this contract.

G.4 INVOICE REQUIREMENTS AND INSTRUCTIONS

The directions below must be followed to meet the requirements of a "proper" payment request pursuant to FAR 32.9, and must be in accordance with the General Provisions clause 52.232-25 Prompt Payment (JAN 2017).

CAPSTONE Page 30 of 103

a. INVOICE CONTENT REQUIREMENTS

(1) The contractor shall use a Standard Form 1034, Public Voucher for Purchases and Services Other Than Personal, as an invoice coversheet when submitting/depositing invoices. An electronic copy of the SF-1034 in PDF format may be found at https://www.ahrq.gov/funding/contracts/1034-form.html?tca=jWR- LY2xeuTDCg9DluOnvNb131SIoJqIQd06MOL6coI.

Instructions how to complete the SF-1034 form can be found at:

https://www.ahrq.gov/funding/contracts/invoice-guide.html?tca=WK1a3t- LpRvys5y_5Vwz2QLqUlfsRvowGd8BVQJqcxM

(2) The contractor shall furnish the following MINIMUM information as an attachment to the SF-1034 in support of the invoiced amounts:

a. Contractor Information – include organization name and address as it appears on the contract (or task order/BPA call as applicable). Include the DUNS and TIN number.

b. Contract Information – include the project number, contract number and if applicable, the task order or BPA call number.

c. Invoice Information – include the invoice number, which shall be a unique number specific to each individual invoice. Letters and numbers may be used. Please note that the invoice number can only be used one time regardless of the number of contracts or orders held by an organization.

Duplicate invoice numbers are not allowed and will result in invoices being returned unpaid.

d. Labor Hours – include all persons, listing the person’s name, title, number of hours worked, labor category and hourly rate, the total cost per person and a total amount for each labor category.

e. Travel – include for each airplane or trip taken the name of the traveler, date of travel, destination, the transportation costs including ground transportation, shown separately, and per diem costs. Other travel costs shall also be listed.

A total amount for this category shall be provided.

f. Other – include a listing of all other direct charges to the contract, i.e., office supplies, telephone, equipment rental, duplication, etc.

g. Current amount billed – by individual cost element and total dollar amount

h. Cumulative amount billed – by individual cost element and total dollar amount.

b. INVOICE SUBMISSION INSTRUCTIONS

Invoices (payment/financing requests) shall be deposited electronically through the AHRQ Invoice Module (AIM) https://docs.ahrq.gov/aim

CAPSTONE Page 31 of 103

Directions/Guide to use the AIM Module can be found at:

https://www.ahrq.gov/funding/contracts/invoice-guide.html?tca=WK1a3t- LpRvys5y_5Vwz2QLqUlfsRvowGd8BVQJqcxM

The SF-1034 and all supporting…