BEST_SOW_Develop_New_Innovative_Adverse_Event_Reporting_Capabilities.pdf

The CBER Biologics Effectiveness and Safety (BEST) Initiative:

Develop New, Innovative Methods for Automation of Blood Product Adverse

Event Reporting

Statement of Work (SOW) July 21, 2017

Table of Contents

1. Introduction

1.1 Background

1.2 Biologics Effectiveness and Safety (BEST) Contract Objectives and Scope

2. Task Description(s)

2.1 Task 1: Program and task Management

2.1.1 Management Plan

2.1.2 Monthly Status Reports (MSRs)

2.1.3 Kickoff Meeting

2.1.4 Annual In-Person Meeting

2.1.5 Workgroup Meetings and Deliverables

2.1.6 Administrative Requirements

2.1.7 Risk Management

2.1.8 Facilities

2.1.9 Delivery Instructions for Deliverables

2.2. Task 2: Transition Services

2.2.1 Transition-out Plan

2.4 Task 4: Resource and Data Requirements: Development of new, innovative Methods for Blood product Adverse Event automated reporting

3. Contract Constraints

4. OTHER REQUIREMENTS

4.1 Key Personnel

4.2 Government Furnished Information (GFI) / Government Furnished Equipment (GFE)

4.3 Travel

4.4 Contract Administration

4.4.1 PLACE OF PERFORMANCE

4.4.2 GOVERNMENT POINTS OF CONTACT (POCS)

1. INTRODUCTION

1.1 BACKGROUND

The U.S. Food and Drug Administration (FDA) is responsible for protecting the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, tobacco products, our nation’s food supply, cosmetics and products that emit radiation. The FDA is also responsible for advancing the public health by helping to speed innovations that make medicines, devices, products, and foods more effective, safer, and more affordable; and helping the public get the accurate, science-based information they need to use medicines and foods to improve their health. As part of its mission, the FDA conducts monitoring of the safety of regulated products.

In response to the FDA Amendments Act of 2007 (FDAAA 2007) with the goal to build an active postmarket risk identification and analysis system, the FDA established the Sentinel system and built a national electronic data system for monitoring the safety of FDA-regulated medical products.

Sentinel is a distributed database system consisting of numerous data partners who retain operational control over their data within a secure environment; the Sentinel Operations Center manages, receives and processes FDA questions in the form of queries and returns summary results back to the FDA in a secure environment. Sentinel is currently run under a five-year contract awarded to Harvard Pilgrim Health Care Institute in 2014. The FDA Center for Biologics Evaluation and Research (CBER) established the Postlicensure Rapid Immunization Safety Monitoring (PRISM) system, within Sentinel, specifically to conduct active postmarket risk identification studies and analyses of vaccine safety. CBER also established the Blood Safety Continuously Active Network (BloodSCAN) as an active postmarket risk identification and analysis system within Sentinel for evaluating the safety of blood and blood products. CBER uses the Sentinel system to conduct some of its epidemiological studies as part of its regulatory responsibilities.

CBER has also conducted a number of epidemiological studies outside of the Sentinel system.

CBER’s Office of Biostatistics and Epidemiology (OBE) initiated postmarket safety studies working collaboratively with federal partners such as the Veterans Health Administration, Indian Health Services and Center for Medicare and Medicaid Services (CMS). Since 2005, FDA has conducted studies with CMS in the areas of vaccine and blood safety and in 2013 in vaccine effectiveness.

OBE has conducted numerous epidemiological studies with private sector partners, such as HealthCore. And these successful collaborations resulted in a number of completed safety studies, peer-reviewed publications and contributed to FDA regulatory decision-making. In 2016, CBER funded a study using artificial intelligence, natural language processing and advanced analytics to evaluate biologic product adverse event reports to more efficiently analyze and review reported adverse events and trends in reports to support and enhance our postmarket safety surveillance work.

CBER is initiating this action as part of its Biologics Effectiveness and Safety (BEST) initiative, in its continuing efforts to develop additional active postmarket risk identification and analysis systems to address surveillance and epidemiological study challenges unique to our regulated biologic products. CBER’s BEST initiative expands work beyond current partners (such as CMS and Sentinel) to provide CBER with new data sources; common data models (CDMs); tools and methods;

infrastructure, capabilities and capacity with the goal of providing deliverables in a more cost-effective, efficient, and timely manner. Overall, the CBER BEST system will be built for conducting surveillance, effectiveness and safety studies in three areas including: 1) Blood and blood product safety as part of FDA’s efforts to build a national biovigilance program; 2) New, innovative approaches to postmarket safety and effectiveness studies for biologic products to include approaches and applications such as query tools, robotics, machine learning, artificial intelligence (AI), natural language processing (NLP) and others to automate blood and blood product adverse event reporting; and 3) Biologics effectiveness and safety surveillance efforts that may include vaccines, blood and blood products, tissues, and other biologic products.

This SOW addresses the second area listed above to build capabilities and capacity for conducting:

2) New, innovative approaches to postmarket safety and effectiveness studies for biologic products to include approaches and applications such as query tools, robotics, machine learning, artificial intelligence (AI), natural language processing (NLP) and others to automate blood and blood product adverse event reporting. Specifically, the goal is to use new innovative methods to mine and identify transfusion-associated adverse event (AE) from electronic health record (EHR) data and similar data and automatically report the AE data to the FDAor FDA systems such as MedWatch.

FDA’s goal with BEST is to gain access to new sources of electronic health record (EHR) data or similar data sources providing more detailed clinical patient information than claims data for patients treated in inpatient and outpatient settings; as well as new common data models (CDM) that CBER’s Sentinel program is not currently using, including the Observational Medical Outcomes Partnership(OMOP) model that can accommodate EHR and richer patient clinical data than available in claims data; and new, innovative tools and methods such as robotics, machine learning, AI, NLP and others; and the infrastructure, capabilities and capacity to conduct this work.

BEST will be part of the growing and evolving network of CBER postmarket safety and effectiveness analysis systems in the Sentinel program and may not repeat work currently underway in Sentinel but is intended to expand CBER’s postmarket safety and effectiveness work, capabilities and capacity in the areas listed above. Access to large sources of patient data, especially inpatient data in Electronic Health Record (EHR) format or similar data sources providing more detailed clinical patient information than claims data (or with rapid access to medical chart data), and a new CDM that can accommodate EHR and richer patient clinical data is required. The focus of this contract is to develop and apply machine learning, AI, NLP and other approaches to automate adverse event reporting. The proposed FDA work may be conducted under a number of possible organizational structures, collaborations and arrangements. Data holder(s) or other organization(s) may have their own team of epidemiologists, subject matter experts, bioinformaticists, programmers and others who are experienced in conducting postmarket safety studies for blood and blood products.

Alternatively, the Contractor may partner with other organizations, such as contract research organizations, academic partners, data partners, or others who have their own teams or may assemble its own team of such experts through teaming or partnership arrangements, which might serve as a Coordinating Center to manage the overall program work.

1.2 BIOLOGICS EFFECTIVENESS AND SAFETY (BEST) CONTRACT OBJECTIVES AND SCOPE

The primary goal of this Contract is to develop and use new innovative methods to identify and mine transfusion-associated adverse event (AE) informationfrom electronic health record (EHR) data and similar data and automatically report the AE data to the FDA or its safety reporting systems.

To accomplish this goal the Contractor shall develop new, innovative tools and methods for mining the AE information; provide EHR data or similar data sources providing more detailed clinical patient information for patients treated in inpatient and outpatient settings; provide the necessary expertise in the proposed methods, as well as epidemiologists, subject matter experts, bioinformaticists, programmers and others who are experienced in conducting postmarket safety studies for blood and blood products.

Objectives

The objectives of this contract are:

1. For the Contractor to provide FDA with the expertise and support to establish infrastructure such as a Coordinating Center or similar organizational structure

2. To provide FDA with data sources including EHR data for millions of patients across a variety of healthcare settings (i.e., inpatient, outpatient, Emergency Department, etc.)

that are readily accessible for use in automated reporting.

3. Develop new capabilities for FDA to conduct datamining and automated reporting of adverse event data, mined from records such as EHRs using accepted case definitions, other text information and codes.

Scope

1. Provide expertise including but not limited to informatics, programming, epidemiological and medical but also experts in areas including but not limited to technologies such as NLP, robotics, machine learning, AI and others

2. Develop methods to identify and mine transfusion-associated adverse event or health outcome information relevant to transfusion of blood or blood products from EHR sources using case tools such as AI, NLP and others as well as definitions, codes and other text information.

3. Use a source of millions of patient EHRs (preferred) or similar detailed data from one or more different types of healthcare settings (e.g., inpatient, outpatient, etc.) and the ability to use records to identify and automatically report adverse events of interest to FDA for blood or blood products.

4. Populate fields in a data table after identifying a product exposure and potential adverse event association for further use in FDA analyses and transfer information and populate corresponding fields on an FDA Report form and automatically submit form to MedWatch;

5. Conduct quality control and validation to ensure valid, accurate identification of transfusion-associated adverse events from EHRs of >90%, and conduct analyses to ensure accurate, timely submission of automated reports to MedWatch;

6. Expand approach so that it may be applied using other EHR data sources or other data sources to build an automated nationwide, representative blood and blood product safety surveillance system.

2. TASK DESCRIPTION(S)

The following tasks constitute the performance of work expectations for BEST support.

2.1 TASK 1: PROGRAM AND TASK MANAGEMENT

The Contractor’s Program Manager and the Principal Investigator (PI) shall provide Program Management (PM) for these Biologics Effectiveness and Safety Contract requirements. These PM responsibilities shall include what is necessary to plan, structure, coordinate, schedule, manage, communicate, report, and steward all respective tasks and activities throughout the Contract’s period of performance. The Contactor shall work with the Government Contracting Officer (CO), Contracting Officer’s Representative (COR), and subject matter experts (SMEs) to ensure that tasks stay on track and important milestones are met.

The Contractor shall ensure smooth operations between its Coordinating Center, Data source(s) / partners, methods development and subject matter experts and work collaboratively with the FDA to ensure delivery of coordination activities such as kick-off meetings, bi-monthly telecons, monthly reports, and in-person meetings; and identify projects, develop protocols, reviewing deliverables such as protocols, interim analyses, interim reports and final reports.

The Contractor shall:

• Develop a plan on processes and procedures for collaborating with FDA in project design and execution. Details of the collaboration will by mutual decision of all collaborative parties. Collaboration includes sharing summary data findings, interpretation of findings, review of manuscripts, design of protocols, and co-authorship of publications.

The order of listing authors on each publication shall be determined on a case-by-case basis, by mutual decision of all collaborative parties and shall reflect appropriately each author's contribution to the project. Where there is no agreement on the above items, the FDA Contracting Officer, through the FDA COR, shall provide the final decision.

• Provide access to a data source(s) with millions of patient electronic health records (EHRs) or similar data sources providing more detailed clinical patient information than claims data; the infrastructure, tools and methods, staffing, expertise and ability to identify and characterize transfusion-associated adverse events for blood and blood products; and the ability to conduct medical chart review to confirm findings.

• Generate a document providing information as to which key personnel w i l l be used on each project and the percentage of time and staff member effort and contribution to the program on a per-task basis.

Additionally, the Contractor shall provide Program Management services to establish control, management, monitoring, and notification mechanisms for Biologics Effectiveness and Safety System Contract requirements. Moreover, the Contactor shall work with the Government Contracting Officer (CO), Contracting Officer’s Representative (COR), and subject matter experts (SMEs) to ensure that tasks stay on track and important milestones are met.

Representative activities include, but are not limited to:

• Administering the program;

• Throughout the period of performance, developing, maintaining, modifying, and updating plans to include: risk management plans, staffing plan, Standard Operating Procedures (SOP), Protocols, and Others;

• Preparing a Management Plan (MP) describing the technical approach, organizational resources and management controls throughout execution.

• Preparing and maintaining the Program Management Plan;

• Preparing monthly progress and financial reports;

• Establishing notification mechanisms for acceptance by the FDA;

• Complying with reporting requirements;

• Following escalation procedures for problems, issues, and recommendations, as encountered;

• Safeguarding information;

• Managing Contractor staff and provide sub-contractor management;

• Coordinating tasks and working collaboratively with FDA CBER staff to carry out the work of the contract;

• Attending an annual face-to-face meeting with FDA CBER staff at the FDA White Oak Campus;

• Communicating, presenting and publishing of any BEST project findings shall be done in coordination and with the concurrence of FDA CBER and the FDA COR;

• Ensuring proper exit criteria are followed when Contractor employees leave projects;

• Providing risk mitigation;

• Conducting and participating in progress meetings, facility meetings, training meetings, performance meetings, program/project reviews, management briefings, stakeholder presentations, demonstration meetings, transition meetings, and COR site visits, and post requirements meetings;

• Responding to ad hoc data calls to refine or confirm project findings or address FDA questions or concerns concerning project findings; and

• Provide training to FDA staff on data sources, study approaches or methods and tools employed and developed as part of the specific tasks. The specific type of training shall be determined based on the tools and methods developed and may include development of training manuals, conduct training sessions, provide presentations or provide other similar activities.

2.1.1 MANAGEMENT PLAN

The Contractor shall deliver a Management Plan. The Management Plan (MP) shall include a Work Breakdown Structure (WBS), which shall contain all of the work elements described. The WBS elements shall also illustrate the sequence and resources required for each work package.

Additionally, the MP shall include a schedule that details activities, dependencies and timeframes for meeting FDA objectives and deliverables; an organizational chart identifying relationships, authorities, and responsibilities. The MP shall include a description of the management controls that the Contractor shall employ to meet the performance, cost, and schedule requirements; and other areas the Contractor deems relevant and important to the management of the Contract.

The MP shall address all the action items from any Kickoff Meetings and other follow-on meetings. The Contactor shall update the MP when re-planning any activities. Changes to the MP are subject to approval by the COR.

2.1.2 MONTHLY STATUS REPORTS (MSRS)

The Contractor shall provide a written Monthly Report (MSR) to the FDA COR, incorporating all tasks at a high level and their progress. Within the first 30 calendar days after award, the Government and the Contractor shall design and agree upon a standard reporting format and timeframe. At a minimum, unless otherwise specified, each MSR shall contain the following information:

• The previous month’s progress

• Deviance from previous month’s planned progress (either positive or negative)

• The current month’s planned progress

• Issues and recommendations

• Performance and Cost Report (P&CR): the P&CR shall include all recurring and non-recurring costs, cumulative cost, total labor hours, and cumulative labor hours for each task area for the previous month

• Program risks, risk mitigation plans, risk mitigation actions taken and resolution of risks.

• Status of deliverables

Lastly, the Contractor shall ensure that all deliverables are sent in electronic formats (i.e. Microsoft Office or Adobe PDF) for the COR’s inspection/acceptance.

2.1.3 KICKOFF MEETING

Following the contract award, the Contractor shall conduct a kickoff meeting at the FDA for the BEST Blood and Blood Safety Surveillance Contract. The kickoff meeting shall be conducted to introduce the Contractor team, discuss roles, responsibilities, other relevant issues and the contractor’s approach to meeting the requirements of the SOW.

2.1.4 ANNUAL IN-PERSON MEETING

The Contractor shall schedule and conduct an annual in-person meeting with FDA personnel a t t he F D A W h it e O ak Cam pu s i n S i l v e r S p r i n g , MD to discuss Contractor’s performance under the contract; and provide a presentation of recently completed or in-progress individual collaborative study or studies .

2.1.5 WORKGROUP MEETINGS AND DELIVERABLES

The Contractor shall work collaboratively with FDA Staff and the COR and shall establish project workgroups and participate in bi-monthly telecons. The Contractor shall participate in the processes and meetings that support the overall efforts of the FDA BEST System program, including general management-related meetings, and meetings related to specific tasks. Generally, there are meetings of the Contractor and FDA Staff working group meetings on an approximately bi-monthly basis. The Contractor shall participate in different capacities, such as leading discussions; contributing questions and answers; raising issues, risks and concerns; and developing and giving presentations.

The contractor shall schedule and provide administrative support for task workgroup meetings. In performing this task, the Contractor shall:

a. Prepare and distribute agendas to include conference call-in information.

b. Prepare and distribute review materials.

c. Take and distribute minutes and action items to meeting participants. Minutes shall be of sufficient detail to accurately document meeting date and location, meeting purpose, items discussed, decisions made, attendees, and action items, and shall be delivered within FDA document repositories.

d. Track and maintain a central repository of minutes and action items and associated materials relating to all meetings/calls

e. Prepare a Draft Contract Annual Technical Report for FDA review and comment and a Final Annual Technical Report consisting of a description of the project status, work performed and results obtained for each annual period of performance, lessons learned in sufficient detail to describe comprehensively the results achieved. The Final Annual Technical Report shall incorporate FDA comments and input.

f. Prepare abstracts, presentations, reports, and manuscripts for publication derived from analyses under the contract.

2.1.6 ADMINISTRATIVE REQUIREMENTS

The Contractor shall ensure smooth operations between its Coordinating Center, Data holders / partners, and work collaboratively with the FDA to ensure delivery of activities such as kick-off meetings, bi-monthly telecons for project meetings, monthly reports, and in-person meetings; and developing query forms, executing and completing queries, generating query reports, developing protocols, generating protocol-based study reports, reviewing deliverables such as protocols, interim analyses, interim reports and final reports.

The Contractor shall:

• Generate a Key Personnel Document providing information as to which key personnel w i l l be used on each project and identifying the percentage of time and staff member effort and contribution to the program on a per-project basis.

Table 1: Task 1 Meetings

Table 2: Task 1 Deliverables/ Work Products

2.1.7 RISK MANAGEMENT

The Contractor shall identify and report for resolution to the Government any issues that prevent the Contractor from performing the work in any task that the Contractor is unable to resolve independently.

For each Task, the Contractor shall document and report the risks and opportunities within a risk registry. The Contractor shall maintain the risk registry and provide it to the COR on request.

Task Meeting / Activity Title Location(s) Estimated Occurrences

Receiving Organization

Task 1 1. Kick-Off Meeting FDA White Oak Campus, Bldg 71 or teleconference

Once: Within 10-calendar days of award

COR

Task 1 2. Annual In-person Meeting FDA White Oak Campus, Bldg 71 One per year COR

Task 1 3. Workgroup Meetings Teleconference Bi-Monthly COR, FDA Project Staff

Task Deliverable/Work Product Title Due Date Quantity / Frequency Delivered To

Task 1 1. Management Plan Initially reviewed at Kick- Off Meeting Ongoing COR

Task 1 2. Monthly Status Reports 5th calendar day of each month Monthly COR

Task 1

3. Workgroup Conference Calls, Meeting Minutes By 10th calendar day following the meeting Monthly COR, FDA project

Staff

Task 1 4. Workgroup Agendas 5 calendar days prior to the meeting Monthly COR, FDA project

Staff

Task 1

5. Central Electronic Repository of Meeting Minutes/ Calls/Action Items

Last business day of the month Ongoing COR, FDA project

Staff

Task 1 6. Draft Annual Technical Report

Draft due 30 calendar days before the last day of the period of annual performance

One per year COR

Task 1 7. Final Annual Technical Report Final due at the end of period of annual performance

One per year COR

Task 1 8. Abstracts and Presentations 5 business days prior to submission or presentation

Anticipate one per year COR

Task 1 9. Reports and Manuscripts 30 calendar days prior to the agreed date

Anticipate one per year COR

Task 1 10. Key Personnel Document 1 month after award date Once COR

The risk registry shall contain, at a minimum, unless otherwise specified in the tasks:

• Issue Number: a sequential numbering of all issues;

• Severity: assessment of issue severity i.e., likelihood and consequences (High/Medium/Low)

• Date Found: – date that issue was identified by the Contractor;

• Duration: number of days that issue has been open;

• Status: representing issue status as open or closed. The Contractor shall maintain a complete list of all issues – no issues typically should be deleted from the Contractor’s delivered list;

• Title: short description of issue;

• Updated: date for updated status;

• Details: narrative description of activities that have been completed to resolve issue;

• Assigned To: identify Government POC or Contractor that is assigned to resolve the issue;

• Solution or Workaround: description of the actions and activities taken to resolve or mitigate issue the issue.

2.1.8 FACILITIES

The Contractor shall provide all facilities and equipment necessary to accomplish the Contract requirements as specified herein. Additionally the Contractor shall ensure that electronic data and medical record data is securely stored, transferred and handled, and retained and protected for future study purposes. The Contractor shall obtain and meet all required review, approval and standards processes that comply with all relevant institutional, federal, state, and local laws and regulations to conduct the work such as data and information technology security, such as Federal Information Security management Act (FISMA), Health Insurance Portability and Accountability Act (HIPAA), Institutional Review Boards (IRB) and any others identified in the contract.

2.1.9 DELIVERY INSTRUCTIONS FOR DELIVERABLES

Unless otherwise specified, all deliverables, reports or copies of reports shall be delivered to the COR electronically via email. Requirements for inspection and acceptance are as follows:

• The COR or the POC designated for the contract, will inspect and accept the services provided to ensure they meet the requirements detailed in the Contract.

• The Government will accept products and services only if they conform to all terms and conditions of the Contract.

• The Government will provide written notification of acceptance or rejection within ten (10) calendar days of receiving the service.

• The Government will reject non-conforming products and services. The Contractor shall correct any deficiencies within thirty (30) calendar days of when the Government issues the rejection notice. If the Contractor cannot correct the deficiencies within this time frame, the Contractor shall immediately notify the COR of the reason for the delay and provide a proposed corrective action plan within ten (10) calendar days.

In addition, the Contractor shall be proactive in informing the designated CBER COR of any issues, problems and recommendations that shall be addressed in order for the overall effective accomplishment of contract goals and deliverables. Recommendations for actions that need to be taken by FDA staff, or other Contractors shall be clearly defined and communicated (via email) to the responsible party and the FDA, and have identified dates for completion.

2.2. TASK 2: TRANSITION SERVICES

Transition activities take place to efficiently transition all activities from an incumbent Contractor to the new Contractor. Currently, there is no incumbent Contractor since this is a new contract. The Contractor may be required to develop transition-out plans and conduct transition activities against this Contract vehicle.

2.2.1 TRANSITION-OUT PLAN

The Contractor shall provide support for transition-out activities of contracted activities and services to the Federal Government or to a follow-on Contractor by the end of the contract period of performance. During this period, the Contractor shall ensure no degradation in support provided under the contract. Between the turnover date designated by the COR and the end of the contract period of performance, the outgoing Contractor shall ensure all contract activities are closed out and provide sustainment support to complete the transition.

Representative deliverables include, but are not limited to:

• Providing the FDA with current versions of all documentation developed (i.e. Protocols , Manual of Procedures, etc.)

• Providing the FDA with a current inventory of all Government-Furnished Equipment (GFE) and Government Furnished Information (GFI) utilized by the Contractor along with full support in the reconciliation of this inventory

• Providing the follow-on contractor with the ability to “shadow” and participate in technical exchange meetings and opportunities to facilitate the transfer of information, processes, and data needed to continue the services being performed by the Contractor

• The Contractor shall develop and submit to the COR a Transition-Out Plan that shall include but not be limited to the following elements:

• Roles and responsibilities for contractor’s staff during the transition, including the identification of a transition lead;

• Transition-out schedule and milestones for completing transition-out activities

• Define schedule for transition-out progress reviews, and define processes for identifying problems, and for ensuring prompt problem resolution;

• Identify methodology used for conducting and ensuring a seamless transition

• Identify activities to acclimate the successor contractor

The Contractor shall conduct and provide and inventory of government-funded equipment (GFE) and government-furnished information (GFI).

Table 3: Task 2 Deliverables/ Work Products

Task Deliverable/Work Product Title Due Date Quantity /

Frequency Delivered To

Task 2 1. Transition-Out Plan 120 calendar days prior to period of performance end date

One COR

Task 2 2. GFE and GFI Inventory 120 calendar days prior to period of performance end date

One COR

2.1 Task 3: Establish Organization and Infrastructure

The primary goal of this Contract is to gain access to methods, tools, expertise and data resources, such as EHRs representing millions of patients, to develop approaches using advance methods such as NLP, Machine learning, AI and others, to advance automated reporting of adverse event data, mined from EHRs using accepted case definitions, other text information and codes. In turn, this information would be used to populate data fields in a data table and/or on an FDA adverse event reporting form, which can then be automatically submitted to the FDA MedWatch system and the FDA Adverse Event Reporting System (FAERS) shall be done in collaboration between the Contractor and CBER staff.

During the course of the contract period of performance, the Contractor shall establish the organizational structure, such as a coordinating center, and infrastructure to provide a work plan(s) to conduct each phase of the blood and blood product safety automation. The Contractor shall not proceed with the work plan until written approval is given by both the Contracting Officer and COR. The following describe in greater detail Contractor activities for each of the objectives of the Contract.

General Requirements: Please note that the requirements below refer largely to blood and blood products.

Organization and Resource Requirements. The Contractor shall:

• Establish a Coordination Center (CC) or similar organizational structure consisting of a team of experts such as epidemiologists, clinical and subject matter experts in NLP, AI and related areas, bioinformaticists, programmers, project managers and others.

• Be capable of executing the work of the tasks including but not limited to identifying the data necessary to conduct the work; developing work plans, protocols and programming and conducting the required development of methods or tools and/or postmarket analyses and safety studies for blood and blood products.

• Provide an EHR data resource/resources or similar sources providing more detailed clinical patient information than claims data for patients treated in inpatient and outpatient settings that can best support population-based studies of blood or blood products and is available for linking each patient to all relevant medical care data including enrollment status, biologic product exposure data, coded medical outcomes, and vital records (desired, but not required). For biologics licensed for distribution in the US see http://www.fda.gov/CBER/products.htm and for vaccines see http://www.fda.gov/CBER/vaccine/licvacc.htm.

• The Contractor shall provide documentation that the Contractor either owns the data or has legal rights to access and disclose analyses of the data to third parties such as the FDA.

• Provide a new common data model that CBER’s Sentinel program is not currently using, including the Observational Medical Outcomes Partnership (OMOP) model that can accommodate EHR and richer patient clinical data than available in claims data alone.

• Provide a document demonstrating full disclosure of HIPAA-compliant data collection, coding, secure handling and configuration methods to allow FDA to appropriately interpret findings.

• Summarize the results of analyses at various phases of the project for FDA use. FDA will require no patient-, provider-, or health plan- specific identifiers. The de-identification of the data shall be solely the responsibility of the Contractor.

http://www.fda.gov/CBER/vaccine/licvacc.htm

• Establish Contract Workgroups consisting of members from the Coordination Center (CC) or similar organizational structures described in the SOW and FDA Staff. In addition the CC team shall consist of at least one member with clinical or specific expertise in the blood transfusion area such as hematology, transfusion medicine, etc. FDA staff will provide questions to be addressed and its own expertise; Workgroup members shall work collaboratively to design and complete the project.

• Develop and provide detailed work plan describing the overall approach and steps to be used to conduct the proposed blood product safety automation work, along with overall management of the workgroup, Contractor staffing and resources to work collaboratively with FDA staff to complete the proposed tasks.

Table 4: Task 3 Deliverables/ Work Products

2.4 TASK 4: RESOURCE AND DATA REQUIREMENTS: DEVELOPMENT OF NEW, INNOVATIVE

METHODS FOR BLOOD PRODUCT ADVERSE EVENT AUTOMATED REPORTING

The Contractor shall provide the necessary data resources including EHR data for millions of patients to develop new, innovative methods to mine adverse event information associated with blood or blood product transfusions. Contractor shall have access to large patient EHR dataset(s) to execute this work.

Contractor shall deliver the new tools and capabilities developed for FDA that will automate reporting of adverse event data, mined from records such as electronic health records using accepted case definitions, other text information, and codes. The tools will in turn be used to populate data tables or fields on FDA adverse event reporting forms, and automatically submit them to FDA or its MedWatch system. Contractor shall determine the accuracy of mining AEs which should be >90% for most

Task Deliverable/Work Product Title Due Date Quantity /

Frequency Delivered To

Task 3

1. Establish Coordinating Center 1 month after award date Once COR

Task 3

2. Provide document that the Contractor either owns the data or has legal rights to access and disclose analyses

1 month after award date Once COR

Task 3

3.Provide a document demonstrating full disclosure of HIPAA-compliant data collection, coding, secure handling and configuration methods

1 month after award date Once COR

Task 3

4. Establish Project Workgroup 1 month after award date Once COR

Task 3

5. Detailed work plan for phases of blood product AE reporting automation

1 month after award date Ongoing, One per project

COR

transfusion-associated AEs, and validate automation process by ascertaining delivery to the MedWatch system and presence of automated reports in the FDA Adverse Event Reporting System (FAERS).

The Contractor shall assemble required documents, including documentation of compliance with HIPAA, FISMA, IRB and other requirements, as well as protocols, workplans and all BEST related documents into the BEST manual of operating procedures (MOP), and update as needed to accurately reflect current and ongoing contract activities. FDA staff will review the draft MOP and protocol documents provide input for incorporation into the final document. The Contractor shall obtain and meet all required review, approval and standards processes that comply with all relevant institutional, federal, state, and local laws and regulations to conduct the work such as data and information technology security, such as Federal Information Security management Act (FISMA), Health Insurance Portability and Accountability Act (HIPAA), Institutional Review Boards (IRB) and any others identified in the contract.

These procedures (i.e. the MOP) shall address at a minimum the following: secure data collection, handling processes, sharing, ownership; management and tracking, consensus definitions, secure data transfer procedures, and sharing considerations for data summaries.

RESOURCE Requirements

The proposed FDA work may be conducted under a number of arrangements. For example, organizations, such as a contract research organization or academic partner, clinical center, etc. which have their own team(s) or may assemble its own team of such experts, which might serve as a Coordinating Center or Operations Center. Contractor shall have capabilities to develop appropriate algorithms and programming for applying methods such as machine learning, artificial intelligence, natural language processing and others to automate mining of transfusion-associated adverse event data from EHRs (preferred). The contractor should also be able to engage EHR vendors to assist with collection and formatting of transfusion data from transfusion centers so that it may be accessible and mined by the automated system.

The Contractor shall have staff with the necessary bioinformatics, programming and advanced computational methods (e.g., NLP, AI or machine learning) expertise to use a combination of case definitions, text information and medical codes to derive the transfusion-associated AE information with a high level of accuracy as determined by confidence levels reflecting the level of alignment of EHR information to the AE case definition.

The Contractor shall identify developed case definitions for transfusion-associated adverse events of interest such as anaphylaxis, thromboembolism, TRALI, TACO, and others from sources such as the International Society of Blood Transfusion (ISBT), Brighton Collaboration, published peer-reviewed literature and other reliable sources for use in identifying key elements for datamining and matching of key case definition terms to the EHR records including non-structured text portions of the record.

The contractor shall transfer the mined information to the appropriate, corresponding fields of an FDA Medwatch report form with a high level of accuracy and populate a data table that can be analyzed using epidemiological tools.

The Contractor shall develop a system that automatically submits reports to MedWatch and shall develop appropriate methods to accurately identify blood and blood product adverse events; populate fields on a report; and verify the submission of automated reports and validate the accuracy of transfusion-associated AEs submitted.

DATA Requirements

The Contractor shall:

• Provide access to large dataset(s) with millions of patient EHRs or similar data sources providing more detailed clinical patient information than claims data and deliver the capability to analyze and assess the incidence, mortality and risk factors for occurrence of severe transfusion-related and plasma derivative-related adverse events in the United States.

• Provide a data resource that is automated with capabilities to link each patient to all relevant EHR data including enrollment status, transfusion exposure, coded medical outcomes, vital records, etc. The Co ntracto r data, tools methods and resources shall provide (but not be limited to) the following:

• Data Sources that demonstrate completeness, timeliness, and accessibility, including estimated times from service delivery;

• Have the ability to identify products by type, lot identification number, or other applicable descriptors.

• Provide level of detail to examine temporal relationships between product administration/and associated adverse events/outcomes.

• Tools shall provide for automated/assisted selection of product, diagnostic and procedure codes that map across different coding systems such as ICD-9, ICD-10, Medra, ISBT, Codabar and others.

• As part of a continuous improvement process, develop, and implement methods for refreshes of data on a periodic basis such as monthly or quarterly for include description of process in the MOP and improve capabilities of system by reducing the lag period in which new data can be evaluated.

• Ability to confirm the accuracy of automated data retrieval through chart review and or other reliable methods.

Table 5: Task 4 Deliverables/ Work Products

Task Deliverable/Work Product Title Due Date Quantity /

Frequency Delivered To

Task 4 1. Manual of Procedures (MOP)

3 months after task order award date

Once, and update as needed

COR

2.5 TASK 5: Build Capabilities to Develop and Implement new, innovative Methods for Automated Reporting of Adverse Events

The Contractor shall develop new, innovative methods to conduct automated reporting of adverse event information associated with blood or blood product transfusions. Contractor shall deliver the new tools, data and capabilities developed for FDA that will fully automate reporting of adverse event data, mined from records such as electronic health records using accepted case definitions, other text information, and codes. The tools will in turn be used to populate data tables or fields on FDA adverse event reporting forms, and automatically submit them to the FDA MedWatch system. Contractor shall determine accuracy of mining AEs which should be >90% for most transfusion-associated AEs, and validate automation process by ascertaining delivery to the MedWatch system and presence of automated reports in the FDA Adverse Event Reporting System (FAERS).

The Contractor shall:

a. Identify and build capabilities and capacities to conduct automated adverse event reporting for blood and blood products using EHR data

b. Work collaboratively with FDA to develop new, innovative methods to address automated reporting of adverse event information for Blood and Blood Products after the contract is awarded.

c. Use the data, methods and expertise to build new capabilities for FDA to conduct automated reporting of adverse event data, mined from records applying machine learning, AI, NLP and other approaches to electronic health records using accepted case definitions, other text information and codes.

d. Identify generally accepted case definitions for transfusion-associated adverse events of interest from sources such as the International Society of Blood Transfusion (ISBT), Brighton Collaboration, published peer-reviewed literature and other reliable sources for use in identifying key elements for datamining and matching of key case definition terms to the EHR records including non-structured text portions of the record.

e. Use mined data to populate fields in a data table used in further studies, and to populate the fields of an FDA adverse event reporting form, and automatically submit to the FDA MedWatch system.

f. Validate the ability of developed methods to accurately identify and mine >90% of the specific transfusion associated adverse events of interest.

f. Validate delivery of AE reports generated from patient data to MedWatch system and retrieval of reports from FAERS.

g. Develop a written project plan detailing staffing of project; data or data subset to be used, such as common data model, etc.; proposed methods development and approaches to be used; test and validation methods for developed algorithms and tools, accuracy of identification and mining of data for reports;

and validation of delivery and retrieval of reports from FDA systems. The plan shall describe the overall approach and steps involved from mining of relevant transfusion-associated AEs to automated reporting to FDA and validation of the accuracy of the tool. The plan shall indicate tools and approaches to be used;

data to be used, including years, estimated numbers of patients/patient records, cohorts to be studied including control and treatment populations;

methods used and rationale; summary of findings; full technical descriptions of project tools and approaches used including strengths, limitations, or challenges in executing the algorithms and automation, and provide results of testing or evaluation demonstrating validation and accuracy of the developed tools in achieving AE reporting automation.

h. Work collaboratively with FDA staff to develop and finalize the project plans, protocols and update as needed to reflect the task activities. These procedures shall address at a minimum the following: data collection, ownership, and security; management and tracking, consensus definitions, transfer, and sharing considerations for developed methods, tools, and data summaries.

Components of the project shall address the following:

1. This project shall focus on developing approaches to automatically report several defined adverse reactions that may be associated with transfusions, these include but are not limited to:

• Transfusion-associated circulatory overload (TACO)

• Transfusion-related acute lung injury (TRALI)

• Transfusion-associated dyspnea (TAD)

• Severe allergic reaction

• Hypotensive transfusion reaction

• Febrile non-hemolytic transfusion reaction (FNHTR)

• Acute hemolytic transfusion reaction (AHTR)

• Delayed hemolytic transfusion reaction (DHTR)

• Delayed serologic transfusion reaction (DSTR)

• Transfusion-associated graft vs. host disease (TAGVHD)

• Post-transfusion purpura (PTP)

• Transfusion-transmitted infection (TTI)

2. Identify, develop and execute study design and methods to achieve project objectives. This will also include:

a. Assessment of the validity of the coded exposure and outcome data using medical record review.

3. Evaluation of blood and blood product safety and special informational considerations for product safety, including the data source's ability to:

• Link product administration/treatment records with inpatient and outpatient data including ER.

o Identify the blood or plasma-derived product information, product name, route and frequency of administration, dose number, and lot information.

o Identify the time interval in hours, when necessary, between product administration and onset of adverse events.

• Accurate identification of blood product type(s) and brand name(s), if possible, transfusion time or treatment time or duration (start and finish) and adverse event onset time and end time.

• Identify prior treatment and medical history.

Describe potential limitations of data and methods, approaches used for mining and automated reporting of transfusion-associated adverse events from EHRs (preferred) or possibly data with rapid access to linked medical records.

The Contractor shall identify medical records with transfusion of blood or blood products recorded in various settings (e.g., inpatient, outpatient, etc.) using ICD-9-CM and/or ICD-10-CM procedure codes, revenue codes, CPTs, HCPCs, and/or quantity used. Contractor shall develop a text mining methodology for record retrieval or conduct a study and verify that ICD and other codes may adequately support this task.

The Contractor shall complete Project Draft Annual Technical Reports and Project Final Annual Technical Reports according to the timelines stated in Task 1.

Table 6: Task 5 Deliverables/ Work Products

3. CONTRACT CONSTRAINTS

The following constraints are applicable to this contract:

1. The Contractor shall ensure that all appropriate administrative and regulatory approvals are obtained and activities are compliant with applicable requirements for security, data security, handling, etc. These include but are not limited to those that address human subject considerations; and clearances including applicable Institutional Review Boards (IRBs), FISMA, HIPAA, and Office of Management and Budget (OMB)).

2. Interim data used in all technical progress reports and other reports developed for the purpose of study monitoring and shall not be released publicly without FDA clearance and approval.

Premature release of such data could result in interpretations that prove to be unreliable or

Task Deliverable/Work Product Title Due Date Quantity /

Frequency Delivered To

Task 5 1. Project Plan, Protocol or Data Analytic Plan

2 months after task order award date

Ongoing, updated as needed

COR

Task 5 2. IRB Documents and Approval

1 month after Protocol or Analytic Plan completion

Once COR invalid once the study is completed and the full context for the data is known and lead the public and medical practitioners to pursue inappropriate measures.

3. The Contractor shall be responsible for knowledge of and compliance with all applicable federal information technology and information management laws, regulations, policies and standards at the government-wide, HHS and FDA levels.

i. The computer programs, data management systems, databases, and other resources used or developed through this contract shall comply with data standards and procedures established by DHHS and FDA (per http://www.fda.gov/ForIndustry/DataStandards/default.htm).

4. The Contractor shall not use confidential or otherwise privileged information shared by FDA with contractors to solicit or in any way encourage industry funding/financial sponsorship of research projects to be performed by the contractor.