Attachment II - CLAT TO SOW AMENDED II.pdf

SOLICITATION NO.: FDA-20-SOL-1223486

Attachment II: Task Order SOW

ATTACHMENT II:

TASK ORDER I STATEMENT OF WORK

Computerized Labeling Assessment Tool

(CLAT) Services

I. BACKGROUND

The mission of the U.S. Food and Drug Administration (FDA) is to protect the public health (www.fda.gov). As part of this mission, FDA is responsible for ensuring the safe use of drug products by minimizing medication errors related to the product’s name, labeling, packaging and design.

The Division of Medication Error Prevention and Analysis (DMEPA) within the FDA, Center for Drug Evaluation and Research (CDER), Office of Surveillance and Epidemiology (OSE) is the scientific lead for medication error prevention and analysis for CDER-regulated prescription and nonprescription drug products. DMEPA’s responsibilities include the premarket (preapproval) and post-market review of drug product labeling (e.g., container labels, carton labeling, prescribing information, Drug Facts, Instructions for Use) to minimize the risk of medication errors. An overview of DMEPA’s mission and responsibilities can be found on the FDA website: https://www.fda.gov/drugs/drugsafety/medicationerrors/.

Biopharmaceutical companies electronically submit drug product labeling to FDA using various file formats (e.g., Word, PDF, XML, JPEG). DMEPA and other offices within FDA review the labeling to ensure the drug product conforms with applicable statues, regulations, standards, FDA guidance for industry, best practices for patient safety, and experience gained through FDA’s post-marketing surveillance program. If the FDAs review of the submitted labeling identifies potential deficiencies or inconsistences, FDA requests the company revise the labeling and resubmit it for additional review.

See Appendix A for a use case that provides an overview of the drug product labeling submission and review process.

Much of the FDA review of drug product labeling is manual, and thus labor- and time-intensive, and may be subjective based on individual reviewer perspective or prone to possible discrepancies related to human factors. FDA reviewers locate and download the files containing the drug product labeling;

manually review the submitted labeling against an internally-generated list of statutes, regulations, standards, FDA guidance for industry, and best practices for medication error prevention; create a written document that includes the review findings and recommendations; and then repeats this process if the company submits revised labeling.

The FDA offices who may participate in this contract include:

CDER, Office of Surveillance and Epidemiology CDER, Office of New Drugs CDER, Office of Generic Drugs CDER, Office of Pharmaceutical Quality CDER, Office of Compliance

FDA has many offices with different responsibilities for reviewing drug product labeling. To this goal, automating all or portions of manual review activities will increase efficiencies, and help standardize http://www.fda.gov/ https://www.fda.gov/drugs/drugsafety/medicationerrors/ reviews to ensure consistency across different products.

As part of FDA’s Technology Modernization Action Plan, FDA intends to apply novel and rapidly evolving technologies at an increasing pace to advance FDA’s public health mission (https://www.fda.gov/about-fda/reports/fdas-technology-modernization-action-plan). As FDA transitions to these technologies, it is expected that tasks under this contract will be impacted. These impacts will be assessed as they occur and modifications to task orders under this contract may need to be considered.

II. SCOPE

The contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, supervision, and other items and non-personal services necessary to perform work as defined in this Performance Work Statement except for those items specified as government furnished property and services. The contractor shall perform to the standards in this contract.

The scope of this task order will address the development and delivery of a prototype for the Computerized Labeling Assessment Tool (CLAT).

III. OBJECTIVES

The objective of this task order is to obtain services to develop and deliver a prototype for a Computerized Labeling Assessment Tool (CLAT) that can be used to perform automated reviews of drug product labeling (e.g., container labels, carton labeling, prescribing information) submitted to the FDA.

This shall also include project management, organization of files and folders, needs assessment of proposed labeling activities for automation, reporting, and determining metrics and measures.

IV. REQUIRED TASKS

A. Task 1: Prototype for a Computerized Labeling Assessment Tool (CLAT)

The below sections list all the activities to be performed under Task 1: Prototype for a Computerized labeling Assessment Tool (CLAT). The Contractor shall validate the requirements with the FDA user under each of the activities. During requirements validation the Contractor shall elaborate each requirement into user story, document and prioritize for the prototype.

i. Develop and Containerize CLAT Prototype

For this task order the contractor shall develop the application offsite on at their facility and containerize the prototype developed. Subsequent task orders shall implement the program in the FDA production environment.

The Contractor shall perform and adhere to the following:

https://www.fda.gov/about-fda/reports/fdas-technology-modernization-action-plan https://www.fda.gov/about-fda/reports/fdas-technology-modernization-action-plan

• After meeting with the FDA project management staff an appropriate Enterprise Performance

Life Cycle (EPLC) Framework with specific stagegates appropriate for the specific project shall be assigned.

• Projects shall not be allowed to move to higher stagegates unless they have satisfied FDA specialists with appropriate document artifacts specific to the stagegate.

• During the Requirements and Analysis Phase the contractor shall validate the technical information that they were given by the FDA and decide on the appropriate technology solutions.

• In the Development Phase the contractor shall create and document the application using publicly available open source tools and/or other technologies.

• User Acceptance Testing (UAT) shall occur iteratively throughout the Development Phase as groups of related requirements are coded and then tested in an agile based methodology.

• The FDA expects that many of the requirements shall require advanced programming, open source software, optical character recognition, artificial intelligence including machine learning, natural language processing and advanced pattern matching techniques in order to perform labeling assessments to meet FDA regulatory standards.

• Multiple UAT throughout the Development phase shall validate the user interface with FDA testers.

• The application shall include a system which shall allow the system to be constantly updated with the latest available drug product labels on at least a monthly basis.

• The application shall provide a robust security enabled environment to protect proprietary and confidential information/508 compliance.

• The system shall be built in a flexible way where new types of labels or new label formats can be configured in the system by appropriate administrators of the system. This should not require changes to the code by programming staff.

• The system shall have Application Programmer Interface capability so that it has the ability to be embedded in other larger system.

• Copies of generated reports shall be retained by the system for audit control.

• Outdated labels shall not be deleted but retained in an archival system for possible later review by users.

• UAT shall be performed with the completed application testing it in an end-to-end fashion.

Appropriate user training documentation shall be provided.

• Measurements and Analysis shall be done on the increasing number of labels that shall accumulate in the system over time and the impact this shall have on performance.

• Scalability UAT shall be done with 50-100 concurrent users to validate system response time.

Total report generation of all requirements shall not exceed 20 minutes after a user has initiated review of candidate labeling information with CLAT.

• For this task order, the Implementation Phase shall consist of planning to implement this software in a production FDA environment.

• Shall initiate all possible documentation and process steps leading up to an FDA authority to operate (ATO) in anticipation of a subsequent task order to implement the software.

• Shall provide artificial intelligence image processing capabilities as needed to remove noise, segmented to produce close object boundaries, analyze to extract a feature vector, and compared to ideal object feature vectors by a classifier to determine the nearest object classification and its associated confidence level.

• Provide a suite of flexible tools within CLAT that can be integrated with FDA’s current and future architecture.

• The contractor shall utilize best practices for user experience and user interface designs.

• Provide Single Sign On (SSO), account administration, including user account privileges, reports on dates when individual users last logged in. User accounts shall be disabled automatically as specified in FDA security policies.

ii. Access and Organization of Files and Folders

The Contractor shall perform the tasks listed below to provide access and organize files and folders for labeling review activities with the Computerized Labeling Assessment Tool (CLAT)). Unless otherwise specified, “labeling” refers to all labels and other written, printed, or graphic matters (1) upon any article or any of its containers or wrappers, or (2) accompanying such an article, including the following types of labeling: container label, carton labeling, prescribing information, Instructions for Use, and Medication Guide.

The system shall function as follows:

• Maintain a repository of the labeling files submitted in pdf and other file formats for review by

CLAT.

• Determine and tag submitted labeling files as to the type of labeling.

• Determine and tag labeling files that are resubmitted with labeling revisions for additional review by CLAT.

• Allow the user to upload labeling submitted in various file formats, including Word, Excel, CSV, PDF, XML, and JPEG.

• Automatically ingest data and populate and organize folder hierarchies.

• For users to create or modify network, local computer, or application folders.

• For default naming convention for application folders and allow user-defined naming conventions for subfolders.

• For users to search CLAT by product application number to retrieve labeling files for their assigned labeling review activities.

• For users to search CLAT by product name to retrieve labeling files for their assigned labeling review activities

• Maintain a repository of labeling that can be used to compare look-alike similarities for submitted labeling with labeling for other products marketed in the United States.

• Maintain a repository of labeling for U.S. marketed products that can be used to compare submitted labeling with previously submitted labeling for the same product.

• For check-in and check-out features of user review during the review process.

• Retain all versions of the labeling files for audit purpose.

• for multiple users to review and edit features in the labeling files under review.

• Provide archival features for labeling files.

iii. Automated Capabilities

The Contractor shall perform the tasks listed below for the Computerized Labeling Assessment Tool (CLAT) to provide the operational needs to satisfy FDA’s business goals for automating labeling review activities. Unless otherwise specified, “labeling” refers to all labels and other written, printed, or graphic matters (1) upon any article or any of its containers or wrappers, or (2) accompanying such an article, including the following types of labeling: container label, carton labeling, prescribing information, Instructions for Use, and Medication Guide.

The system shall function as follows:

• Use open-source software and other technologies to provide automated reviews of prescription drug product labeling submitted to the FDA.

• Use open-source software and other technologies to provide automated reviews of over-the-counter (OTC) drug product labeling submitted to the FDA.

• Automate the labeling review activities listed in the attached CLAT Labeling Review Activities file.

• Perform multiple simultaneous automated review activities for the same product.

• Perform multiple simultaneous automated review activities for multiple products, executed by the same user.

• Perform multiple simultaneous automated review activities for multiple products, executed by multiple users.

• Incorporate updates to the activities listed in the attached CLAT Labeling Review Activities file.

• Evaluate the submitted labeling for deficiencies or inconsistencies with the CLAT Labeling Review Activities file.

• Highlight each identified deficiency or inconsistency for each type of labeling reviewed.

• Provide a textual descriptor for each identified deficiency or inconsistency for each type of labeling reviewed.

• Enable the user to multi-select specific review activities performed by CLAT for manual review by the user.

• Enable the user to review labeling activities determined by CLAT as “meets” expectation.

• Enable the user to review labeling activities determined by CLAT as “doesn’t meet” expectation.

• Enable the user to review labeling activities determined by CLAT as “requires additional review” by the user.

• Enable the user to manually override or change a CLAT determination for “meets”, “doesn’t meet”, or “requires additional review”.

• Track manual overrides and changes to a CLAT determination.

• Enable the user to add a comment regarding a manual override or change to a CLAT determination.

• Enable the user to manually override or change a CLAT determination from “meets” to “doesn’t meet” or “requires additional review”.

• Enable the user to manually override or change a CLAT determination from “doesn‘t meet” to “meets” or “requires additional review”.

• Enable the user to manually override or change a CLAT determination from “requires additional review” to “doesn’t meet” or “meets”.

• Compare labeling with the initial or previously submitted labeling.

• Apply FDA-defined thresholds to assign look-alike similarity between selected labeling as high, moderate, and low similarity.

• Apply the entire CLAT to submitted labeling for review.

• Enable the user to select and apply portions of the CLAT to submitted labeling for review.

• Enable the user to multi-select labeling types for CLAT review.

• Enable the user to select a specific section(s) of the prescribing information for CLAT review.

• Enable the user to filter and exclude the manufacturer name and trade dress from the container labels for review.

• Enable the user to filter and exclude the manufacturer name and trade dress from the carton labeling for review.

• Provide annotation features for the user to annotate submitted labeling.

• Retain user annotations with the user name and date of annotation.

• Enable the user to add and save comments to submitted labeling.

• Separate text analytics from image analytics as part of the labeling review.

• Compare the product name, dosage form, route of administration, and NDC stated in the prescribing information with that information stated on the container label and carton labeling.

• Provide zoom in and zoom out features for manual review by users.

• Provide 3D rotation viewing features for 3D images.

• Enable the user to search the system by product name to identify labeling reviews.

• Enable the user to search the system by application name to identify labeling reviews.

• Enable the user to search the system by created or completed dates to identify labeling reviews.

• Enable the user to search the system by user names to identify labeling reviews.

• Enable the user to search the system by one or multiple review criteria such as “is the product identifier present on the carton labeling” to identify all products that had been categorized as meets, doesn’t meet, or requires additional review.

• Enable the user to search the system using fuzzy search logic to identify labeling reviews of interest.

iv. Needs Assessment

The Contractor shall perform the tasks listed below to define and prioritize labeling review activities for automation with the Computerized Labeling Assessment Tool (CLAT). Unless otherwise specified, “labeling” refers to all labels and other written, printed, or graphic matters (1) upon any article or any of its containers or wrappers, or (2) accompanying such an article, including the following types of labeling:

container label, carton labeling, prescribing information, Instructions for Use, and Medication Guide.

The Contractor shall perform and adhere to the following:

• Provide a written assessment of available open-source software and other technologies that can be used to automate labeling review activities.

• Develop a rubric (evaluation criteria) that can be used to assess and determine whether a labeling review activity is suitable for automation.

• Apply the rubric to evaluate the labeling review activities listed in the attached CLAT Labeling Review Activities file.

• Evaluate the labeling review activities listed in the attached CLAT Labeling Review Activities file, and recommend specific open source software or other technology for automating each activity.

• Evaluate the labeling review activities listed in the attached CLAT Labeling Review Activities file, and describe the risks and pertinent considerations for automating each activity.

• Evaluate the labeling review activities listed in the attached CLAT Labeling Review Activities file, and categorize each activity as low, medium, or high complexity based on time, effort, risks and other considerations, and technology needed for automation.

• Perform an assessment to determine gaps between the CLAT capabilities and desired labeling review activities.

v. Reporting, Analytics, and Data Visualization

The Contractor shall perform the tasks listed below to provide robust reporting, analytics, and data visualization for the Computerized Labeling Assessment Tool (CLAT). Unless otherwise specified, “labeling” refers to all labels and other written, printed, or graphic matters (1) upon any article or any of its containers or wrappers, or (2) accompanying such an article, including the following types of labeling:

container label, carton labeling, prescribing information, Instructions for Use, and Medication Guide.

The system shall function as follows:

• Support preparing, saving, and executing ad hoc reports.

• Enable the user to export, print, and share reports.

• Generate a report of the types of labeling submitted for review.

• Generate a report for individual labeling reviews that includes the reviewer name(s), date and time of review, product application number, product description, and submitted labeling that was subject of the review.

• Generate a report that provides the number of completed, in progress, or not started status labeling reviews by user name.

• Generate a report, based on an FDA template, of findings from the labeling review.

• Generate a report, based on an FDA template, that lists the review activities performed by CLAT for the product of interest.

• Generate a report that categorizes initial findings from the labeling review as meets, doesn’t meet, or requires additional review.

• Generate a report that categorizes findings from the labeling review as meets, doesn’t meet, or requires additional review, and includes user overrides and changes made to the CLAT determinations.

• Generate a report that provides the regulation or other FDA-provided data source as the reference for determining the labeling meets, doesn’t meet, or requires additional review.

• Generate a report that highlights the specific location of the deficiency or inconsistency identified in the reviewed labeling.

• Generate analytics and data visualization reports with the number and types of deficiencies identified by the system, for each reviewed labeling type for a specific product.

• Generate analytics and data visualization reports with the number and types of deficiencies identified by the system, for each section of the Prescribing Information, for a specific product.

• Generate analytics and data visualization reports with the number and types of deficiencies identified by the system, for each reviewed labeling type for a specific product over a user-defined time period.

• Generate analytics and data visualization reports with the number and types of deficiencies identified by the system for multiple products selected by the user.

• Generate an analytics and data visualization report that compares, describes, and illustrates revisions to the labeling over a user-defined time period.

• Generate a comparison report that compares labeling with the initial or previously submitted labeling.

• Enable the user to view findings from the labeling review.

• Enable the user to export findings from the labeling review to Word, pdf, ppt, or Excel formats.

• Enable the user to provide user and content tracked change annotation, editing, or similar functionality to labeling review reports.

• Enable the user to add comments to labeling review reports.

• Enable the user to view and compare labeling review findings for two or more products.

• Enable the user to generate and save a report of the review findings that includes a default system-generated name as per the system configuration.

• Generate FDA-defined metric and measure reports.

• Generate FDA-defined quality management reports.

• Provide a dashboard format for users to access the reporting, analytics, data visualization, metric, measure, and quality reports.

vi. Assessment of Requirements for Production Environment

The Contractor shall develop and deliver an assessment of the infrastructure requirements for a development, test, pre-production and production environments within FDA, but not limited to server provisioning and open source software that will be required to perform Task 2: Development Modernization and Enhancement (DME). The Contractor shall perform the tasks listed below to develop and deliver an assessment of the Computerized Labeling Assessment Tool (CLAT) requirements for development, test, pre-production and production environments.

The Contractor shall perform and provide the following:

• A written assessment that defines the resources and path for the FDA authority to operate certificate.

• Infrastructure requirements for development, test, pre-production and production environments.

• Requirements for central processing units, memory, servers and software.

• An assessment to meet 508 compliance and security documentation and certification.

• System shall enable with API capabilities to connect or integrate with other FDA systems.

• Provisioning requirements to scale the software for other CDER offices and the public.

vii. Metrics and Measures

The Contractor shall perform the tasks listed below to provide metrics and measures related to the Computerized Labeling Assessment Tool (CLAT). Unless otherwise specified, “labeling” refers to all labels and other written, printed, or graphic matters (1) upon any article or any of its containers or wrappers, or

(2) accompanying such an article, including the following types of labeling: container label, carton labeling, prescribing information, Instructions for Use, and Medication Guide.

The system shall provide metrics and measures based on the following:

• Number of labeling reviews performed by CLAT.

• Showing the number of labeling reviews performed using CLAT by each user in each activity with the amount of time taken to complete each review.

• Number of labeling reviews performed by CLAT, categorized by the labeling that meets, doesn’t meet, or required additional review.

• Number and type of labeling deficiencies or inconsistencies commonly identified by CLAT.

• Number and type of labeling deficiencies or inconsistencies by dosage form type.

• Number and type of labeling deficiencies or inconsistencies by labeling type.

• Number and type of labeling deficiencies or inconsistencies by application number or regulatory citation for monograph products, marketing category (ANDA, NDA, ANDA, over-the-counter [OTC] monograph), and product type (prescription, OTC, unapproved product).

• Number and type of labeling deficiencies or inconsistencies by product manufacturer names.

• Number of labeling activity, product name, and user name for overrides and changes to CLAT determinations.

• Number and type of reviewer edits and annotation made to labeling review findings by CLAT. The system shall provide metrics on the number of labeling reviews performed by CLAT, categorized by the number of reviews by user name and office.

• Number of labeling reviews performed by CLAT, categorized by the labeling type, and product name or type.

• Usage, including the number of users, simultaneous use, and sharing of files.

• Amount of time taken to perform labeling reviews using CLAT, with categories for labeling type and product name or type.

• Related to CLAT process improvement.

• Related to trends over time with increasing number of users.

• Provide role-based privileges for accessing metric data.

• Provide a dashboard format for accessing metric data.

• Evaluate and recommend other measures that can be used to track efficiencies gained or lost with CLAT.

viii. Quality Assurance, Quality Control, and Quality Improvement

The Contractor shall perform the tasks listed below to provide quality management functionality related to the Computerized Labeling Assessment Tool (CLAT). Unless otherwise specified, “labeling” refers to all labels and other written, printed, or graphic matters (1) upon any article or any of its containers or wrappers, or (2) accompanying such an article, including the following types of labeling: container label, carton labeling, prescribing information, Instructions for Use, and Medication Guide.

The Contractor shall perform and provide the following:

• A quality management plan that includes quality measures, monitoring frequency, and actions required for each component used by or activity performed by CLAT

• A quarterly CLAT quality management summary report

• Enable users to run quality parameters to generate quality management information

• Enable users to run quality parameters on open-source software and other technologies used for

CLAT

• Periodically test CLAT using “dummy” labeling data to ensure the accuracy, reproducibility, and other quality measures for labeling review activities

• Monitor and identify trends to detect errors and possible failures related to CLAT

• Enable users to report possible CLAT failures using “one-click” notification functionality

B. Task 2: Task Order Level Project Management

The Contractor shall provide task order project management to ensure that the scope, schedule, quality, and price are monitored and controlled. The Contractor Project Manager (PM) shall provide overall task order management and assist in the following IT related management activities as they pertain to this Task Order: release, change control, configuration, and IT investment management. The Contractor shall use internal processes consistent with the FDA’s Enterprise Performance Lifecycle (EPLC) and IT standards.

Specifically, the Contractor PM shall perform the activities described in the following sections below.

i. Project Management

The Contractor shall provide Task Order project management services necessary to plan, staff, coordinate, schedule, manage, communicate, report, and steward all tasks, activities, and deliverables throughout this task order execution.

ii. Project Management Plan (PMP)

The Contractor shall provide and maintain a PMP within 4 calendar weeks of award that describes the technical approach, organizational resources and addresses policies, processes, procedures necessary and is inclusive of the following plans: communication approach, staffing plan, quality control, risk and mitigation management, change management, configuration management, build management, integration management, data migration and transition, deliverable management, and project milestone schedule, to be employed to meet the cost, performance, quality, schedule and deliverable requirements throughout this task order execution.

Additionally, the PMP shall include the reviewed Work Breakdown Structure (WBS) composed of the major and minor milestones within 4 calendar weeks of award.

The Contractor shall include the reviewed Project Schedule which shall be completed in accordance with Project Management Institute (PMI) and CMMI Level 3 standard practices, and incorporate all deliverables, EPLC milestones and stage gates as drafted in the Project Process Agreement (PPA) within 15 calendar days of award.

The PMP shall include a communication plan that identify roles and responsibilities of FDA personnel and contractors and submit to the COR for approval within 30 calendar days of task order award.

iii. Monthly Progress Reports

The Contractor’s PM shall provide monthly status reports on the progress of activities identified to the FDA Project Manager and/or Contracting Officer Representative (COR). Within the first 30 calendar days after contract award, the Government and the Contractor shall design and agree upon a standard reporting format. Formal monthly status reports shall be delivered to the FDA Project Manager and/or the COR in accordance with the delivery table at the time on or before the 10th of the calendar month following the reporting month.

This report shall include at a minimum the following information:

• The overall progress of the project

• Progress on deliverables against the project plan

• Identified risks and issues as well as the mitigation strategy

• Accounting for deliverables

• Action items for the following period

• Any current or anticipated problems with scheduled activities

• The status of the previous month’s invoice

iv. Work Breakdown Structure (WBS) and Project Schedule

Maintain COR approved Work Breakdown Structure (WBS) and Project Schedule – Includes modules, components, builds/sprints, milestones, and all major deliverables; updated to indicate planned and actual start and completion dates for each.

v. Weekly Project Status Reports

Weekly project status reports to COR – Presents on-going activities, new activities, activities completed;

progress to date on all above mentioned activities; problems and corrective actions taken; risks, issues or concerns and proposed resolutions to address or mitigate them and, Government actions required.

Within the first 30 calendar days after contract award, the Government and the Contractor shall design and agree upon a standard reporting format.

vi. Final Report

The final report shall include a summation of the work performed and shall be in sufficient detail to describe comprehensively the results achieved for the entire contract period of performance. The final report shall include all deliverables that shall fully document the outcomes of the tasks outlined in the scope of work and lessons learned. The final report shall be submitted 30 days prior to the last day of the contract performance period. The report and model must comply with OMB and HHS guidelines related to Information Quality (available at http://aspe.hhs.gov/infoquality) and Information Technology and Section 508 Compliance requirements.

Prototype Demonstration The Contractor shall provide a final demonstration of the CLAT Prototype via an FDA webinar or if the Contractor is local (within 50-mile radius of the FDA White Oak Campus, Silver Spring MD) an onsite presentation in coordination with (and concurrence from) FDA COR.

The contractor shall also load the prototype onto the FDA cloud pre-prod environment and be able to run the demonstration from there if requested. This will need to be planned in advance. This will include having all software approved through normal FDA security procedures and making sure that the cloud application runs on AWS FedRamp cloud infrastructure https://aws.amazon.com/compliance/fedramp/

vii. Communication Management

The Contractor Project Manager shall identify how they intend to communicate with stakeholders by submitting a communication approach to address interactions between contractor staff, the COR. The Contractor shall ensure that the communication approach provides contact information including names, roles and responsibilities, staff changes, and progress meeting schedules. The communication approach shall be submitted with the Project Management Plan (PMP) to the COR.

viii. Escalation Procedures

The Contractor Project Manager shall escalate issues and risks related to any impediment to progress towards any deliverables defined in the Task Order. Issues and risks shall be based on the severity and probability of the specific issue or risk. The Contractor shall use the following order for escalation when reporting to FDA personnel:

• Task Order Contracting Officer’s Representative (COR)

• Contracting Officer

ix. Meetings https://aws.amazon.com/compliance/fedramp/

As an integral part of task order management, the Contractor shall prepare materials for and participate in or lead briefings, meetings, and other interactions with multiple FDA government points of contact for specific activities regarding project progress, issues, and concerns as directed by the COR. The Contractor shall participate in meetings in different capacities such as coordinating meetings, facilitating or leading discussions, contributing questions and answers, raising issues, risks and concerns, developing and giving presentations, and other active oral and written communications. These meetings shall include weekly status meetings, meetings based on the WBS and any other ad-hoc meeting that may be required.

The Contractor shall document the meeting minutes for all contractor attended meetings. The meeting minutes shall include all discussion points, summary, actions items and next steps and shall be provided to the COR in accordance with the delivery table no later than 2 business days after the meetings.

The Contractor shall also coordinate review meetings as directed by COR, to discuss the status, progress and any issues regarding this TO.

Representative tasks for a meeting shall include activities such as:

• Schedule meeting place and time

• Provide supporting information that may be required such as meeting agenda

• Ensure all necessary materials are distributed far enough in advance to allow for review

• Prepare meeting Room (Meeting can be conducted remotely)

• Give presentations

• Facilitate Discussions

• Summarize Discussions

• Draft meeting minutes

• Facilitate kickoff meeting for the project

x. EPLC Support and Documentation

The Contractor shall adhere to the FDA EPLC process for the duration of this Task Order. Within thirty

(30) calendar days after award, the Contractor Project Manager shall provide a signed FDA EPLC Project Process Agreement (PPA) to the COR for each release.

The PPA shall outline in detail the list of EPLC artifacts/deliverables, per FDA EPLC framework, to be delivered with each phase of the lifecycle process. At a minimum, the release artifacts shall include either new or updated versions of the following:

• Project Schedule

• Work Breakdown Structure

• Project Management Plan

• Business Requirements Document

• System Requirements Specifications

• System Design Document

• UAT Test Cases

• UAT Test Results

• Requirements Traceability Matrix

• System Test Report

The Contractor shall provide technical coordination (initiating updates to EPLC documentation, facilitating build/sprint schedules, and discussions) among stakeholders and documentation management. The Contractor shall develop and provide EPLC and other documents (as appropriate) to demonstrate compliance with data quality, EPLC, IT security and other standards and constraints. The Contractor shall ensure all EPLC documentation is free from typographical errors, grammatical errors and navigational problems. The Contractor shall create new documents in accordance with EPLC if there is no older version available or the older version is obsolete.

Release Management

The contractor shall work with Office of Information Management and Technology (OIMT) to support the development and deployment of system releases to the appropriate environments. The FDA categorizes releases as Initial, Major, Minor, or Emergency/Patch. These release types are defined as:

• Initial Release: Scheduled first release of a new system.

• Major Release: A major release is a planned release that provides major functional changes to or extensions of the product. Major releases are usually not backward compatible with previous releases.

• Minor Release: Scheduled release to include defect corrections and/or minor enhancements, including software changes necessary to support system or patches. Minor releases are backward compatible with all releases up to the last prior major release.

• Emergency Release: Release to correct one or more problems with a system that is negatively impacting functionality and requires prompt resolution. Emergency Releases are added to the project schedule as needed upon approval by COR and/or Project Manager.

Documentation

The contractor PM shall provide documentation specific to the technical solution applied. Documentation may include software technical specifications, test scripts, configuration and installation guides to support a secure and authorized solution.

The contractor shall work with the COR to determine the appropriate EPLC documentation required for implementation, Operations and Maintenance (O&M), and Development, Modernization and Enhancement (DME) work performed on the CLAT tool under this IDIQ contract. The contractor shall create agreed upon documentation and submit to the COR. Typical documents include but are not limited to (See RFP Attachment 10: EPLC Artifacts for more detail on Agile methodologies):

• Project Charter / Project Process Agreement (PPA);

• Work Breakdown Structure (WBS);

• Risk Management Plan and Risk Tracking Log;

• Business Requirements Document (BRD);

• Project Management Plan and Schedule;

• Systems Requirement Specification (SRS);

• System Design Document;

• Requirements Traceability Matrix;

• Test Planning and Reporting;

• Training Plan and User Help;

• Version Description Document (VDD)

• Authorization to Operate (ATO) - The contractor shall support FDA stakeholders to identify and document security requirements associated with obtaining the Authorization to Operate (ATO) including NIST SP 800-53 Rev. 3. As tasked, the contractor shall document updated control descriptions for any NIST SP 800-53 Rev. 3 controls that will be impacted due to the work performed.

V. TRANSITION OUT (OPTIONAL CLIN)

Transition Out will be a separately priced out line item that shall commence over a period of sixty (60) calendar days prior to the end of the period of performance based in accordance with FAR 52.217-7 Option for Increased Quantities of separately priced line items. The contractor shall provide support services to the orderly transition of functions from this task order to the FDA. The FDA will exercise this option during the last period of performance that it intends to fund under this contract. It is the responsibility of the incumbent to ensure a smooth transition out from this contract and the successor contractor. Should a new contract not be in place, the incumbent shall provide this transition to the designated FDA employees. The incumbent contractor shall be the lead on the transition activities during the transition period, leading Technical Interchange Meeting (TIMs) with the successor to impart knowledge transfer of the effort under the scope of this contract. At a minimum, these TIMs shall divulge the technology and processes employed, provide licensing requirements/status (if any), and conduct demonstrations of tasks. During this period, there shall be no degradation in support to the government while the outgoing contractor focuses on contract closeout activities and providing sustainment support to complete the transition. At the COR designated turnover date, approximately 60 days after award, the successor contractor shall assume full responsibility of the contract.

Representative Transition activities are as follows:

• Develop Transition-In or Transition-Out Plan to include a training program to ensure full operating accountability and responsibility of the operational task order

• Understand roles and responsibilities of incumbent and successor staff

• Engage in regularly scheduled meetings

• Plan and engage in knowledge transfer meetings

• Receive or transfer all system documentation

• Receive or transfer all workflows and business process development documentation

• Transfer source codes, if applicable

• Receive or transfer system administration responsibilities

• Receive or transfer current Change Requests

• Discuss change management processes and any other outstanding issues

• Receive or transfer Government Furnished Property or Information

As part of the transition/closeout activities, the contractor shall confirm via email to the COR/PM the completion of all transition out activities as follows:

• Ensure the accuracy of asset management records, software documentation, and training materials, and delivery of these documents to the government.

• Provide FDA with a list of users (along with their organization/affiliation) granted access to the systems, along with their role.

• Ensure that all system documentation is updated as of the most recent release (e.g., system architecture, user guides, release documentation, design documentation, run books)

• Participate in transition meetings as requested by the COR/PM or their designated representative.

• Assist with documenting and returning all government furnished equipment (GFE) as directed by the COR/PM and FDA Property Officer.

• Provide information about any changes to staffing to the COR or their designated representative.

• Provide Subject Matter Expert (SME) support to the FDA.

• Provide the FDA with training on producing metrics for FDA weekly, monthly and six-month program reviews.

VI. PLACE OF PERFORMANCE

Place of performance will be at the contractor’s locations. Contractor employees may be required to periodically interact with FDA staff at FDA facilities at the at White Oak facility in Silver Spring, MD.

VII. Contract Type

This is a Labor-Hour (LH) Task Order to be issued against the IDIQ Contract for Computerized Labeling Assessment Tool (CLAT) services. The terms and conditions of the IDIQ contract will be applicable to this task order.

VIII. CONTRACT ADMINISTRATION DATA

a. Contracting Officer’s Representative (COR)

The COR is responsible for: (1) monitoring the Contractor's technical progress, including the surveillance and assessment of performance and recommending to the Contracting Officer changes in requirements;

(2) interpreting the contract and any other technical performance requirements; (3) performing technical evaluation as required; (4) performing technical inspections and acceptances required by this contract;

and (5) assisting in the resolution of technical problems encountered during performance.

The Contracting Officer is the only person with authority to act as agent of the Government under this contract. Only the Contracting Officer has authority to: (1) direct or negotiate any changes in the RFP; (2) modify or extend the period of performance; (3) change the delivery schedule; (4) authorize reimbursement to the Contractor any costs incurred during the performance of this contract; or (5) otherwise change any terms and conditions of this contract.

b. Government Points of Contact

Contracting Officer:

Steven Gagnon 4041 Powder Mill Road Beltsville MD 20705 E-mail: Steven.Gagnon@fda.hhs.gov

Contracting Specialist:

Laurel Woest 4041 Powder Mill Road Beltsville MD 20705 E-mail: Laurel.Woest@fda.hhs.gov

Contracting Officer Representative:

TBD

[INSERT ADDRESS]

xxx-xxx-xxxx

Email: xxx.xxx@fda.hhs.gov mailto:xxx.xxx@fda.hhs.gov

The Contractor is responsible for providing the following key personnel with the appropriate expertise to accomplish the activities and prepare the deliverables identified in this Statement of Work.

Labor Category Role Name IT Project Manager II Software Engineer I

IX. GOVERNMENT FURNISHED EQUIPMENT/GOVERNMENT FURNISHED INFORMATION

The FDA will provide the Contractor with any updated documented standards and policies including publicly available labeling files, FDA Office of Information Management (OIM) Policy and Guidance, Office Policy, EPLC templates and IT security policies for which the Contractor is expected to comply.

The FDA will provide the following hardware when required per task order requirement:

• FDA laptop - The Contractor shall maintain a formal inventory of any Government furnished equipment and at a minimum shall track property number, equipment type, and assignee.

Security access badges are to be used only for the purposes of accessing FDA facilities, networks, and PCs. At the end of the period of performance or termination of the contract, or termination of an employee, the Contractor must return the badges and any other Government owned property. Access to FDA systems is contingent on successful completion of the badging process; it should be started as soon as possible after award.

X. Performance Management

Performance metrics designed to measure contractor effectiveness shall be utilized. Explained baseline goals are noted in the Table 4: Performance Metrics.

Table 4: Performance Metrics Type Description Performance Standard Acceptable Quality Level Surveillance

Method

Overall Client Satisfaction

Client Satisfaction

Requirements and final Prototype Application is configured to meet client needs.

Overall 7 or above on a scale of 10 or subjectively measured with service ratings of Excellent, Satisfactory, or Unsatisfactory.

Stakeholder Feedback

Communication Emails Timely response to stakeholder communication regarding the CLAT Prototype Application

All FDA correspondence will be responded within 24 hours

Stakeholder Feedback, Random Inspection

Reliability and Quality (UAT)

CLAT Prototype module defect occurrences

Change Management procedures, Implementation Guidelines, and standard operating procedures are adhered for UAT. All new or modified code is thoroughly inspected and tested against the system and functional requirements.

Less than 10 Low or Medium defects, less than 3 High defects, and less than 1 Major defect are discovered during UAT.

Workaround for high and major defects must be submitted within 5-business days of discovery and resolved within 10-business days.

100% Inspections, Stakeholder Feedback

Deliverables Final Report and Demonstration

Final CLAT demonstration in contractor’s environment with no Interface or functionality issues All associated activities and deliverables are completed.

EPLC documentation is complete, stage gates are approved, documents are uploaded to the project repository

100% Inspection, Stakeholder Feedback

XI. SECURITY AND PRIVACY

IX.I FDA Information Technology Security – Information Security and/or Physical Access

SECTION 1

A. Baseline Security Requirements

1. Applicability. The requirements herein apply whether the entire contract or order (hereafter “contract”), or portion thereof, includes either or both of the following:

a. Access (Physical or Logical) to Government Information: A Contractor (and/or any subcontractor) employee will have or will be given the ability to have, routine physical (entry) or logical (electronic) access to government information.

b. Operate a Federal System Containing Information: A Contractor (and/or any subcontractor) will operate a federal system and information technology containing data that supports the HHS mission. In addition to the Federal Acquisition Regulation (FAR) Subpart 2.1 definition of “information technology” (IT), the term as used in this section includes computers, ancillary equipment (including imaging peripherals, input, output, and storage devices necessary for security and surveillance), peripheral equipment designed to be controlled by the central processing unit of a computer, software, firmware and similar procedures, services (including support services), and related resources.

1) Safeguarding Information and Information Systems. In accordance with the Federal Information Processing Standards Publication (FIPS)199, Standards for Security Categorization of Federal Information and Information Systems, the Contractor (and/or any subcontractor) shall:

a. Protect government information and information systems in order to ensure:

• Confidentiality, which means preserving authorized restrictions on access and disclosure, based on the security terms found in this contract, including means for protecting personal privacy and proprietary information;

• Integrity, which means guarding against improper information modification or destruction, and ensuring information non-repudiation and authenticity; and

• Availability, which means ensuring timely and reliable access to and use of information.

b. Provide security for any Contractor systems, and information contained therein, connected to an FDA network or operated by the Contractor on behalf of FDA regardless of location. In addition, if new or unanticipated threats or hazards are discovered by either the agency or contractor, or if existing safeguards have ceased to function, the discoverer shall immediately, within one (1) hour or less, bring the situation to the attention of the other party. This includes notifying the FDA Systems Management Center (SMC) within one (1) hour of discovery/detection in the event of an information security incident.

c. Adopt and implement the policies, procedures, controls, and standards…