Attachment_1_-_NCI_EVS_Draft_SOW_1.pdf

National Cancer Institute (NCI) Enterprise Vocabulary Services Support (EVS)

STATEMENT OF WORK

2.1 PURPOSE

The purpose of this requirement is to acquire Enterprise Vocabulary Services (EVS) for the National Cancer Institute’s (NCI) Center for Biomedical Informatics and Information Technology (CBIIT) and its support of integrated Information Technology (IT) and biomedical informatics coordination and management efforts.

2.2 BACKGROUND

NCI is the Federal Government's principal agency that conducts and supports research, training, and health information dissemination with respect to the cause, prevention, diagnosis, and treatment of cancer, rehabilitation from cancer, and the continuing care of cancer patients and the families of cancer patients. NCI is the largest institute within the National Institutes of Health (NIH), a world-renown medical research facility and one of the most visible operating divisions of the Department of Health and Human Services (HHS). The NCI consists of six Divisions, four Centers, and several Offices. There are approximately 7,834 employees and contractor staff, including research scientists, medical doctors, visiting international researchers, program administrators, and administrative staff. The NCI staff is currently located in over 20 buildings in Bethesda, Rockville, and Frederick, MD. Further information about NCI is available on its public website at https://www.cancer.gov/.

The CBIIT is an important NCI component entrusted to lead and coordinate efforts in the fields of informatics, data science, and information technology. CBIIT is defining, designing, developing, and deploying key infrastructure, standards, and application-level components of the IT fabric to support a modern vision for integrated cancer and biomedical research, translational and population science, and personalized medicine. CBIIT establishes and develops the framework, infrastructure, standards, processes, and tools for data collection and management to ensure the long-term value of NCI’s investment in research through data accessibility, interoperability, sharing and reuse. CBIIT creates mechanisms to build and support information networks enabling all constituencies in the cancer community – NCI programs, other institutions, researchers, physicians, and patients – to facilitate new discovery, integrate data, implement knowledge, make research results available for the community, minimize the burden of cancer, and improve the lives of patients.

The NCI, its partners, and wider cancer research community share a critical reliance on the comprehensive, consistent, current, and harmonized biomedical terminology and ontology resources. High-quality semantic content and associated services are “mission-critical” components essential for the NCI activities.

NCI Enterprise Vocabulary Services (EVS) is a premier NCI terminology and ontology resource. It provides foundational content, consulting and technical services, and tools to meet NCI and NCI partners’ needs for controlled terminology and terminology standards, and to https://www.cancer.gov/ facilitate data interoperability and support knowledge management technologies across the Institute’s information systems, NIH Centers, federal regulatory agencies, the larger biomedical community, and health care organizations.

NCI EVS (https://evs.nci.nih.gov/) produces, maintains, and publishes multiple semantic products including two major ones: the NCI Thesaurus (NCIt), the core NCI's reference biomedical terminology/ontology containing more than 192,000 concepts and more than 2,000 terminology value sets generated according to the requirements of the EVS stakeholders and users, and the NCI Metathesaurus (NCIm), a comprehensive integrated database of 7.1 million terms from 116 source biomedical vocabularies condensed into 2.7 million concepts.

Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT), The Cancer Medications Enquiry Database (CanMED), Chemical Entities of Biological Interest (ChEBI), The Human Genome Organization's (HUGO) Gene Nomenclature Committee (HGNC) Terminology, Gene Ontology (GO), Logical Observation Identifier Names and Codes (LOINC), Medication Reference Terminology (MED-RT), The International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM), Medical Dictionary for Regulatory Activities Terminology (MedDRA), and multiple cross-terminology mappings.

NCI EVS performs semantic requirements gathering, scientific/biomedical content and technical consultations, and facilitates multi-stakeholder content and technical collaborations and user support to address the broad spectrum of terminology needs of NCI and its partners.

The EVS Program encompasses terminology evaluation, science-driven content development and maintenance, and terminology licensing; terminology software evaluation, development, and licensing; terminology publications and downloads through multiple file formats, browsers, and application programming interfaces (APIs).

EVS products and services have been adopted by research institutions, regulatory authorities, data standards development organizations (SDOs), healthcare organizations, and industry as a shared standards development and coding environment, allowing participants to compare and harmonize with each other's content while taking advantage of controlled terminology, ontology, structures, full-text definitions, codes, and other features for effective data interpretation, management, retrieval, sharing, reuse, and knowledge representation.

EVS Specialized Software and Workflow

EVS currently develops and uses the following specialized terminology tools, which can be found at https://evs.nci.nih.gov/tools and https://wiki.nci.nih.gov/display/EVS/EVS+Wiki/ .

Endpoints, tools, and applications to support terminology development, searching, metadata and content retrieval for EVS-hosted terminologies, terminology mapping, and semantic support of data harmonization and sharing include:

• NCI Protégé - creation and maintenance of NCIt terminology and ontology

• NCI Metathesaurus editing tool - Metathesaurus Enhancement and Maintenance

Environment (MEME)

• EVS Term Suggestion Application https://evs.nci.nih.gov/ https://evs.nci.nih.gov/tools https://wiki.nci.nih.gov/display/EVS/EVS%2BWiki/

• EVS Report Writer

• EVS EXPLORE Browser

• NCI Report Exporter

• EVSREST API

• EVS File Transfer Protocol (EVS FTP) site

• EVS Genomic Data Commons (GDC) Metadata Validation Service (EVS GDC MVS)

• EVS Semantic Integration Platform (EVS-SIP)

• Triplestore database containing NCIt and other terminologies (currently in Enterprise

Knowledge Graph Platform Stardog).

EVS core terminology products are currently supported by the following workflows:

NCI Thesaurus

• Developed and maintained in NCI Protégé version 3.4 (built on top of Protege 5) including NCI Edit Tab and a package of plugins specific to the NCI editing and production process.

• Conflict resolution and history generated through Revision History plugin for Protégé.

• Published in OWL2 format with history files, including comprehensive logical subset branch files and value sets.

• OWL files promoted to the AWS environment and loaded into a triplestore database.

• Content of the triplestore database is indexed into Elasticsearch and used by

EVSREST API to support the EVS EXPLORE Browser functionalities.

• Browse with EVS EXPLORE Browser

(https://evsexplore.semantics.cancer.gov/evsexplore/welcome).

• OWL files processed for insertion into NCI Metathesaurus.

• NCIt versions posted on EVS ftp site (https://evs.nci.nih.gov/ftp1/) in multiple formats and sent to NLM UMLS.

• NCIt content pulled monthly by The National Center for Biomedical Ontology

(NCBO, https://bioportal.bioontology.org/) to publish in Bioportal and by The Renaissance Computing Institute (RENCI) at the University of North Carolina at Chapel Hill; Oregon State University (OSU) to produce and publish NCIt Open Biological and Biomedical Ontology Foundry (OBO) edition.

NCI Metathesaurus

• Editing of NCI Metathesaurus in the Metathesaurus Enhancement and Maintenance Environment (MEME) in the AWS environment.

• Indexed in Elasticsearch.

• Production release through EVSREST API

(https://evsexplore.semantics.cancer.gov/evsexplore/evsapi).

• Browsing with EVS EXPLORE Browser

(https://evsexplore.semantics.cancer.gov/evsexplore/welcome).

• Distribution NCI Metathesaurus mapping files posted on ftp site

(https://evs.nci.nih.gov/ftp1/).

https://evsexplore.semantics.cancer.gov/evsexplore/welcome https://evs.nci.nih.gov/ftp1/ https://bioportal.bioontology.org/ https://evsexplore.semantics.cancer.gov/evsexplore/evsapi https://evs.nci.nih.gov/ftp1/

Standalone Terminologies and Mappings Workflows

• Standalone terminologies and terminology products are obtained from various sources (terminologies marked with an asterisk are extracted from NCI Metathesaurus), indexed in Elasticsearch, and published by EVSREST API (https://evsexplore.semantics.cancer.gov/evsexplore/evsapi) to support the EVS EXPLORE Browser (https://evsexplore.semantics.cancer.gov/evsexplore/welcome) functionalities.

o The Cancer Medications Enquiry Database (CanMED) o Chemical Entities of Biological Interest (ChEBI) o Common Terminology Criteria for Adverse Events (CTCAE) o Data Use Ontology (DUO) o The Human Genome Organization’s (HUGO) Gene Nomenclature

Committee Terminology (HGNC) o Logical Observation Identifier Names and Codes (LOINC)* o Microarray Gene Expression Data Ontology (MGED) o Medication Reference Terminology (MED-RT)* o Medical Dictionary for Regulatory Activities Terminology (MedDRA)* o NanoParticle Ontology (NPO) o The International Classification of Diseases, Tenth Revision, Clinical

Modification (ICD-10-CM)* o The International Classification of Diseases, Tenth Revision (ICD-10)* o The International Classification of Diseases, Nineth Revision, Clinical

Modification (ICD-10-CM)* o Gene Ontology (GO) o Ontology for Biomedical Investigations (OBI) o The Ontology for Biobanking (OBIB) o Reference Ontology for Radiology (RadLex)* o The Systematized Medical Nomenclature for Medicine–Clinical

Terminology (SNOMED CT)* o Unified Medical Language System (UMLS) Semantic Network o Zebrafish Model Organism Database (Zebrafish)

• Pairwise mappings extracted, used by EVSREST API and Elasticsearch to be published through the EVS EXPLORE Browser functionalities.

o GO to NCIt Mapping o ICD-10 to MedDRA Mapping o Anatomical Dictionary for the Adult Mouse (MA)to NCIt Mapping o NCIt to ChEBI Mapping o NCIt to HGNC Mapping: NCIt to HGNC Mapping o SNOMED CT US Edition to ICD-10-CM Mapping o SNOMED CT US Edition to ICD-10 Mapping

2.3 OBJECTIVE

https://evsexplore.semantics.cancer.gov/evsexplore/evsapi

The objective of this Statement of Work (SOW) is to acquire contractor services for EVS Operations support. This requires providing professional staff with clinical, biomedical, scientific, and informatics expertise, as well as strong biomedical terminology/ontology and information technology knowledge and experience to provide a variety of semantic services.

Services include controlled biomedical vocabularies and other semantic products analysis;

semantic support of biomedical research and clinical data modeling and design; full cycle of development, documentation, operations, publication, and maintenance of ontology and controlled terminology for variety of biomedical research, and development of associated processes, applications, and tools. It also requires providing user community services for the NCI Enterprise Vocabulary Services (EVS) Program. The goal is to comprehensively review, maintain, and improve EVS content resources, infrastructure/architecture, workflows, data access, communications, and services while ensuring uninterrupted operations.

2.4 SCOPE

The Contractor shall provide several general categories of consulting, planning, process improvement, management, systems and software development, information technology operations, subject matter expertise, and terminology analysis, development, maintenance, publishing, and support services to CBIIT.

These services shall include, but not be limited to: overall Task Order and project management and high level coordination among subprojects and tasks; analysis of semantic content and technical aspects of all EVS resources and many related and external terminology/ontology sources, mappings, and biomedical data standards; working with NCI and community partners to analyze and address their terminology needs; developing recommendations for the EVS Program, including technical and process improvements; and performing the development, modeling, refinement, quality assurance, maintenance, publication, and deployment of EVS scientific, clinical, and administrative terminology resources, tools, and services.

The Contractor shall evaluate EVS requirements, plans, processes, and technologies and identify opportunities for optimization of current processes, and areas and targets for potential beneficial changes, enhancements, and expansion of EVS semantic content and services. When recommended improvements are accepted by the government, the Contractor shall assist in planning and supporting their implementation.

The contractor shall perform continuous process and EVS semantic products enhancement, integration of new content developed inside and outside of NCI; terminology editing, quality assurance, testing, maintenance, harmonization, mapping, and publications of NCI vocabularies and other appropriate terminology and ontology sources. The contractor shall also participate in testing of new versions of the NCI-specific versions of Protégé and other EVS software including EVS browsers, APIs, semantic content extraction applications used for terminology editing, mapping, display, and publishing, as well as data integration support tools as needed. The contractor shall provide technical support for terminology production, such as loading a variety of terminologies in the terminology servers and coordinating the promotion of data and software through the relevant NCI system tiers and publishing according to the defined schedules.

Therefore, in support of continuing EVS operations the contractor shall provide professional skills and services in the following broad categories:

• EVS Architecture, Process, Technical and Contractors’ Relationships Services o NCI Enterprise Vocabulary Services architecture o Process reengineering and improvement o Support of development, operations, and maintenance of EVS tools o Innovating and updating EVS technologies and tools o Technical coordination and contractors’ relationship management support

• Terminology Lifecycle Support: Creation, Maintenance, Processing, and Publication Services, and Contractors’ Relationships Services o Editing NCI Thesaurus, Metathesaurus, standalone EVS-hosted terminologies and ontologies, cross-terminology mappings, and other EVS content o Maintaining currency of terminology content to support rapidly advanced cancer research and evolving users’ semantic needs o Terminology data conversion, mapping, extension, and integration including analysis, adoption, processing, and representation of biomedical research and clinical data standards o Terminology/ontology and mappings publications o Technical coordination of terminology development and management, and contractors’ relationship management support o Implementation of best practices of semantic representation of scientific and clinical data for reporting, coding, data management, and analysis in cancer and biomedical research

• Quality Assurance/Control and coordination of Data, Tools, and Systems Services

• Subject matter expertise in ontology/terminology, knowledge representation, and scientific and biomedical consulting services o Guidance to educate users on best practices of the semantic content use o Consulting on data standards utilization and data modeling and coding

• Scientific and technical program and project management

• Technical and Product Documentation Services.

Work products shall include terminology content, data files, scripts or software code, analysis reports, recommendations, analyses, management reports, systems and database requirements documents, software code documents, standard operating procedures documents, and briefings and presentations.

2.5 TASK AREAS

The Contractor shall furnish all the necessary services, qualified personnel, materials, equipment, and facilities, not otherwise provided by the Government, as needed to perform this SOW at the technical direction of the Contracting Officer’s Representative (COR) in the Task Areas specified below. This effort may require interactions, meetings, calls, training and working sessions with CBIIT, NCI and other federal and non-governmental staff and contractors. The Contractor shall be providing mission critical services in a fast paced and highly integrated environment.

2.5.1 Task Order Management

The Contractor shall provide the technical and functional activities necessary for the management of this SOW and all tasks provided under this SOW. The Contractor shall ensure that personnel are sufficiently qualified to execute the tasks listed in this SOW. The Contractor shall assume responsibility to meet the cost, performance, and schedule requirements throughout the Task Order execution. The Contractor shall provide a Task Order Management Plan outlining the approaches to be used to provide the required services including the specific deliverable documents, analyses, and reports to be developed. When approved by the COR, this plan shall be used to monitor, control, and report progress, issues, and resource expenditure. This plan shall be updated as required and approved by the COR as required to maintain currency with project objectives, activities, and direction.

Additionally, the Contractor shall participate in task management meetings, and provide weekly verbal status updates and monthly status reports monitoring the performance and progress toward successful completion of this SOW.

The Contractor shall ensure that all personnel complete appropriate training. This includes, but is not limited to, the use of training necessary for this project and all applicable confidentiality, privacy, information security, and conflict of interest regulations and corresponding NIH, HHS and Federal policies and requirements.

2.5.1.1 Kickoff Meeting

An initial kickoff meeting will be held within 7 calendar days of award or as agreed to by the COR. This will be attended by the COR and may be attended by the Government’s Contracting Officer (CO). Key Contractor personnel as well as a representative from the Contractor’s contracts organization are requested to attend. The intent of the kickoff is for all key personnel to meet to discuss the project’s overall technical and contractual requirements.

At this meeting, the Contractor shall be prepared to discuss the following:

• Technical Objectives

• Task Order Management Plan

• Deliverables and Deliverable Acceptance Criteria

• Reporting and Invoice Requirements.

2.5.1.2 Coordination

The contractor shall provide mission critical services in a fast moving and highly integrated environment. The need to maintain a team culture in the maintenance of NCI EVS is paramount.

The Contractor shall interact with the NCI COR; attend EVS weekly and ad hoc meetings;

and interact with other Task Order and NCI staff who accept and load and manage EVS semantic content including but not limited to the NCI Thesaurus and NCI Metathesaurus files, perform quality assurance (QA) on EVS tools, edit or otherwise update NCI Thesaurus and other EVS terminologies and mapping, and the NCI hardware and software systems on which EVS content, software and applications reside.

The Contractor shall deal routinely with NCI EVS users and external terminology developers and software developers and shall establish and maintain effective and ethical customer relationships as befits a customer-focused service enterprise.

2.5.1.3 Status Meetings and Reports

The Contractor shall participate in recurring management meetings and provide status updates and monthly status reports monitoring the performance and progress toward successful completion of this SOW. Known changes to work schedules and costs must be identified and discussed in these meetings.

2.5.1.4 Contractor/Subcontractor Personnel

The Contractor shall be responsible for managing and overseeing the activities of all Contractor personnel, as well as subcontractor efforts used in performance of this effort. The Contractor's management responsibilities shall include all activities necessary to ensure the accomplishment of timely and effective support, performed in accordance with the requirements contained in the SOW. Resumes submitted for employees assigned to perform under this SOW shall contain documented experience directly applicable to the functions to be performed. Further, these prior work experiences shall be specific and of sufficient variety and duration that the employee is able to effectively and efficiently perform the functions assigned.

2.5.1.5 Project Manager

The Contractor shall provide a Project Manager to facilitate Government-Contractor communications. The Project Manager shall be fully assigned to the EVS Program. The Project Manager shall be the primary technical and managerial interface between the Contractor, the CO and the COR. The name of the Project Manager shall be designated in writing to the CO.

The Project Manager shall have full authority to act for the Contractor on all Task Order matters relating to daily operations. The Project Manager must have fluent ability to read, write, speak, and understand English.

The Contractor’s Project Manager must be available during normal duty hours, as specified herein and available to meet with Government personnel within 24 hours to discuss problems.

The Contractor's Project Manager shall meet with the CO/COR as necessary to maintain satisfactory performance and to resolve other issues pertaining to Government/Contractor procedures. At these meetings, a mutual effort will be made to resolve any and all problems identified. Action items stemming from these meetings shall be prepared by the Contractor, ownership assigned, and timeframe proposed, and furnished to the Government within two (2) workdays of the subject meeting.

2.5.2 EVS Architecture, Process, Technical, and Relationships Services

2.5.2.1 NCI Enterprise Vocabulary Services Architecture

The Contractor shall provide consulting and technical services in support of the definition, design, development, implementation, operations, and management of a comprehensive NCI Enterprise Vocabulary Services architecture. This architecture shall encompass NCI EVS operations as well as activities supported by NCI CBIIT that require semantic, scientific, and technical support provided by EVS, to include biomedical research, healthcare care delivery, public health, and NCI research data collection and integration initiatives including, but not limited to Cancer Research Data Commons and relevant components, and NCI administrative activities. This architecture effort shall be integrated with the specific activities in tasks 5.2.2 –

5.2.4 and shall be designed to facilitate and support the full range of tasks and activities specified in this SOW. Implementation planning shall include provision for continuous operation of all core EVS services.

2.5.2.2 Process Reengineering and Improvement

The contractor shall provide expert technical and consulting services to review, update, innovate and validate EVS processes and workflows, aligning these with the NCI Enterprise Vocabulary Services architecture and EVS users’ semantic and technical needs. The Contractor shall:

• Analyze current EVS terminology resources, content, processes, and services;

• Propose process improvements to improve performance, workflows, semantic content access, and increase quality, efficiency, and utility;

• As approved by the COR, work with NCI staff and collaborators, EVS staff and stakeholders to implement such improvements and expand EVS services.

2.5.2.3 Support of Operations and Maintenance of EVS Tools, and Innovating and

Updating EVS Technologies and Tools The contractor shall provide expert technical and consulting services to review, update, innovate, validate, and maintain EVS systems and tools to meet NCI requirements. These services shall include:

• Technical support of terminology creation, editing, maintenance, mapping, publication, and semantic content access, delivery, and distribution services;

• Technical support of terminology data analysis and reporting services to include coverage of data quality, utilization, effectiveness, and standards support;

• Development, implementation and operation of validation services;

• Improvement and expansion of documentation and other forms of communication for adopters and other users of EVS systems and tools.

• Operations and maintenance of existing EVS software components and development of new features and functionalities (if needed) including identifying, triaging, debugging, developing, and deploying bug fixes, minor enhancements, and quality assurance/testing of the toolset; providing operational support to the tools and services for the NCI production deployment of the tools; providing technical leadership to the community on the toolset. The following are in scope for this effort:

o Software development, maintenance, and enhancement of EVSREST API, associated applications, web pages, and the EVS EXPLORE Browser.

o Software development and maintenance of EVS Report Writer tool.

o Software development and maintenance of EVS Term Suggestion Application.

• Software design, development, enhancement, and maintenance including requirements gathering; use case identification, validation and prioritization;

development and optimization of new features and functionalities; identifying, triaging, debugging, developing and deploying bug fixes, minor enhancements, and quality assurance/testing of the toolset; providing operational support to the tools and services for the NCI production deployment of the tools; providing technical leadership to the community on the toolset. The following are in scope for this effort:

o Design, development and maintenance of an enhanced EVS EXPLORE Terminology Browser and report generation application utilizing the Triplestore terminology server.

o Design, development and maintenance of the EVS Triplestore based terminology server and related applications, and API.

o Design, development and maintenance of additional applications if needed to support EVS activities requested by the COR.

• EVS tool technology enhancements to improve EVS software and tools functionality, reliability, performance, and maintainability including, but not limited to, the following:

o Cloud support: The deployment, customization and maintenance of EVS tools and services deployed to the NCI CloudOne environment. Working with the NCI Cloud Engineers, expand the current EVS Cloud based infrastructure in support of the growing data size and computational requirements of the biomedical research community EVS supports.

o Backend persistence mechanisms (file systems and databases) improvements and Representational State Transfer (REST) services.

o Deployment and Test Automation: Build upon current continuous integration techniques to develop a more efficient and responsive development and deployment process for EVS software and data.

o Maintenance of the software defined as in scope for this effort including changes and enhancements as required to ensure that the software product is functioning and up-to date with the NCI tech stack as well as resolving any key security or other operational issues. The Contractor shall support prioritization and maintenance of the software defined as in scope for this effort.

o Providing required updates and addressing security vulnerabilities to reduce the risk of security issues in the existing applications and tools and in the software in development.

o As needed, support for planning transitions from one tool to another as part of architecture evolution.

Successful completion of this task requires personnel with a minimum of:

• Five (5) years of technical experience supporting editing and production of ontology and terminology products using Protégé tools;

• Five (5) years of technical experience supporting editing and production of ontology and terminology products using the MEME environment including three (3) years of using MEME 5 editing tool;

• Five (5) years of technical experience with Triplestore technologies and proficiency with the SPARQL querying language;

• Five (5) years of experience in the medical informatics industry with focus on terminology tools to support a full life-cycle necessary for the production, maintenance and delivery terminology and ontology content including a minimum of five (5) years of experience with MEME software support and maintenance.

2.5.2.4 Coordination and Relationship Management Services

The Contractor shall work with non-EVS domain experts and other stakeholders to analyze their terminology requirements, identify ways in which those requirements might be addressed through existing or new EVS resources and services, and participate in planning and carrying out such initiatives. In particular:

• Such work will often include advising, coordinating, and reviewing with such stakeholders’ test insertions of stakeholder terminology into NCI vocabularies and terminology systems and cross-terminology mapping.

• Contractor shall initiate and support expert terminology working groups in the development of new terminology and terminology standards. Such work will include assisting with defining the scope of work, creating and defining terms, providing guidance on logical placement, and supporting the overall working group process including administrative tasks.

• Contractor shall work closely with the CBIIT staff and collaborators, NCI programs and initiatives, and NCI partners to provide terminology advice and annotations for NCI-supported data, metadata, data standards, and models.

• The Contractor shall provide support to other NCI divisions, offices, centers and collaborators, and NCI partners including regulatory agencies and SDOs on an as needed basis, at the request of the COR.

Successful completion of this task requires personnel with an intensive background in biomedical terminology and ontology development. Understanding of the NCI Thesaurus and other terminologies and ontologies, biomedical data standards and reference sources, good terminology practices, and ability to curate and work with submitters to get the terminology evaluated, reviewed, inserted, and integrated expeditiously and efficiently are required.

Biomedical domain expertise is required.

2.5.3 Terminology Creation, Processing, and Publication Services The Contractor shall provide support for the creation, processing and publication of controlled terminologies, vocabularies and ontologies for the support of operations and development activities in the high priority knowledge domains including but not limited to:

• Diseases, findings, and related entities with a special focus on clinical, pathologic, and molecular characteristics of neoplasms and associated abnormalities and biologic processes

• Clinical procedures, treatment, and prevention

• Molecular biology

• Radiology and imaging

• Chemicals, reagents, drugs and drug regimens, and medicinal products

• Food and nutritional components

• Biomarkers including molecular markers

• Anatomy including microanatomy

• Genetics, genomics, and clinical genomics

• Gene products and proteomics

• Nanotechnology

• Animal models and veterinary clinical research

• Laboratory procedures, biobanking, clinical tests and measurements

• Preclinical, translational, and clinical research

• Epidemiology, cancer surveillance, and public health

• Clinical trials: design, structured and computable representation of protocols, policy formulation, regulatory terminology, data management, data collection and reporting standards

• Informatics, data management, and data science

• Healthcare regulatory activities and science administration.

The Contractor shall receive, analyze, respond to, and act on incoming terminology requests from NCI and its partners and stakeholders; such requests are normally received several times each day, and may include or require sets of terms. The Contractor shall also be proactive in identifying and addressing areas where new terminology support may be needed or old terminology resources may need improvement and shall work in coordination with EVS staff to prioritize and address such needs.

2.5.3.1 Editing NCI Thesaurus, Metathesaurus, and Other EVS Terminology Content The Contractor shall provide editing support for NCI Thesaurus, NCI Metathesaurus, and other NCI terminology content, both current and potentially modified as described under the tasks in section 5.2. This support shall be provided in such a way that production and publication processes are not interrupted.

The Contractor shall model the NCI Thesaurus, and occasionally other terminology content, in description logic, including “is_a” and “part_of” relationships, preferred names, synonyms, other inter-concept relationships, special properties, term types, semantic types, other terminology/ontology attributes, and other relevant information. NCI Thesaurus is maintained in production with open source Protégé OWL2 with NCI Edit Tab and Revision History and other related plugins. The flavor of OWL2 used in the NCI Edit Tab plugin is essentially OWL DL. NCI will provide training for any new updates to NCI Edit Tab should they occur during the period of performance.

To meet NCI requirements for prompt response to requests for terminology coding, typically involving NCI Thesaurus support of NCI programs such as the NCI Drug Dictionary, the Clinical Trials Reporting Program (CTRP), The Cancer Data Standards Registry and Repository (caDSR) and similar projects and initiatives of strategic importance to NCI, such as components of Cancer Research Data Commons, the Contractor shall turn around such requests for concept codes within one to two full business days, except for large or difficult requests for which NCI approval for a longer response period is obtained. These requests usually occur several times per day.

The Contractor shall review, process and edit terminology from sources outside of the NCI

EVS, sometimes requiring conversion into Web Ontology Language (OWL), Rich Release Format (RRF), or other standard formats, ensuring format compliance, accuracy, and suitability to be used as terminology sources, value sets, or mappings in EVS terminology servers.

The Contractor shall edit the NCI Metathesaurus, including regular single source insertions and updates, and an annual Unified Medical Language System (UMLS) update (two per year or as mutually agreed upon by the Contractor and the COR), in MEME. This includes editing concepts, synonyms, definitions, semantic types, term types and relationships, processing demotions of concepts that don’t automatically merge after insertions and reviewing the edit checks. The Contractor shall provide quality control and post-production services to assure the NCI Metathesaurus data are correct (see Quality Assurance task 5.4 below). The Contractor shall notify the NCI COR when editing is complete and ready for export.

Successful completion of this task requires personnel with a minimum of five (5) years’ experience in terminology development in Protégé 5 (OWL2) and/or NCI Protégé 3.4 based on Protégé 5 (which use the OWL 2.0 specification) and Revision History. Additionally, personnel with a minimum of five (5) years’ experience with the Metathesaurus Enhancement and Maintenance Environment (MEME) and a minimum of three (3) years’ experience with the MEME 5 editing tool are required to ensure success. Senior staff assigned also require advanced (doctoral and/or master's) degrees (including but not limited to degrees in medicine, osteopathic medicine, pharmacy, nursing, biology and related fields, public health, chemistry, informatics, computer science and related fields, library science), and training and experience in relevant medical and/or nursing and/or scientific fields and biomedical informatics, with current knowledge in a discipline such as clinical and translational research, oncology, pathology, molecular biology, genomics, proteomics, pharmacology, non-neoplastic diseases, healthcare, regulatory affairs, biomedical data standards and best terminology practice, as well as familiarity with the NCI and NIH missions and activities.

2.5.3.2 Maintaining Currency of Terminology Content

The Contractor shall ensure that provided subject matter expert staff:

• Remain aware of ongoing scientific advances in areas related to their field as it applies to cancer, healthcare research and related scientific areas;

• Assess how new knowledge will affect the need for related terminology and presentation of knowledge in EVS terminology resources;

• Assess how new advances and technologies in data science affect the need for semantic content, tools, standards utilization, data modeling, coding, integration and reporting.

• Develop and present recommendations to EVS project management regarding changes in terminology modeling and content to accommodate new knowledge.

2.5.3.3 Terminology Data Conversion, Mapping, and Integration The Contractor shall prepare the NCI Thesaurus data files for NCI Metathesaurus and UMLS submission by conducting vocabulary cleanup as needed, and excluding various parts of NCI Thesaurus, as requested by the Contracting Officer’s Representative (COR). The submission format will be RRF, or another format defined by technical requirements of the submission.

Terminologies and terminology products such as mappings require conversion from native editing and file formats to formats that EVS terminology tools can load and process, including but not limited to RRF or OWL. The Contractor shall convert terminologies and terminology products to formats required for loading and publication, either via existing programs or scripts or via modification or creation of new programs or scripts. Other conversions may also be required.

2.5.3.4 Terminology Publication

The contractor shall provide technical support for publishing NCI and other terminology to the development, QA, staging, and production terminology servers (triple store terminology databases and other servers and tools as needed) and shall assist in modifications and updates to those servers, and shall provide any additional related support necessary for keeping this terminology in production and updated.

The Contractor shall coordinate with and assist NCI staff as necessary to make this process happen smoothly, correctly, and efficiently. To successfully complete this task, staff knowledgeable in database loads, software deployment, web technologies, shell scripting, programming, and operations support are required. Experience in working with terminology files and formats is desired.

Metathesaurus Production The Contractor shall provide, on an approximately 6-8 month cycle, NCI Metathesaurus update files to NCI. NCI will provide access to the source files, either new sources or updates to existing sources. Production of the update files for NCI Metathesaurus involves inversion of the source files into appropriate format for insertion into MEME in the AWS environment;

test and final insertion into MEME; and a variety of quality control steps using the MEME software.

After editing occurs, the Contractor shall export the updated NCI Metathesaurus in RRF. The RRF files shall be placed on an NCI ftp site for loading by NCI staff on NCI servers.

To successfully complete this task, Contractor staffing requires personnel able to manage the NCI Metathesaurus production process through its entire life cycle from software installation, updating, maintenance, and enhancement in the AWS environment, to data quality control, inversion, insertion, and export, with no disruption of ongoing production.

2.5.4 Quality Assurance/Control of Data, Tools, and Systems Services The Contractor shall provide comprehensive quality assurance and control of EVS terminology data, including:

• Use EVS software tools to check NCI Thesaurus, Metathesaurus, and custom-generated terminology during each production cycle and implement changes to procedures and reports as required to improve error detection.

• Formulate QC methodologies, scripts, tools, and reports to provide QC for OWL files from NCIt in Protégé, and for terminologies loaded into the EVS servers, databases, and tools and viewed and searched through EVS EXPLORE Terminology Browser.

• Provide recommendations to the COR to formulate requests to other EVS contactors for running additional scripts and/or reports on content that will assist with quality assurance.

• Work with the NCI and NCI contractor technical staff to help ensure that the quality of sources loaded into EVS servers, databases, and tools is maintained throughout the entire production lifecycle, including reversion capability where possible, i.e. ability to get back to the original source format after conversion.

For additional details on EVS QA processes, the article by de Coronado et al., “The NCI Thesaurus Quality Assurance Life Cycle” in Journal of Biomedical Informatics 2009 June;42(3):530-539 available via PubMed at http://www.ncbi.nlm.nih.gov/pubmed/19475726

The Contractor shall test releases of EVS software such as browsers, editors, and reporting, metadata validation and semantic integration tools, including user acceptance testing, shall register bugs and enhancement requests in appropriate Jira or other tracker as appropriate locations provided by NCI, and may contribute to development of such software if needed.

2.5.5 Technical and Product Documentation Services

The Contractor shall maintain the currency of documentation for NCI Thesaurus, NCI Metathesaurus, and other EVS terminology resources, EVS tools, as well as documentation related to other resources, activities, and EVS operations and processes covered by this

SOW.

The Contractor shall update the NCI Thesaurus Editor’s Guide (Standard Operating Procedures and Style Guidelines for Editing) as editorial policies change or require revised description.

The Contractor shall maintain the currency of all NCI Metathesaurus documentation, including:

the NCI Metathesaurus Editing Guide; Standard Operating Procedures for emergencies;

documentation of responsibilities by various Contractor and NCI parties for maintenance of the server(s), database, and software; documentation of the MEME server and editing software; and other relevant information about status and releases.

The Contractor shall log customer terminology requests in JIRA or other designated location(s) to provide information on users and uses of NCI or NCI distributed terminologies.

The Contractor shall ensure that staff are familiarized with, and operate in accordance with, existing NCI EVS project guidelines, standards and established processes (see https://wiki.nci.nih.gov/display/EVS/EVS+Wiki ).

2.5.6 Other Tasks Necessary to Support SOW Requirements

The Contractor shall provide other support services and provide deliverables in support of NCI EVS operations and maintenance tasks and projects within the scope of this SOW as required.

http://www.ncbi.nlm.nih.gov/pubmed/19475726 https://wiki.nci.nih.gov/display/EVS/EVS%2BWiki

2.5.7 Optional Task 1: EVS Support to the Food and Drug Administration If or when tasked, the contractor shall provide EVS support to the U.S. Food and Drug Administration (FDA) including providing semantic support to FDA via development, maintenance, and updates of controlled terminology for, but limited to, the following:

• Center for Devices and Radiological Health (CDRH) Event Problem Codes and Global Unique Device Identifier Database (CDRH-GUDID).

• Center for Veterinary Medicine (CVM); Electronic Common Technical Documents (eCTD) and eManufacturing terminolo.

• Individual Case Safety Reports (ICSR).

• Electronic Vaccine Adverse Event Reporting (eVAERS and Pharmaceutical Quality /

CMC Terminology (PQCMC).

• Quality Metrics (QM) and US Counties terminology.

• Geopolitical Entities, Names, and Codes (GENC).

• International Standards Organization (ISO) terminology.

• Regulated Product Submission (RPS) and Structured Product Labeling (SPL) terminology.

• Unique Ingredient Identifier (UNII) codes, and European Directorate for the Quality of

Medicines & Healthcare (EDQM) terminology.

2.6 DELIVERABLES AND DELIVERY SCHEDULE

All written deliverable products shall be submitted in draft format for the COR’s review and comment. Final copies shall be delivered to the COR five (5) working days after receipt of the Government’s comments. Deliverables shall be submitted with an accompanying letter describing the deliverable product and delivered to NCI CBIIT COR.

All written draft deliverable products shall be submitted in a current version of Microsoft Office unless prior approval for another format has been obtained from the COR.

2.6.1 Deliverables

The delivery schedule of products is outlined in a delivery schedule table below. Other written and technical deliverables shall be submitted according to the schedules identified in the approved Task Order Management Plan developed under this SOW. The Contractor shall deliver the following:

2.6.1.1 Task Order Management Plan

The Contractor shall prepare and submit a Task Order Management Plan specifying approaches to be used to provide the services and products specified, including the deliverable documents, analyses, and reports necessary to fulfill the requirements of this SOW. A draft version of this Plan shall be delivered with the Offeror’s proposal, with the final version due five working days after receipt of Government comments.

2.6.1.2 Monthly Task Order Status and Progress Summary Report The Contractor shall prepare and submit a draft Monthly Task Order Status and Progress

Summary Report by the 10th calendar day of the month following the reporting period. A final version shall be submitted within five working days after receipt of Government comments to the draft. This Report shall be a compilation of weekly management issues and meetings during the month reported and in sufficient detail to ensure understanding of performance progress and issues. The format and content of the Report shall include the following:

• Activities planned for the reporting period;

• Work and deliverables completed during reporting period;

• Status of ongoing activities including percentage of completion;

• Activities planned for the following reporting period;

• Planned travel during the following reporting period;

• Problems or issues projected or identified;

• Alternatives and/or recommended solution(s) for identified or projected problems or issues;

• Known or projected resources (staff and funding) and schedule impacts; and

• Status of project funds including monthly and total expenditures and funds remaining.

2.6.1.3 Monthly Financial Management Status Report

The Contractor shall prepare and submit draft and final Monthly Financial Management Status Reports. Draft reports shall be forwarded to the COR by the 10th calendar day of the month following the reporting period. A final version shall be submitted within five working days after receipt of Government comments to the draft. Reports shall be broken out by CBIIT Portfolio Item (major program or service) for review by the appropriate NCI manager. Currently CBIIT financial tracking reports on 14 portfolio items. A final report shall be submitted five days after receipt of Government comments.

The delivered Report must be suitable for forwarding to NCI executive management. The format and content of the Report shall include the following as a minimum:

• Actual hours expended for the reporting period including breakdown by labor category, name and the Task Order total.

• Actual hours expended to date and Task Order total

• Actual total costs to date and for the reporting period (based on actual hours) and Task

Order total.

• ODC monthly costs and Task Order total.

2.6.1.4 Staffing Plan

The Contractor shall prepare a draft Staffing Plan to be submitted with their proposal with a final detailed Staffing Plan specifying how they plan to staff the specific service areas within 5 days of receipt of Government comments.

2.6.1.5 Quality Assurance Plan

The Contractor shall provide and follow a Quality Assurance Plan specifying the tasks, subtasks, and approaches it will use to ensure accuracy and quality of products delivered and services provided. The Contractor shall deliver this plan in draft upon submission of their proposal, and a final copy within 5 business days of receipt of Government’s comments.

2.6.1.6 Transition-In Plan

The Contractor shall prepare and submit draft and final Transition-In Plans as prescribed in Section 2.6.2 - Delivery Schedule. This Transition-In Plan shall include detailed background information, workflow and processes, work plans and procedures, and specific activities the Contractor shall use and conduct to adequately and effectively transfer roles and responsibilities, ensure required knowledge transfer is conducted, and smoothly and effectively transition work and ongoing activities from an outgoing contractor.

2.6.1.7 Transition-Out Plan

The Contractor shall prepare and submit draft and final Transition-Out Plans as prescribed in

2.6.2 - Delivery Schedule. This Transition-Out Plan shall include detailed background information, workflow and processes, work plans and procedures, and specific activities the Contractor shall use and conduct, if not selected for a follow-on Task Order, to adequately transition the successful contractor’s employees and smoothly and effectively transition CBIIT work activities to the new…